Managing Go Module Pseudo-Versions in Go 1.13

Go modules have helped bring order to Go development, but there’s been some disorder lurking. Managing module pseudo-versions can be difficult, especially with some of the latest changes to Go. Let’s take a look at how pseudo-versions work, and what you can expect from those changes. We also offer some guidance on keeping your Go builds working as you upgrade to Go 1.13 and later. Go Module Versioning The ability to version is a key feature, providing developers a way to make sure their applications use the dependencies they intend. When modules are versioned, an app can specify the use of a module version they know will be compatible with the rest of their runtime. Go modules A Go module version is assigned by tagging its revision in the underlying source repository. The go command uses of the standard form vX.Y.Z to describe the module version. The version number changes based on the changes made in the API : semantic versioning From this standard format, module versions can be compared to identify which should be considered most or least current. Using Pseudo-Versions A versioned Go module is one that has been released for general use and should be preferred by most developers. However, there are some cases where you cannot release the most current version of a module. For example, a team may need to share an interim version during development. This is especially the case when a dependent project has no released versions yet, so it hasn’t been tagged with a version. Similarly, you may need to develop against a commit that hasn’t yet been tagged. To use an untagged version of a module as a dependency, it must be referenced by its pseudo-version identifier. A pseudo-version has the following format: There are : 3 acceptable forms of pseudo-version vX.0.0-yyyymmddhhmmss-abcdefxyz when no earlier versioned commit with an appropriate major version before target commit vX.Y.Z-pre.0.yyyymmddhhmmss-abcdefxyz when most recent versioned commit before the target commit is vX.Y.Z-pre vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdefxyz when most recent versioned commit before the target commit is vX.Y.Z As a best practice, a pseudo-version string should never be typed by hand. The go command will accept the plain commit hash and translate it into a pseudo-version automatically. This method helps to compare revisions based on the generated timestamp. For example, a go get command might use just the commit hash for the module query: -> go get # records v0.0.0-20180517173623-c85619274f5d github.com/x/sys@c856192 There are problems with not letting the go command automatically generate the pseudo-version: The pseudo-version participates in minimal version selection. If its version prefix is inaccurate, the pseudo-version may appear to have higher precedence than the releases that follow it, effectively pinning the module to that commit The commit date within pseudo-version provides a total order among pseudo-versions, so if it gets edited it will mess up the ordering Despite this recommendation, sometimes a pseudo-version may exist in a go module that has been edited by hand. In other instances, the full pseudo-version string may be generated by a third-party tool. Stricter Rules Through release 1.12, Go was forgiving for pseudo-version references. Most operations involving pseudo-versions accepted any arbitrary combination of version string and date, and would resolve to the underlying revision (typically a Git commit hash) as long as that revision existed. The release of Go 1.13 brought stricter enforcement, in order to address the problems noted above. Go 1.13 restricts the pseudo-versions that ‘go’ command accepts, rendering some previously accepted but not canonical versions invalid. The go client now performs some validation on different elements of the pseudo-version against the version control metadata: The tag from which the pseudo-version derives points to the named revision or one of its ancestors as reported by the underlying VCS tool, or the pseudo-version is not derived from any tag (i.e. has a “vX.0.0-” prefix before the date string) and uses the lowest major version appropriate to the module path The date string within the pseudo-version matches the UTC timestamp of the revision as reported by the underlying VCS tool The short name of the revision within the pseudo-version is the same as the short name generated by the go command. The pseudo-version includes a ‘+incompatible’ suffix only if it is needed for the corresponding major version, and only if the underlying module does not have a go.mod file Even after resolving the module from proxy, the go client will try to fetch the checksum content from the checksum server, which enforces the same pseudo-version validation rules and will refuse to serve the checksum content. How to Fix Improper Pseudo-versions In order to move to Go 1.13, a developer must correct all pseudo-version references that don’t align with the above requirements. Otherwise the go client will flag an exception: go get : invalid pseudo-version: does not match version-control timestamp (2019-08-13T06:44:41Z) golang.org/x/sys@v0.0.0-20190726090000-fde4db37ae7a Fortunately, this is pretty easy to do through your go.mod file where your pseudo-version references are made. If the go.mod file’s require directive has an incorrect pseudo-version, this can be corrected by Replace the full pseudo-version reference with just the commit hash string equire { golang.org/x/sys@fde4db37ae7a } 2. If one of the transitive dependencies references an invalid pseudo-version, you can use the replace directive in your go.mod file to force the correction: replace => golang.org/x/sys@v0.0.0-20190726091711-fde4db37ae7a fde4db37ae7a golang.org/x/sys GoCenter Can Help An important principle of GoCenter, the is being version agnostic. JFrog’s Community Engineering team has made important updates to GoCenter to support all versions of Go through 1.13, and we are in the process of further updates to accommodate Go 1.14 requirements. GoCenter can help you comply with the by redirecting to the correct pseudo-version. GoCenter changes the metadata in the .info with the correct version when the module download was requested for incorrect pseudo-version. free repository of versioned Go Modules pseudo-version validation -> curl https://gocenter.io/golang.org/x/sys/@v/fde4db37ae7a.info { “name” : “v0.0.0-20190813064441-fde4db37ae7a”, “shortName” : “v0.0.0-20190813064441-fde4db37ae7a”, “version” : “v0.0.0-20190813064441-fde4db37ae7az”, “time” : “2019-08-13T08:03:52Z” } To make use of GoCenter, set your GOPROXY GOPROXY= https://gocenter.io/ For Go 1.12 For Go 1.12 users, GoCenter will update the go.mod file held in its repository with the correct pseudo-version. GoCenter will still serve the incorrect pseudo-version that was processed in GoCenter before this change. -> go version go version go1.12.14 darwin/amd64 -> go get golang.org/x/sys@v0.0.0-20190726090000-fde4db37ae7a go: finding v0.0.0-20190726090000-fde4db37ae7a golang.org/x/sys -> cat go.mod module example go 1.12 require v0.0.0-20190813064441-fde4db37ae7a // indirect golang.org/x/sys For Go 1.13 Go 1.13 users will receive an error message that points to the correct pseudo-version. -> go version go version go1.13.5 darwin/amd64 -> go get golang.org/x/sys@v0.0.0-20190726090000-fde4db37ae7a go: finding golang.org v0.0.0-20190726090000-fde4db37ae7a go: finding v0.0.0-20190726090000-fde4db37ae7a golang.org/x/sys go: finding golang.org/x v0.0.0-20190726090000-fde4db37ae7a go get : : proxy returned info for version v0.0.0-20190813064441-fde4db37ae7a instead of requested version golang.org/x/sys@v0.0.0-20190726090000-fde4db37ae7a golang.org/x/sys@v0.0.0-20190726090000-fde4db37ae7a -> cat go.mod module example.com/mitali go 1.13 In order to update the correct pseudo-version in the go.mod file, Go 1.13 users need to change the go get to include only the commit hash part of pseudo-version. -> go get golang.org/x/sys@fde4db37ae7a go: finding golang.org fde4db37ae7a go: finding fde4db37ae7a golang.org/x/sys go: finding golang.org/x fde4db37ae7a -> cat go.mod module example.com/mitali go 1.13 require v0.0.0-20190813064441-fde4db37ae7a // indirect golang.org/x/sys If you want to override this behavior and have GoCenter serve the incorrect pseudo-version that was processed earlier, then you can set GOSUMDB= off. What Go 1.14 Changes For Modules As we noted, JFrog is working on changes to GoCenter to support Go 1.14. Here are some of the changes to Go in that release that affect the operation of modules that you may want to be aware of: go command flags go get command no longer accepts the -mod flag -mod=readonly is set by default if there is no top level vendor directory and go.mod file is read only -modfile=file new flag introduced which instructs go command to read/write an alternate go.mod file and an alternate go.sum file will also be used. Although a file named go.mod must still be present in order to determine the module root directory go.mod file changes go get will not upgrade to an +incompatible major version unless it is requested explicitly or already required go commands (except go mod tidy) will not remove a require directive that specifies a version of an indirect dependency that is already implied by other dependencies of the main module When -mod=readonly flag is set then the go command will not fail due to a missing go directive or any error Module download go command now supports Subversion repositories in module mode. Go command now includes snippets of plain-text error messages from module proxies and other HTTP servers. An error message will only be shown if it is valid UTF-8 and consists of monopoly graphic characters and spaces. Using Pseudo Versions with Confidence We hope that the steps outlined help you manage your pseudo versions more easily and effectively. As more versions are released, more will change and JFrog will do all that it can to ensure that we are ahead of the curve! Look forward to more helpful blogs from us in the very near future and If you haven’t explored GoCenter’s free repository of Go modules yet, we invite you to do so! With a rich UI that helps you examine data about all 600,000+ Go modules, it can help you gain powerful command over the GoLang dependencies you use