In a recent article “Thirteen Years of Go” published by Russ Cox on behalf of the Go core team, he mentioned that “In Go’s 14th year, the Go team will continue to strive to make Go the best environment for large-scale software engineering, with a special focus on supply chain security, improved compatibility, and structured logging. Of course there will be many other improvements, including profile-guided optimization.

The current version under development is Go 1.20, expected to be officially released in February 2023, and this version will also be the first version of Go released in its 14th year. Many people didn’t expect Go to actually go to Go 1.2x instead of Go 2.x. Remember that Russ Cox said there would probably never be a Go2. After all, the Go generic syntax landed such a big syntax change that it didn’t invalidate the Go1 compatibility promise.

Go 1.20 is currently in full swing, and many gophers are curious about what new features Go 1.20 will bring. In this article, I’ll take you through the list of issues in the Go 1.20 milestone to see what new features will be added to Go.

1. Syntax changes

Go saw its biggest syntax change since open source in version 1.18, and then what? Go has once again fallen silent on syntax evolution, and yes, this is the style Go has long maintained.

If there is a syntax change in Go 1.20, it is probably this issue: “spec: allow conversion from slice to array”, i.e. allow type conversion from slice to array.

Prior to Go 1.20, we wrote the following code using Go 1.19 as an example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

package main

import "fmt"

func main() {
    var sl = []int{1, 2, 3, 4, 5, 6, 7}
    var arr = [7]int(sl) // Compile Exception: cannot convert sl (variable of type []int) to type [7]int
    fmt.Println(sl)
    fmt.Println(arr)
}

In this code, we perform a []int to [7]int type conversion, but in Go 1.19 the compiler reports an error for this conversion! That is, explicit conversion of slice types to array types is not supported.

Prior to Go 1.20 if you wanted to implement a slice-to-array conversion, there was a trick, see the following code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

func main() {
    var sl = []int{1, 2, 3, 4, 5, 6, 7}
    var parr = (*[7]int)(sl)
    var arr = *(*[7]int)(sl)
    fmt.Println(sl)  // [1 2 3 4 5 6 7]
    fmt.Println(arr) // [1 2 3 4 5 6 7]
    sl[0] = 11
    fmt.Println(sl)    // [11 2 3 4 5 6 7]
    fmt.Println(arr)   // [1 2 3 4 5 6 7]
    fmt.Println(*parr) // [11 2 3 4 5 6 7]
}

The theory behind the trick is that Go allows to get the address of the underlying array of the slice. In the above example parr is a pointer to the underlying array of the slice sl, and any changes to the elements of the underlying array via sl or parr will be reflected in each other. But arr, on the other hand, is a copy of the underlying array, and subsequent changes to the slice via sl or to the underlying array via parr do not affect arr, and vice versa.

But this trick syntax is not that intuitive! So the above issue of “allowing direct conversion of slices to arrays” was raised. We can use the latest go tip code by selecting “go dev branch” in the go playground, and let’s try the latest syntax.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

func main() {
    var sl = []int{1, 2, 3, 4, 5, 6, 7}
    var arr = [7]int(sl)
    var parr = (*[7]int)(sl)
    fmt.Println(sl)   // [1 2 3 4 5 6 7]
    fmt.Println(arr)  // [1 2 3 4 5 6 7]
    sl[0] = 11
    fmt.Println(arr)  // [1 2 3 4 5 6 7]
    fmt.Println(parr) // &[11 2 3 4 5 6 7]
}

We see that directly converting sl to array arr no longer reports an error, but its semantics is the same as the previous “var arr = ([7]int)(sl)”, i.e., it returns a copy of the underlying array of slices, and arr is not affected by subsequent changes in the elements of the slices.

However, there is a constraint that the length of the converted array should be less than or equal to the slice length, otherwise it will panic.

1
2

var sl = []int{1, 2, 3, 4, 5, 6, 7}
var arr = [8]int(sl) // panic: runtime error: cannot convert slice with length 7 to array or pointer to array with length 8

2. Compilers/linkers and other tool chains

profile-guided optimization

The Go compiler team has been working on optimizing the Go compiler/linker for a long time, and this time in Go 1.20 the team will probably bring us “profile-guided optimization”.

What is “profile-guided optimization”? The original Go compiler implemented optimizations, such as inlining, were based on fixed rule-based decisions, with all information coming from the compiled Go source code. The “profile-guided optimization”, as the name implies, requires information outside the source code to “guide” the decision on which optimizations to implement, and this information outside the source code is the profile information. The information outside the source code is the profile information, i.e., the data collected by the pprof tool during the program runtime, as shown in the following figure (from the profile-guided optimization design document):

profile-guided optimization

So pgo optimization actually requires programmer participation. The programmer takes the program to the production environment and runs it, the profile performance collection data generated by the program is saved, and this profile collection data is then provided to the Go compiler to aid in optimization decisions the next time the same program is built. Since these profiles are data from the production environment or simulated production environment, it makes this optimization more targeted. And, the results of implementing PGO optimization for other languages (C/C++) in Google’s data center show that the performance improvement after optimization is conservatively estimated at 5% to 15%.

Like other newly introduced features, Go 1.20 will include this feature, but it is not on by default, we can manually turn it on for experience, and only in future versions will the pgo feature be auto on by default.

Dramatically reduce the size of Go distribution packages

With the evolution of the Go language, the size of the Go distribution is also increasing, from a few tens of M at the beginning to hundreds of M today. a few more Go versions installed in the local computer, (after decompression) a few G will be gone, in addition to the large size also makes the download time longer, especially in some areas with poor network environment.

Why is the Go distribution size getting bigger and bigger? This is largely due to the fact that the Go distribution contains pre-compiled .a files for all packages under GOROOT. Take the go 1.19 macos version as an example, under $GOROOT/pkg, we see the following .a files, and use du to check the disk space occupied, which amounts to 111M.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

$ls
archive/    database/   fmt.a       index/      mime/       plugin.a    strconv.a   time/
bufio.a     debug/      go/     internal/   mime.a      reflect/    strings.a   time.a
bytes.a     embed.a     hash/       io/     net/        reflect.a   sync/       unicode/
compress/   encoding/   hash.a      io.a        net.a       regexp/     sync.a      unicode.a
container/  encoding.a  html/       log/        os/     regexp.a    syscall.a   vendor/
context.a   errors.a    html.a      log.a       os.a        runtime/    testing/
crypto/     expvar.a    image/      math/       path/       runtime.a   testing.a
crypto.a    flag.a      image.a     math.a      path.a      sort.a      text/

$du -sh
111M    .

The size of the entire pkg directory is 341M, which is nearly 70% of the total size of 495M in Go 1.19.

So at the suggestion of the Go community, the Go team decided that starting with Go 1.20 the distribution would no longer provide pre-compiled .a files for most of the packages in GOROOT, and that the new version would only include .a files for the few packages in GOROOT that use cgo.

So with Go 1.20, the source code under GOROOT will be cached in the native cache after the build like other user packages. Also, go install will not install .a files for GOROOT packages, except for those packages that use cgo. This will reduce the size of the Go distribution by up to two-thirds.

Instead, these packages will be built when needed and cached in the build cache, just as is already done for non-main packages outside of GOROOT. In addition, go install will also not install .a files for GOROOT packages, except for those packages that use cgo. These changes are intended to reduce the size of Go distributions, in some cases by as much as two-thirds.

Extend code coverage reporting to the application itself

I’m sure you’ve all used go test’s output for code coverage. go test will inject code into the unit test code to count the paths of the tested packages covered by the unit test, here is an example of code injection.

1
2
3
4
5

func ABC(x int) {
    if x < 0 {
        bar()
    }
}

After injecting the code.

1
2
3
4
5

func ABC(x int) {GoCover_0_343662613637653164643337.Count[9] = 1;
  if x < 0 {GoCover_0_343662613637653164643337.Count[10] = 1;
    bar()
  }
}

Code like GoCover_xxx will be placed under each branch path.

But go test -cover also has a problem that it is only suitable for collecting data and providing reports for packages, it cannot give code coverage reports for the application as a whole.

The “extend code coverage testing to include applications” proposal in Go 1.20 is to extend code coverage to support coverage statistics and reporting for the application as a whole.

This feature will also be available as an experimental feature in Go version 1.20 and is off by default. The proposal generates applications injected with coverage statistics code by means of go build -cover, and during the execution of the application, the report will be generated to the specified directory, and we can still view this holistic report by means of go tool cover.

In addition, the new proposal is similar to go test -cover in principle, it is a source-to-source solution, so that it can be maintained uniformly. Of course, the Go compiler will also have some changes.

Deprecate -i flag

This is a planned “deprecation”. Since the introduction of the go build cache in Go 1.10, go build/install/test -i no longer installs compiled packages under $GOPATH/pkg.

3. Go Standard Library

Support wrap multiple errors

Go 1.20 adds a mechanism to wrap (wrap) multiple errors into a single error, making it easy to get information from the Error method of a packed error containing a series of related errors about the error at once.

This mechanism adds an (anonymous) interface and a function.

1
2
3
4
5

interface {
    Unwrap() []error
}

func Join(errs ...error) error

Also enhances the semantics of functions like fmt.Errorf. When using multiple %w verb’s in Errorf, e.g.

1

e := errors.Errorf("%w, %w, %w", e1, e2, e3)

Errorf will return an instance of the error type that wraps e1, e2, e3 and implements the above interface with the Unwrap() []error method.

The semantics of the Join function is to pack all incoming err into an instance of the error type that also implements the above interface with the Unwrap() []error method, and the error method of the type of the error instance returns a list of newline-spaced errors.

Let’s look at the following example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

package main

import (
    "errors"
    "fmt"
)

type MyError struct {
    s string
}

func (e *MyError) Error() string {
    return e.s
}

func main() {
    e1 := errors.New("error1")
    e2 := errors.New("error2")
    e3 := errors.New("error3")
    e4 := &MyError{
        s: "error4",
    }
    e := fmt.Errorf("%w, %w, %w, %w", e1, e2, e3, e4)

    fmt.Printf("e = %s\n", e.Error()) // error1 error2, error3, error4
    fmt.Println(errors.Is(e, e1)) // true

    var ne *MyError
    fmt.Println(errors.As(e, &ne)) // true
    fmt.Println(ne == e4) // true
}

We first compile and run the above program in Go 1.19.

1
2
3
4

e = error1 %!w(*errors.errorString=&{error2}), %!w(*errors.errorString=&{error3}), %!w(*main.MyError=&{error4})
false
false
false

Apparently Go 1.19’s fmt.Errorf function does not yet support multiple %w verb.

And Go 1.20 compiles the above program to run as follows.

1
2
3
4

e = error1 error2, error3, error4
true
true
true

Replace the line fmt.Errorf with the following

1

e := errors.Join(e1, e2, e3, e4)

The result of running it again is.

1
2
3
4
5
6
7

e = error1
error2
error3
error4
true
true
true

That is, the Error method of the Join function’s packed error type instance type returns a list of newline-spaced errors.

New arena experiment package

Go is a language with GC, although Go GC has continued to improve in recent years, the majority of occasions are not a big problem. But in some performance-sensitive areas, the GC process takes up considerable arithmetic power that still overwhelms applications.

The main idea of reducing GC consumption is to reduce heap memory allocation, reduce repeated allocation and release. some projects in the Go community, in order to reduce memory GC pressure, build another set of simple memory management mechanisms on top of mmaped memory that are not perceived by GC and apply them in appropriate occasions. However, all these self-implemented, GC-independent memory management have their own problems.

Go 1.18 version release, arena this proposal was on the agenda, arena package and is a google internal experimental package, it is said that the effect is not bad (in improving the grpc protobuf deserialization experiments), can save 15% of cpu and memory consumption. However, once the proposal came out, it received comments from all sides, the proposal in Go 1.18 and Go 1.19 once in the hold state, until Go 1.20 was included in the experimental features, we can turn on the mechanism through GOEXPERIMENT=arena.

The main idea of arena package is actually “overall allocation, piecemeal use, and then overall release”, in order to minimize the pressure on GC. About arena package, after further improvement, there may be a special article to analyze.

time package changes

The time package adds three time layout format constants, which I believe do not need to be explained, we all know how to use.

1
2
3

DateTime   = "2006-01-02 15:04:05"
DateOnly   = "2006-01-02"
TimeOnly   = "15:04:05"

The time package also adds a Compare method to Time for >= and <= comparisons between times.

1
2

// Compare returns -1 if t1 is before t2, 0 if t1 equals t2 or 1 if t1 is after t2.
func (t1 Time) Compare(t2 Time) int

In addition, the time package’s RFC3339 time format is the most widely used time format, and its parsing performance has been optimized in Go 1.20, improving by about 70% and formatting performance by 30%.

4. others

5. Ref

  • https://github.com/golang/go/milestone/250
  • https://www.polarsignals.com/blog/posts/2022/09/exploring-go-profile-guided-optimizations/
  • https://twitter.com/mvdan_/status/1588242469577117696
  • https://tonybai.com/2022/11/17/go-1-20-foresight/