Rob opposes changes to the Go 1.18 generic standard library

Rob Pike, the father of the Go language, previously created issue to oppose the introduction of generic support in the standard library in Go 1.18. Rob’s move was both unexpected and justified. Rob’s rationale was twofold: the scope of the change was too large, and the lack of real-world experience. So Rob suggested not to change the standard library in Go 1.18, but to provide a generic version of the library at golang.org/x or golang.org/exp first. After one or two versions, we will gain enough experience to consider updating the standard library at that time. What controversial changes did the community originally want to make to the standard library? Please allow me to elaborate.

Introduction to the syntax of generics

Before we start, let’s briefly introduce the syntax of Go language generics. For example, to implement a simple Max function, you can write it like this.

func Max[T any](a, b T) T {
  if a < b {
    return b
  }
  return a
}

Here you need to insert [T any] between the function name Max and the argument list (a,b). where T is the generic argument, followed by any which is the constraint of the generic argument. As we’ll get into later, you know that the Go language requires generic parameters to specify a range. After declaring generic parameters, you can use them in parameter lists, return value lists, and inside functions.

The actual type needs to be specified when calling a generic function:

1
2
3

fmt.Println(Max[int](1, 2)) // 2
fmt.Println(Max[float32](1.1, 2.2)) // 2.2
fmt.Println(Max[string]("a", "b")) // b

For ease of use, Go generics also support type inference. That is, Go can infer the type of a generic argument based on the type of the function argument, so the above example can be rewritten as follows

1
2
3

fmt.Println(Max(1, 2)) // 2
fmt.Println(Max(1.1, 2.2)) // 2.2
fmt.Println(Max("a", "b")) // b

Does it look no different from a normal function!

Here’s a question, can we execute the following call?

`1`	`fmt.Println(Max([]int{1}, []int{2}))`

The answer is no. This is because there is a comparison judgment like if a < b {} in the Max function, and slice is not comparable with each other (see comparable for details). That is, although we want to use T to represent different argument types, the implementation of Max dictates that only types that can be compared with each other can be passed. This is the problem that constraint needs to solve.

As we said earlier, generic parameters need to specify a constraint range. So how do you define constraint? This requires a variant of interface. For example, the types that support comparison can be written like this.

type Ordered interface {
  int | int8 | int16 | int32 | int64 |
  uint | uint8 | uint16 | uint32 | uint64 | uintptr |
  float32 | float64 |
  string
}

So, the previous Max function should be rewritten as Max[T ordered](a, b T) T. Because of the added ordered restriction, Max([]int{1}, []int{2}) will report an error at compile time.

A special syntax is also supported when defining constraint

1
2
3

type Float interface {
	~float32 | ~float64
}

The ~ here means that the underlying type is float32 or float64. That is, if you define type MyFloat float32, you can also match Float. But if you don’t add ~, then it won’t match.

The interface used here, you should be able to guess that you can use the interface to limit the scope of generic parameters (you must implement the corresponding interface). If you want to support passing in all types (i.e., no restrictions), then you have to use the famous interface{}. But writing it as func foo[T interface{}])(a T) is ugly, and T must be followed by interface{}, and most generic types have interface{} as a parameter restriction, so the official word any was agreed to replace interface{}. This echoes the way Max[T any](a, b T) T is written at the top.

Generic transformation of standard libraries

With generics, many things that couldn’t be done before or were not elegant are now better.

Introducing the new standard library

To make it easier for you to write generic code, the first thing to introduce is the constraints standard package.

package constraints
// 有符号整数
type Signed interface {
	~int | ~int8 | ~int16 | ~int32 | ~int64
}
// 无符号整数
type Unsigned interface {
	~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr
}
// 所有整数
type Integer interface {
	Signed | Unsigned
}
// 浮点数
type Float interface {
	~float32 | ~float64
}
// 复数
type Complex interface {
	~complex64 | ~complex128
}
// 支持比较的类型
type Ordered interface {
	Integer | Float | ~string
}
// 任意类型的 slice
type Slice[Elem any] interface {
	~[]Elem
}
// 任意类型的 map
type Map[Key comparable, Val any] interface {
	~map[Key]Val
}
// 任意类型的 channel
type Chan[Elem any] interface {
	~chan Elem
}

where comparable is a built-in generic restriction that matches all types that allow equal comparisons. But the constraints package has just been merged, and fzipp has proposed to simplify the generic restriction syntax. Simply make [T nonInterfaceType] equivalent to [T interface{~nonInterfaceType}]. As an example.

A generic argument that matches all maps can be written as [M interface{~map[K]V}, K comparable, V any] and can also refer to the constraints package as [M constraints.Map[K, V], K comparable, V any]. However, if the fzipp proposal is supported, it can be written as [M map[K]V, K comparable, V any] . Similarly, if you want to match all slice, you can write [S []T, T any] . If you just want to simply match multiple types, you can write [T int8|int16], which is a bit of a union type.

We could care less about the details of the fzipp proposal, but one thing needs to be clear: if the fzipp proposal is accepted, then Slice/Map/Chan in the constraints package is not very necessary. This is where Rob’s concern comes in. We need real world experience to test the current generic design.

Also introduced is the slice package of operations.

package slices
import "constraints"
// 相等
func Equal[T comparable](s1, s2 []T) bool {}
func EqualFunc[T1, T2 any](s1 []T1, s2 []T2, eq func(T1, T2) bool) bool {}
// 比较
func Compare[T constraints.Ordered](s1, s2 []T) int {}
func CompareFunc[T1, T2 any](s1 []T1, s2 []T2, cmp func(T1, T2) int) int {}
// 查找
func Index[T comparable](s []T, v T) int {}
func IndexFunc[T any](s []T, f func(T) bool) int {}
func Contains[T comparable](s []T, v T) bool {}
// 插入
func Insert[S ~[]T, T any](s S, i int, v ...T) S {}
// 删除
func Delete[S ~[]T, T any](s S, i, j int) S {}
// 复制
func Clone[S ~[]T, T any](s S) S {}
// 去重
func Compact[S ~[]T, T comparable](s S) S {}
func CompactFunc[S ~[]T, T any](s S, eq func(T, T) bool) S {}
// 扩容
func Grow[S ~[]T, T any](s S, n int) S {}
// 缩容
func Clip[S ~[]T, T any](s S) S {}

Except for Index/Contains/Insert which is more commonly used, the other methods are very niche. The code for Grow, for example, has only the following line.

1
2
3

func Grow[S ~[]T, T any](s S, n int) S {
	return append(s, make(S, n)...)[:len(s)]
}

It is not known exactly how they will work in practice, and it is really too early to integrate them into the standard library.

The same introduced map operation.

package maps
// 所有 key
func Keys[M constraints.Map[K, V], K comparable, V any](m M) []K
// 所有值
func Values[M constraints.Map[K, V], K comparable, V any](m M) []V
// 判断相等
func Equal[M1, M2 constraints.Map[K, V], K, V comparable](m1 M1, m2 M2) bool
func EqualFunc[M1 constraints.Map[K, V1], M2 constraints.Map[K, V2], K comparable, V1, V2 any](m1 m1, m2 M2, cmp func(V1, V2) bool) bool
// 清空
func Clear[M constraints.Map[K, V], K comparable, V any](m M)
// 复制
func Clone[M constraints.Map[K, V], K comparable, V any](m M) M
// 把一个 map 的所有 k-v 复制到另一个
func Copy[M constraints.Map[K, V], K comparable, V any](dst, src M)
// 删除指定元素
func DeleteFunc[M constraints.Map[K, V], K comparable, V any](m M, del func(K, V) bool)

This is more commonly used is Keys/Values, other functions need to be further verified through practice.

These are the parts that introduce new standard libraries and do not create compatibility problems with existing code. There is still a part that needs to be adapted from the existing standard library, which may cause compatibility problems.

Retrofitting old standard libraries

This part can be mainly found in the discussion how to update APIs for generics started by Russ Cox. The core of the problem is that many packages already take up good names and can’t be directly retrofitted to support generic versions. For example.

The type of sync.Pool should be defined as sync.Pool[T].
The type of sync.Map should be defined as sync.Map[K, V].
The type of atomic.Value should be defined as atomic.Value[T].
List should be defined as list.List[T].
Abs should be defined as math.Abs[T].
math.Min should be defined as math.Min[T].
math.Max should be defined as math.Max[T].

Unfortunately, these types or functions are strongly associated with interface{}.

sync.Pool as an example. If you force it to sync.Pool[T], then everything that declares a Pool needs to be changed to var p sync.Pool[interface{}].

For math.Max, if you change it to a generic type directly, you will have problems with type derivation again. For example.

1
2
3

var i int
Min(i, 5) // 对应 Min[int]
Min(3, 5) // 对应 Min[float64]

So Russ proposes to add a version with an Of suffix to the relevant type or function! For example, sync.Pool corresponds to sync.PoolOf[T any]. The sync package proposal is in here. Min.MinOf feels a bit odd to be written as math.

Of course, there are proposals to add v2 suffixes or prefixes to the standard library, and even arguments over whether to use v2/math or math/v2.

Personally, I prefer DeedleFake’s solution, which doesn’t change the name, but decides whether to use generic code based on the go version of go.mod. If someone wants to upgrade to go 1.18, they can just run go fix once and upgrade all the code to the generic version. I think this is the most clean solution.

Summary

Enough content, back to Rob’s proposal. After the previous analysis, it’s easy to conclude that we lack practical experience with generics, and rushing to modify the standard library now will only create more maintenance burden for the community. So Rob’s proposal is slightly conservative, but very scientific. I’ll end this article with a quote from Rob: I strongly believe it is best to take it slow for now. use, learn, study, and move cautiously.

Table of Contents

Introduction to the syntax of generics

Generic transformation of standard libraries

Introducing the new standard library

Retrofitting old standard libraries

Summary