go module

I recently received a question about golang from a reader that reads as follows:

In a directory, I wrote a.go and a_test.go, after go mod init main and execute go test, it will report error: could not import main( can not import "main"). I know its solved by changing the package name. My questions are:

  1. is it impossible to execute package tests on main package.
  2. what is the underlying reason for the error reported here.

This article will do a brief analysis of the problem, which will involve the concepts of go module, go package and package import as well as the working principle of go test.

1. Build a test environment and reproduce the problem

Let’s set up a test environment to reproduce the problem this reader encountered.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

func Add(a, b int) int {
    return a + b
}

$cat pkg_test.go
package main

import (
    "testing"
)

func TestAdd(t *testing.T) {
    n := Add(5, 6)
    if n != 11 {
        t.Errorf("want 11, got %d\n", n)
    }
}

There! We execute go test to run the test.

1
2
3
4

$go test
# main.test
/var/folders/cz/sbj5kg2d3m3c6j650z0qfm800000gn/T/go-build1902276879/b001/_testmain.go:14:8: could not import main (cannot import "main")
FAIL    main [build failed]

We see: Here the go test command executed with Go version 1.20 reports an error! The error is the same as the reader’s question! Next, let’s analyze why the error is reported!

2. go module, go package, and import path

Before analyzing the problem, we still need to sort out the concepts of go module, go package and import path.

The concept of go package is familiar to everyone, it is the basic compilation unit of Go. The later go module, import path and package concepts are all related.

Go introduced go module in version 1.11, after which go module replaced gopath build mode and became the standard Go build mode.

The definition of a go module in the Go mod reference manual is: “A module is identified by a module path. The go.mod file declares the module path and the dependency information about the module ( require, replace, etc.). The directory containing the go.mod file is called the module root directory. main module is the module that contains the directory where the go command is invoked.

Note: This article only discusses the go module pattern, the obsolete GOPATH pattern is no longer discussed.

Note: The term main module is extremely confusing and may be changed to work module in Go 1.21 or subsequent versions.

Go module was introduced to solve the dependency management problem, so go module is a collection of packages, and the version of this package is bound to the module version. However, the introduction of go module also has a slight impact on the determination and meaning of the package import path.

In the era of GOPATH build mode, the import path (import path) of a go package is determined by the path of the directory where the package is located relative to $GOPATH/src. For example, if your package is placed under $GOPATH/src/github.com/user/repo, then the import path of your package is import “github.com/user/repo”. This is the same as the path rule for downloading packages from go get at that time.

In the Go module era, $GOPATH/src is no longer mandatory and there is no longer any coupling between go module and $GOPATH/src. At this point, go package import paths are determined by both the go module path and the package’s relative path in the module .

  • If your module path (declared in the go.mod file) is github.com/user/yourmodule, and your package is in the foo/bar directory under the root path of yourmodule, then the import path for your package is github.com/user/ yourmodule/foo/bar.
  • If your module uses a custom module path, e.g. example.com/go/yourmodule, then again, if your package is in the foo/bar directory under the root path of yourmodule, the import path for this package will be example.com/go/ yourmodule/foo/bar.
  • If your module uses a “local path” like tonybai/yourmodule instead of the above two URLs, then if your package is in the foo/bar directory under the root path of yourmodule, the package will be imported to tonybai/yourmodule/foo/bar.

Note: In addition to being a prefix for the package import path, the module path can also be used to indicate the url address of the version hosting service where the module is stored.

How do the above concepts and their relationships help to solve the problem at the beginning of our text? Don’t worry, the following corollary is strongly related to that question in this paper.

3. what is the import path of the package in the module root directory

Okay, here is the question that is most relevant to the one at the beginning of this article: What is the import path of the package in the module root directory of go module? According to the definition of package import path in go module mode above: the import path of packages in the root directory of go module is module path.

Take our test project above as an example, the root path of main module is module-path-main directory, and a package main(pkg.go) is stored under this directory, then the import path of this main package is the module path of go module: “main “. Even if you change the package name in pkg.go from “main” to “demo”, the package path of the demo package will still be “main”.

Note: The reader’s statement in the question that changing the go package name will solve the problem is incorrect. Change the package name main above to demo, go test will still report the same error.

4. the principle of go test

Okay! We are almost done reviewing the concepts of go module, package, and package import path. The review of these concepts is a prerequisite for solving the problem at the beginning of the article, so let’s put them in our brains for now. Let’s talk about the other half of the knowledge: go test.

Go test is Go’s built-in testing framework, which we can use to drive unit tests, integration tests, and even automated tests.

After executing go test inside a package, go test will first compile the target package, then compile the test package (the test package and the target package may be one package or different packages), i.e. all the source files in the directory with the _test.go suffix; go test will compile the test package into an executable, the main package of this executable will depend on and import the test package, and will call the TestXxx exported methods in the test package to execute the test. and will call the TestXxx exported methods in the test package to execute the test.

Note: go test -c can get this executable file pkg.test

5. The truth comes out

Well, with the above knowledge about the two preparations, let’s uncover the truth of the matter!

We use go test -work to see the file where the main function of the executable generated by go test execution is located (the purpose of passing in the -work flag is to allow the go compiler to keep the temporary directory where the test source files are built after compilation).

1
2
3
4
5
6
7

# Execute under module-path-main

$go test -work
WORK=/var/folders/cz/sbj5kg2d3m3c6j650z0qfm800000gn/T/go-build2039841248
# main.test
/var/folders/cz/sbj5kg2d3m3c6j650z0qfm800000gn/T/go-build2039841248/b001/_testmain.go:14:8: could not import main (cannot import "main")
FAIL    main [build failed]

Open _testmain.go under b001 in the temporary path, this file is generated by the go test tool.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

// Code generated by 'go test'. DO NOT EDIT.

package main

import (
    "os"

    "testing"
    "testing/internal/testdeps"

    _test "main"

)

var tests = []testing.InternalTest{

    {"TestAdd", _test.TestAdd},

}

var benchmarks = []testing.InternalBenchmark{

}

var fuzzTargets = []testing.InternalFuzzTarget{

}

var examples = []testing.InternalExample{

}

func init() {
    testdeps.ImportPath = "main"
}

func main() {

    m := testing.MainStart(testdeps.TestDeps{}, tests, benchmarks, fuzzTargets, examples)

    os.Exit(m.Run())

}

We see that this is the contents of the main package and main function of the test executable that we want to compile out, the most critical line of which is as follows:

1
2
3
4
5
6
7

import (
    ... ...

    _test "main" // This line is the "culprit" for the error in the execution of `go test`

    ... ...
)

According to the definition of package import path under go module we reviewed before, “main” here is actually the import path of the package under the root path of module-path-main, as we said before: the import path of this top-level package is module path (no matter what the package name is, whether it is main or demo), and here we define the module path is main, so the path here is “main”.

According to the package import path rule, if the import path is like “fmt” or “io”, the go compiler will search from the standard library; if it is “main”, then is considered as main package.

Well, here’s the problem! This _testmain.go is the main package of the test executable generated by go test, and it now imports a “main” package, which is not allowed in Go. Because main packages and main functions are usually used to integrate your code units (i.e. packages), if your other code units depend on main again, it will cause a “circular import”, which is absolutely forbidden in Go. This is the real reason for the problem at the beginning of the article.

Note: The main package supports unit testing, but it is generally recommended that you do not unit test against the main package. If you have code in main that is worth testing (for unit testing; not for integration testing), consider moving it to a library package.

After knowing the real cause, the solution is also very simple, that is, rename the module path, for example, to demo, so that go test will be executed successfully. After changing to demo, the imported code in _testmain becomes something like this:

1
2
3
4
5
6
7

import (
    ... ...

    _test "demo" 

    ... ...
)

6. Ref

  • https://tonybai.com/2023/04/08/the-reason-why-go-test-fails-when-module-path-is-main/