Go Embed Brief Tutorial

Go compiles programs that are perfect for deployment, and if no other libraries are referenced through CGO, we generally compile executable binaries that are single files, perfect for copying and deployment. In practice, in addition to binaries, you may also need some configuration files, or static files, such as html templates, static images, CSS, javascript, etc. How these files can also be typed into the binary would be fantastic, and we could just copy and follow the single executable.

Some open source projects started doing this a long time ago, such as gobuffalo/packr, markbates/pkger, rakyll/statik, knadh/stuffbin, etc., but anyway these are all third-party provided features, and it would be nice if Go could officially build in support. late 2019 a proposal was raised in issue#35950, expecting the official Go compiler to support embedded static files. Russ Cox later wrote a design document for Go command support for embedded static assets, and eventually implemented it.

Go 1.16 already includes the go embed feature, so you can try it out to your heart’s content.

In this article, we will go through examples and introduce the various features of go embed in detail.

Embed

For single files, embedding as strings and byte slice is supported
For multiple files and folders, support for embedding as a new file system FS
For example, importing an “embed” package, even without explicitly using the
The go:embed directive is used for embedding and must be followed by the variable name immediately after embedding
FS only supports embedding as string, byte slice and embed.FS three types, these three types of alias (alias) and named types (such as type S string) are not available

1, Embed as string

For example, there is a hello.txt file under the current file, and the content of the file is hello,world! With the go:embed directive, the value of the s variable in the following program becomes hello,world! after compilation.

package main
import (
	_ "embed"
	"fmt"
)
//go:embed hello.txt
var s string
func main() {
	fmt.Println(s)
}

2, Embed as byte slice

You can also embed the contents of a single file as a slice of byte, which is an array of bytes.

package main
import (
	_ "embed"
	"fmt"
)
//go:embed hello.txt
var b []byte
func main() {
	fmt.Println(b)
}

3, Embed as fs.FS

You can even embed as a file system, which is very useful when embedding multiple files.

For example, to embed a file:

package main
import (
	"embed"
	"fmt"
)
//go:embed hello.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
}

Embedding another local file hello2.txt, supporting multiple go:embed directives on the same variable (embedding as string or byte slice is not possible with multiple go:embed directives):

package main
import (
	"embed"
	"fmt"
)
//go:embed hello.txt
//go:embed hello2.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("hello2.txt")
    fmt.Println(string(data))
}

The current duplicate go:embed command embedded as embed.FS is supported and will automatically remove duplicates:

package main
import (
	"embed"
	"fmt"
)
//go:embed hello.txt
//go:embed hello.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
}

It is also possible to embed files under subfolders.

package main
import (
	"embed"
	"fmt"
)
//go:embed p/hello.txt
//go:embed p/hello2.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("p/hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("p/hello2.txt")
	fmt.Println(string(data))
}

Pattern matching can also be supported for embedding, as the following section is dedicated to.

Embed the same file as multiple variables

For example, in the following example, the s and s2 variables are embedded in the hello.txt file.

package main
import (
	_ "embed"
	"fmt"
)
//go:embed hello.txt
var s string
//go:embed hello.txt
var s2 string
func main() {
	fmt.Println(s)
	fmt.Println(s2)
}

Exported/unexported variables are supported

Go can embed a file either as an exported variable or as an unexported variable.

package main
import (
	_ "embed"
	"fmt"
)
//go:embed hello.txt
var s string
//go:embed hello2.txt
var S string
func main() {
	fmt.Println(s)
	fmt.Println(S)
}

Both package level variables and local variables are supported

The preceding examples are all package level variables, even if they are local variables within a function, and all support embedding.

package main
import (
	_ "embed"
	"fmt"
)
func main() {
	//go:embed hello.txt
	var s string
	//go:embed hello.txt
	var s2 string
	fmt.Println(s, s2)
}

The value of the local variable s is embedded at compile time, and although s and s2 are embedded in the same file, their values are compiled using different values in the initialization fields

0x0021 00033 (/Users/....../main.go:10)        MOVQ    "".embed.1(SB), AX
0x0028 00040 (/Users/....../main.go:10)        MOVQ    "".embed.1+8(SB), CX
0x002f 00047 (/Users/....../main.go:13)        MOVQ    "".embed.2(SB), DX
0x0036 00054 (/Users/....../main.go:13)        MOVQ    DX, "".s2.ptr+72(SP)
0x003b 00059 (/Users/....../main.go:13)        MOVQ    "".embed.2+8(SB), BX
......
"".embed.1 SDATA size=16
       0x0000 00 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00  ................
       rel 0+8 t=1 go.string."hello, world!"+0
"".embed.2 SDATA size=16
       0x0000 00 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00  ................
       rel 0+8 t=1 go.string."hello, world!"+0

Note that the values of the s and s2 variables are determined at compile time, so even if you change the hello.txt file at runtime, or even delete hello.txt, it will not change or affect the values of s and s2.

Read Only

The contents of the embedding are read-only. That is, whatever is embedded in the file at compile time is also what is embedded at runtime.

The FS file system value provides open and read methods, and no write method, which means that the FS instance is thread-safe and multiple goroutines can be used concurrently.

type FS
    func (f FS) Open(name string) (fs.File, error)
    func (f FS) ReadDir(name string) ([]fs.DirEntry, error)
    func (f FS) ReadFile(name string) ([]byte, error)

go:embed command

1, The go:embed command supports embedding multiple files

package main
import (
	"embed"
	"fmt"
)
//go:embed hello.txt hello2.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("hello2.txt")
	fmt.Println(string(data))
}

Of course you can also write it as a multi-line go:embed like the previous example:

package main
import (
	"embed"
	"fmt"
)
//go:embed hello.txt
//go:embed hello2.txt
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("hello2.txt")
	fmt.Println(string(data))
}

2, Folder support

The folder separator uses a forward slash /, even for windows systems.

package main
import (
	"embed"
	"fmt"
)
//go:embed p
var f embed.FS
func main() {
	data, _ := f.ReadFile("p/hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("p/hello2.txt")
	fmt.Println(string(data))
}

3, Relative paths are used

The root path of the relative path is the folder where the go source file is located.

Support for applying double quotes" or backquotes to embedded file or folder names or schema names, which is useful for files and folders with spaces or special characters in their names.

package main
import (
	"embed"
	"fmt"
)
//go:embed "he llo.txt" `hello-2.txt`
var f embed.FS
func main() {
	data, _ := f.ReadFile("he llo.txt")
	fmt.Println(string(data))
}

4, Matching mode

The go:embed command allows you to write only the folder name, and all files and folders in this folder except for the . and _ will be embedded, and subfolders will be recursively embedded, forming a file system for this folder.

If you want to embed files and folders starting with . and _ files and folders, such as the .hello.txt file in the p folder, then you need to use *, for example go:embed p/*.

* is not recursive, so subfolders under . and _ will not be embedded unless you are embedding specifically using the * of the subfolder:

package main
import (
	"embed"
	"fmt"
)
//go:embed p/*
var f embed.FS
func main() {
	data, _ := f.ReadFile("p/.hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile("p/q/.hi.txt") // Not embedded p/q/.hi.txt
	fmt.Println(string(data))
}

Embed and embed mode do not support absolute paths, paths containing . and .. If you want to embed the path where the go source file is located, use *:

package main
import (
	"embed"
	"fmt"
)
//go:embed *
var f embed.FS
func main() {
	data, _ := f.ReadFile("hello.txt")
	fmt.Println(string(data))
	data, _ = f.ReadFile(".hello.txt")
	fmt.Println(string(data))
}

File System

embed.FS implements the io/fs.FS interface, which opens a file and returns fs.File:

package main
import (
	"embed"
	"fmt"
)
//go:embed *
var f embed.FS
func main() {
	helloFile, _ := f.Open("hello.txt")
	stat, _ := helloFile.Stat()
	fmt.Println(stat.Name(), stat.Size())
}

It also provides ReadFileh and ReadDir functions, traversing the information of files and folders under a file.

package main
import (
	"embed"
	"fmt"
)
//go:embed *
var f embed.FS
func main() {
	dirEntries, _ := f.ReadDir("p")
	for _, de := range dirEntries {
		fmt.Println(de.Name(), de.IsDir())
	}
}

Since it implements the io/fs.FS interface, it can return its subfolders as a new filesystem:

package main
import (
	"embed"
	"fmt"
	"io/fs"
	"io/ioutil"
)
//go:embed *
var f embed.FS
func main() {
	ps, _ := fs.Sub(f, "p")
	hi, _ := ps.Open("q/hi.txt")
	data, _ := ioutil.ReadAll(hi)
	fmt.Println(string(data))
}

Applications

1, net/http

Previously, when we served a static file, we used.

`1`	`http.Handle("/", http.FileServer(http.Dir("/tmp")))`

Now, io/fs.FS file system can also be converted to http.FileServer parameters:

type FileSystem
    func FS(fsys fs.FS) FileSystem
type Handler
    func FileServer(root FileSystem) Handler

So, the embedded file can be used in the following way:

`1`	`http.Handle("/", http.FileServer(http.FS(fsys)))`

2, text/template and html/template.

Similarly, templates can be parsed from the embedded file system.

1
2

func ParseFS(fsys fs.FS, patterns ...string) (*Template, error)
func (t *Template) ParseFS(fsys fs.FS, patterns ...string) (*Template, error)

Reference https://colobu.com/2021/01/17/go-embed-tutorial/

Table of Contents