Developers familiar with the Go language are generally very familiar with the principles of Goroutine and Channel, including how to design applications based on the CSP model, but the Go language’s plug-in system is a module that few people know about. With the plug-in system, we can load dynamic libraries at runtime to implement some of the more interesting features.

Design Principles

The plug-in system of Go language is implemented based on C dynamic libraries, so it also inherits the advantages and disadvantages of C dynamic libraries. We will compare static libraries and dynamic libraries in Linux in this section and analyze their respective features and advantages.

  • Static libraries or static linked libraries are composed of programs, external functions and variables determined at compilation time. The compiler or linker copies the contents of the programs and variables, etc. to the target application and generates a separate executable object file.
  • dynamic libraries or shared objects can be shared among multiple executables, and the modules used by the program are loaded from the shared objects at runtime rather than packaged into separate executables when the program is compiled.

Due to the different characteristics, the advantages and disadvantages of static and dynamic libraries are also obvious; binaries that rely only on static libraries and are generated by static linking can be executed independently because they contain all the dependencies, but the compilation result is also larger; while dynamic libraries can be shared among multiple executables, which can reduce the memory footprint, and their linking process is often triggered during loading or running, so they can contain some modules that can be hot-plugged and reduce the memory footprint.

Compiling binaries using static linking has very obvious deployment advantages, and the final compiled product will run directly on most machines. The deployment benefits of static linking are far more important than the lower memory footprint, so many programming languages, including Go, use static linking as the default linking method.

Plug-in systems

Today, the low memory footprint benefits of dynamic linking are no longer very useful, but the mechanism of dynamic linking can provide more flexibility in that the main program can dynamically load shared libraries after compilation to implement a hot-pluggable plugin system.

By defining a set of conventions or interfaces directly between the main program and the shared library, we can dynamically load Go shared objects compiled by others with the following code. The advantage of this is that the developers of the main program and the shared library do not need to share code, and there is no need to recompile the main program after modifying the shared library as long as the conventions remain the same for both parties.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

type Driver interface {
    Name() string
}

func main() {
    p, err := plugin.Open("driver.so")
    if err != nil {
	   panic(err)
    }

    newDriverSymbol, err := p.Lookup("NewDriver")
    if err != nil {
        panic(err)
    }

    newDriverFunc := newDriverSymbol.(func() Driver)
    newDriver := newDriverFunc()
    fmt.Println(newDriver.Name())
}

The above code defines the Driver interface and assumes that the shared library must contain the func NewDriver() Driver function. When we read the shared library containing the Go plugin via plugin.Open, we get the NewDriver symbol in the file and convert it to the correct function type, and we can use this function to initialize the new Driver and get its name.

Operating Systems

Different operating systems implement different dynamic linking mechanisms and shared library formats. The shared objects in Linux use the ELF format and provide a set of interfaces to operate the dynamic linker.

1
2
3
4

void *dlopen(const char *filename, int flag);
char *dlerror(void);
void *dlsym(void *handle, const char *symbol);
int dlclose(void *handle);

dlopen loads the corresponding dynamic library based on the incoming filename and returns a handle; we can search for a specific symbol, i.e. function or variable, in the handle directly using the dlsym function, which returns the address where the symbol was loaded into memory. Since the symbol to be searched may not exist in the target dynamic library, we should call dlerror after each search to see the result of the current search.

Dynamic libraries

The full implementation of the Go language plug-in system is contained in plugin, a package that implements the loading and resolution of the symbol system. A plugin is a package with public functions and variables, and we need to compile the plugin using the following command.

1

go build -buildmode=plugin ...

This command generates a shared object .so file, which when loaded into the Go program is represented by the following structure plugin.Plugin, which contains information such as the path to the file and the symbols it contains.

1
2
3
4
5

type Plugin struct {
	pluginpath string
	syms       map[string]interface{}
	...
}

The two core methods related to the plugin system are plugin.Open for loading shared files and plugin.Plugin.Lookup for finding symbols in plugins.

CGO

Before analyzing the public methods in the plugin package, we need to understand the two C functions plugin.pluginOpen and plugin.pluginLookup used in the package; plugin.pluginOpen simply wraps the dlopen and dlerror functions from the standard library and returns a handle to the dynamic library when After a successful load, it returns a handle to the dynamic library.

1
2
3
4
5
6
7

static uintptr_t pluginOpen(const char* path, char** err) {
	void* h = dlopen(path, RTLD_NOW|RTLD_GLOBAL);
	if (h == NULL) {
		*err = (char*)dlerror();
	}
	return (uintptr_t)h;
}

plugin.pluginLookup uses dlsym and dlerror from the standard library to get specific symbols in dynamic library handles.

1
2
3
4
5
6
7

static void* pluginLookup(uintptr_t h, const char* name, char** err) {
	void* r = dlsym((void*)h, name);
	if (r == NULL) {
		*err = (char*)dlerror();
	}
	return r;
}

Both of these functions are relatively simple to implement, and they serve to simply encapsulate C functions in the standard library, making their signatures look more like function signatures in Go, and making it easier to call them in Go.

Loading process

The function used to load a shared object, plugin.Open, takes the path to the shared object file as an argument and returns the plugin.Plugin structure.

1
2
3

func Open(path string) (*Plugin, error) {
	return open(path)
}

The above function will call the private function plugin.open to load the plugin, which is the core function of the plugin loading process, and we can split the function into the following steps.

  1. prepare the parameters of the C function plugin.pluginOpen.
  2. calling plugin.pluginOpen via cgo and initializing the loaded module.
  3. finding the init function in the loaded module and calling that function.
  4. building the plugin.Plugin structure from the plugin’s filename and symbol list.

The first step is to prepare the arguments needed to call plugin.pluginOpen using some of the structures provided by cgo. The following code converts the filename to a variable of type *C.char, which can be passed as an argument to the C function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

func open(name string) (*Plugin, error) {
	cPath := make([]byte, C.PATH_MAX+1)
	cRelName := make([]byte, len(name)+1)
	copy(cRelName, name)
	if C.realpath(
		(*C.char)(unsafe.Pointer(&cRelName[0])),
		(*C.char)(unsafe.Pointer(&cPath[0]))) == nil {
		return nil, errors.New(`plugin.Open("` + name + `"): realpath failed`)
	}

	filepath := C.GoString((*C.char)(unsafe.Pointer(&cPath[0])))

	...
	var cErr *C.char
	h := C.pluginOpen((*C.char)(unsafe.Pointer(&cPath[0])), &cErr)
	if h == 0 {
		return nil, errors.New(`plugin.Open("` + name + `"): ` + C.GoString(cErr))
	}
	...
}

Once we have a handle to the dynamic library, we call plugin.lastmoduleinit, and the linker links it to the runtime.plugin_lastmoduleinit function, which parses the symbols in the file and returns the directory of the shared file and all the symbols it contains.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

func open(name string) (*Plugin, error) {
	...
	pluginpath, syms, errstr := lastmoduleinit()
	if errstr != "" {
		plugins[filepath] = &Plugin{
			pluginpath: pluginpath,
			err:        errstr,
		}
		pluginsMu.Unlock()
		return nil, errors.New(`plugin.Open("` + name + `"): ` + errstr)
	}
	...
}

At the end of this function, we build a new plugin.Plugin structure and iterate through all the symbols returned by plugin.lastmoduleinit, calling plugin.pluginLookup for each one.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

func open(name string) (*Plugin, error) {
	...
	p := &Plugin{
		pluginpath: pluginpath,
	}
	plugins[filepath] = p
	...
	updatedSyms := map[string]interface{}{}
	for symName, sym := range syms {
		isFunc := symName[0] == '.'
		if isFunc {
			delete(syms, symName)
			symName = symName[1:]
		}

		fullName := pluginpath + "." + symName
		cname := make([]byte, len(fullName)+1)
		copy(cname, fullName)

		p := C.pluginLookup(h, (*C.char)(unsafe.Pointer(&cname[0])), &cErr)
		valp := (*[2]unsafe.Pointer)(unsafe.Pointer(&sym))
		if isFunc {
			(*valp)[1] = unsafe.Pointer(&p)
		} else {
			(*valp)[1] = p
		}
		updatedSyms[symName] = sym
	}
	p.syms = updatedSyms
	return p, nil
}

The above function returns a plugin.Plugin structure with a mapping of symbol names to functions or variables, which the caller can use as a handle to look up the symbols in the structure.

Symbol lookup

Plugin.Lookup looks for the symbol plugin.Symbol in the structure returned by plugin.Open, which is an alias of type interface{} that we can convert to the real type of a variable or function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

func (p *Plugin) Lookup(symName string) (Symbol, error) {
	return lookup(p, symName)
}

func lookup(p *Plugin, symName string) (Symbol, error) {
	if s := p.syms[symName]; s != nil {
		return s, nil
	}
	return nil, errors.New("plugin: symbol " + symName + " not found in plugin " + p.pluginpath)
}

The private function plugin.lookup called by the above method is relatively simple to implement, it directly uses the symbol table in the structure and returns an error if the corresponding symbol is not found.

Summary

The Go language plug-in system makes use of the operating system’s dynamic libraries to achieve a modular design, which provides features that are interesting but encounter more restrictions in actual use. The current plug-in system also only supports Linux, Darwin and FreeBSD, and there is no way to use it on Windows. Because the implementation of the plug-in system is based on some black magic, so the cross-platform compilation will also encounter some rather odd problems, the author also encountered a lot of problems when using the plug-in system, if the Go language is not particularly well understood, it is still not recommended to use the module.