Using Mock and Interface for Golang Unit Testing

In my work I often find that many engineers’ Golang unit tests are written in a problematic way, simply calling the code for output and including various IO operations, making it impossible to run the unit tests everywhere.

Golang Unit Testing with Mock and Interface

This article explains how to do unit tests properly in Golang.

What is unit testing? Features of unit testing

Unit testing is a very important part of quality assurance. A good unit test not only finds problems in time, but also facilitates debugging and improves productivity, so many people think that writing unit tests takes extra time and will reduce productivity, which is the biggest prejudice and misunderstanding of unit testing.

Unit testing will isolate the module corresponding to the test for testing, so we have to remove all relevant external dependencies as much as possible and only unit test the relevant modules.

So some of the unit tests you see in the business code repository that call HTTP in the client module are actually irregular, because HTTP is an external dependency and your unit tests will fail if your target server fails.

func Test_xxx(t *testing.T) {
 DemoClient := &demo.DemoClient{url: "http://localhost:8080"}
 DemoClient.Init()
 resp := DemoCliten.DoHTTPReq()
 fmt.Println(resp)
}

In the above example, DemoClient will call the http.Get method when doing the DoHTTPReq method, which contains an external dependency that means he will request the local server, if a new colleague just got your code and there is no local server, then your unit test will fail.

And in the above example, the function DoHTTPReq is simply an output without any check on the return value. If the internal logic is modified and the return value is modified, although your test can still pass, your unit test will not work.

From the above example, we can summarize two unit test features:

No external dependencies, no side effects as much as possible, and the ability to run everywhere
Checking the output

Another point I would like to mention is that there is actually a ranking of the difficulty of writing unit tests.

UI > Service > Utils

So for writing unit tests, we will give priority to the unit tests for Utils, because Utils will not have too many dependencies. Next is the unit test for Service, because the unit test for Service mainly depends on the upstream service and database, so we only need to separate the dependencies and then we can test the logic.

So how do you isolate the dependencies? Let’s go to the next section, where we will cover how to separate out dependencies.

What is Mock?

For IO dependencies, we can use Mock to mock the data so that we don’t have to worry about unstable data sources.

So what is Mock? And how do we Mock it? Think of a scenario where you and your colleague are working on a collaborative project and your side is progressing faster and is almost done with your development, but your colleague is progressing a little slower and you are still dependent on his services. How do you continue development without block your progress?

Here you can use Mock, that is, you and your colleague can work out the data format you need to interact with in advance, and in your test code, you can write a client that can generate the corresponding data format, and the data is false, then you can continue writing your code, and when your colleague finishes his part of the code, you just need to replace the Mock Clients with the real Clients and you’re done. That’s what Mock does.

Similarly, we can use Mock to mock the data on which the module needs to be tested. The following is an example.

package myapp_test
// TestYoClient provides mockable implementation of yo.Client.
type TestYoClient struct {
    SendFunc func(string) error
}
func (c *TestYoClient) Send(recipient string) error {
    return c.SendFunc(recipient)
}
func TestMyApplication_SendYo(t *testing.T) {
    c := &TestYoClient{}
    a := &MyApplication{YoClient: c}
    // Mock our send function to capture the argument.
    var recipient string
    c.SendFunc = func(s string) error {
        recipient = s
        return nil
    }
    // Send the yo and verify the recipient.
    err := a.Yo("susy")
    ok(t, err)
    equals(t, "susy", recipient)
}

We will have a MyApplication that also depends on a YoClient that sends reports. In the above code we will replace the dependent YoClient with TestYoClient, so that when the code calls MyApplication.Yo, it actually executes TestYoClient.Send, so that we can customize the input and output of the external dependency.

It is also interesting to note that we have replaced SendFunc with func(string) error in TestYoClient so that we can control the input and output more flexibly. For different tests, we only need to change the value of SendFunc. This way we can control the input and output of each test as much as we want.

Another problem you will find at this point is that if you want to successfully inject TestYoClient into MyApplication, the corresponding member variable needs to be either the concrete type TestYoClient or an interface type that satisfies the Send() method. But if we use a concrete type, there is no way to replace the real Client with the Mock Client.

So we can use the interface type to replace it.

What is Interface?

In Golang the interface may be different from the interfaces of other languages you’ve come across, in Golang the interface is a collection of functions. And Golang interfaces are implicit and do not need to be defined explicitly.

Personally, I agree with this design, because after much practice, I have found that pre-defined abstractions often do not accurately describe the behavior of concrete implementations. So you need to do abstraction afterwards, instead of writing types to meet interface, you should write interfaces to meet the usage requirements.

Always abstract things when you actually need them, never when you just foresee that you need them.

My personal recommendation is that for several similar processes, we can first organize the code by writing several structures, and then after we find that these structures have similar behaviors, we can abstract an interface to describe these behaviors, which is the most accurate.

At the same time, the number of methods included in the interface in Golang should be limited, not too many, 1-3 methods is enough. The reason for this is that if your interface contains too many methods, you will have a lot of trouble adding a new code that implements the type, and the code is not easy to maintain. Likewise if your interface has many implementations and many methods, it will be difficult to add one more function to the interface, and you will need to implement those methods in each structure.

Back to the topic, for YoClient, initially if we don’t use the TDD approach, then MyApplication must depend on a formal concrete type, at this point we can write an instance of TestYoClient type in the test code, extract the common functions to extract the interface, and then go to replace YoClient in MyApplication with the interface type.

This will accomplish our goal.

package myapp
type MyApplication struct {
    YoClient interface {
        Send(string) error
    }
}
func (a *MyApplication) Yo(recipient string) error {
    return a.YoClient.Send(recipient)
}

Some other examples

I have also provided an example for reference, mostly from official production code, that masks sensitive information.

This example is a mock of the external dependency etcd.

type ETCD interface {
 GetWithTimeout(key string, opts ...clientv3.OpOption) (*clientv3.GetResponse, error)
 Watch(ctx context.Context, key string, opts ...clientv3.OpOption) clientv3.WatchChan
}

type MockEtcdClient struct {
 GetWithTimeoutFunc func(key string, opts ...clientv3.OpOption) (*clientv3.GetResponse, error)
 WatchFunc          func(ctx context.Context, key string, opts ...clientv3.OpOption) clientv3.WatchChan
}

func (m MockEtcdClient) GetWithTimeout(key string, opts ...clientv3.OpOption) (*clientv3.GetResponse, error) {
 return m.GetWithTimeoutFunc(key, opts...)
}

func (m MockEtcdClient) Watch(ctx context.Context, key string, opts ...clientv3.OpOption) clientv3.WatchChan {
 return m.WatchFunc(ctx, key, opts...)
}

An example of a unit test:

func Test_saveTestConf(t *testing.T) {
 etcd := store.MockEtcdClient{
  GetWithTimeoutFunc: func(key string, opts ...clientv3.OpOption) (*clientv3.GetResponse, error) {
   return &clientv3.GetResponse{
    Kvs: []*mvccpb.KeyValue{
     {
      Key:   []byte("/xxxx/xxx/config"),
      Value: []byte("{\"xxx\":\"xxx\"}"),
     },
    },
   }, nil
  },
  WatchFunc: func(ctx context.Context, key string, opts ...clientv3.OpOption) clientv3.WatchChan {
   return nil
  },
 }

 configKey, err := saveTestConf(etcd ,"xxxx", "/xxxx/xxx/config")
 if err != nil {
  t.Error(err)
 }
 assert.Equal(t, "/xxxx/xxx/config", configKey)
}

other tips

There are some tips you may not know about testing on my end

golang’s interal and external testing

For a package’s exported methods and variables you can create test files under the same package to test them, just by changing the package name suffix to _test. This way you can do black box testing. The advantage is that you can describe your test from the caller’s point of view, rather than writing your test from an internal point of view. It can also be used as an example to show users how to use it.

// in example.go
package example

var start int

func Add(n int) int {
  start += n
  return start
}

// in example_test.go
package example_test

import (
 "testing"

 . "bitbucket.org/splice/blog/example"
)

func TestAdd(t *testing.T) {
  got := Add(1)
  if got != 1 {
    t.Errorf("got %d, want 1", got)
  }
}

Alternatively you can test unexported methods and variables by creating a file with the suffix _internal_test to identify that you want to test unexported methods and variables.

Summary

Features of Golang unit tests.
- No external dependencies, no side effects as much as possible, ability to run everywhere
- Need to check the output
- Can be used as an example to show users how to use
Golang can use interfaces to replace dependencies

Table of Contents