The Golang language syntax has a very distinctive design of error handling mechanism, which is based on the defensive programming idea. But today’s article will not discuss the syntax design of Golang error handling. Instead, today I would like to think about how error logging should be handled and printed in Golang.

5 suggestions for error handling and log printing in Golang

  1. use the error stack approach.
  2. use logical stack information instead of the code call stack.
  3. use fmt.Errorf instead of the pkg/errors third-party module.
  4. avoiding logging methods that rely on the standard library fmt formatting strings.
  5. convert external errors, based on internal error type determination.

The way to use the error stack

Since I switched to Golang development, from the Golang code I’ve seen and from my own practice, there are probably a few ways of printing error logs that I don’t think make much sense.

  1. printing an error message at every function call when an error is found.
  2. the convention that only the innermost or outermost function call prints an error message when an error is found, and if further subdivided, whether or not it carries call stack information in the error.
  3. there is no explicit specification that an error message may be printed when an error is found at any one or more calls in the entire call chain.

In the first way, the advantage is that the information of all calling nodes on the calling link is not missed, but in the real application scenario, the service threads are executed concurrently and the log lines printed by different threads are interleaved, so the logs on the same link printed in this way are very scattered. This makes it difficult to easily and quickly filter out relevant rather than interfering log lines, despite the fact that the logs contain all error-related logs. The so-called benefits exist in name only and take up a lot of disk space.

The biggest problem with the second approach is that some of the contextual information needed for error troubleshooting may be missing. Most function calls occur on calls across layers of code logic, and if the error is printed only at the innermost call, most of the parameter information for the outermost request is generally missing, imagine an example of a storage layer code call. Another way of thinking is to record the code call stack, which can help developers restore the execution path of the program, and then restore the request context information by reading the source code and reasoning, which can really improve the efficiency of problem solving. However, if the code call stack information is pure, on the one hand, there will be a lot of business-irrelevant code stack information that may be recorded to the logs resulting in wasted storage space, and on the other hand, some key contextual information may still be missing, which may also be a necessary element for problem location.

The third way, which is essentially a lack of thought on the part of the developer about error handling itself and a lack of relevant coding specifications on the part of the team, may seem like a fairly low-level problem, but it is not uncommon. This kind is naturally the most important one to avoid. I hadn’t given much thought to this myself until then.

The first and second way to effectively locate the root cause of an error is essentially the need to record the call stack information when the error occurs so that we know how the error appeared along the way, so we get the first consensus: errors need to carry call stack information.

Use logical stack information, not code call stack

Following the first point, we understand the importance of call stack information. One of the most intuitive ways of looking at the call stack is the program’s function call stack, which is not necessarily human-oriented, although it does detail the source code file where each call stack is located and the number of lines. For example, the call stack information printed by a Golang program when it encounters a panic.

1
2
3
4
5
6
7

panic: a problem

goroutine 1 [running]:
main.main()
    /tmp/sandbox4213436970/prog.go:15 +0x27

Program exited.

This approach often looks like just a bunch of stack information with file names and function names, which avoids the need to go back to the source code to read, and may make it difficult for developers who are not familiar with the business to quickly understand the cause of the problem.

Another way of thinking, in my opinion, is that if I can artificially and actively record in the code where the error was found and the parameters, etc., isn’t that also a call stack idea? And, in this way, I could additionally add the necessary contextual information. For example, I’d expect to have something like this log to retrace the process of error occurrence, which has the great advantage of being developer-friendly as well as business-descriptive.

1
2
3
4

handle upload failed, caused by:
    parse file failed, format: JSON,caused by:
    open file failed, caused by:
    file not found, path: /path/to/file

With this log, the information is easy for the developer to understand, and it is easy to read and understand the purpose of the program and the exceptions encountered. Logs such as “handle upload failed” are a logical link to the call, while “format: JSON” and “path: /path/to/ file” are necessary contextual information.

Specifically for Golang design considerations, if you need to get the call stack information of the called function in an error, you need to rely on Golang’s runtime implementation, which will result in a significant performance overhead.

So, taking into account both the bootability of error information and the performance-friendliness of the program, logical stack information should be used, and the code call stack should be avoided.

Use fmt.Errorf without the pkg/errors third-party module

This point is an extension of point 2.

In earlier versions of Golang, there was no support for error stack information in the standard library. Starting with Golang 1.13, Golang added support for a new formatting placeholder %w to the fmt.Errorf standard library function, w being short for Wrap, meaning a layer of wrapping around the raw error object. For example, to implement the logic stack in the previous section, the code is similar.

1
2
3
4
5
6
7

func main() {
    cause := errors.New("file not found, path: /path/to/file")
    err := fmt.Errorf("open file failed, %w", cause)
    err = fmt.Errorf("parse file failed, format: JSON, %w", err)
    err = fmt.Errorf("handle upload failed, %w", err)
    fmt.Println(err) // output: handle upload failed, parse file failed, format: JSON, open file failed, file not found, path: /path/to/file
}

Golang 1.13, in addition to this new feature of fmt.Errorf, correspondingly added two new functions to the errors standard library, errors.Is, which is used to determine whether a specific error value exists in the error chain for a given error, and errors.As, which is used to try to convert the error value to a specific error values.

Golang 1.13 added two new functions, errors.Is and errors.As, to the errors standard library in addition to this new feature of fmt.Errorf. The former is used to determine if a specific error value exists in the error chain for a given error, while the latter is used to try to convert the error value to a specific error value.

It is worth mentioning that this new feature of Golang 1.13 is supposed to be derived from pkg/errors, a third-party package by design, so early on people may use it to implement the error stack above.

1
2
3
4
5
6
7

func main() {
    cause := errors.New("file not found, path: /path/to/file")
    err := errors.WithMessage(cause, "open file failed")
    err = errors.WithMessage(err, "parse file failed, format: JSON")
    err = errors.WithMessage(err, "handle upload failed")
    fmt.Println(err) // output: handle upload failed: parse file failed, format: JSON: open file failed: file not found, path: /path/to/file
}

However, since Golang already implements this error stack, pkg/errors has archived the project and recommends that developers use the official Golang implementation, in order to make the application itself more forward compatible with Golang 2.0.

Avoid using logging methods that rely on the standard library fmt formatted strings

In the standard library logging implementation, which is based on the fmt standard library implementation, and the latter relies heavily on reflection, these result in relatively high CPU overhead and fine-grained memory allocation, the former directly affecting performance by crowding CPU time slices, and the latter indirectly affecting performance by increasing the burden of runtime garbage collection.

So what should we do? Consider a third-party logging library like uber-go/zap for performance optimization. zap optimizes performance mainly from several angles:

  1. using a delayed loading mechanism to avoid unnecessary calculations, e.g. some logs need to be printed at the Debug log level, but in a program started at the Info log level, this part of the log is not actually printed, and there is no need for calculations if it is not printed, so delayed loading helps to directly omit the work of formatting logs in high-level logging scenarios.
  2. using the explicit Fields mechanism, zap avoids the need for extensive reflection, plus improves performance in combination with a zero-allocation JSON serialization encoder.

Convert external errors and determine based on internal error types

External errors are errors defined outside the source code of the application, such as errors in system calls, errors returned by rpc services, and errors in database read/write operations. Application design is about layering and decoupling, and failure to convert the external errors encountered by the underlying function calls means that the upper logic is coupled with the lower implementation, destroying the low coupling. For example, for a business logic layer code, the database layer functions it depends on should give it uniform internal errors like DBConnectFailed, instead of the latter relying on errors like MySQLError or PGError defined by some SDK.