Golang http.Server graceful exit: the easily misused Shutdown() method

Writing an HTTP service in Go is easy, but getting a running service to exit safely is not so straightforward.

If you are new to the term graceful shutdown, it refers to an HTTP service that stops accepting new requests after receiving a user’s exit command, and then actively exits after processing and responding to the batch of requests it is currently processing. Unlike SIGKILL (kill -9 or force stop), a safe exit minimizes service jitter as the program rolls over for updates.

The user’s exit command is typically SIGTERM (k8s implementation) or SIGINT (often corresponding to bash’s Ctrl + C).

Listening for signals

Use the standard library signal to complete the signal listening, small field

var waiter = make(chan os.Signal, 1) // 按文档指示，至少设置1的缓冲
signal.Notify(waiter, syscall.SIGTERM, syscall.SIGINT)

// 阻塞直到有指定信号传入
<-waiter

It is worth noting that when signal.Notify() is not used, Go has a default set of signal handling rules, such as SIGHUP , SIGINT or SIGTERM will cause the program to exit directly.

Stop HTTP service

Calling the Shutdown() method of a running Server instance allows the service to be safely exited.

type GracefulServer struct {
	Server           *http.Server
	shutdownFinished chan struct{}
}

func (s *GracefulServer) ListenAndServe() (err error) {
	if s.shutdownFinished == nil {
		s.shutdownFinished = make(chan struct{})
	}

	err = s.Server.ListenAndServe()
	if err == http.ErrServerClosed {
		// expected error after calling Server.Shutdown().
		err = nil
	} else if err != nil {
		err = fmt.Errorf("unexpected error from ListenAndServe: %w", err)
		return
	}

	log.Println("waiting for shutdown finishing...")
	<-s.shutdownFinished
	log.Println("shutdown finished")

	return
}

func (s *GracefulServer) WaitForExitingSignal(timeout time.Duration) {
	var waiter = make(chan os.Signal, 1) // buffered channel
	signal.Notify(waiter, syscall.SIGTERM, syscall.SIGINT)

	// blocks here until there's a signal
	<-waiter

	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()
	err := s.Server.Shutdown(ctx)
	if err != nil {
		log.Println("shutting down: " + err.Error())
	} else {
		log.Println("shutdown processed successfully")
		close(s.shutdownFinished)
	}
}

GracefulServer has this usage.

func main() {
	flag.Parse()

	var err error
	defer func() {
		if err != nil {
			log.Println("exited with error: " + err.Error())
		}
	}()

	// 各种各样的初始化和依赖注入...
	//
	// defer tearDown()
	// defer beforeClose() // 注册各种各样的主程序退出时的清理工作
	
	server := &GracefulServer{
		Server: &http.Server{
			Addr:    fmt.Sprintf(":%d", port),
			Handler: myAwesomeHandler,
		},
	}

	go server.WaitForExitingSignal(10 * time.Second)

	log.Printf("listening on port %d...", port)
	err = server.ListenAndServe()
	if err != nil {
		err = fmt.Errorf("unexpected error from ListenAndServe: %w", err)
	}
	log.Println("main goroutine exited.")
}

Some readers may ask at this point, why do we need to encapsulate a layer of Server.ListenAndServe() it, plus shutdownFinished where the meaning of this channel? Don’t worry, Server.Shutdown()’s documentation has a sentence.

When Shutdown is called, Serve, ListenAndServe, and ListenAndServeTLS immediately return ErrServerClosed. Make sure the program doesn ’t exit and waits instead for Shutdown to return.

ListenAndServe()returns immediately afterShutdown()is called, and if we putListenAndServe()in the main functionmain(), the main function will exit soon after. In Go, no matter what the state of the other goroutines is at this point, [main function exit exits the entire program](https://golang.org/ref/spec#Program_execution), so we can't safely ensure thatServer.Shutdown()is finished. So,shutdownFinished` is placed here to provide protection.

Experienced readers may ask at this point, why not write the Server.ListenAndServe() call to the goroutine and the signal listening and Server.Shutdown() to the main function main()? Like this example written in this way, but also do not need to wrap another layer of Server, is not beautiful?

Here is a matter of opinion, ListenAndServe() in the goroutine, the error handling probability is log.Fatal(err) such an operation, if the service is not the initiative to exit (such as the start immediately encountered the port occupation error), the main function main() in the defer is not executed. I’ve used some extra complexity here to round out the safe exit logic a bit.

If you’re interested, I’ve put a copy of the full implementation on Github, so you can compile it and request another simultaneous exit for yourself to experience.

Subtle API

In fact, the use of shutdownFinished to ensure that Server.Shutdown() is finished is something I only recently realized. For about two years, I had been ignoring the fact that Server.ListenAndServe() would return immediately after Server.Shutdown() started executing, and had been using the incorrect implementation Gracefully-Shutdown-Done-Right/blob/master/wrong-way/main.go). And due to some chance coincidence, this buggy implementation has a great probability to exit correctly and safely if there are no HTTP requests being processed at the time of exit. _(:зゝ∠)_

In my personal opinion, Server.Shutdown() is an easy API to misuse.

Server.Shutdown() was discussed in 2013 and finally introduced in Go 1.8 in 2016. And the unexpected feature in the documentation we mentioned above stating that Server.ListenAndServe() would return immediately after Server.Shutdown() started executing was specifically added to the documentation six months later +/37161/), so maybe I’m not the only victim of this subtle and easy to accidentally misuse API. ``Later

Designing and adding external APIs is a really hard thing to do. The Go Team has recently been discussing how to make the standard library support generics, and I wish them the best of luck.

Table of Contents

Listening for signals

Stop HTTP service

Subtle API