Golang Common Concurrency Programming Tips

Golang was one of the first languages to incorporate the principles of CSP into its core and to introduce this style of concurrent programming to the general public.CSP refers to Communicating Sequential Processes, or communicating sequential processes, where each instruction needs to specify exactly whether it is an output variable (the case of reading a variable from a process), or a destination (in the case of sending input to a process).

Golang provides not only a CSP-style concurrency approach, but also supports the traditional approach of synchronization via memory access. This article provides a summary of the most commonly used Golang concurrent programming tools.

sync package

The sync package contains the most useful concurrency primitives for low-level memory access synchronization, and is the most beneficial tool for “memory access synchronization”, as well as a common tool for traditional concurrency models to solve critical zone problems.

WaitGroup

WaitGroup is a method that waits for a set of concurrent operations to complete and contains three functions.

1
2
3

func (wg *WaitGroup) Add(delta int)
func (wg *WaitGroup) Done()
func (wg *WaitGroup) Wait()

Add() is used to add the number of goroutines, Done() is used by the goroutine to indicate that execution is complete and exit, decreasing the count by one, and Wait() is used to wait for all goroutines to exit.

The usage is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

func main() {
	wg := sync.WaitGroup{}

	wg.Add(1)
	go func() {
		defer wg.Done()
		fmt.Printf("goroutine 结束\n")
	}()

	wg.Wait()
}

Note that the Add() method needs to be executed before the goroutine.

Mutual exclusion locks and read/write locks

Mutual exclusion is a way to protect critical areas in your program. A mutex can only be locked by one goroutine at a time, and the other goroutines will block until the mutex is unlocked (recontested lock on the mutex).

The usage is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

func main() {
	var lock sync.Mutex
	var count int
	var wg sync.WaitGroup

	wg.Add(1)
	// count 加 1
	go func() {
		defer wg.Done()
		lock.Lock()
		defer lock.Unlock()
		count++
		fmt.Println("count=", count)
	}()

	// count 减 1
	wg.Add(1)
	go func() {
		defer wg.Done()
		lock.Lock()
		defer lock.Unlock()
		count--
		fmt.Println("count=", count)
	}()

	wg.Wait()
	fmt.Println("count=", count)
}

Note that it is a common usage convention to call Unlock with defer in goroutine to ensure that the call is always executed even if there is a panic, preventing deadlocks.

Read-write locks are conceptually the same as mutually exclusive locks: protecting access to memory, read-write locks give you more control over memory. The biggest difference between read and write locks and mutually exclusive locks is that you can lock for reads and writes separately. It is generally used in cases where there are a lot of read operations and a few write operations.

Lock() and Unlock() for read and write locks are locking and unlocking for write operations; Rlock() and RUnlock() are locking and unlocking for read operations and need to be used in pairs.

The relationship between read locks and write locks is as follows.

1. only one goroutine can get a write lock at the same time.
2. Any number of gorouinte can be read locked at the same time.
3. only write lock or read lock can exist at the same time (read and write are mutually exclusive).

channel

Channel is one of the CSP-derived synchronization primitives, and is the most beneficial tool for Golang’s philosophy of “using communication to share memory, not communicating through shared memory”.

The basic use of Channel is not discussed here, but a summary of the results of the different operations of Channel in different states.

for-select

The select statement is the glue that binds Channels together, enabling a goroutine to wait for multiple Channels to reach readiness at the same time.

The select statement is a Channel-specific operation that looks syntactically like switch, but differs in that the case statement in the select block has no test order and execution does not fail if any condition is not met. The usage is as follows.

1
2
3
4
5
6
7

var c1, c2 <-chan interface{}
select {
  case <- c2:
    // 某段逻辑
  case <- c2:
    // 某段逻辑
}

The select control structure above will wait for the return of any of the case conditional statements and execute the code in the case immediately regardless of which one returns, but if both cases in the select are triggered at the same time, one case will be randomly selected for execution.

for-select is a very common use, usually in the “send iterator to Channel” and “loop wait stop” cases, and is used as follows.

Sends an iterative variable to the Channel.

1
2
3
4
5
6
7
8

func main() {
	c := make(chan int, 3)
	for _, s := range []int{1, 2, 3} {
		select {
		case c <- s:
		}
	}
}

Loop Waiting Stop.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

// 第一种
for {
  select {
  case <- done:
    return
  default:
    // 进行非抢占式任务
  }
}
// 第二种
for {
  select {
  case <- done:
    return
  default:
  }
  // 进行非抢占式任务
}

The first means that when we enter the select statement, if the completed channel is not closed, we will execute the default statement; the second means that if the completed channel is not closed, we will exit the select statement and continue with the rest of the for loop.

done channel

While goroutines are cheap and easy to use, and multiple goroutines can be reused at runtime for any number of OS threads, we need to be aware that goroutines are resource intensive and are not garbage collected at runtime. If there is a goroutine leak, it can lead to severe memory utilization degradation.

The done channel is a great tool to prevent goroutine leaks. The done channel creates a “signal channel” between the parent and child goroutine, which the parent goroutine can pass to the child goroutine and then close the channel when it wants to cancel the child goroutine. usage The usage is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

func main() {
	doneChan := make(chan interface{})

	go func(done <-chan interface{}) {
	   for {
		  select {
		  case <-done:
		    return
		  default:
		  }
		}
	}(doneChan)

	// 父 goroutine 关闭子 goroutine
	close(doneChan)
}

The way to ensure that a goroutine does not leak is to specify a convention that if the goroutine is responsible for creating the goroutine, it is also responsible for ensuring that it can stop the goroutine.

context Packages

The Context package is designed to simplify operations related to request-scoped data, cancellation signals, deadlines, etc. between multiple goroutines handling a single request, which may involve multiple API calls. the purpose of the Context package is twofold: to provide an API that can cancel branches in your call graph, and to provide packets for transferring request-scoped data over calls The purpose of the Context package is twofold: to provide an API that can cancel branches in your call graph, and to provide packets for transferring request scope data over the call.

If you use the Context package, then each function downstream of the top-level concurrent call takes context as its first argument.

The type of Context is as follows.

1
2
3
4
5
6

type Context interface {
  Deadline() (deadline time.Time, ok bool)
  Done() <-chan struct{}
  Err() error
  Value(key interface{}) interface{}
}

The Deadline function is used to indicate whether the goroutine will be cancelled after a certain time; the Done method returns the Channel that was closed when our function was preempted; the Err method returns the reason for the cancellation error, because of what Context was cancelled; and the Value function returns the key or nil associated with this Context.

Although Context is an interface, we don’t need to implement it when we use it, there are two methods built into the context package to create instances of the context.

1
2

func Background() Context
func TODO() Context

Background is mainly used in the main function, initialization and test code, as the top-level Context of the tree structure of Context, which cannot be canceled; TODO, if we do not know what Context to use, we can use this, but in practice, this TODO has not been used yet.

Then the parent Context is used as the top level, and the child Context is derived to start the call chain. These Context objects form a tree, and when the parent Context object is cancelled, all its child Contexts are cancelled. context package also provides a series of functions to generate child Contexts.

1
2
3
4

func WithCancel(parent Context) (ctx Context, cancel CancelFunc)
func WithDeadline(parent Context, deadline time.Time) (Context, CancelFunc)
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc)
func WithValue(parent Context, key, val interface{}) Context

WithCancel returns a new Context that closes its done channel when the returned cancel function is called; WithDeadline returns a new Context that closes its done channel when the machine’s clock exceeds a given deadline; WithTimeout returns a new Context that closes its done channel after a given timeout; WithValue generates a Context that binds a key-value pair of data that can be accessed through the Context. WithTimeout returns a new Context that closes its done channel after a given timeout; WithValue generates a Context with a key-value pair of data bound to it, which can be accessed via the Context.Value.

Here’s how to use it.

WithCancel

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

func main() {
	wg := sync.WaitGroup{}
	ctx, cancel := context.WithCancel(context.Background())

	wg.Add(1)
	go func(ctx context.Context) {
		defer wg.Done()
		for {
			select {
			case <-ctx.Done():
				fmt.Println("Err:", ctx.Err())
				return
			default:
			}
		}
	}(ctx)

	cancel()
	wg.Wait()
}

WithDeadline

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

func main() {
	d := time.Now().Add(1 * time.Second)
	wg := sync.WaitGroup{}
	ctx, cancel := context.WithDeadline(context.Background(), d)
	defer cancel()

	wg.Add(1)
	go func(ctx context.Context) {
		defer wg.Done()
		for {
			select {
			case <-ctx.Done():
				fmt.Println("Err:", ctx.Err())
				return
			default:
			}
		}
	}(ctx)

	wg.Wait()
}

WithTimeout

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

func main() {
	wg := sync.WaitGroup{}
	ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
	defer cancel()

	wg.Add(1)
	go func(ctx context.Context) {
		defer wg.Done()
		for {
			select {
			case <-ctx.Done():
				fmt.Println("Err:", ctx.Err())
				return
			default:
			}
		}
	}(ctx)

	wg.Wait()
}

WithValue

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

func main() {
	wg := sync.WaitGroup{}
	ctx, cancel := context.WithCancel(context.Background())
	valueCtx := context.WithValue(ctx, "key", "add value")

	wg.Add(1)
	go func(ctx context.Context) {
		defer wg.Done()
		for {
			select {
			case <-ctx.Done():
				fmt.Println("Err:", ctx.Err())
				return
			default:
				fmt.Println(ctx.Value("key"))
				time.Sleep(1 * time.Second)
			}
		}
	}(valueCtx)

	time.Sleep(5*time.Second)
	cancel()
	wg.Wait()
}

s