Read and write network packets in batches for high performance - Addendum

The previous article introduced the sendmmsg system call, which sends network packets in batches, and this article talks about how this differs from writeev.

In fact, looking at their definitions, you can see where the difference lies.

1
2

int sendmmsg(int sockfd, struct mmsghdr *msgvec, unsigned int vlen, int flags);
ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

The sendmmsg() system call is an extension of sendmsg(2) that allows the caller to transmit multiple messages on a socket using a single system call. (This has performance advantages for some applications.) The sockfd parameter is the file descriptor of the socket on which the data is to be transferred. The msgvec parameter is a pointer to an array of mmsghdr structures. The size of this array is specified in vlen. sendmmsg is mainly used for network batch writing, if you want to operate on file IO batches, you can use writev and readv.

The writev() system call writes iovcnt cached data to the file descriptor fd (“gather output”). The data it handles is “atomic” and is not disturbed by other concurrent read and write operations. This means that when using these functions in a multi-threaded or multi-process environment, data can be safely transferred without fear of data fragmentation or confusion.

Under regular circumstances, the write() function does not write only part of the data. It either writes all of the data as a single operation, or returns an error indicating that the write failed.

However, there is a special case where only partial writes may occur, namely when a non-blocking socket is used for a write operation and the write buffer is full. In this case, writev() may only write part of the data, and the remaining data will be returned for writing later. In this case, further processing is required based on the returned result to ensure that all data is written correctly.

Therefore, when using the writev() function, it is recommended to check the number of bytes returned for writing to ensure that all data is written correctly, and to handle partial writes with a retry operation if needed.

In fact, sending data using sendmmsg and writeev is boring and anti-human, requiring the construction of specific data structures such as mmsghdr and iovec. In the last article we covered Go’s wrapping of sendmmsg, which simplifies calls to sendmmsg. In this article we introduce Go’s wrapping of the writev system. Go does not wrap readv, even though some people have contributed code that implements it, mainly because people feel that they have not yet seen the performance gains and benefits that readv brings.

Go internally wraps Writev in src/internal/poll/writev.go, and the wrtev system call for the Linux environment is implemented in fd_writev_unix.go.

You can also directly use Writev and Readv, which are more on the bottom.

If you find the Go wrapper Writev still cumbersome, you can use Buffers, which is optimized for conn’s that implement writev, and call writeBuffers first.

func (v *Buffers) WriteTo(w io.Writer) (n int64, err error) {
    if wv, ok := w.(buffersWriter); ok {
        return wv.writeBuffers(v)
    }
    for _, b := range *v {
        nb, err := w.Write(b)
        n += int64(nb)
        if err != nil {
            v.consume(n)
            return n, err
        }
    }
    v.consume(n)
    return n, nil
}
// buffersWriter is the interface implemented by Conns that support a
// "writev"-like batch write optimization.
// writeBuffers should fully consume and write all chunks from the
// provided Buffers, else it should report a non-nil error.
type buffersWriter interface {
    writeBuffers(*Buffers) (int64, error)
}

Here is an example of sending a batch message using Buffers.

package main
import (
    "fmt"
    "net"
    "strconv"
)
func main() {
    conn, err := net.Dial("udp", "192.168.0.1:8972")
    if err != nil {
        panic(err)
    }
    var buf net.Buffers
    for i := 0; i < 10; i++ {
        buf = append(buf, []byte("hello world: "+strconv.Itoa(i)))
    }
    _, err = buf.WriteTo(conn) // as a datagram packet
    if err != nil {
        panic(err)
    }
    var data = make([]byte, 1024)
    for {
        n, err := conn.Read(data)
        if err != nil {
            panic(err)
        }
        fmt.Println(string(data[:n]))
    }
}

It adds 10 messages to Buffers and then writes them all at once. In fact, these ten messages are sent to the server as one UDP packet, so consider this if you have a lot of send requests and need to do it in batches.

Ref

https://colobu.com/2023/05/21/batch-read-and-write-udp-packets-in-Go-2/

Table of Contents

Ref