The specific issue to be analyzed today is ‘why DNS uses the UDP protocol’. DNS acts as a phone book for the entire Internet, translating domain names that can be understood by humans into IP addresses that can be understood by machines, eliminating the need for users of the Internet to have direct access to IP addresses that are difficult to read and understand.

I believe that it is common knowledge among software engineers that DNS uses the UDP protocol, and anyone who knows a little about computer networks knows that DNS uses the UDP protocol to transfer data, but this is not entirely true. Here we will analyze “why DNS uses UDP to transfer data” and “why DNS does not only use UDP to transfer data” in detail, hoping to help you understand the whole picture of DNS protocol.

Overview

The two problems we are about to discuss are not conflicting, in most cases DNS uses the UDP protocol for communication, and the DNS protocol was designed to recommend that we use UDP first for domain name resolution, which does solve many needs, but not all.

In fact, DNS uses not only the UDP protocol, but also the TCP protocol, but before we get into the specifics of today’s issue, let’s give a brief introduction to the DNS protocol: DNS queries contain not only common queries such as A records and CNAME records, but also special queries of the AXFR type, which are mainly used for DNS zone transfers, which is used to quickly migrate records between multiple nameservers. Since the response returned by the query is relatively large, the TCP protocol is used to transfer the packets.

As a widely used protocol, we can find a lot of DNS-related RFC documents, DNS Camel Viewer lists nearly 300 RFC documents related to DNS protocol, among which 6 are current Internet standards and 102 are DNS-related proposals. These documents together constitute our current understanding of the design of the DNS protocol, and the author has not been able to read them one by one, and has only selected some of the important documents to help us understand the history of DNS and its relationship with the UDP/TCP protocol, and will only excerpt from the documents related to the UDP/TCP protocol.

  1. RFC1034 - Domain Names - Concepts and Facilities Internet Standard, 1987-11
    1. that DNS queries can be transmitted via UDP packets or TCP connections.
    2. because the DNS zone transfer function has a strong need for accurate data, we must use TCP or other reliable protocols for AXFR-type requests.
  2. RFC1035 - Domain Names - Implementation and Specification
    1. the Internet supports nameserver access via TCP or UDP protocols.
    2. The UDP protocol should not carry more than 512 bytes of messages; messages exceeding that are truncated and set the TC bit of the DNS protocol. The UDP protocol is unacceptable for zone transfer functions, but is the recommended protocol for standard queries on the Internet. Queries sent over the UDP protocol may be lost, so a retransmission policy is required to address this issue.
  3. RFC1123 - Requirements for Internet Hosts - Application and Support Internet Standard, 1989-10
    1. the new DNS record types defined in the future may contain more than 512 bytes of information, so we should use the TCP protocol to transfer DNS records; therefore resolvers and naming services need to use the TCP protocol as a backup in case UDP cannot meet the demand.
    2. DNS resolvers and recursive servers must support the UDP protocol and should support sending non-zone-transfer queries using the TCP protocol; that is, when a DNS resolver or server sends a non-zone-transfer query, it must first send a UDP query, and if the response to that query is truncated, it should attempt to re-request using the TCP protocol.
  4. RFC3596 - DNS Extensions to Support IP Version 6 Internet Standard, 2003-10
    1. support for the IPv6 protocol through DNS extensions, each of which occupies 16 bytes four times as much as IPv4.
  5. RFC5011 - Automated Updates of DNS Security (DNSSEC) Trust Anchors Independent, 2007-10
    1. the addition of multiple resource records to authenticate the source of DNS data for DNS clients, with records often containing larger data.
  6. RFC6376 - DomainKeys Identified Mail (DKIM) Signatures Internet Standard, 2011-09
    1. choosing the appropriate key size for encryption is a trade-off between cost, performance and risk, however large keys (4096-bit) may not have a way to be put directly into the DNS UDP response packet and returned directly.
  7. RFC6891 - Extension Mechanisms for DNS (EDNS(0)) Internet Standard, 2013-04
    1. DNS queries and responses transmitted using UDP cannot exceed a maximum of 512 bytes and cannot support the transmission of large numbers of records such as IPv6 addresses or DNS security signatures.
    2. EDNS provides extensions for DNS to carry up to 4096 bytes of data over the UDP protocol.
  8. RFC7766 - DNS Transport over TCP - Implementation Requirements Proposed Standard, 2016-03
    1. when a client receives a truncated DNS response, it SHOULD determine via the TC field whether it needs to repeat the DNS query request over TCP protocol.
    2. the introduction of DNSSEC has made truncated UDP packets very common.
    3. that the use of UDP to transmit DNS with packet sizes exceeding the maximum transmission unit (MTU) may result in fragmentation of IP packets, and that the future predicted in the RFC1123 document has arrived where the only EDNS mechanism for increasing the packet size that UDP can carry is not considered reliable enough.
    4. that all generic DNS implementations must support both UDP and TCP transport protocols, including authoritative servers, recursive servers, and stub resolvers.
    5. that staked resolvers and recursive resolvers may choose to use TCP or UDP queries to request the target server directly, as appropriate, that it is no longer mandatory to initiate DNS requests with the UDP protocol, and that the TCP and UDP protocols may be substituted for each other in DNS queries rather than as a retry mechanism.
  9. Specification for DNS over Transport Layer Security (TLS) Proposed Standard, 2016-05
    1. introducing TLS into the DNS protocol to provide privacy for users and reduce eavesdropping and tampering with DNS queries, but the introduction of the TLS protocol introduces some additional overhead in terms of performance.
  10. RFC8484 - DNS Queries over HTTPS (DoH) Proposed Standard, 2018-10
    1. defines a protocol for sending DNS queries and obtaining DNS responses over HTTPS.

To briefly summarize the history of DNS, the original version of the DNS protocol was defined in 1987 by RFC1034 and RFC1035 The DNS was designed to use both UDP and TCP protocols, with UDP datagrams being used for most DNS queries and TCP protocols being used only in zone transfer scenarios, where UDP packets would only transmit a maximum of 512 bytes of data and any excess would be truncated; two years later, RFC1123 was released. tools.ietf.org/html/rfc1123), published two years later, predicted that more and more data would be stored in DNS records, and for the first time explicitly stated that UDP packets should be retried via TCP when they were found to be truncated.

After almost 20 years, as the Internet grew, it was discovered that IPv4 was no longer sufficient for allocation, so the longer IPv6 was introduced, and DNS was supported protocol-wise in the 2003 release of RFC3596; the subsequent releases of RFC5011 and RFC6376 added support for authentication and security, but also brought huge DNS records. RFC5011](https://tools.ietf.org/html/rfc5011) and RFC6376 added support for authentication and security, but also introduced huge DNS records, and UDP packets were truncation became very common.

The DNS extensions provided by RFC6891 helped to address the problem of truncated large packets to some extent, reducing the need for retries using the TCP protocol, but this did not solve all the problems due to the maximum transmission unit limit.

It was more than 30 years after the advent of DNS that RFC7766 finally proposed the use of TCP as the primary protocol to solve problems that UDP could not, and TCP was no longer just a mechanism to be used for retries, followed by DNS over TLS and The subsequent DNS over TLS and DNS over HTTP were also a complement to the DNS protocol.

From this development, DNS does not only use UDP packets for communication, but also the TCP protocol has been seen in the DNS standard, and we would like to analyze it today from a historical point of view - “why DNS queries choose to use UDP/TCP protocol”.

Design

In this section, we will present in two parts what are the advantages and disadvantages of the two different protocols, UDP and TCP, in supporting DNS queries and responses, depending on the protocol used for DNS, and during the analysis we will also restore the scenarios when making design decisions, taking into account the historical context.

UDP

The UDP protocol has been the primary protocol used by DNS for decades, and as an Internet standard, the majority of current DNS requests and responses use the UDP protocol for data transmission.

The data of DNS requests are encapsulated in binary form in the UDP packet as shown below, the following is a request to call the DNS server to obtain the IP address of the www.baidu.com domain, starting from the 05 byte in the fourth line to the end is the content of the DNS request, the entire packet contains Ethernet, IP and UDP protocol headers in addition to DNS protocol-related content. IP and UDP protocol headers.

1
2
3
4
5

0000   b0 6e bf 6a 4c 40 38 f9 d3 ce 10 a6 08 00 45 00   .n.jL@8.......E.
0010   00 3b 97 ae 00 00 40 11 0b 0a c0 a8 32 6d 72 72   .;....@.....2mrr
0020   72 72 f3 27 00 35 00 27 6b ee 0c 5a 01 00 00 01   rr.'.5.'k..Z....
0030   00 00 00 00 00 00 03 77 77 77→05 62 61 69 64 75   .......www.baidu
0040   03 63 6f 6d 00 00 01 00 01                        .com.....

Although each UDP packet contains many Ethernet, IP, UDP, and DNS protocol-related elements, the DNS request above is only 73 bytes in size, and the response to the above DNS request is only 132 bytes, which is a very small packet for other common requests today.

1
2
3
4
5
6
7
8
9

0000   38 f9 d3 ce 10 a6 b0 6e bf 6a 4c 40 08 00 45 00   8......n.jL@..E.
0010   00 76 00 00 00 00 96 11 4c 7d 72 72 72 72 c0 a8   .v......L}rrrr..
0020   32 6d 00 35 f3 27 00 62 5b c2 0c 5a 81 80 00 01   2m.5.'.b[..Z....
0030   00 03 00 00 00 00 03 77 77 77 05 62 61 69 64 75   .......www.baidu
0040   03 63 6f 6d 00 00 01 00 01 c0 0c 00 05 00 01 00   .com............
0050   00 02 cb 00 0f 03 77 77 77 01 61 06 73 68 69 66   ......www.a.shif
0060   65 6e c0 16 c0 2b 00 01 00 01 00 00 01 18 00 04   en...+..........
0070   3d 87 a9 7d c0 2b 00 01 00 01 00 00 01 18 00 04   =..}.+..........
0080   3d 87 a9 79                                       =..y

The communication mechanism between UDP and TCP is very different. As a reliable transport protocol, TCP requires three handshakes to establish a TCP connection between the two communicating parties before they can communicate, but in the 30-year-old DNS query scenario we don’t really need a stable connection (or think we don’t), each DNS query sends UDP datagrams directly to the nameserver, and at the same time the packets of common DNS queries are very small, and TCP connection establishment brings the following additional overheads.

  • TCP connection establishment requires three network communications.
  • TCP connection establishment requires ~130 bytes of data to be transferred.
  • TCP destruction of the connection requires four network communications.
  • TCP destruction of the connection requires the transfer of ~160 bytes of data.

Assuming that the time consumed by network communication is negligible, if we consider only the data transferred when a TCP connection is established, we can make a simple calculation.

  • Using TCP protocol (330 bytes total)
    • Three handshakes - 14x3(Ethernet) + 20x3(IP) + 44 + 44 + 32(TCP) bytes
    • Query Protocol Header - 14(Ethernet) + 20(IP) + 20(TCP) bytes
    • Response protocol header - 14(Ethernet) + 20(IP) + 20(TCP) bytes
  • Using UDP protocol (84 bytes total)
    • Query Protocol Header - 14(Ethernet) + 20(IP) + 8(UDP) bytes
    • Response protocol header - 14(Ethernet) + 20(IP) + 8(UDP) bytes

Note 1: The overhead of TCP headers is request and environment specific, and the exact result may fluctuate slightly in size to about 120 bytes.

Note 2: We calculate the results here on the assumption that a DNS resolver needs to communicate with only one nameserver or authoritative server to obtain a DNS response, but in real-world scenarios, DNS resolvers may communicate recursively with multiple nameservers, which doubly magnifies the disadvantage of the TCP protocol in terms of additional overhead.

If the request body and response of a DNS query are 15 and 70 bytes respectively, TCP will add ~250 bytes and ~145% additional overhead compared to UDP protocol, so when the size of the request body and response is small, transmission via TCP protocol not only requires more data to be transmitted, but also consumes more resources, and the time cost of multiple communications and information transmission cannot be ignored when the DNS query is small, and the reliability brought by TCP connection does not play a big role in the DNS scenario.

TCP

Today’s network is not as simple as it was designed to be decades ago. Not only are we experiencing the impending unavailability of IPv4, but we also need to introduce mechanisms such as DNSSEC to ensure the integrity of DNS queries and requests as well as transmission security. Why do we have to use the TCP protocol when more data needs to be transferred? If we continue to use the UDP protocol, can’t we do DNS resolution.

In theory, the size of a UDP packet can reach up to 64KB, which is actually a very large value for a common DNS query; however, in practice, once the data in a packet exceeds the maximum transmission unit (MTU) of the transmission link, which is the upper limit of a single packet size, usually 1500 bytes, the current packet may be fragmented and discarded, and some network devices may even refuse to process requests containing the EDNS(0) option, which can lead to DNS instability using the UDP protocol.

TCP, as a reliable transport protocol, can solve this problem very well. Through mechanisms such as sequence numbering and retransmission, it can ensure that the message is not heavy or missing, and the TCP stack of the message recipient will reassemble the fragmented data, and application layer protocols such as DNS can directly use the processed complete data. At the same time, when the packet is large enough, the percentage of additional overhead from the three TCP handshakes becomes smaller and smaller, converging to zero compared to the size of the entire packet: * When the DNS packet size is 500

  • When the DNS packet size is 500 bytes, the additional overhead of the TCP protocol is ~41.2%.
  • When the DNS packet size is 1100 bytes, the additional overhead of the TCP protocol is ~20.7%.
  • An additional overhead of ~10.3% for the TCP protocol when the DNS packet size is 2300 bytes.
  • an additional overhead of ~5.0 percent for the TCP protocol when the DNS packet size is 4800 bytes.

So, when we store more content in DNS, the three TCP handshakes and the additional overhead from the protocol headers are not a critical factor, but there is still no way to avoid the three network transfer times from our three TCP handshakes, which is something we have to accept in the current scenario.

Summary

Many people believe that DNS uses the UDP protocol to obtain the IP address corresponding to a domain name.

  1. the DNS was initially designed to introduce the TCP protocol in zone transfers and use the UDP protocol in queries.
  2. when DNS exceeded the 512-byte limit, we specified for the first time in the DNS protocol the specification that “when a DNS query is truncated, it should be retried using the TCP protocol”.
  3. the subsequent introduction of the EDNS mechanism, which allowed us to transfer up to 4096 bytes of data using UDP, but this feature was not reliable due to data fragmentation and loss caused by the MTU limit.
  4. in recent years, we have re-specified that DNS should support both UDP and TCP protocols, and that the TCP protocol is no longer just an option for retries.

This article has already detailed the history of DNS and the key points to consider when choosing different protocols, here we revisit the main reasons why DNS queries choose between the two different protocols, UDP or TCP.

  • UDP protocol
    • Smaller packets and simple mechanisms for DNS queries.
    • UDP protocol has less additional overhead and better performance.
  • TCP protocol
    • DNS queries are rapidly expanding due to the introduction of DNSSEC and IPv6, resulting in DNS responses often exceeding MTU and causing data fragmentation and loss, and we need to rely on the more reliable TCP protocol to complete the data transfer.
    • As the data contained in DNS queries continues to grow, the percentage of additional overhead from TCP protocol headers and three handshakes gradually decreases and is no longer a major part of the total transferred data size.

Whether we choose UDP or TCP, the core conflict lies in the size of the packets to be transmitted. If the packets are small to a certain extent, the UDP protocol is definitely the best choice, but when the packets gradually increase until they break the 512-byte and MTU 1500-byte limits, we can only choose to use the more reliable TCP protocol to transmit DNS queries and corresponding. Finally, let’s look at some of the more open-ended related issues, and the interested reader can ponder the following questions.

  • How can we make some optimizations to reduce some additional overhead for DNS using the TCP protocol?
  • What are the similarities and differences between the four methods of DNS transmission, UDP/TCP/TLS/HTTPS? Is it also possible to implement DNS queries through other protocols?