GoBGP Principles and Practices

GoBGP is an open source tool developed in Go language and running on Linux systems that provides control plane functionality for the BGP protocol. Compared with Quagga/FRRouting, GoBGP has better performance and shorter convergence time, and can be applied to larger networks, such as acting as an IXP router. GoBGP can be configured via the gRPC API using multiple languages such as Python, C++, and of course the CLI. GoBGP also supports OpenConfig, and its YANG model conforms to draft-ietf-idr-bgp-model-03. Because GoBGP can easily interfere with routing manually and is more involved, it is a good tool for experimentation. In this paper, we will introduce the main features and practices of gobgp.

Background

Installation and Composition

The installation of GoBGP is very simple, just download the tar.gz file from https://github.com/osrg/gobgp/releases and unzip it. The one selected here is v2.27.0.

`1`	`$ tar -xzf gobgp_2.27.0_linux_amd64.tar.gz`

gobgpd
- daemon for Gobgp, complete implementation of BGP protocol.
- Can interact with gobgpd via the gRPC API.
- You can also configure bgp through configuration files.
gobgp
- Full-featured CLI.
- You can view BGP-related information and configure BGP.
Configuration file: supports multiple formats toml/yaml/json, etc.

Support Features

Full-featured CLI
Multiprotocol Support
- IPv4/Pv6
- Labeled IPv4/IPv6
- Labeled IPv4/IPv6
- EVPN
- Flowspec IPv4/IPv6/L2
Flexible Policy
Graceful Restart
- Both restarting/helper speak role
Route Reflector
Route Server
MRT Dumping
BMP
RPKI Validation
FIB manipulation
gRPC API
Standard configuration format

Performance Test

Compared with Quagga/FRRouting, GoBGP has better performance and shorter convergence time, and can be applied to larger networks, such as acting as an IXP router. For more performance tests on BGP, see Comparing Open Source BGP stacks with internet routes.

Comparing Open Source BGP stacks with internet routes

Integration with Quagga/Zebra, etc

GoBGP supports only one routing protocol, BGP, but it can be integrated with Zebra to work with Quagga/FRR by way of an API to support multiple routing protocols.

Integration with Quagga/Zebra, etc

GoBGP can be integrated into the Quagga/Zebra system.

GoBGP integrated into the Quagga/Zebra system

Using GoBGP

Basic operation

We can start gobgpd as a bgp server to establish a BGP connection with the switch.

Here is the network topology diagram

For example, for

[global.config]
  as = 1002
  router-id = "172.25.0.136"

[[neighbors]]
  [neighbors.config]
    peer-as = 1002
    neighbor-address = "172.25.0.129"
    auth-password: "xxxxx"

Start gobgpd.

/ # ./gobgpd -t yaml -f gobgpd.yaml
{"level":"info","msg":"gobgpd started","time":"2022-03-03T07:28:56Z"}
{"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2022-03-03T07:28:56Z"}
{"level":"info","msg":"Peer 172.25.0.129 is added","time":"2022-03-03T07:28:56Z"}
{"Topic":"Peer","level":"info","msg":"Add a peer configuration for:172.25.0.129","time":"2022-03-03T07:28:56Z"}
{"Key":"172.25.0.129","State":"BGP_FSM_OPENCONFIRM","Topic":"Peer","level":"info","msg":"Peer Up","time":"2022-03-03T07:29:01Z"}

Check the peer information via gobgp, here the Establ of the State means the connection has been established, if the State is Active then you need to check if the switch configuration is correct.

/ # ./gobgp neighbor
Peer           AS  Up/Down State       |#Received  Accepted
172.25.0.129 1002 00:01:29 Establ      |        8         8
/ # ./gobgp neighbor 172.25.0.129
BGP neighbor is 172.25.0.129, remote AS 1002
  BGP version 4, remote router ID 172.25.100.4
  BGP state = ESTABLISHED, up for 00:01:34
  BGP OutQ = 0, Flops = 0
  Hold time is 90, keepalive interval is 30 seconds
  Configured hold time is 90, keepalive interval is 30 seconds

  Neighbor capabilities:
    multiprotocol:
        ipv4-unicast:   advertised and received
    route-refresh:  advertised and received
    extended-nexthop:   advertised
        Local:  nlri: ipv4-unicast, nexthop: ipv6
    4-octet-as: advertised and received
  Message statistics:
                         Sent       Rcvd
    Opens:                  1          1
    Notifications:          0          0
    Updates:                0          7
    Keepalives:             4          4
    Route Refresh:          0          0
    Discarded:              0          0
    Total:                  5         12
  Route statistics:
    Advertised:             0
    Received:               8
    Accepted:               8

View the global table.

/ # ./gobgp global rib
   Network              Next Hop             AS_PATH              Age        Attrs
*> 10.0.0.0/24          172.25.0.129         801 45090 45090      00:04:16   [{Origin: ?} {LocalPref: 100}]
*> 10.0.2.0/24          172.25.0.129         801 45090 45090      00:04:16   [{Origin: ?} {LocalPref: 100}]
*> 172.25.0.0/25        172.25.0.129         801 1001             00:04:16   [{Origin: i} {LocalPref: 100}]
*> 172.25.0.128/25      172.25.0.129                              00:04:16   [{Origin: i} {Med: 0} {LocalPref: 100}]
*> 172.25.100.1/32      172.25.0.129         801                  00:04:16   [{Origin: i} {Med: 0} {LocalPref: 100}]
*> 172.25.100.2/32      172.25.0.129         801                  00:04:16   [{Origin: i} {Med: 0} {LocalPref: 100}]
*> 172.25.100.3/32      172.25.0.129         801 1001             00:04:16   [{Origin: i} {LocalPref: 100}]
*> 172.25.100.4/32      172.25.0.129                              00:04:16   [{Origin: i} {Med: 0} {LocalPref: 100}]

Check out adjacent rib-in and rib-out.

/ # ./gobgp neighbor 172.25.0.129 adj-in
   ID  Network              Next Hop             AS_PATH              Age        Attrs
   0   10.0.0.0/24          172.25.0.129         801 45090 45090      00:07:18   [{Origin: ?} {LocalPref: 100}]
   0   10.0.2.0/24          172.25.0.129         801 45090 45090      00:07:18   [{Origin: ?} {LocalPref: 100}]
   0   172.25.0.0/25        172.25.0.129         801 1001             00:07:18   [{Origin: i} {LocalPref: 100}]
   0   172.25.0.128/25      172.25.0.129                              00:07:18   [{Origin: i} {Med: 0} {LocalPref: 100}]
   0   172.25.100.1/32      172.25.0.129         801                  00:07:18   [{Origin: i} {Med: 0} {LocalPref: 100}]
   0   172.25.100.2/32      172.25.0.129         801                  00:07:18   [{Origin: i} {Med: 0} {LocalPref: 100}]
   0   172.25.100.3/32      172.25.0.129         801 1001             00:07:18   [{Origin: i} {LocalPref: 100}]
   0   172.25.100.4/32      172.25.0.129                              00:07:18   [{Origin: i} {Med: 0} {LocalPref: 100}]
/ # ./gobgp neighbor 172.25.0.129 adj-out
Network not in table

The route can be declared with the following command.

`1`	`gobgp global rib -a ipv4 add 192.168.1.0/24`

Route Reflector

To ensure connectivity between iBGP peers, full connectivity relationships between IBGP peers are required . The Route Reflection mode is a mature alternative to the Full Mesh mode, which will become dramatically less efficient as the cluster size increases. The RR scheme allows a BGP Speaker (that is, a Route Reflector) to broadcast the learned route information to other BGP Peers, greatly reducing the number of BGP Peer connections.

For gobgpd, the BGP Server can be supported as a Route Reflector by modifying the configuration file by adding the RouteReflector.RouteReflectorConfig configuration as follows.

node 172.25.0.7 as an RR node to establish a bgp peer with switch 172.25.0.1
Node 172.25.0.6 as RR client node, establish bgp peer with RR node 172.25.0.7
node 172.25.0.8 as RR client node, establish bgp peer with RR node 172.25.0.7

[global.config]
  as = 1001
  router-id = "172.25.0.7"

[[neighbors]]
  [neighbors.config]
    neighbor-address = "172.25.0.1"
    peer-as = 1001
    auth-password = "xxxxxx"

[[neighbors]]
  [neighbors.config]
    neighbor-address = "172.25.0.6"
    peer-as = 1001
    auth-password = "xxxxxx"

  [neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "172.25.0.137"

[[neighbors]]
  [neighbors.config]
    neighbor-address = "172.25.0.8"
    peer-as = 1001
    auth-password = "xxxxxx"

  [neighbors.route-reflector.config]
    route-reflector-client = true
    route-reflector-cluster-id = "172.25.0.137"

Route Server

There are some scenarios in the existing network, In order to achieve network traffic interoperability, it is usually necessary to perform full connectivity through eBGP . Full connectivity between border devices is more demanding in terms of cost consumption and device performance, and is not conducive to network topology and device expansion. Route Server is similar to IBGP full connection using route reflector , which is a device (or multiple devices) used to perform routing services, and its main function is to propagate routes to individual clients (border devices), and the routes published to clients do not modify path attributes such as AS_PATH, Nexthop, MED, etc., thus reducing the border router full connection consumption.

As shown in the figure below, an IX (Internet eXchange) contains multiple independent SPs (service providers) that want to interoperate traffic. Each SP has a border router connected to the common switching network. Each SP has its own AS number, and the BGP Router addresses range from 10.0.0.1 to 10.0.0.8.

Internet eXchange

In this case, the 8 BGP Peers are required to establish a full connection. Like iBGP, this full mesh connection is more demanding in terms of cost consumption and device performance, and is not conducive to the expansion of network topology and number of devices.

ix eBGP full mesh

BGP Route Server can simplify SP connectivity. This is shown below.

BGP Route Server

The following figure shows the transparent route propagation implemented by route server.

transparent route propagation implemented by route server

For more information about route server, you can refer to Route Server.

For GoBGP, Route Server is also supported.

[global.config]
  as = 64512
  router-id = "192.168.255.1"

[[neighbors]]
  [neighbors.config]
    neighbor-address = "10.0.255.1"
    peer-as = 65001
    auth-password = "hoge1"
  [neighbors.transport.config]
    passive-mode = true
  [neighbors.route-server.config]
    route-server-client = true

[[neighbors]]
  [neighbors.config]
    neighbor-address = "10.0.255.2"
    peer-as = 65002
    auth-password = "hoge2"
  [neighbors.transport.config]
    passive-mode = true
  [neighbors.route-server.config]
    route-server-client = true

BGP Policy

Policy is a way to control how BGP routes are inserted into the RIB or broadcast to BGP Peers, and is divided into two parts Condition and Action. When the Policy is configured and the Condition condition is triggered, an Action action is performed to modify the route.

Condition includes prefix, neighbor (source/destination of the route) and aspath, etc.
Action includes accept , reject , MED/aspath/community manipulation , etc.

Policy Model

Policy model includes Import Policy and Export Policy ：

Import policy is invoked before best path calculation and pushing routes to RIB.
Export policy is invoked after that.

Policy Model

You can view the policy with the following command.

1
2

$ gobgp global policy import
$ gobgp global policy export

Route Server Policy Model

For the Route Server model, Import and Export policies are both specific to a Peer.

The Import policy defines what routes will be imported into the master RIB.
The Export policy defines what routes will be exported from the master RIB.

Route Server Policy Model

1
2

$ gobgp neighbor <neighbor-addr> policy import
$ gobgp neighbor <neighbor-addr> policy export

Policy Structure

A Policy contains multiple Statements, each with its own Condtions and Actions.

Policy Structure

Conditions includes.

prefix
neighbor
aspath
aspath length
community
extended community
rpki validation result
route type (internal/external/local)
large community
afi-safi in

Actions includes.

accept or reject
add/replace/remove community or remove all communities
add/subtract or replace MED value
set next-hop (specific address/own local address/don’t modify)
set local-pref
prepend AS number in the AS_PATH attribute

You can view the Policy configuration with the following command.

$ gobgp policy
$ gobgp policy statement
$ gobgp policy prefix
$ gobgp policy neighbor
$ gobgp policy as-path
$ gobgp policy community
$ gobgp policy ext-community
$ gobgp policy large-community

Policy Configuration

Policy configuration is more complicated, here are the steps to configure it, you can refer to here for details.

define defined-sets
1. define prefix-sets
2. define neighbor-sets
define bgp-defined-sets
1. define community-sets
2. define ext-community-sets
3. define as-path-setList
4. define large-community-sets
define policy-definitions
attach policies to global rib (or neighbor local rib when neighbor is route-server-client).

Graceful Restart

[global.config]
  as = 64512
  router-id = "192.168.255.1"

[[neighbors]]
  [neighbors.config]
    neighbor-address = "10.0.255.1"
    peer-as = 65001
  [neighbors.graceful-restart.config]
    enabled = true

BMP

GoBGP supports BGP Monitoring Protocol (RFC 7854) for real-time monitoring of the operational status of BGP sessions, including the establishment and closure of peer relationships, routing information, etc.

[global.config]
  as = 64512
  router-id = "192.168.255.1"

[[bmp-servers]]
  [bmp-servers.config]
    address = "127.0.0.1"
    port=11019

Dynamic Neighbors

In a BGP network, when multiple peers change frequently, if the peers are configured statically, the peers need to be added or deleted frequently at the local end, which is a great maintenance workload. In this case, you can configure the BGP dynamic peer function to make BGP listen to BGP connection requests from the specified network segment and dynamically establish BGP peers, and add these peers to the same peer group. This reduces the workload of network maintenance by eliminating the need to add or remove BGP peers at the local end when peer changes occur.

Switches generally support the configuration of Dynamic Neighbors, such as here is the method of configuring Dynamic Neighbors for Huawei switches Dynamic Neighbors are also supported for gobgp.

As shown below, there are two main parts to configure.

Create a peer group, describing the basic information of the peer group
Configure the peer group to listen on the 172.40.0.0/16 network segment

[global.config]
  as = 65001
  router-id = "172.40.1.2"

[[peer-groups]]
  [peer-groups.config]
    peer-group-name = "sample-group"
    peer-as = 65002
  [[peer-groups.afi-safis]]
    [peer-groups.afi-safis.config]
      afi-safi-name = "ipv4-unicast"
  [[peer-groups.afi-safis]]
    [peer-groups.afi-safis.config]
      afi-safi-name = "ipv4-flowspec"

[[dynamic-neighbors]]
  [dynamic-neighbors.config]
    prefix = "172.40.0.0/16"
    peer-group = "sample-group"

Others

There are a lot of other features on GitHub about MRT/BMP/EVPN, so you can check out the documentation if you need to.

GoBGP Programming

Basic Server

Referring to the gobgp library documentation provided, we can implement a simple go bgp server as shown below.

package main

import (
    "context"
    "time"

    "github.com/sirupsen/logrus"
    apb "google.golang.org/protobuf/types/known/anypb"

    api "github.com/osrg/gobgp/v3/api"
    "github.com/osrg/gobgp/v3/pkg/log"
    "github.com/osrg/gobgp/v3/pkg/server"
)

func main() {
    log := logrus.New()

    // 创建 BGP Server 实例
    s := server.NewBgpServer(server.LoggerOption(&myLogger{logger: log}))
    go s.Serve()

    // global configuration
    if err := s.StartBgp(context.Background(), &api.StartBgpRequest{
        Global: &api.Global{
            Asn:         65003,
            RouterId:   "10.0.255.254",
            ListenPort: -1, // gobgp won't listen on tcp:179
        },
    }); err != nil {
        log.Fatal(err)
    }

    // monitor the change of the peer state
    if err := s.MonitorPeer(ctx, &api.MonitorPeerRequest{}, func(p *api.Peer) { log.Print(p) }); err != nil {
        log.Fatal(err)
    }

    // neighbor configuration
    n := &api.Peer{
        Conf: &api.PeerConf{
            NeighborAddress: "172.17.0.2",
            PeerAsn:          65002,
        },
    }

    if err := s.AddPeer(context.Background(), &api.AddPeerRequest{
        Peer: n,
    }); err != nil {
        log.Fatal(err)
    }

    // add routes
    nlri, _ := apb.New(&api.IPAddressPrefix{
        Prefix:    "10.0.0.0",
        PrefixLen: 24,
    })

    a1, _ := apb.New(&api.OriginAttribute{
        Origin: 0,
    })
    a2, _ := apb.New(&api.NextHopAttribute{
        NextHop: "10.0.0.1",
    })
    a3, _ := apb.New(&api.AsPathAttribute{
        Segments: []*api.AsSegment{
            {
                Type:    2,
                Numbers: []uint32{6762, 39919, 65000, 35753, 65000},
            },
        },
    })
    attrs := []*apb.Any{a1, a2, a3}

    _, err := s.AddPath(context.Background(), &api.AddPathRequest{
        Path: &api.Path{
            Family: &api.Family{Afi: api.Family_AFI_IP, Safi: api.Family_SAFI_UNICAST},
            Nlri:   nlri,
            Pattrs: attrs,
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    v6Family := &api.Family{
        Afi:  api.Family_AFI_IP6,
        Safi: api.Family_SAFI_UNICAST,
    }

    // add v6 route
    nlri, _ = apb.New(&api.IPAddressPrefix{
        PrefixLen: 64,
        Prefix:    "2001:db8:1::",
    })
    v6Attrs, _ := apb.New(&api.MpReachNLRIAttribute{
        Family:   v6Family,
        NextHops: []string{"2001:db8::1"},
        Nlris:    []*apb.Any{nlri},
    })

    c, _ := apb.New(&api.CommunitiesAttribute{
        Communities: []uint32{100, 200},
    })

    _, err = s.AddPath(context.Background(), &api.AddPathRequest{
        Path: &api.Path{
            Family: v6Family,
            Nlri:   nlri,
            Pattrs: []*apb.Any{a1, v6Attrs, c},
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    s.ListPath(context.Background(), &api.ListPathRequest{Family: v6Family}, func(p *api.Destination) {
        log.Info(p)
    })

    // do something useful here instead of exiting
    time.Sleep(time.Minute * 3)
}

// ...

As you can see, the sample code is relatively simple and mainly uses the following APIs.

// 创建 BGP Server 实例
func NewBgpServer(opt ...ServerOption) *BgpServer

// 启动 BGP Server
func (s *BgpServer) Serve()

// global 配置
// BGP Server 的 AS 是 65003，RouterId 是 10.0.255.254
api.Global{
  Asn:         65003,
  RouterId:   "10.0.255.254",
  ListenPort: -1, // gobgp won't listen on tcp:179
}

// 根据传入 global 配置，开启 BGP Server 的 BGP 协商
func (s *BgpServer) StartBgp(ctx context.Context, r *api.StartBgpRequest) error

// 观察 BGP Peer 状态变化
func (s *BgpServer) MonitorPeer(ctx context.Context, r *api.MonitorPeerRequest, fn func(*api.Peer)) error

// Peer 信息
api.Peer{
        Conf: &api.PeerConf{
            NeighborAddress: "172.17.0.2",
            PeerAsn:          65002,
        },
    }

// 建立 BGP Peer 连接
func (s *BgpServer) AddPeer(ctx context.Context, r *api.AddPeerRequest) error

// 传递 BGP 路由信息
func (s *BgpServer) AddPath(ctx context.Context, r *api.AddPathRequest) (*api.AddPathResponse, error)

Route Reflector

The api.Peer structure can be configured in more detail so that the joined BGP Peer is acting as an RR client.

n := &api.Peer{
    Conf: &api.PeerConf{
        NeighborAddress: "172.25.0.6",
        PeerAsn:          1001,
    },
    RouteReflector: &api.RouteReflector{
        RouteReflectorClient: true,
        RouteReflectorClusterId: "172.25.0.7",
    }
}

BMP

Here is a list of several common BMP messages.

type BMPMessage struct {
    Header     BMPHeader
    PeerHeader BMPPeerHeader
    Body       BMPBody
}

type BMPRouteMonitoring struct {
    BGPUpdate        *bgp.BGPMessage
    BGPUpdatePayload []byte
}

type BMPPeerDownNotification struct {
    Reason          uint8
    BGPNotification *bgp.BGPMessage
    Data            []byte
}

type BMPPeerUpNotification struct {
    LocalAddress    net.IP
    LocalPort       uint16
    RemotePort      uint16
    SentOpenMsg     *bgp.BGPMessage
    ReceivedOpenMsg *bgp.BGPMessage
}

// ...

Table of Contents

Background

Installation and Composition

Support Features

Performance Test

Integration with Quagga/Zebra, etc

Using GoBGP

Basic operation

Route Reflector

Route Server

BGP Policy

Policy Model

Route Server Policy Model

Policy Structure

Policy Configuration

Graceful Restart

BMP

Dynamic Neighbors

Others

GoBGP Programming

Basic Server

Route Reflector

BMP