Cumulus Basics Part IV - BGP introduction

Updated: May 21, 2019

The goal of this post is to introduce you, the reader, to BGP on a Cumulus box and then move towards a BGP unnumbered design.

We'll be using the following network topology for this post:

First, we will try to create a traditional BGP scenario with OSPF as an IGP. For now, OSPF is up and running and I have learnt the loopback of each LEAF switch.

I can use these loopbacks now to form an iBGP session between LEAF1 and LEAF2 to provide connectivity between PC1 and PC2.

An iBGP session is now up between the two:

Advertise the host subnets now into BGP:

Let's try and ping from PC1 to PC2 now:

All pings timeout. This is a common problem that you may run into - look at what is happening here with more thought. PC1 sources a packet with with an IP destination of, with an ICMP header trailing it. This reaches the default gateway, LEAF1. LEAF1 does a mac lookup, realizes it owns the destination mac and thus moves into the IP header to do a RIB lookup and forwards it towards SPINE1 (assuming the packet hash is in such a way that it goes to SPINE1).

SPINE1 has no entry for this prefix and thus your packets get blackholed here. To get around this problem, you either redistribute your BGP table into your IGP (which doesn't make sense considering your BGP table might grow substantially) or you have some sort of a meshed iBGP peering to ensure all boxes receive this route. The cleanest way of doing this would be with route reflectors. So, let's go ahead and make SPINE1/SPINE2 as RRs and have LEAF1/2/3 peer with them as route reflector clients.

I have now modified the configuration appropriately:

Go back in and advertise the host subnets again and now we see the intermediate devices (SPINE1/SPINE2) also having the prefixes in RIB.

PC1 can now reach PC2:

In the next post, we'll take a look at the ingenious BGP unnumbered design and understand how it truly works.

For any queries, concerns or conversations, please email