Now that we've covered this new routing paradigm that LISP introduces in the previous post and understood some commonly used terms, we will move on to basic LISP operations.
The topology that we will be using is the following:
Some generic details about the setup - R1 and R5 have a loopback each (loopback0) with IP addresses of 220.127.116.11/24 and 18.104.22.168/24 respectively. There is an OSPF adjacency between R1 and xTR2 as well as R5 and xTR4. Loopback0 is advertised into OSPF by both R1 and R5. xTR2 and xTR4 learn these respective loopbacks.
xTR2, xTR4 and the MS/MR have a /32 loopback each:
The LISP cloud has some form of underlay (we don't need to go into details of that) that allows for these loopbacks to talk to each other and have complete reachability between them.
Apart from basic IP addressing between each device (and details described earlier), there is nothing else that has been pre-configured on these devices.
Let's go ahead and start configuring each device role in the topology now. We are going to start with the MS/MR.
On the MS/MR, outside of configuring it as a Map Server and Map Resolver, you also need to define what prefixes it is allowed to accept and store in its EID to RLOC mapping database and against what site.
Next, configure the xTRs. This is where you setup the EID advertisement and point to the MS/MR.
Similar configuration for xTR4, only the database-mapping line changes a bit:
Take a look at the database-mapping statement closely - what it does is advertise the EID (which is in red) along with the RLOC (which is in yellow) to the MS/MR (we are setting the RLOCs to be the loopback0 IP address). The MS/MR now stores this in its EID to RLOC mapping database.
Now that we've configured the xTRs and the MS/MR, let's take a look at what is really going on behind the scenes.
When you configure the xTRs and advertise an EID to RLOC mapping, it sends a Map Register to the MS/MR.
The Map Register packet looks like this:
From the Map Register, the MS/MR pulls out relevant information to populate its EID to RLOC mapping database.
Additionally, MS/MR sends a Map Notify back (only if the 'M' bit is set in the Map Register which essentially means it expects to get a Map Notify back).
The Map Notify packet looks like this:
This process ends here. The MS/MR does not 'push' this information to anyone else.
From a xTR perspective, there are two important tables that you should be looking at - the LISP database table and the LISP map-cache. The LISP database table is where it stores the locally attached EIDs that are being advertised to the MS/MR and the LISP map-cache is what is actively used to build the LISP forwarding/data-plane.
The LISP database table:
It is important to understand how the database determines if an EID is 'reachable' or not - this is not a data packet driven derivative. There is no active reachability check that is done against this EID. It is purely from the existence of the prefix in RIB.
For example, I shut down R1s interface to xTR2. With an intermediate dumb device between R1 and xTR2, xTR2s interface stays up despite R1 being down. Eventually, this causes OSPF peering to go down and 22.214.171.124/24 to be pulled from RIB. At this point, the state of this EID in the LISP database is moved to unreachable:
I can now configure a static route towards that subnet and it becomes reachable again, thus proving what we had theorized earlier:
The LISP map-cache:
Remember how we talked about LISP being a pull model? Well, at this point, with just the EID registrations being done with the MS/MR, the xTRs have no knowledge of each others EIDs in their map-caches (which is used for LISP data-plane).
Thus, at time=0 (where time=0 is the point when xTRs have just registered EIDs with the MS/MR and no data-plane traffic is flowing right now) the map-cache is empty except for one, very important, default entry.
This map-cache entry triggers a 'Map Request' to the MS/MR (this is the 'pull' that forms the basis of the LISP model), essentially requesting for information about the EID subnet it is trying to reach. The MS/MR looks at its site database for the EID. If it finds it, it forwards the Map Request to the RLOC that registered the EID. This RLOC now responds to the source RLOC and the source RLOC installs the EID to RLOC mapping (information gleaned from the Map Reply) in its map-cache. This new entry is now used for data-plane forwarding.
LISP process - end to end
Now that we have a general understanding of the tables in use, let's look at an entire flow, end to end by having R1 ping R5s loopback (126.96.36.199) sourcing its own loopback (188.8.131.52).
R1s default gateway is xTR2, so after determining that the destination is not in its subnet, R1 ARPs for its default gateway (xTR2), resolves that and sends the packet to xTR2.
xTR2 looks at the destination mac, realize it owns it and strips the L2 header. It then does a lookup against the destination IP address in the IP header. This is where some of the magic begins. While RIB shows no route to the destination, look closely at the CEF entry:
While the route is still a 'no route' entry, it has appropriate actions set to leak the packet to the CPU and into the LISP process. LISP now processes the packet by looking at its map-cache table:
A lookup in the LISP map-caches results in a hit against the 0.0.0.0/0 entry that LISP installs by default. This is a catch-all entry that never expires and its sole purpose is to trigger a Map Request to the MS/MR.
Having hit this entry, the LISP process on xTR2 now generates a Map Request for EID 184.108.40.206 and sends that to the MS/MR.
The Map Request packet looks like this:
The MS/MR looks at its site database to determine if any RLOC has registered this EID:
It finds an active entry for this against SITE_B, with a RLOC of 220.127.116.11. However, notice one of the flags that is set against the eTR - no proxy-reply. This means that the MS/MR cannot proxy for this EID and it must forward the Map Request to the RLOC that registered this EID - 18.104.22.168 in this case.
That is what it does next:
When xTR4 gets this, it looks at its database. It is the authoritative owner for this EID so it sends a Map Reply back to the iTR RLOC, 22.214.171.124 in this case.
The Map Reply packet looks like this:
xTR2 processes the Map Reply and inserts a new, more specific (when compared to 0.0.0.0/0) entry for the EID in its LISP map-cache:
A lot of good information here - you can see the EID that has been added to the map-cache, along with its RLOC. Outside of this, the source of this was a Map Reply and this entry is valid for the next 23 hours, 59 minutes and 45 seconds.
Where does this expiry time come from? This is actually specified in the Map Reply itself - it is the 'Record TTL' field. Look closely at the packet - this field has a value of 1440 (in minutes), which is 24 hours.
We're not done yet - once the map-cache has been built, it is pushed down to CEF as well:
Remember that earlier, we were hitting the 0.0.0.0/0 entry for the same prefix. Now, a more specific entry has been added for 126.96.36.199/24; subsequent packets hit this instead and do not require any further map requests to be generated. This entry forces a LISP encapsulation of the packet with a next-hop of 188.8.131.52 as you can see from the output above.
This encapsulated packet is sent towards the MS/MR, since recursively, that is the next hop for 184.108.40.206.
The encapsulated packet looks like this:
** quick side note: look at the source IP address in the outer IP header. It is not 220.127.116.11. This is because the LISP process populates the outer source IP address with the IP address of the egress interface. To change this behavior (and make it more consistent/predictable), configure the command 'ip lisp source-locator <interface>' on each egress LISP interface **
At this point, regular routing takes over (the destination IP address in the outer IP header carries the packet to the destination RLOC) and the packet is forwarded to xTR4.
The destination mac and the destination IP address in the outer IP header is owned by xTR4 itself (18.104.22.168), so it strips off the outer L2 and L3 headers (decapsulates the packet). It parses through the UDP and LISP headers and then does a route lookup against the inner destination IP address. The packet is then forwarded based on this lookup, towards R5.
This entire process now happens in the reverse direction, starting from R5.
Result? We have reachability between R1s and R5s loopbacks.
Why were two pings lost? One ICMP timeout is how long it took for one map resolution to be completed - this needs to be done in both directions, thus two ICMP requests did not get any responses back.
This concludes a very long post. See you in the next one!