We live in an IP world, at least insofar as the way the network appears to users. IP remains a largely adaptive protocol, one that makes traffic engineering more difficult. Some of the proposals to modernize IP networks, including SDN/OpenFlow, have focused on centralizing route control for the purpose of improving traffic engineering and potentially reducing “convergence time” as a network adapts to a failure mode. We may now be seeing developments that combine the revolutionary aspects of separating the IP control plane, with the kind of evolutionary changes that IP’s broad use almost dictates.
There have long been initiatives to improve how IP itself works, including multi-protocol label switching (MPLS) and the SPRING initiative (Source Packet Routing In NetworkinG). Both these concepts have evolved from early initiatives and ideas. Have they, or will they, relieve the pressure on transforming IP for better traffic efficiency? Will some form of SDN do that? Will something make the whole topic moot? That’s what we’ll look at today.
IP networks are based on connectionless, per-packet forwarding. There is really no “route” in vanilla IP, only a series of hops to the next-best place to be, which inevitably get you where you need to be. Topology and conditions advertising build a routing table in the devices, and that table is then used to determine where to send the packet next. The problem with this is that you don’t really know where anything is going, which makes it hard to optimize resources. SDN attempted to address this, in its OpenFlow variation, by centralizing the control of forwarding-table entries. That way, with a god-like view of the overall topology and conditions in the network, the central controller could work to balance traffic optimally.
There is, and in fact has been, another way of using a single source of routing intelligence. Packets all enter at the edge, by definition. Suppose we have a database (forget how it’s created for the moment) containing the labels of nodes/trunks that made up all the possible paths source-to-destination. If, on entry, a packet’s destination is looked up, and the list of labels associated with its route is then prepended to it, you could get from source to destination by following the list of labels. That’s “source routing”.
Source routing is a very old concept, one that was adopted by both Frame Relay and ATM standards a couple decades ago. In its most modern form, source routing assumes that a network consists of a series of linked black-box domains. When an “item” (a packet/message/session, depending on just what we’re talking about) has to connect across these domains, the source of the item determines the best path as a series of domain hops and codes the packet header accordingly, to follow the route. At each domain entry, the domain edge device pops its own header segment and replaces it with instructions on how to transit the domain to the designated next domain on the vector.
The nice thing about this approach, from a perspective of modernizing networks, is that it allows a network to be divided into domains that can represent different implementations. When a packet enters the network, a stack of labels defines its path, but a label can represent a domain, too. That lets different implementations within each domain to exist, while retaining source-side path selection.
Obviously, this corresponds at least somewhat with the notion of segment routing in IP today, and SPRING is a refinement of (or implementation of) source routing concepts. The advantage of source/segment routing is that packets can be directed along a specific route, which means that it’s possible to do more traffic engineering than would be possible in a purely adaptive IP network, where packets work their way to their destination through the interaction of a series of dynamically defined routing tables they meet along the way.
MPLS is a younger concept; the first draft goes back to about 1997. The roots of MPLS really came about because of a startup, Ipsilon, who proposed to use ATM networks as the “core” of an IP network. When the Ipsilon edge device recognized a session (which they called a “persistent flow”), they’d set up a virtual circuit to the destination’s Ipsilon edge device, and forward along that, bypassing all the interior IP stuff. Cisco picked up the concept as Tag Switching, and that became the basis for a workgroup that produced MPLS.
MPLS also works loosely on source routing, and its goal was different from that of Ipsilon; MPLS paths or tunnels were designed more to carry aggregate traffic, meaning it was first adopted to support traffic engineering. It’s been adopted as a means of controlling actual lower-layer behavior, including optical-layer, via “generalized” MPLS or GMPLS. Most think of MPLS as a new protocol layer below IP and above any physical or data-link protocols.
There are two significant differences between segment routing and SPRING, and MPLS. The first is the control plane, and the second is the way that forwarding state is maintained. MPLS is more IP-ish, and segment routing is a bit more source-routing-ish.
In MPLS, routers involved in MPLS forwarding have a forwarding state (think of it as a set of tables) that is exchanged among themselves via LDP or RSVP-TE, which is in effect an MPLS control plane. In effect, these form a different forwarding rule set from normal IP, and so to make data forwarding work in MPLS, the states of these tables are synchronized via the control-plane exchanges (LDP, RSVP-TE). In segment routing, the packets themselves maintain a label stack (source routing) that identifies the forwarding state explicitly on entry. The intermediate segment-routing elements just process the source route, so they don’t need to maintain forwarding states for each tunnel.
The segment routing approach lets you do path computation (building the label stack) as a cooperative function among edge routers, or through a central controller. It’s thus philosophically a kind of bridge between IP and OpenFlow-modeled SDN. That makes segment routing a potentially critical bridge between the IP-modeled network and a network in which SDN is used to build the interior routes.
Source routing, in the most flexible form I described, is good for another reason, which is that it could be a means of integrating different forwarding approaches into a common IP network. If, in my example earlier, we think of the “domains” as different network forwarding models (some IP, some SDN, or whatever), then each is a black box. We push a route at the domain level, and have each domain pop their contribution and convert it to something internal (SDN, MPLS, SPRING), and so on until the next domain entry point.
Right now, source routing is mostly an interior behavior, meaning within a network, and extensions to the interior gateway protocols (OSPF, IS-IS) are proposed. BGP-LS Extensions for Segment Routing allow for the necessary forwarding-state information to be carried between AS domains via BGP. However, the implementation of the SR extensions to BGP is necessary for this to work, not just the definitions of how the implementation should be done.
What I’m not clear on, frankly, is exactly what BGP-LS is advertising. As I see it, there would be two options. It could advertise a label representing the entry-to-egress path in aggregate, in which case it would have to translate that to an IGP label stack on entry. It could also advertise its segment structure overall, in which case the source route from the originator would/could include the segment label stack end to end. The latter approach would necessarily give another AS some control over traffic engineering, which I think is unlikely to be met with much enthusiasm.
Everything in BGP, pretty much, is optional under policy control. That gets us back to implementation. Not every ISP will elect to support any form of source routing, and if there are indeed different approaches to what might be advertised by BGP-LS, then there are different implementations that could be chosen. This might mean that source routing and even BGP-LS extensions to support it would be most valuable in large data-center-and-VPN combinations, where everything is within a single true network administration, even if there are multiple AS’s involved.
The central point to all of this change is that we’re evolving our sense of how IP networks are built, but more perhaps through “Brownian movement” than through a specific drive toward a goal. The segment routing initiatives seem to bring out a couple of important points, particularly that a different model of the routing control plane is indeed helpful and that source routing could be applied to IP to gain many of the benefits claimed for OpenFlow SDN. It just doesn’t seem to be pursuing these points directly, and segment routing could bring us a lot of benefits if some of its critical points were addressed more systematically.
But let’s now get to that “mootness” point I opened at the start of this blog. Traffic engineering is one of two possible responses to congestion. The other is to bury the problem in capacity. As fiber transport improves, and more significantly as transport economies of scale grow, aggregating traffic is less likely to create congestion even with primitive mechanisms to allocate capacity. We could, at some point, build a more meshed, more efficient, higher-capacity transport network, and that could take a lot of the pressure off on traffic engineering enhancements to IP. It could also reduce the benefit of the alternative approaches.
We can do virtual networking above IP—look at SD-WAN. If we can minimize traffic engineering’s value through capacity augmentation, we might be able to degrade or even eliminate the MPLS mission entirely. But if we create a formal virtual-network mission, we’ll then have to decide how that mission is harmonized across the world, across different implementations, across different sets of needs. It’s a challenge not unlike “network slicing” in 5G.
I can’t give you a firm proof of this, but I think that segment routing is going to be important in the future, because I think we’re in for a long period of divergence in approaches to networking before we see convergence. That alone makes segment routing valuable. Segment routing in any form isn’t going to sweep MPLS aside; there are just too many things done with it and too large a skills commitment to it. I do hope that wat segment routing does accomplish is making the way we structure IP networks more friendly with regard to the introduction of new technologies like SDN, and it does seem to be doing just that so far. This could be a very important space to watch.