Applying Google’s Service Orchestration Lessons to NFV

When I blogged about the Google federated layers model I had a request from a LinkedIn contact to blog about applying the approach to NFV.  I said I’d blog more on the topic next week, but it’s important enough to slot in for today.  It is indeed possible to view NFV’s coordinated orchestration or decomposition as a federation problem, and there are several options that could be applied to the approach, all of which have benefits.  And perhaps issues, of course.  Let me warn you now that we’ll have to ease into this to be sure everyone is up to speed.

For purposes of this discussion we need to broaden the traditional definition of “federation” to mean a relationship between network domains that supports the creation of services or service elements using contributed resources or services.  This would include the traditional operator mission of federation, which is focused on sharing/cooperating across administrative domains or business units.

OK, let’s now move to the orchestration coordination point.  “Orchestration” is the term that’s arisen in NFV to describe the coordinated combining of service elements to create and sustain services.  Generally, orchestration has a lot of common with DevOps, the process of deploying and managing an application lifecycle in the cloud or data center.  DevOps for ages has recognized two basic models, “declarative” and “imperative”.  The former defines deployment in terms of the goal-state—what are you trying to get as your end-game.  The latter defines a set of steps that (presumably) lead to that goal-state.  One defines intent, in modern terms, and the other defines process.

A further complication in the area of NFV is the jurisdictional division that’s been created.  NFV explicitly targets the process of combining virtual functions and deploying them.  A service would rarely consist entirely of virtual functions, and so NFV implies that there exists an end-to-end orchestration process that would do the stuff needed beyond what the NFV ISG defines.  This approach is explicit in the AT&T ECOMP and Verizon architectures for NFV.

If you have end-to-end orchestration of something that itself is an orchestrated process (as NFV function-deployment is) then you have an orchestration hierarchy and layers.  That raises the question of how many layers there are and how the layered structure is decomposed to drive deployment.  Let’s then start with the two overall models—declarative and imperative—inherited from DevOps and see if we can make any useful judgements.

I think it’s clear that an imperative approach—a deployment script—works only if you assume that you have a fair number of layers or abstractions.  Imagine having to write a recipe for service deployment that had no interior structure.  You’d end up with one script for virtually every customer order of every service, or certainly a very large number of configuration-dependent and order-parameter-dependent steps.  This is what software people call a “brittle” structure because a little change somewhere breaks everything.

If we do have an imperative approach with a hierarchy of layers, then it’s obvious that the layers at the “bottom” would be really deploying stuff and those at the top organizing how the bottom layers were organized and connected.  Thus, IMHO, even the imperative approach would depend on higher-layer descriptions that would look more and more like declarations.

This is why I don’t like the YANG-driven thinking on orchestration.  YANG has limitations even for describing connectivity, and it has more serious limitations describing virtual function deployment.  It is, IMHO, unsuitable for describing open abstractions—declarations.  TOSCA is far better for that.  But for this discussion let’s forget the details and focus on the structure.

Right now, we have a kind of fuzzy view that there are two or three specific levels of orchestration.  There’s service-level, end-to-end stuff.  There’s functional orchestration, and then there’s the device diddling at the bottom in some models.  Is that the right number?  What, if anything, could actually set the number of layers, and if something did would the setting be useful or arbitrary?

Here we get back to the notion of a model—declarative or imperative.  A model has to be processed, decomposed if you will, to some substructure.  While it’s true that we could develop software to decompose multiple model structures, it’s unlikely that would be a logical approach.  So, we can say that the number of different decomposition approaches equals the number of different models we use.  But what is the optimum number?  I think Google’s approach, which has five layers of decomposition, illustrates that there probably isn’t an optimum number.

Does this then argue for an infinitely layered and infinitely complicated orchestration approach?  The answer is that it depends.  In theory, a hierarchical model could be decoded by any number of orchestrators as long as each understood the model at the level that it was applied.  If every model represents a series of intent-or-declarative elements, then you could have any element decoded by anything, and it wouldn’t even have to decode into the same deployment as long as the SLA offered to the higher layer was met.

This is the lesson Google teaches.  You can structure a layer of the network in any way that optimizes what you value, and if you present its capabilities in an abstract way (one that cedes details of implementation for firmness of SLA) then it doesn’t matter.  You build layers and models to represent elements in layers as you will.

My personal view has always been that the best approach would be to have a uniform modeling down to the point where resources are committed (roughly to what ETSI calls the Virtual Infrastructure Manager).  You compose upward to services by aggregating whatever is useful in whatever way you like; layers would be unimportant because there’d be no functional subdivision to create them.  TOSCA could support this approach all the way to the bottom, which is one reason I like it.

The relationship between “layers” of orchestration and different models or different orchestration software tools is pretty clear.  A fixed layer exists where there’s a boundary, a different model or a different orchestrator.  I think boundaries of any sort are a risk to an emerging technology, so I think we either have to demand a single model/orchestrator from top to bottom, or we have to mandate an open model and orchestration strategy where any element in a service structure could be decomposed in any convenient way.

This puts me in gentle opposition to the approach that’s embodied in both AT&T’s and Verizon’s NFV models, because they do have fixed layers with implicitly different models and explicitly different orchestrators.  I think both operators have an interest in advancing NFV, which I share, but I think that since service modeling and orchestration is the cornerstone of software automation, we can’t afford to get this wrong.  I’d hope that somebody revisits this issue, quickly.  We could get locked by inertia into a limited approach with limited benefits.