What’s next in DevOps, or orchestration or automation or whatever you like to call it? That’s a question that was asked (in a more limited form) in a new DevOps LinkedIn group, and I think it’s an important one if we address it the right way. At the DevOps or tool level, a question like that is just a decent into hype, a speculation about what might happen based on the premise that technology evolves by itself. It doesn’t. We have to step beyond the old hype that technology drives reality and explore where reality is driving technology.
The term “DevOps” is a problem in the current age, because it embodies two fundamental errors. The first is that this is all about transporting developer knowledge into application operations practices. Developers know best, they know what the application needs, and so if we communicate that need to operations, we get optimality. The second is that the goal is a transfer of knowledge, when the real goal is the automation of what we know we need to do.
The first problem is one of “inclusion”. Imagine the deployment of yesteryear (when, yes, the Lone Ranger rode), a time when we deployed transactional applications in a very basic (often monolithic) form on bare metal. Now imagine a world of componentization, microservices, lambdas and functional computing, virtualization, containers, clouds, serverless, events, and more. How likely is it that the practices of that early era would, though simple evolution, meet the current needs of the IT market?
Are developers of applications the drivers behind cloud adoption? Do they understand the differences between containers and bare metal? Do they know why we scale applications, why stateless behavior is good for some components? The first truth of the current era is that we need OpsDev as much as DevOps. But that truth doesn’t go far enough.
Application scalability and resiliency aren’t just things that operations people can introduce on a whim. They are ways of meeting application requirements set by the ultimate arbiter of truth, the line department users and management. That’s also true of whether we need to empower mobile workers, what we empower them with, how sensors and controllers could fit into business processes, and other stuff we think of as developer territory. We need UseOps, UseDev, as well as DevOps and OpsDev. There’s a trinity here; users, operations, development. They all need to be represented.
And they all need to be orchestrated or automated. We can’t talk about automated tools without considering how the humans work with them. Optimum “automation” has to look at the business process efficiency overall, not simply at what the most efficient IT operation might be. That might be awful for the workers. Similarly, automation of tasks has to automate enough to ensure that what’s left for the worker isn’t harder to do than the job would have been without any automation at all.
Early in my career as a software developer/architect, I spent a lot of time working out a way of making a certain IT process more efficient. My boss looked at it, then showed me that by simply doing something totally different at the IT level created a better business process overall. It was an important lesson.
The second problem is simpler in one sense and more complex in the other. My favorite-of-all-time CIO quote was “Tom, you need to understand. I don’t want decision support, I want to be told what to do!” Put another way, we don’t want “knowledge” or “guidance” as much as we want things taken care of. That’s what automation is all about, after all.
The question, then, is how we go about automating something. DevOps has always had two different models, one that describes the steps needed to accomplish something, and the other that describes the “something” we’re trying to accomplish. “Prescriptive” versus “declarative”, or “transformation” versus “end-state”. If you took a fairly loose definition of what kinds of tools fit into traditional DevOps, you could probably say that the majority of it is prescriptive today. But prescriptive models are very hard to define in highly complex environments. What, exactly, is the script trying to do? The more variable the starting and ending points, the more scripts you need and the harder it is to organize things.
Declarative approaches have had their own challenges, because once you decide that you’re not going to build scripts/instructions for handling something, you need to decide how the automated process knows what to do to get from where you are to that glorious, desired, end state you’ve declared. We are seeing clear recognition of this challenge in “zero-touch automation” in cloud-hosted carrier services today. Do we think that services are inherently more complex than applications, and so have a need for this heavy thinking that apps don’t? Why is that, given that what we are hosting future services in is the same cloud as what we’ll host future applications in? DevOps, at the basic level of deployment of features, has been way ahead of comparable initiatives in the service provider space (like Network Functions Virtualization’s Management and Orchestration or MANO processes). Why get behind now? DevOps has to be careful not to ignore “orchestration”, just as orchestration needed to embrace DevOps.
Two things are emerging in the service provider orchestration discussions that need to be in DevOps discussions as well. The first is the notion of intent modeling and hierarchical orchestration. This lets you define abstract steps that can be refined according to the specifics of where deployment is taking place or what the SLA needs of a service/application happen to be. The second is the notion of scalable, distributed, execution. It makes little sense to deploy scalable, resilient, applications using monolithic tools.
The reason these two points should be important to DevOps is that we’re already seeing a major shift of DevOps focus, from static deployment to dynamic and continuous lifecycle management. DevOps tools have dabbled in this shift through the support of “events”, but not gone far enough. Modeling a service lifecycle in an environment where deployment and redeployment and scaling are almost continuous, and where resource pools include not only the data center but multicloud, is beyond current tool capabilities.
Event recognition is really a key to a future DevOps transformation. First, events combine with intent-model principles to establish the notion of a coordinated multi-layer, multi-element, lifecycle management process. This, as I’ve said, is the key point that the service provider orchestration advocates are contributing to lifecycle management. Second, events combine with AI to create the kind of automated system people are seeing as the future of deployment and lifecycle management.
You can’t have AI without events, because events in some form are what AI systems react to. But most important, you can’t have AI without goal-state awareness. The only rational mission for AI in lifecycle management is the convergence of a complex system on the goal state. State/event systems recognize the goal state by defining it and defining the “non-goal-states” and their event progression toward the goal. AI systems would presumably replace explicit state/event structures with an automated state-event-interpretation process, something like complex event processing. Whatever happens at the detail level, nothing can come out of this unless the system knows what the goal-state is. That means that DevOps should be aggressively moving toward an end-state-and-event approach. It’s moving there, but hardly aggressively.
The future is complicated, and if there’s a failing that’s plagued tech for decades, it’s that we’ve always underestimated the complexity of new technology, particularly in networking. Automation is critical to sustaining reasonable operating costs and preventing human error from contaminating service reputations. DevOps has led the way, and it needs to continue that, because applications and service features are all going the same direction—componentization and cloud hosting.