Sometimes it’s useful to take an extreme case, a kind of end-game view, to gain insight into how a technology shift might be happening and what it might mean to the rest of the tech world. I’ve said several times in this blog that the cloud, to be truly revolutionary, had to be more than hosted virtualization. What does it have to be? It’s often hard to see the direction when you look at subtle details of application and cloud evolution, so let’s try direction-finding through something I mentioned earlier this week, Amazon’s AWS Lambda.
In programming, a Lambda function is one that operates only on what’s passed to it. You give it arguments and it does its thing with them, using nothing else. There’s ample documentation on how to use Lambda functions in all the popular programming languages, and Java 8 has specific support for them. You may have seen reference to the programming style that’s based on Lambda—“functional programming”.
Obviously Lambda functions are a whole new style of programming. It’s hard for me, a former software architect and programmer, to see an easy way of converting something to Lambda form—you’d pretty much have to start over. It’s also probably hard for most non-programmers to see much of a value in Lambda. Even in the IT space, some have told me that they wonder if the whole idea is just the invention of some geeks looking for novelty over utility.
But it’s not the abstract notion of Lambdas I want to get into here, but how Amazon seems to be using them in the cloud. AWS Lambda is a service unlike any other; it’s not about hosting at all. You don’t get a VM or container. What you get is the right to run some Lambdas, and if you find that confusing I’m pretty sure visiting the Amazon site won’t help you much. I’m going to get a bit inventive to try to explain why I think that Lambda is important to the cloud.
The world is a kind of giant event-space, a place where stuff happens. That’s true of the global ecosystem and also of subsets like companies, offices, teams, households, even people. Typically, we think of IT as a set of tools to handle information needs, to get stuff for us, store things, process things. In most of these cases the IT tools are defined processes that you interface with through some device. We think of a flow of processes, a workflow.
An event-space view is different. Stuff happens, and the stuff provokes a process response. Events trigger processes directly. The event can be an outside stimulus coupled in some way into the cloud, from IoT or a device. It could also represent a piece in an ongoing data flow, somewhere between origination and destination. In a sense, any condition that can be detected and expressed can be an event, and can trigger a Lambda process.
Amazon’s basic material doesn’t make it clear, but it appears that their Lambdas are a bit looser than the strict definition; you can do a database access inside one, for example. However, it appears that all Lambda processes are “RESTful” in that they don’t maintain state inside them, but process each request they get atomically. That means that the AWS Lambda service can scale to handle a stream of events by instantiating multiple parallel Lambda processes.
You can probably see the similarity between Lambdas and microservices already. The difference, I think, is that a microservice is represented by a URL and invoked by somebody when they need what it represents. It sits there waiting for action. A Lambda is bound directly to an event, and it’s spawned when the event is recognized and disappears when it has done its work.
Amazon calls this “serverless” processing, and others would say it’s a step toward “NoOps” meaning no operations processes are needed because there’s no explicit infrastructure to maintain. I don’t think either of these notions do the concept justice, and in fact both can be distracting. What AWS Lambda is, is a framework for distributed-real-time event handling that can scale fully. And because you can visualize any business process as a generator of events, it’s a way to build applications for the cloud that are fully elastic.
It will probably be a long time before current applications are redesigned to be fully Lambda-ized, if they ever are. However, virtually all IoT applications, process control applications, many mobile empowerment applications, and every piece of NFV could be done in Lambda form and gain significant value for their users. Even today’s application deployment tasks, automated routinely in DevOps tools, are starting to become event-driven in part so that they can respond to things like excess work by spawning some new component instances. In the long run, those spawned processes themselves could be Lambdas, but in the short term you could use Lambdas linked to the key events to activate the necessary scale-in-or-out steps.
Amazon offers a number of case studies on their Lambda page, and if you wade through them you can catch a sense of the power of the approach. If elasticity is the essential property of the cloud, then Lambdas are the ultimate cloud.
You also can catch a fundamental truth, which is that you have to develop the applications in Lambda form. You can’t just “run” something as a Lambda. This demonstrates that the cloud, as the future of computing, has its own rules for application-building and you’ll have to conform to gain the most from the cloud itself. It also demonstrates that Lambda-izing something is a major change of thinking.
One of my big concerns is that opportunities like NFV or IoT, both of which scream to be developed in Lambda form, are going to get pigeon-holed into a legacy model by the specifications that are developed. There is no “interface” to a Lambda in a traditional sense, for example, so do standards that mandate interfaces end up foreclosing or diminishing Lambdas?
Exploiting Lambdas is the path to optimum usage of the cloud, and yet we don’t know a lot about how they might be used. I think IoT, for example, should start as a vast cloud (literally) of Lambdas that are feeding repositories from both raw sensor data and from correlated/processed data created by other Lambdas. How exactly do you architect this sort of thing? What are the networking requirements as a user, and as a provider? For sure the only durable thing about a Lambda cloud is the elastic connectivity, the virtual network.
The downside to Lambdas is that “I don’t know” problem. Even in the software world, Lambdas or functional programming isn’t exactly sweeping the world. I’ve seen a lot of interest in the software organization of startups, particularly those who explicitly intent to deploy on the cloud. In the broader space, not so much. A straw poll I took of OpenStack supporters found less than 10% believed that Lambdas could help OpenStack (it could be crucial in handling infrastructure events).
That might make Lambdas perversely attractive to some, though. If this model is as good as I think it is, then using it would confer an enormous competitive advantage, not to mention a lot of bragging rights in the media. I suspect this will take a bit more than 8 months to shake out, but we might see signs by the end of this year, if Lambdas are really going to catch on.