Month: December 2022

I’ve blogged about the value of utilizing consumer broadband access and the Internet for more business connectivity. What about the Comcast decision to deploy specialized technology for business services via a newly defined architecture model? Does this mean that Comcast is going another way, or even that it’s actually trying to avoid that consumer broadband leverage? Then, how do you explain their 10G DOCSIS decision? Split personality? Is Comcast pursuing two relatively independent architecture strategies for consumer and business, is it moving toward a tactical link between the two for infrastructure economy, or does it have a strategy?

Cable providers like Comcast rely on CATV cable for most of their last-mile connectivity to customers. The capabilities of CATV vary with the version of DOCSIS in use, but generally CATV is shared media and thus has a higher risk of QoS impacts created by other traffic, and at least a theoretical security risk. However, business services based on CATV have been popular with SMBs and even with some enterprises, who want to connect small branch/remote locations.

Comcast is upgrading a lot of its plant to support the DOCSIS 4.0 specification, which offers 10G/6G maximum download/upload speeds. This would make CATV competitive with much of the fiber broadband plant, in particular for uploads. Since uploads are usually more critical to businesses than to residential users, that would make the service more broadly useful to both SMBs and to enterprises for small-site connection.

Comcast also offers other business services via Ethernet, suitable for larger sites, but it’s not universally available. The new converged optical model of delivery would improve optical broadband connectivity to businesses, and it might also have benefits in edge computing. It might even be beneficial in the creation of a universal metaverse model.

First, let’s look at the two upgrades together. Wireline broadband suppliers face a challenge in that residential broadband and TV services need residents and households, neither of which are growing fast enough. Comcast has already started to increase its rates to raise revenues (ARPU) because the number of users isn’t growing. In fact, competition from other technologies (telco fiber, 5G FWA) is starting to tap off some existing customers. But business users are under-penetrated, so growth there is still possible. My data suggests that there are about four million business sites in the US that are or could be connected, and about a million and a half are satellite sites of multi-site businesses. You can sell to perhaps fifty thousand companies to address all those satellite sites.

If you have the right stuff to sell, which is the aim of the announcements. You need to improve DOCSIS to improve your connectivity to those satellite sites, but also to be more effective in single-site business sales. You also need a way of building the equivalent of an MPLS VPN for multi-site businesses, which is the aim (I believe) of the optical convergence architecture.

The essence of the architecture is a new terminating box that can reside on the customer’s premises or in theory in a Comcast data center. It links via fiber to a wavelength integration shelf that’s located with the Comcast hub, and from there back to the metro center. This provides a fast and low-latency way of connecting the commercial fiber service with the CATV access network, and onward to the metro point where the Internet connection is made. It also improves end-to-end visibility overall, particularly for business customers who use CATV/DOCSIS to connect their smaller sites and optical Ethernet to connect larger ones.

One thing this setup could do is improve the latency and connection management for multi-site companies, and that would be positive for Comcast’s business aspirations. The same setup could, in theory, be used to couple edge computing hosting provided either by Comcast or by a third party, and that could enhance the value of the offering for IoT and (gasp!) the metaverse.

I’m sure everyone who reads my blog regularly knows that I believe that we need to think about the metaverse concept more broadly than the social-media context you see most often. A metaverse is a digital-twin, communal-use, visualization, model. Each of those things can have a specific implementation that’s tuned to the mission, but one thing I think we can assume is that all metaverses will be “communal”, meaning that they will represent a series of interdependent digital-twin elements whose behaviors need to be reflected across the community as a whole. That means defining both a compute/hosting model and a network/connectivity model, and both have to be able to deliver results with minimal latency or the system loses sync with the real world.

It’s interesting to speculate on whether Comcast sees metaverse deployment as a viable opportunity, or sees it at all, for that matter. The material I’ve reviewed doesn’t make any explicit connection between their architectural changes and the metaverse, and generally the cable companies have been even more hidebound in the old traditional service models than the telcos have been. If I had to guess, I’d say that the potential metaverse applications of their architectural shifts are an accidental byproduct of going after the business market in a more serious way.

That raises another question, which is whether Comcast would push a modern over-the-Internet SD-WAN approach to business VPNs along with this change. They do offer SD-WAN today, but their diagrams show it as a small-site adjunct to a “private data network”. Might Comcast make a specific push for an SD-WAN-over-their-optical-architecture model as a total VPN solution? Given the reluctance of the telcos to go that far, it might be a very interesting play for them.

The potential issue with using the new Comcast architecture as a metaverse connectivity framework, or even just an IoT framework, is the latency. Potentially, DOCSIS 4.0 still has greater latency than many alternative access technologies, including 5G. That could limit the utility of Comcast’s capabilities. It might even impact the credibility of their business SD-WAN delivery, though most SD-WAN applications have greater latencies than MPLS VPNs.

Analyzing just what properties a new business service based on these architectures would present, and what applications it might facilitate, will have to wait for release and use. I suspect that when that happens it may also be possible to determine what Comcast thinks about the future of services, and its own revenues.

Probably the most dangerous thing a company can do in tight financial times is to chase buzzwords. Given that view, you won’t be surprised when I say that I was less than enthusiastic about the Great Telco Debate on the importance of being a “digital service provider”. I’ve never been wild about the term itself, perhaps because I remember when “digital service” meant T/E-carrier leased lines, and the fact that all services today are carried digitally. But rather than dig up ancient history, let me talk about what I think are the real issues the term raises.

According to the piece I referenced above, the attributes of a digital service provider are “flexible customer engagement, integrated operations and end-to-end automation”. It’s not that I dispute that these are important points, but that I think they take second place to a more critical way we could define what being a digital service provider should mean.

My beef with the attributes is simple; they make three important presumptions; that changes in the actual services being sold are not a requirement, that you need to engage customers better to sell your stuff, and you need to wring out all possible excess opex, because profit enhancement depends on cost reduction. The last two are necessary because of the first, which is why I emphasized that first point. The reason is that we’ve likely reached a point of diminishing returns on investment to reduce opex.

The integrated operations and end-to-end automation attributes, if they’re to reduce opex, have to reduce staff, and the primary problem with these two is that the majority of human-related opex was focused on outside plant, the “craft” piece of telco. There is little that remains in that area now, and the specific focus of the two attributes would be network operations, whose staff is a minimal piece of opex overall. There are benefits that might accrue from addressing these two cost factors, but frankly I doubt they’d be worth having a conference. Operators need to address profit per bit decisively, and I don’t think re-operationalizing existing services is the way to do it.

The right way would be to change the service set, and not just diddle with pricing models like “usage” versus “leased”. Experience has shown that if you offer a different way of pricing a service, users will adopt it if it lowers their cost overall. If that’s the only reason for adoption, then “success” in the offering would mean that operators suffered an overall revenue loss. Even the as-a-service model that I personally believe is the right way to target networks faces a challenge if you can’t define something new to be offered in as-a-service form. VPN as a service? Get real.

Here’s the thing. The network operator community wants to take root in the current service set and become trees. Give them a story that seems to promise they can do that and they’re all over it. Operators are facing a revolutionary problem and they want to address it with slow, orderly, comfortable, evolution, and that simply is not going to be effective.

What could actually link the three attributes for “digital service provider” to some rational model for operators? The best option and the easiest option are quite different, but both are illustrative and both are NaaS in some form. To me, “NaaS” would have to mean providing a form of logical addressing, and the difference between easy and optimal versions would be based on just how much work the user had to undertake to use the service.

The easy NaaS model would be one where the user could assign IP addresses to logical elements that could be hosted in various places, and the service would route the IP address to the current location of the element. This would mean that the user would have to tell the service that it had relocated an element, of course, and that would mean that there could be a lag between the instantiation of an element in a new location and the direction of network traffic there. The user, or software tools provided to the user, could take on the role of deploying and redirecting to simplify use. This mechanism would assume that an element was relocatable but that there could be only one instance of it.

A more complex example would be to allow the logical address to represent a community of elements 1:n in makeup. The NaaS service would be responsible for maintaining the instances associated with the community based on the user’s having provided a “community label” for an element and adding it to the community. Now, the service could employ one of multiple options for load-balancing among the community members.

The ultimate, perhaps, would be to frame NaaS as a “service mesh service”. In a service mesh, there’s a sidecar that service elements communicate through, and this sidecar element provides logical addressing and load balancing capability. The problem with current service mesh technologies (like Istio) is that they’re not highly integrated with the network and they don’t present themselves as a service. The challenge in a service mesh NaaS is figuring out what the service would look like. Since service mesh technology is really about message exchange within a component community, it would appear that you’d have to include a community-building function similar to the intermediate-level NaaS model above.

I think the NaaS evolution described here demonstrates a deeper truth as well. To be relevant, future services in general, and the “facilitating services” that I’ve blogged about and that AT&T has suggested it’s planning to deploy, have to be tied more to the cloud/edge hosting and message exchange model. There’s a tendency for operators to think only in terms of connectivity, and both the NaaS concept and the facilitating service concept mandate presenting something a bit more elevated, and thus a bit more useful in building retail experiences.

The challenge with all of this, and thus the challenge for operators who want to be true “digital service providers”, is that OTTs and cloud software providers and tools could “build down” to provide the same sort of thing. If we focus for a moment on facilitating services (presented, likely, as a NaaS), it’s pretty obvious that the stuff that’s being facilitated could build down as easily as the connection layer could build up. Kubernetes, for example, already has a virtual-network plugin/API. The service mesh concept in particular could be generalized, and we have a number of service mesh implementations already.

This is why I get frustrated by the determination of operators to take root in their comfort zones. There’s little comfort in eroding financials, and absent some strategy change that’s exactly what we can expect for operators worldwide.

Just how serious a problem is cloud provider lock-in to buyers? We hear a lot about it, often in connection with justifying “multi-cloud” or the practice of using multiple cloud providers rather than one. Cloud software vendors like Red Hat and VMware often cite the need for multi-cloud and the desire to avoid lock-in as a reason to use their software rather than the web service features of cloud providers themselves. I’ve been digging through my conversations for 2022 as part of my year-end process, and here’s what I found.

Let’s start with an easy question; do users take specific steps to avoid cloud provider lock-in? Only 29% of enterprises told me they did. Of the remainder, roughly half said they were “concerned about” or “planning to reduce” provider lock-in risk, and the rest said they had only “heard” or “thought” about it.

Of the 29% who said they took specific steps to limit lock-in risk, half said they designed their cloud components to be “as portable as possible”, a quarter said they used cloud-portable technology (Red Hat and VMware, primarily), and a quarter said they used multi-cloud. The smaller the company, the more likely they were to say that they tried to be as portable as possible, meaning they weren’t really doing much. The largest enterprises were likely to adopt both cloud-portable software and multi-cloud, though the biggest driver of multi-cloud in this group was a desire to get optimum coverage and pricing over a global geography.

There’s some indication that, over the last three years, the population of cloud users who took specific steps to avoid lock-in was shifting slightly toward multi-cloud as a strategy, but the biggest shift was that more companies were leaving and entering the group (keeping that 29% figure unexpectedly stable). There are strong indications that as companies get more committed to the cloud (note that means the hybrid cloud front-end model), they are more likely to want to focus on a single cloud provider whose cloud tools/services meet their needs. Likewise, enterprises who are raising their reliance on the cloud and at the same time trying to control costs may adopt multi-cloud to divide applications among providers based on cost.

I’ve tried running my model on the data, and it’s telling me that the long-term trend is going to be away from specific measures to avoid lock-in. One factor is that about two-thirds of users in 2022 said that the cost of cloud services was “substantially the same” across the major providers, which is up from their views of every past year since 2017. Another factor is that small enterprises and mid-sized businesses represent a big piece of the untapped cloud opportunity, and these prospective users don’t have the skill or staff to support any complicated development or operations. But neither of these are the biggest factor.

The number one reason why users are gradually moving away from provider lock-in fears is the increased complexity of hybrid cloud application goals, combined with the growing cloud provider trend toward “PaaS-or-SaaS-like integrated offerings.

At the risk of oversimplification, the cloud customer base can be divided into two groups, the big and tech-literate players and everyone else. It’s largely been the former group who have driven the cloud up to now, and this group of customers has the skill needed to support cloud-portable platform tools, multi-cloud, and custom development techniques to improve portability. The fact that more of the latter group are starting to use the cloud is enough to change the lock-in picture by itself, because that group has little interest in managing any of the avoidance options.

That less-literate group isn’t the only group that’s avoiding complexity, though. Over 80% of the bigger cloud users I talked with told me that they were trying to reduce their development and operations complexity, particularly with respect to multi-cloud. If we focus on users who went to multi-cloud primarily or exclusively to reduce lock-in, over half say they’re “re-evaluating” their multi-cloud decision. Obviously there are other reasons to do multi-cloud, but that’s still an interesting observation.

Multi-cloud issues include both development and operations complexity, but development complexity alone is apparently a factor even with tech-literate cloud users. There are a lot of cloud tools; the average literate cloud user says there are “dozens” to be evaluated and almost a dozen are used, on the average, by this group. All of these cloud users say that their cloud provider(s) are moving toward a more integrated “suite” approach, and more than half think that their provider(s) are developing what’s essentially a PaaS targeted at an application set. A few say they’ve heard that at least one of their providers (Amazon is almost always the one named) is starting to create vertical-market or horizontal-market SaaS by augmenting collections of their web service tools. Every one of these cloud-literate users say they’d be interested in reviewing a PaaS/SaaS offering, particularly for new applications.

A final point on the lock-in story comes from growing user awareness that “moving to the cloud” isn’t the goal. Those who have tried it report that application costs go up noticeably. Some enterprises have done some limited redevelopment of applications to make them more cloud-aware, but a large majority say that even if you rewrite an application for the cloud, it’s not going to be cheaper to run than the original data center application. That means that the hybrid front/back-end model is becoming the norm, and since most enterprise applications weren’t developed for that split, new development of the front-end piece is required. You can see why, for most enterprises, anything that makes that easier is valuable, and it’s easier to build hybrids from a single cloud provider, forgetting both portability and lock-in. As before, though, I need to emphasize that there are reasons for creating portable or multi-cloud applications beyond the lock-in justification.

I think that cloud providers are seeing the shifts I’ve noted here, just as they’ve seen that cloud adoption growth is slowing a bit. The big driver is that while you’re not hearing public cloud providers dismiss the popular (but totally incorrect) notion that everything will move to the cloud, they’re privately aware that the true cloud value proposition is the hybrid cloud and that they need to get on that bandwagon. Part of the reason is that both IBM and Oracle are (at the sales level, at least) pushing the hybrid model, and IBM has been very aggressive in positioning that model even to Wall Street.

Cloud providers might be heartened by news that users were less interested in preventing lock-in, but I think that if they look deeper at the reasoning behind the shift, they could find something to be concerned by. If lock-in isn’t a meaningful risk to buyers, then they’re likely to pay close attention to all the cloud providers’ features and prices before making a choice, which would surely make the sales cycle longer. In addition, any dissatisfaction with a cloud provider in any application could result in having the user move everything to a competitor rather than just moving the offending application. The combination of these two forces could make it harder for cloud providers to gain market share, and to avoid price competition that would reduce profits. That, in the long term, is probably where the greatest threat to cloud revenue growth can be expected to develop.

While it’s not an argument made out in the open very often, sellers have always debated the value of “truth” relative to the value of hype and exaggeration. The debate has increased in volume in the last decade as sellers competed for media visibility, given that the media is focused on what gets the most clicks not what’s the objective truth. One of those objective truths is that often hype and exaggeration end up hurting things. “Bulls**t has no inertia”, as I’ve often said, so it gets out there faster than any real product, and changes with the whim of interests faster than anything tangible could change.

So it is with mobile network evolution, 5G and beyond. Talking with vendors, network operators, and even enterprises over the last year has let me identify some specific ways where hype and exaggeration has hurt 5G and mobile services, and the hurt is continuing. The question is whether the hurt will be just pain, or be fatal.

The first of my hype issues is 5G was hot in the media years before there was any hope of deploying it. What’s the difference between a modern stealth fighter and a UFO? Answer: A UFO isn’t going to land nearby and present itself for inspection, so everyone is free to assign UFOs any characteristics they find interesting. Well, 5G was a UFO, and so all manner of promises were made for it. The chance of any of them being realized in the near term was zero, and many of these early myths continue to be featured in 5G stories and even marketing. All of this set up expectations in the market, and all of this also influenced both vendors and network operators to base future growth on 5G.

This first hype issue was a driver for the second issue, because you can’t keep saying the same thing to the tech media and expect to get continued coverage. To make things worse, the fact that 5G was hyped prematurely was no barrier to stories criticizing 5G progress. That encouraged our second issue, the promise that 5G Standalone would finally realize 5G’s potential.

5G has always had two potential models, the “standalone” and “non-standalone” or SA and NSA. The latter model deployed 5G’s radio network on top of 4G/LTE packet core, and the latter deployed the full 5G core. Many of the 5G features promised were in truth attributes of the 5G Core technology, but the features that were visible to the average smartphone user were almost entirely delivered in the NSA version. Today, there is almost zero chance you’d know, as a consumer, which 5G model you were using, but there are stories that promise that when 5G SA deploys, all will be grand. This has the effect of pushing unrealistic views of 5G impact into the future.

A further obfuscation of the impact of mobile standards changes is the growing 6G hype. We do not, at this point, have fully resolved 5G specifications and there’s already a wave of publicity on 6G. Obviously, if 5G’s UFO properties are waning in the face of actual service deployment, it makes sense to move on to the next thing. The troublesome thing about the 6G hype is that it’s coming when we know less about 6G than we did about 5G at the same point in its evolution.

6G stories have the same impact as 5G stories did, with regard to the actual progress of 6G deployment. They also have a new and truly unfavorable impact on 5G, something that didn’t happen with 5G and its predecessor 4G/LTE. When 5G came along we had already deployed 4G/LTE, but because 5G isn’t really fully deployed at this point, there’s a risk that 6G stories will overhang 5G. The good news is that such an overhang might actually help by diverting the “UFO” stories to a technology that’s not going to happen for years, and letting both operators and vendors get on with the uninteresting task of actually deploying something.

But then there’s the next of our hype-and-exaggeration points, 5G deployment is moving forward by applying cloud-native technology to networking. Every step we take in 5G, so the story goes, takes us closer to moving telcos to the “telco cloud” and cloud-native technology. That, by implication, moves us closer to edge computing and the expansion of applications that the edge would bring about. But is any of this true? No, probably not.

The first problem is that even within the cloud community there’s some debate on what “cloud-native” means, and there’s even more uncertainty within the telecom community. The fact that the term isn’t even reliably and consistently defined makes surveys difficult, but an analysis of my interactions with telcos over 2022 suggests that well over 70% still believe that 5G uses NFV. About the same percentage believe that “cloud-native” means “containerized” as opposed to virtual machines. Almost all of those who believe that 5G uses NFV think that the NFV ISG’s initiative toward “containerized network functions” will make everything cloud-native.

If there’s more to “cloud-native” than containers, it’s arguable that it has to be related to serverless or functional computing. A function is fully scalable, totally stateless, and in many respects an ideal element of a state/event processing model. That’s what most protocol handling is like, and in particular it’s a good approach to handling signaling versus data path, which is the primary mission of hosted functions in 5G. But unlike containers, which have largely settled on Kubernetes for the operationalization/orchestration model, there is no single model for functions and the suitability of serverless/functional computing for telecom issues varies with the implementation.

The second problem is that 5G feature hosting doesn’t necessarily mean hosting in the telco cloud. Some of the RAN features, for example, would likely run in places where little else was being hosted, and thus could well end up simply being white boxes or server-hosted features, not depending on a resource pool. Many telcos are looking at using public cloud providers to host features, which could mean that “telco cloud” isn’t going to happen at all, and thus won’t drive edge computing.

The third problem is that we can’t really define what “edge computing” is beyond saying it’s at the edge. Why is it there? If we can’t answer that question, we face the problem of justifying deployment of a more highly fragmented resource pool, which is inevitable if we host at the edge. If there really is a special class of applications that require edge hosting, we need to ask what specific features make that true, and how platform software in the edge locations would support those features.

All of this combines to drive us toward a messy evolution of 5G infrastructure, and the truly sad thing is that this was unnecessary. The cloud existed very early in the 5G standardization process, and it existed before NFV was launched. A reasonable architectural vision of feature hosting could have given us an abstract model of service feature hosting that we could have specificized to any new cloud thinking as well as applying it to the cloud’s state of the art.

Is it too late for that? That’s a hard question to answer. We have a lot of 5G deployed, which might have to be retrofitted if our abstract model didn’t conform to the current implementations. On the other hand, we also have only limited 5G Core implementations deployed, and it’s likely that the major impact of 5G on edge computing would come at the point where RAN and Core link, which is the metro area. That, as it happens, is also the sweet spot for edge hosting, because it’s close enough to the edge to ensure latency is low, but deep enough to serve the number of users needed to justify a real resource pool. We may still have time to think about “5G/6G”, “cloud native”, “telco cloud”, and “edge computing” in a realistic and holistic way, and that would be a good thing for all.

Our podcasters have been pretty accurate in their predictions about the economy and the stock market, and they update their views in this podcast. They also look at two key tech questions, which is what’s happening to the cloud and what we can expect in business networking.

Podcast: Things Past and Things to Come December 12 2022

As I looked through the details of my discussions on 2023 technology trends, one thing I found interesting was that there were some questions that many people in the enterprise, service provider, and vendor communities thought needed answering next year. I’d classify most of them as technology-strategic in nature, meaning they represent things that network types in many roles think might drive change in our industry. OK, let’s try to answer them here, and I’ll present them in what I think is a top-down way.

Question one is what role will the metaverse play in business IT. People think this is important because there’s a general sense that business technology needs to somehow jump up to a new level in order to continue to enhance productivity and justify additional spending. This was the question that was rated most often, spontaneously, as the most important of the lot, and it’s also the most difficult to answer.

The broad market view of the metaverse is that it’s a virtual reality, visualized through some immersive display technology and linked in some way to at least the individual viewer/user, and perhaps to a group of users. This view likely evolved from Meta’s social-metaverse model, and it does have business value in applications like conferencing and training. However, users don’t rate those applications high in terms of impact on their business efficiency.

I’ve presented the metaverse in my blogs as a mixture of digital twinning technology and visualization technology. Any virtual reality is a reality first and a virtual visualization second, IMHO. The digital-twin concept is a mechanism that links sensor and potentially controller technology to a real-time model of a real-world system. A human wearing a sensor suit is one example, but an assembly line or even a smart building is another. The visualization piece shouldn’t be automatically be considered an avatar-based representation of the model, but rather a presentation of the model in a useful form. This model seems broadly applicable; you could fit the traditional view into it as well as almost any IoT application, but it’s not widely accepted.

So what’s the answer to the metaverse-impact question? Well, there is little chance of meaningful business impact of a traditional metaverse model in 2023, or even later, in my view. There is little chance that my digital-twin approach will be available to exploit in 2023 either, so there is no likely significant metaverse impact on information technology next year. I’d be happy if we got the the point of understanding what the digital-twin approach to the metaverse could offer us.

The next question is how much impact can we expect from artificial intelligence and machine learning? Here we can be positive; the answer is a lot of impact. AI/ML is the use of information systems technology to deliver insights, make decisions, or take actions without specific human intervention (hence “artificial” intelligence). We’re doing that now, and the use of AI/ML is going to explode in 2023, making it the most important technical element for the year.

The reason for this explosion is complexity. When I started programming, an application ran on a computer in a singular sense. You loaded and ran a program, not multiple programs at once or multi-component programs. It ran on a computer, not a resource pool of connected elements. You fed it data and the data was accepted at the pace the program could accept it. Data did not present itself and demand servicing. Nothing is simple these days, and there’s little question that if we want to continue to improve productivity and quality of life via IT, we will need even more complexity.

Complexity makes operations expensive at first, and eventually impossible. People can’t absorb the full range of relationships that exist today for even a single major application. If something goes wrong, they can’t easily identify what happened or select the proper response. AI/ML is a complexity manager and we are irrevocably committed to more complexity. Given this, the most important thing we need is a model for applying AI/ML, one that reduces complexity and the risks it creates. Do we get something like root cause analysis, recommendations on actions, or automatic responses? Probably all of these, or one of them whose selection depends on just what’s happening and how critical the system it impacts is to our operations or lives. What we need to look for in 2023 is less the explosion in AI/ML, less the technology, than the way we organize it into those operations and lives. If we don’t get that right, we may have our systems battling each other and creating even more complexity.

The third question relates to that point. People are wondering how we educate buyers, workers, providers, of technology as the complexity of elements and systems of elements are both expanding. This, to me, may be the most important question of all, even though it’s rated lower by the people I’ve talked with.

Those who read my blog regularly (or know me personally) know that I’m a poetry fan. So you won’t be surprised if I quote Milton here: “Who best bears his mild yoke, he serves him best.” Or maybe Emerson: “The world is like a multiplication table or a mathematical equation that, turn it as you will, balances itself”. Our work and our lives are a vast ecosystem, one that technology is expanding. Does anyone understand it all, or should our goal be to properly play the role we’ve selected or been assigned?

You could argue that the real metaverse value proposition is representation to facilitate simplification. You could say that AI/ML is really the same thing under the covers. We can cope with massive organizations, massive problems, if they can be divided into a hierarchy. That allows us to grasp the total picture as a sum of high-level things, and to decompose the details of the specific piece where we find ourselves. Companies rely on organizational hierarchy in their operation, but they usually fail to apply that principle to technology for two reasons.

Reason One is that information conduits today are driven by sales objectives. The tech world is a collection of products, in information terms, because product information is what’s presented in an ad-sponsored world. You can find information about programming languages, operating systems, middleware, servers, network devices…the list goes on. Suppose you want to run your company as cost-effectively as possible?

That leads us to Reason Two, we lack an intuitive view of the hierarchy into which tech elements fit to support a given mission. The process of knapping a stone knife is complex, but it’s easy to link the mission, the product goal, the materials, and the processes. If we were to assemble all the parts that make up an automobile in a big pile, I doubt even an automotive engineer could assemble them into a car without a blueprint. Where’s the blueprint that defines an IT ecosystem? The sum of any arbitrary set of parts don’t make a whole, it makes a pile of junk.

I’ve recounted my favorite CIO remark in past blogs. I was sitting with the CIO of a giant financial company and listening as he explained his challenges. All of a sudden, his eyes lit up as though he’d been struck with a great insight. He leaned forward, slapped his palms on his desk, and said “Tom, you have to understand my position. I don’t want decision support, I want to be told what to do!” So do we all, or at least we want to learn what to do. Who teaches us? That’s the educational question in a nutshell, and I have no answer, and it may be that until we can address this last question, answering the earlier ones will prove difficult.

Salesforce is the premier provider of SaaS, and their stock recently took a hit because the Street didn’t like their guidance in their earnings call and their executive changes. The company has been a bit of a darling of the Street, as have most of the cloud companies, but unlike the major cloud providers, Salesforce is pretty much a pure play on the cloud. So what’s really going on? Do we have a Salesforce problem, a SaaS problem, or a cloud problem?

If you dig into public cloud services, you likely find terms that end in “aaS”, which of course means “as-a-service”. We have all sorts of aaSs (be careful how you pronounce that!) but one of them is SaaS, for “software-as-a-service”. I’ve not talked about this particular form of cloud computing as much because SaaS is an application set, not a general cloud resource, but there are some reasons we need to talk about it now.

As a cloud model, the advantages of SaaS are pretty clear. It presents users with an application and not just a resource, so it’s easy to establish a business case. You essentially subscribe to an online application, so there’s not as much (if any) development work associated with it. It’s ideal for organizations who want an application but don’t want to go through their IT organization to get it. There’s some administration needed, but it’s way easier than using cloud hosting in infrastructure- or platform-as-a-service mode.

There are also risks, of course. In theory, almost anything can be run in the cloud on IaaS/PaaS, and anything can be written to run there using all manner of custom tools and features. SaaS is an application set, so its breadth of utility depends on just how many applications are in the set and what feature flexibility is offered in each.

Do the pluses outweigh the minuses? Well, there’s one key minus we need to add, one that’s only implied, and that’s the fact that SaaS is outside the area of cloud services where the largest business driver of the cloud is focused.

Why do companies adopt cloud computing? The simple fact is that the great majority of public cloud services target the creation of an enhanced, broadened, user interface to company applications, what I’ve continually called a “front-end” to those applications. Companies started this because they wanted to open a direct online conduit to their customers, and have expanded their interest to include creating a partner connection and even to support their own workers. If you look at public cloud application development, this is almost a universal target, and SaaS doesn’t fit into it.

Could it? Companies like Salesforce, the market leader, certainly think that something could be done. Their SaaS tools already integrate with company data and applications, and they’re offering more general tools for integration as well. They offer at least some customer support and marketing applications too, but the enterprise contacts I have rarely mention them as candidates to support a cloud front-end mission. We need to explore why that is to answer the question of whether SaaS could fit.

A facet of the problem can be laid to a theme I’ve mentioned before, which is marketing and “trajectory management”. If you go to the website for Salesforce or another SaaS player, and if you play the role of a prospect who’s kicking the SaaS tires, it’s easy to see how you’d abandon your consideration quickly. It’s not easy to navigate the site from a position of ignorance about SaaS and the company’s offerings. If you wanted, or knew about, Salesforce offerings like CRM you could pick out what you need, but I took the time to play the role of an enterprise exploring whether there was a “SaaS front-end” tool, and came away with no answer. Salesforce and others in the space aren’t developing the demand for a SaaS front-end.

The “why” associated with that truth is another facet of the problem. Public cloud dependence on a hybrid cloud model where data center applications and databases are front-ended by cloud elements has been swept under the surface by the “everything moves to the cloud” theme. IBM recently gained traction in cloud applications because they at least understood what buyers were trying to do. In the last year, perhaps spurred in part by IBM’s traction, the three major cloud providers have stepped up their hybrid cloud story, but that hasn’t happened among the SaaS players.

And the “why” for that particular point is our last facet. SaaS is really application-as-a-service, and the front-end mission is not one that’s particularly easy for a company to adopt in “stock software” form. A cloud front-end has to be tightly integrated with the data center piece of the hybrid cloud, and also to the company website. That multi-dimensional integration requirement combines with the simple fact that the front-end element of applications is literally the face of the company to mean that companies are pretty particular with regard to how it looks and works. It’s my view that SaaS as a model isn’t helpful in the great majority of cloud front-end missions.

That doesn’t necessarily mean that SaaS-centric companies like Salesforce are cut out of the cloud bonanza. There are three possible ways they could get into the game.

The first way would be to go directly head to head with Amazon, Google, and Microsoft. That would mean a major amount of development, big changes in sales and marketing, and a major set of new risks, all to join a field that many think is already crowded. Non-starter.

The second way would be to develop what I’ll call a “front-end SaaS” model targeted to smaller-than-enterprise businesses. These companies are less likely to have a need for highly dynamic websites, and may already use site-building tools. The problem with this approach is that it targets prospects that would be harder to close through direct sales, and harder to support if you succeed. Possible.

The third way would be to develop a “front-end-PaaS” instead. It might be possible to build a platform that would facilitate hybrid cloud front-end implementation without unduly constraining it. I stress “might” here because nobody has really tried to do that. If I’m correct in presuming that the general trend in hybrid cloud has been obscured until recently, then the nature of a specific model for cloud front-ending has also been obscured. Right now, organizations build their own model from more primitive tools, offered as web services by public cloud providers, and Amazon’s extensions to AWS suggest that the big cloud providers may be enhancing those tools. Many of the changes Amazon recently announced deal with integration, noting that having service features created by semi-autonomous product units can create something only an architect can understand and assemble. Are they backing into PaaS?

A broader question is whether a general PaaS be defined that would make development easier and faster, and offer enough flexibility to accommodate the different visions of the GUI that inevitably emerge in a competitive market. Maybe. Some constraint is inevitable in a PaaS model because platform features, to facilitate development, must presume something about the direction development will take. So the real question is whether it’s possible to develop a PaaS model whose constraints are acceptable to cloud users, and whether that model will add to cloud revenues or just cannibalize the current cloud front-end base.

Those are the things I plan to explore with both cloud users and providers, and I’ll open it again when I think I have some useful answers.

I’ve said several times in my blog that I’m a fan of the notion of network operators’ providing “facilitating services”. This is a notion raised by AT&T, one I blogged about HERE, and the reason I like it is that it seems to strike a balance between operators’ fixation on providing traditional services and the fact that the revenue per bit for those services is (let’s face it) doomed. The question is what these services could be and how operators would go about offering them. I did a multi-blog series on the technology aspects of facilitating services, but I want to take a step back and look at the properties a facilitating service would have to offer to be valuable. From that, we may be able to derive some tracks that facilitating services could follow.

The first property is broad market utility. Putting it another way, the facilitating services must be targeted at the consumer space, because (as I’ve noted in my blog on Tuesday) business-specific data services are doomed niches. Something targeted there is simply not going to generate enough incremental revenue to make any investment worthwhile. Achieving a broad market utility goal divides into two paths. The first exploits what is already being done online, and the second exploits what is likely to come.

What’s already been done is a mixture of general Internet elements and content delivery elements. In the first category, DNS is already offered by almost every operator, but operators don’t charge for DNS so it’s not much of an opportunity. Same with DHCP. Many operators offer security services for consumers and business, but uptake is mixed because their offerings are usually simply the resale of something available elsewhere.

Content delivery facilitation is another option, particularly mobile content. Current CDN technology works well in wireline/fixed services because current CDN suppliers can peer with the broadband provider at a fixed point. Mobile roaming means that optimum caching may have to be changed depending on where the user is located. However, people consuming video don’t tend to roam far or quickly. In short, this area is currently covered enough that it’s probably too late for operators to jump in.

The second of our areas shows more promise, but also generates more risk. I’ve blogged for years about an area I called contextual services, services that would be designed to integrate applications and information tightly with the life-context of users. It’s probably not escaped many of those who’ve been reading my blog that the “digital-twin” framework I assert is the essential element in the metaverse concept is perhaps a modern basis for these new services.

Any service that’s intended to integrate with the real world needs to know what’s going on there, and needs to be able to organize and reference that knowledge in the way it interacts with the user. That means, IMHO, creating a “digital twin” of those real-world elements that are essential in supporting that interaction, and this is the piece of the metaverse concept that I believe is essential. Other layers build on it, in fact, and that means that a service that supported collection and modeling for digital twinning is a poster child for “facilitating service”.

Speaking of layers, there are opportunities for facilitating services above the basic digital-twinning layer. Most digital twins, especially those that support an individual-subjective service set, would likely reside close to the individual contracting for the service. However, we could say that above the digital-twinning layer is a visualization layer. The metaverse models reality, and how reality should be presented would depend on the nature of the higher-layer service. The Meta concept of a social metaverse is based on creating an alternate or virtual reality, and one attribute of that reality is a remapping of user geographies. A social metaverse creates a shared virtual place whose inhabitants are not sharing a physical location. Other metaverse concepts could have more limited need for “location remapping”, but all require some form of visualization, which is another facilitating service opportunity, and so is the “location remapping”.

A final point here is that digital-twin applications of any sort are likely to be very latency-sensitive, and that favors those who can provide hosting proximate to the inside edge of the access network. Obviously, network operators can do that, and so this point would favor their participation in the application set via facilitating services—edge hosting in this case. That has the advantage of being a very general service, one that doesn’t link the operator to a specific application and take it into competition with OTTs.

In all, some sort of digital-twin-metaverse concept seems to be the best opportunity for operators to target with their facilitating services.

The second of our properties is a close relationship to current operator services. That doesn’t mean voice; there’s no future opportunity there because OTT voice is simply too easy a market to enter. It means current broadband Internet, both wireless and wireline/fixed.

The question here is just what might constitute a “close relationship”. One possibility is that a facilitating service would exploit features of a current service. The feature of connectivity is an example, and it’s probably the most insidious trap of all traps for operators, and so of course they fall into it continually. We have connectivity today, and we’ve built whole industries, maybe whole economies, on the connectivity we have. So what new connectivity feature do operators think we need?

There is a feature that might be useful, though, and that’s the feature of location awareness. Wireline services carry the knowledge of where they terminate. Mobile services carry the knowledge of what towers they’re connecting through. The trick is our last property, avoiding regulatory pitfalls. Selling or even using the specific location of a user is a minefield of public policy issues. The good news is that there’s an alternative.

If lemmings are marching to the sea, versus happily grazing in a meadow, you don’t need to survey lemmings to find out where they’re going; the herd is going and it’s that truth that matters. Same is true, for example, for auto or pedestrian traffic. Do you care who, specifically, is congesting your route, or do you care that your route is congested and that patterns of traffic in and off your route suggest it will clear in twelve minutes? There is value in the knowledge of the broad movements of mobile users.

You could make a similar case for the patterns of usage of calls and the Internet, plotted against location. A given group of people will make and receive a number of calls and texts, and will perform a number of Internet searches. Suppose that activity levels suddenly change radically? Something is going on, and the something is visible enough to change mass behavior. That’s useful knowledge, which means it’s exploitable.

Another possibility is that the deployment of a facilitating service would be aided by real estate and technology associated with broadband deployment. That means, IMHO, that the smarts associated with facilitating services would reside in metro centers where economies of scale and service opportunities would both be high, and latency could be minimized.

The third property is a relatively high first cost and low ROI. Operators want to sell facilitating services at a profit, but their tolerance for low ROIs is higher than that of OTT players. Operators can also accept a fairly high “first cost”, meaning the cost to create a credible initial service over a viable market footprint. A high tolerance for marginal returns on investment is critical if you’re going to offer a wholesale service element and expect to make a profit. A high first cost is critical because you don’t want the retail providers rolling their own features, you want them to buy yours. If first costs are high it discourages those retail providers from cutting you out.

This is a more serious issue than most, including most operators, realize. There’s no point in investing in something that ends up being an attractive paper weight. The person who owns the customer, the retail provider operators are hoping to facilitate, has the most critical asset. That means the operator has to have a counterbalancing asset, and that asset (like the retail customer ownership) comes down to a financial value. You do what the other company needs, what they don’t want to do themselves. Otherwise you’re throwing money away.

The final property is avoiding regulatory pitfalls. Given the second of our properties, any meaningful facilitating service will have to be related to broadband Internet, and that area is fraught with regulatory uncertainty and has been for decades. Operators are justifiably reluctant to enter a service market whose investment is at risk if there’s a change of policy.

Facilitating service goals are easier to manage to dodge regulatory issues for the simple reason that most regulatory policy focuses on what the consumer is offered. However, there is a second-level vulnerability in terms of use and misuse of information.

On the “use” side, there’s all manner of regulations protecting consumer information and privacy. These rules would likely prevent a network operator from leveraging information they had as a result of the customer relationships they supported. If a hacker might want something, you probably don’t want to offer it as a facilitating service.

Relative to “misuse” the issue is that you could expect any facilitating service offered to be examined by malefactors to identify ways it could be exploited. If one is found, the repercussions could be dire if someone suffers financial loss or injury, or if it turns out that some aspect of the service doesn’t pass regulatory requirements.

This issue may be the most problematic for facilitating services, particularly for “digital-twin” services, because by nature these services require a level of personalization, which means they have to know who the user is and something about their life.

I think we could, as an industry, navigate through the issues and create valuable facilitating services, but doing that almost surely starts with recognizing and accepting the concept of a service-level partnership between operators and OTTs, something that’s really not part of the current mindset of either party. I hope that AT&T’s interest, which seems to me to be flagging a bit, is enough to get everyone thinking, at least. The concept, or lack of it, could change networking.

Our weekly podcast looks at economic conditions, the stock market, and tech in general, but also at the recent Wall Street interest in Juniper, a major competitor to Cisco in the IP networking space.

Podcast: Things Past and Things to Come December 5 2022

Recent discussions on LinkedIn that relate to how NFV and O-RAN fit seem to show that there’s still a wide range of views on the way that hosted features relate to network services. Since that may well be the most important topic in all of networking, and surely is for network infrastructure, I want to weigh in on the issue.

NFV, or “network functions virtualization”, came about because of a 2012 Call for Action paper put out by ten network operators. If you look at the paper, and at the early end-to-end architecture model of NFV that’s still largely in place today, you find that the high-level goal of NFV was to substitute “virtual network functions” or VNFs for network appliances, or “physical network functions”. The paper in particular seems to focus on “appliances” rather than on the broader “network devices” category. The Call For Action paper, for example, talks about “many network equipment types” as the target. It also focuses on equipment other than switches and routers (see Figure 1 of the paper if you have access to it) and the functional software elements that replace “appliances” are called “virtual appliances” in that figure. All this is why I contend that NFV was targeted at replacing discrete devices with virtual devices.

The architectural focus of the white paper and the end-to-end model means in part that NFV management strategy was aligned with the goal of meshing VNFs with existing element management systems (EMSs) that were used for the appliances the VNFs would replace, and with OSS/BSS tools. Most of the early proof-of-concept projects approved for NFV also targeted “universal CPE”, which of course is almost exclusively relevant in the business service space. It’s that combination of alignments that I contend focuses NFV on customer-specific business service deployments.

This sort of stuff is understandable in light of the original NFV mission, but it collides not only with the way that hosting is used in later service standards (notably 5G), and even in some ways with the original Call for Action white paper. Some of the appliance types that the Figure 1 I previously referenced include are community- rather than customer-focused, and I contend those demand a different management model. That was in fact one of the things I addressed in the PoC I submitted (on behalf of a collection of vendors). I called services directed at community elements like 5G components “Infrastructure Services”, meaning that they represented services that were deployed as part of overall infrastructure, for community use, rather than per-user.

A traditional element management model implies that a single management element controls a specific network element, and where that network element is a shared resource that cannot be the case. In fact, any management interface to a shared element poses a risk that shared use will create a kind of “management-denial-of-service” attack, where too many users/services reference a management API and the workload causes the element to collapse. The notion of “derived operations”, which was based on the (sadly abandoned) IETF draft called “Infrastructure to Application Exposure” or i2aex, addressed this by maintaining a management database that was queried and updated by users through one of any number of software/virtual MIB proxies, but was alone referencing the actual MIBs.

The other issue presented by the NFV architecture is one I’ve mentioned before, which is a deviation from the cloud’s onward evolutionary processes. Cloud computing at the time the NFV launched was largely based on virtual machines and “infrastructure as a service” or IaaS. Today it’s much broader, but a major part of NFV was explicitly tied to the IaaS/VM approach. The notion of creating a functional unit (a virtual device or a VNF) by composing features was actually raised in the spring 2013 NFV ISG meeting, but it would have required vendors to “decompose” their current appliance software since applications weren’t built that way then. Today, of course, cloud applications are regularly implemented as compositions of feature components, and standards like 5G and O-RAN presume that same sort of feature-to-service relationship. To get terminological on you, NFV was about functions which are collections of features, and the cloud is about features and how to collect them.

My references to 5G and O-RAN here should demonstrate why I’m bothering with this topic now. The fact is that when NFV launched, we had network devices that we were trying to virtualize. Now, we have network features that we’re trying to integrate, and that sort of integration is exactly what the cloud is doing better and better every year. We have orchestration tools and concepts for the cloud that make NFV’s Management and Orchestration (MANO) look like Star Trek’s “bearskins and stone knives”. Projects like Nephio and Sylva are advancing cloud principles in the network space, and Sylva is explicitly targeting edge computing applications, in which group I’d place O-RAN.

As one of my LinkedIn contacts noted recently, we do have successful applications of NFV, examples of how it can be modernized. That doesn’t mean that we have a justification to continue to try to adapt it to current feature-hosting missions when we have cloud technology that’s not only addressing those missions already, it’s evolving as fast as new missions are emerging.

Standards groups are the wrong place to try to introduce software, and open-source projects are the right place. That doesn’t mean that simply morphing NFV into an open-source project would have done better, though. In my view, both Nephio and Sylva are current examples of open-source projects directed at a telco mission, and neither of them is proceeding in what I believe to be the optimum way. It’s always difficult to understand why that sort of thing happens, but my personal view is that telco participation in the projects is essential because telcos are the target, but telcos don’t have software architects driving their participation. As a result, they drive sub-optimal decisions without realizing it.

So is there no hope? I think there is, but I think that the “hope” we have is the hope that the processes we need will evolve without the telcos’ involvement, and that because of that those processes will be focused more on the public cloud than on the telco. There will be renewed cries of “disintermediation” from the telco side, of course, but in this evolving edge and function hosting model as in past service initiatives, the telcos have succeeded in disintermediating themselves. If they want to stabilize their profit per bit, they need to get with the software program in earnest.