Remember “shadow IT?” Even the name seems a little sinister, and for many CIOs it was a threat from the get-go. What shadow IT is all about is the dispersal of information technology purchasing and control to line organizations, creating a parallel set of technology centers. We don’t hear as much about it these days, but a recent article in Protocol made me look back at what I’ve heard from enterprises this year, and it’s interesting.
Despite the seemingly recent origin of shadow IT, it’s actually been around for sixty years. A company I used to work for, a giant publishing firm, had a half-dozen very separate lines of business, and they tried out dispersing IT to the business units—several times in fact. Over a period of 20 years, they went back and forth between central and distributed IT, and gradually ended up in a kind of uneasy middle ground. The reason for all the vacillation is relevant to the current situation.
Line organizations often see the IT department as an enemy. In the 177 firms I’ve talked with this year, over a hundred said that they had “tension” on goals and costs between IT and line organizations. Almost a hundred said that they had some “shadow IT” activity in place, and none of them indicated they were trying to stamp it out, though half said they were trying to “control” it.
The reasons for the tension vary, but the top two items on the list are pretty consistent across enterprises. The number one issue was delay in getting critical line-sponsored projects implemented. Companies said that it took IT “over a year” in most cases to get a project completed, and the line organization targets were less than half of that. Issue number two was that IT costs were substantially higher than expected, often so high that the business case was threatened. Line departments felt that there was too much IT overhead, and that their allocated costs covered resources and activities that didn’t benefit the line organizations at all.
Line departments reacted to these issues in two specific ways. First, they promoted the notion of low-code/no-code development to take more control over project implementations. Second, they looked increasingly to the use of cloud computing, because as an expense, cloud computing bypassed some executive limitations on just what line organizations could do. “I can’t buy computers but I can rent computing” was a typical line comment.
There’s no question that low-code/no-code “citizen developers” have revolutionized the way line organizations handle many of their projects. Interestingly, enterprises often don’t consider this a form of shadow IT; twenty percent more companies say their line organizations use low-code/no-code than say they have shadow IT. IT organizations have gradually accepted the citizen-developer trend as well; the majority of the enterprises that use it said they didn’t put any special restrictions on use, though most did say that IT played a major role in selecting the tools.
The cloud is another matter. When asked whether cloud projects driven from line organizations were more successful or more likely to have issues, enterprises (and even a majority of CFOs) said that “citizen-cloud” projects often failed to deliver on expectations, were more likely to experience significant cost overruns, and “usually” required IT organizations step in to correct problems. CIOs’ biggest problem with citizen-cloud projects was the security/compliance issues, which came about because of data stored in the cloud without proper precautions.
The specific trend that Protocol talked about in the referenced article, the notion that software and even hardware vendors would start selling (or trying to sell) to line organizations rather than IT, wasn’t particularly common in my sampling. Only 27 of the 177 enterprises indicated that this had happened, but of course there’s always the chance that the attempts weren’t all recognized even by CIOs and CFOs. Both the CIO and CFO organizations indicated that they believed that their current policies on software and hardware purchase by line departments were satisfactory. In general, those policies required that the dollar amount be small (usually somewhere between five and ten thousand dollars maximum) and that the software and hardware be used entirely within the purchasing department. Network equipment purchasing by line organizations was rarely allowed (11 out of 177), and the same policy held with software that had to be run on IT-owned or multi-tenant facilities, though that was allowed by 38 of 177 enterprises if the hosting was done in the cloud.
I was interested in how the sellers might view the idea of going around IT, and there was considerably less consensus there than among the buyers. My sampling here is more limited (75 vendors), but over two-thirds of hardware and software vendors said that they would be “reluctant” to prospect line organizations, partly because they were afraid of alienating their major buyer (IT) and partly because they were afraid of creating a very visible failure that could taint their reputation within the company, or even in other firms in the same vertical market.
Cloud providers didn’t share this reluctance, though they were reluctant to talk about it and their policies might vary significantly across providers, sales regions, and even individual salespeople. The best I could do here was to identify 49 of 177 companies who said their line organizations had been prospected by cloud providers or cloud resellers.
Perhaps the most interesting thing about all these shadow-IT, citizen-IT plays is that they dependably spark IT initiatives to enhance IT responsiveness and manage costs. Every enterprise that supported citizen developers also had a “growing” community of low-code/no-code developers within the IT organization, aimed at providing a rapid response to the line project requests. These resources were primarily directed at projects associated with long-running applications rather than the one-off applications that (slightly) dominate line department projects. About a quarter of enterprises either had, or were exploring, ways of supporting citizen-cloud requirements out of IT.
If you dig into citizen/line/shadow IT, you find the classic irresistible force versus the classic immovable object principle in action. Line organizations see business problems and opportunities first, and because most have direct profit-and-loss responsibility, they also see considerable pressure to address them. Given that enterprise IT has enough problems in acquiring skilled technology workers, it would be surprising if line organizations had a rich pool of IT skills, so there’s rarely a good understanding of how to run an IT project, or how to assess whether one being run by the CIO is being run right. Some bickering here is inevitable, but that isn’t the main issue.
The main issue is that we’re continually trying to move IT closer to the worker. We’ve gone from batched punch-card retrospective reporting to “mainframe interactive” transaction processing, to distributed computing, to hybrid cloud. That evolution frames the capability to host more worker-interactive models, but it’s the applications that really establish the way workers and IT relate. What we have in shadow IT today is an attempt to make applications as dynamic as application hosting. That’s partly a low-code/no-code problem and partly a data architecture problem.
It’s hard to advance applications toward citizen development, even with coding tools, without creating “data middleware” that makes information access easy. The great majority of citizen developer projects are really data analysis problems, and so enterprises who are trying to use citizen developers or support the activity should take a hard look at their data analysis and modeling strategies, and try to create a unified data model to support line department information use. Without that, it may be more difficult to get full benefit of low-code/no-code and even citizen cloud empowerment.