Why the time could be up for enterprise middleware?
Since the onset of Information Technology, there has been multiple solutions addressing a very common problem any enterprise had to face and that is “Integration”. Basically to transfer data or information from one application to another essentially to simplify, automate or optimize business processes. This took many forms such as a centralized middleware platform to domain specific or unit specific middleware or pattern based integrations. The premise however for Integration was its mandate and it eventually grew within the enterprise. Even today many of the leading analysts’ reports on IT expenses put the integration expenses to vary between 30% to 60% of the overall IT budget which is quite a conspicuous cost under the radar for any CIO.
There are many names given to the architectures, patterns, usages to solve this “integration” whether Point to Point, File Transfers, Landing zones, Message bus, MoM, EAI, ESB, SOA, EDA, Message Broker, Pub-Sub, EDI, ETL etc. Probably someone can write a 1000 page book on how every company uniquely choose to have certain constructs, bought leading proprietary software of those times and even continues today with decades old technologies either in part or whole. At the same time this middleware or integrations have been so infamous in a IT organization that any production issue first will be routed to the middleware support correctly or incorrectly.
Over the years, the maturity has improved among the software vendors, IT organizations and systems Integrators. Effective technologies, standardized protocols and proven architectures have significantly improved the middleware operations. The availability of low cost open source software as an alternative has also challenged the status quo and given enterprise leaders a leverage to deal with proprietary vendors. All sounds favorable to the Enterprise Middleware.
But!, the nature of middleware as it stands today is not relevant to the needs of a Digital enterprise and future of Information technology. You may ask “why”?
More than half of my professional life has been spent as a middleware specialist integrating systems, applications and processes using myriad ways in a variety of enterprises. In the past couple of years, having discussed with many CTOs, VPs and senior directors who directly manage this type of portfolio, there is a common concern about the middleware being the bottleneck in this age of digital projects requiring quick releases.
To summarize the key drivers and trends going against the grain of middleware:
1 - Enterprise will and must evolve from on premise to numerous Cloud Applications. Each Cloud app will be self-sufficient to handle the set of services. The integrations between these cloud-apps will be on standard protocols (RESTful APIs) which doesn’t require a conventional ESB or middleware product.
2 - Small will be big. There is no appetite for large programs, big ERPs, multi-year journeys and yearly releases. Forward looking multi-year plans laying out the visions and strategy will obviously exist and needed for any enterprise but the executions will be broken down in small independent projects and functionalities. To execute such small programs, independent but complete teams will be needed. Bureaucratic hurdles accompanying centralized systems and middleware teams and platforms will no longer be tolerated.
3 - Application owners will expose APIs or create micro-services to expose the right APIs. One of the key requirement of middleware was to translate messages from one system to another. This will not be needed if every application exposes standard APIs governed by acceptable principles.
4 - Automation of business processes. More and more functions will be automated requiring less and less workflows and user interventions. Automatically, integrations will be needed but easily solvable by standard APIs.
5 - Leading Silicon Valley companies don’t have any middleware. Businesses are seriously considering their IT units to adapt progressive technology practices.
For modern enterprises to take the next leap, they need to dissolve these centralized middleware constructs and adopt Microservices, DevOps and APIs to accelerate enterprise integrations.