Middleware Explosion

Two decades ago Nicklaus Wirth, the designer of the Pascal language, wrote a book entitled Programs=Algorithms+Data Structures. His thesis was that writing a program required defining the data structures and designing the algorithms required to manipulate them. In retrospect, it seems that all one needed to get started was to understand the programming language and the way it interacted with the environment. This is in contrast to the situation

today.

If one were to pen a similar equation today, it’d probably read,

Programs=Algorithms+Data

Structures+Middleware+Development Framework+Database Engine+Domain Knowledge. Even this risks being a simplification. The development environment today is so complex that a large part of the effort in designing and implementing software is in understanding and managing processes external to the actual coding. 

Let’s see what it takes to mount a medium-sized project in the MIS/ERP segment. Two types of skills are required: implicit and explicit. In implicit I’d put knowledge of the application domain (financial accounting/production planning; general programming, networking and DBMS concepts; and knowing the basics of client-server computing).

About explicit knowledge, one needs good knowledge of the RDBMS being used and it’s associated middleware. Understanding a product such as Oracle calls for knowing its basic architecture, the way it implements a protocol stack and the important configuration options. In most cases you also need to understand the associated ODBC driver and its limitations. Then you need to have a good idea of concurrency-related issues, like transactions and locking.

Development tools are complex beasts and mastering them calls for a complete understanding of most of the underlying language and its object mechanism, the core class libraries, handling of common controls and interfacing with middleware.

Much of this is acquired the hard way by being exposed to the deficiencies in the default mechanisms. 

Increasing environment complexity impinges on every aspect of software development, but the two areas that suffer the maximum impact are human resources and product quality. Let’s start with human resources. To train a fresh recruit, you have to start from ground up. Smaller organizations do not have formal training and most training occurs by mentoring. This means that not only is the new recruit unproductive, but he also reduces the productivity of seniors who have to teach him.

Now let’s turn to product quality. It is axiomatic that reliability decreases with increasing environment complexity. You may have gotten your logic right but your application is not going to run if the middleware you use has a bug. Worse is the case where the middleware behaves differently from how you expect it to.

The Bottom Line

Every development tool in the market today promises to increase programmer productivity. None do.

Gautama Ahuja runs a turnkey software company,

AHC Infotek