Manually typed lines of code (MTLOC.)

Thoughts of an amateur philosopher.

Recently I have been working on couple of small projects. One consists from one class and one jsp page that demonstrates usage of the class ( /rndnode_war/test.jsp ). And second has about 5 manually coded classes. One class is a simple Velocity based code generator, which produces persistence layer (6 other classes). (Why no persistence framework was used is an irrelevant story).

Those unremarkable “projects” suddenly forced me to think about evolution of information technologies and how they affect my work and life. My first thought was: look I have created about 150 lines of code (LOC) and they do so much work. But second thought was: hey, how about millions of LOC which have been used by my code, they have made that possible. There is nothing new, we rely on libraries and operation systems, but I think that complexity of projects should be redefined somehow.

I started thinking about some simple but meaningful measurement that may express relationships between those different LOC.

So, our projects consist from:

- MTLOC - Manually typed LOC; - that is what developers actually type in redactors and IDEs.

- GLOC - Generated LOC; that is some kind of“plumbing” code that various code generators and helpers create to support certain technologies: CORBA, WS, EJB, persistence, etc.

- ULOC - Used LOC; - number of code lines in all used libraries. I would differentiate among them

o Trusted code: TULOC ? stuff like JDK or OS, some widely used and well known libraries (Oracle JDBC for instance)

o Risky code: RULOC ? some stuff from a small vendor, first time used framework etc.

Note: What is TULOC and what is RULOC is a completely personal/team decision which might not have apparent reasons.

I assume good code quality and smart developers here, that means some direct correlations between LOC, project complexity and man-months.

There is a question:

- Which project is more complex: one that uses 1000 MTLOC or another that has 100 MTCLOC and 10000 RULOC?

One might argue that second project is a clear winner because it is easy even to recreate 100 lines from scratch if there will be available TULOC replacement. Not so simple in the “real” world, especially corporate one.

- There no need to rewrite at all since there is nothing to be gone;

- It is so damn hard to move anything; it requires endless meetings and tons of documents.

<q><origin> http://www.robelle.com/library/smugbook/paperwrk.html</origin>

The late Dr.Richard Feynman of Cal Tech did a famous study of the Challenger shuttle disaster. He ranged widely throughout NASA and its contractors, talking to anyone who could shed light on the quality problems. A group of production workers had found a simple way to improve the calibration of the rocket engines, but it was never implemented.

The foreman said he wrote a memo with this suggestion to his superiors two years ago, but nothing had happened yet. When he asked why, he was told the suggestion was too expensive. "Too expensive to paint four little lines?" I said in disbelief. They all laughed, "It's not the paint; it's the paperwork. They would have to revise all the manuals.

Quoted from "Personal Observations on the Reliability of the Shuttle" in What Do You Care What Other People Think? by Richard P. Feynman, (W. W. Norton, 1988). This book is enthusiastically recommended, as is his earlier collection of stories, Surely You're Joking, Mr. Feynman!.

</q>

Personally I do not know answer and I think that every approach might be perfectly justifiable in a particular case.

I was thinking about some kind of Individuality coefficient that might be calculated as MTLOC/ULOC. Primitive, isn't it? Yes, and does not make much sense, although might be used to create nice charts. I value Agile approach to software development, so I said stop inventing that foolish coefficients and I drew a pyramid.

The pyramid gave food for further thoughts.

1st - I would say that every decent user application needs all those layers. Do not take me literally, I do not mean that every application should be based on some kind of virtual machine, no, I mean that we usually create some layers of abstraction from underlying OS. Such repetitive job was summarized in POSIX standard and in various VM.

Lets look back in time. Layers #1 and #2 were very volatile, and others were nonexistent or not well established. Result: all projects had to build all those layers virtually from scratch. So, every project naturally required many man-hours to complete. Fixed deadlines and rush caused high demand for programmers.

Over time IT industry established (more or less) requirements and granularity of APIs at every level from #1 to #8. As soon as requirements were established, implementations arrived on the market.

That busted productivity and produced waves of geek's complaints: That limits our abilities! That is a stupid implementation! Etc...

Some complaints make sense, but many of them come from the sad realization: I am expert, but there is there is no need for my expertise.

Lets try to find some parallels in other industries. We may look at any industry but automotive one would be the best example. Lets try to draw?. a pyramid.

Even it might be hard to come up with exact comparisons between layers of those pyramid, we may notice that they are (strangely ?) similar. If we look back in the history of automotive industry, or industry in general, then we may recall time when robotic assembly lines were introduced on factories. What has happened with workers?

Well, the same that is happening right now with programmers. Before this time programmers can migrate to upper layers because market was expanding and there was a demand for bunch of applications on layers #6-#8. Now the market pretty much has reached its limits and a natural (sometimes not) competition established leaders are and we realized that we do not need different 100 RDBMS, 200 text editors, etc.

We need a variety, but on a smaller scale, because by natural reasons there cannot be one version of a product that fits to everybody (even MS delivers different versions).

Taking the car example: even all cars suddenly will have the same price, I doubt that all people will choose exactly the same model.

Lets take market/nature seriously: we are not rushing to throw away our well working TVs just because there is a newest model with two more options available in onscreen menu.

Lets face it: Increasing productivity kills jobs. Sad but true, because it affects individuals, human beings, which have families to feed.

Real consultants (not temporary code-monkeys with consultant's label) cause layoffs, because they bring technologies/products that increase productivity.

If a company buys an excavator then there is no more a need for 100 diggers. Only couple of them might be trained as excavator operators and about 10 will be left to carry light/precision digging. So, roughly 88 workers have to be fired?

There are economical theories which express the idea, that there are enough jobs for everybody anyway. I tend to believe in that, but in every single case that is somebody's tragedy.

That pyramid gives me another idea: similar trends we may help to predict future.

Prediction #1: very powerful programming language(s) will be necessary to that small number of people, who will stay/survive on the IT market.

Why limitations of a framework or a language are so important nowadays? Why do they boost productivity?

I think mostly because they prohibit inexperienced people from doing potentially dangerous things. What will happen when there are only highly trained and experienced developers/architects? Limitations become obstacles and they decrease productivity. So, professionals will demand power to boost productivity (aspect oriented programming and adaptive frameworks will shine).

Prediction #2: before IT market come to its “stable” stage, there will be time when more programmers will be necessary again. Reason: there is way too much poorly written code, which soon will reach limits of its supportability. I mean that every system comes to the point when it is easier to rewrite it from scratch that to maintain and extend. Many systems will be simply abandoned in favor of market leaders, for instance home grown accounting systems will be replaced by SAP class applications.

Prediction #3: Most IT specialists will be temporary consultants and companies will hire them to do the job. (Continuing our car example: companies use service centers to take care of their cars rather then have own garages and technicians in staff).

By no means I am trying to say that everything is done and we have best possible APIs, tools and applications. No, but I guess that IT revolution is done and it is time for evolution.

What is my point? Just thoughts.