Software productivity is incredibly important, but hard to measure and hard to achieve. Nonetheless, good people seek ways to make their software productivity better. They seek out productivity multipliers.

All too often, widely hailed advances in method and technique not only fail as productivity multipliers, they are actually productivity divisors!

There is a whole family of software productivity divisors whose obvious impact is to double (or more!) the labor for any given body of code. They don't double the pleasure or double the fun — they double the work.

And yet, these methods are taken seriously by all too many people in the field.

What are software productivity divisors?

The field of software is awash with a plethora of tools and techniques, each of which claims amazing benefits, usually boiling down to producing better code with less effort. They are, in effect, promoted as productivity multipliers.

Some of the techniques are simply useless. But a surprisingly large number of supposed software improvements actively make things worse. They aren't productivity multipliers — they are productivity divisors.

Doubling the Work

One large class of fancy new software methods essentially involve doubling the work. There are two main methods for doing this.

One favorite method of work doubling is very simple. Instead of using one programmer to do a job, use two.

Another widespread method of work doubling is to write the code twice, in different forms. Usually, one form of the code is the code that you actually want to write. The second form of the code is normally a mirror of the original code, whose nominal purpose is to test the original code.

Each of these methods is completely brain-dead. They normally double the labor of everything that is done, while not improving the value of the results at all. Costs more … later completion … no greater value … Hmmmmmmmm … what's wrong with this picture??

Pair Programming, the Hot New Productivity Divisor

Programming is, by definition, a task that requires great mental concentration. Kind of the opposite of what you do in social situations. Two people working on the same task relate to each other socially. In fact they are supposed to interact. If one person codes and the other one sits there, you've doubled the labor with no gain at all. To the extent that the second person interrupts (as he/she is supposed to), you've made things even worse.

I know, I know, the whole idea is that the whole is greater than the sum of the parts. The second person catches mistakes the first person missed. The second person comes up with a much better way of doing something than the first person got part way done doing. Etc. It's all a bunch of Bologna. I'm relieved that silly parodies of this silly idea are beginning to appear.  

Unit Testing, the Classic Productivity Divisor

There are many variations of unit testing for code, from test-driven development on. People get into arguments about whether you should write the test before or after you write the code, whether the person who writes the code should also write the test, and endlessly on and on. They all amount to arguing which is worse: dirty stinking rotten, or slimey awful junk. Who cares?? It's all bad!!!

No one claims it's easy to write good code. Code that's accurate, fast, does the job, does only the job, has no side-effects and doesn't crash. Still, there may be bugs. The idea with test code is that someone, in some order, writes test code that assures that the original code is accurate, fast, does the job, does only the job, has no side effects and doesn't crash. If you leave out any one of these items (as is typically the case), you haven't tested everything.

The person who writes the test code has to understand the requirements just as thoroughly as the person who writes the base code. Just as the person who writes the base code can make a mistake, so can the person who writes the test. It is just as hard to write the test as it is to write the code being tested. So you've doubled the amount of work, at minimum.

Now think about making changes and debugging. Instead of just changing the code, you've got to change the test code, and either one can have bugs. The test code can fail to test something that is supposed to work, or declared that something works when in fact it does not, as a result of a bug in the test code. So changes are now twice as much work, with twice as many places for bugs, in addition to bugs of interaction.

Any variation of unit testing has the certain result of at least doubling the cost of building a piece of code, usually increasing the elapsed time, with vague promises that are never proven and never measured about the improved value of the results.

Conclusion

In many organizations, there is a dead-simple way to improve software productivity: STOP the madness of institutionalized software productivity divisors! You don't have to be best-in-class; you just have to avoid doing things that for sure double the work while doing some enthusiastic but empty arm-waving about the future benefits.

People
in every field are concerned about productivity. They want to know how
to increase their own productivity and the productivity of their group.

Software
is a peculiar field in which history is ignored and fashion too often
trumps objective fact. In software, we don't even think about productivity, much less have any idea how to measure it.

What is Productivity?

Productivity measurement is central to our economy as a whole, to industries as a whole, and to individual companies.

In general, the more productive you are, the more outputs you generate for a given set of inputs.

Productivity can be measured by various means, depending on the purpose of the measurement. At the level of the firm, the most basic measurement is comparing the cost of the inputs to the value of the outputs, adding in overhead and the cost of the capital and labor required to produce the outputs.

Increased productivity can be accomplished by greater automation, improved processes, reduced cost of inputs, labor or capital, and by more effective use of labor, labor that is higher quality, more discliplined, trained and motivated, etc.

This is basic economics. It is applied from the national level down to mom-and-pop firms.

What is Productivity in Software?

Measuring
software productivity is incredibly important. But practically no one
actually does it! It's not even studied in any meaningful way — if it
were, you'd find competing theories, and experiments to settle the
issue. While there are a couple studies, their paucity and narrowness
prove the case that software productivity is largely ignored, both in
theory and in practice. GIven that software is increasingly the engine
that drives our enterprises, this is amazing.

The situation with software productivity is a bit different than in your basic widget factory.

• The cost of the inputs is pretty much zero
• For all the talk of software factories, the cost of "machines" is generally insignificant
• While the quantity of output (lines of code, LOC) is tangible, it's not really correlated with value
• Except in a few cases of commercial software products, the value of the output can be hard to measure, and even then it can be dicey.
• The overwhelmingly important factor in cost tends to be labor!
  

Software Productivity: the best case

To
understand software productivity, let's start with an extreme case, but
one that does happen a small percentage of the time.

A
single person understands the requirements of what is to be written.
That individual, perhaps with the assistance of one or two other people,
writes, tests and delivers the code. The code runs in production for a
number of years, and the few bugs that arise and the enhancements that
are needed are quickly supplied by the original author(s).

In a case like this, the value of the outputs can still be hard to measure. But at least the other costs are easy: it's basically the cost of the people, with overhead.

Is
this a fantasy or does it happen in real life? Well, all I can say is
that it happened in my life, more than once, and I know people who have done similar things. In my first job after
college, I wrote a commercial FORTRAN 66
compiler and run-time system (in assembler language) for a computer
company. I had an assistant for part of the run-time environment. It was
in production use for about ten years. I fixed a couple minor bugs in
the first year, and that was it. I personally did a couple similar
projects in different software fields.

In the case of the compiler, the cost was easy to measure. The value was typically challenging, but not too hard. The company sold the compiler. But its greater value was enabling sales of the display computer on which it ran. Potential buyers were refusing to buy because there was no high level language available at the time; my compiler enabled many sales that otherwise would have been lost.

It's
easy to imagine the software productivity in this simple case. You can
pick your measure of value (pick a few — why not?), including lines of
code, number of users, value of the code, indirect value. You pick your measures of cost
(again, pick a few) including person-days, elapsed time, salaries, etc. And finally, you relate them into simple, calculable numbers that you can track over time.

Software Productivity: the Reality

In practice, software productivity is an absolute bear to measure. Cost isn't too hard. But valuing the outputs? That's the big problem. The other problem is that software just isn't like a widget factory. The items ("pieces" of software) cranked out are simply not comparable to each other, as I've discussed.

Conclusion

Productivity is important in general, perhaps even more in software than other fields because it is so hard to measure. Software won't get better until we increase its productivity, and that won't happen until we are hard-nosed and objective about exactly what software productivity is. All the discussions in this blog about software quality, estimating and other things are best understood in the broader context of software productivity, the great unknown frontier of computer engineering.

Some of the biggest mistakes in software architecture do not involve making the wrong choice; they involve failing to consider an important choice altogether, by assuming (without consideration) the approach to take. A great example of this is whether software components should be "loosely coupled" or "tightly coupled."

What is Coupling?

"Coupling" is closely related to layering in software, which I've discussed elsewhere. Given that two bodies of software are going to "talk" with each other, how "tight" is the communication layer?

There is a spectrum of how tight the coupling is.

At one end of the spectrum is "tight" coupling, which is like when two people are physically together and talk with each other.
Just a little in from that end is like when two people are not physically together, but still communicate in real time, like a phone conversation.

At the far loose end of the spectrum is when the people are separated by space, time and perhaps other factors. A newspaper is an example of loose coupling in this way, particularly when you throw in letters to the editor. (Credit.)
The more you stretch the time and how personal the communications are, the looser it is. Notice that when you stretch time, normally there isn't just a communications medium involved, but also a storage medium of some kind, like the newspaper.

Tight Coupling in Software

When two computer modules are tightly coupled, they communicate with each other in real time. If they're in the "same room," the communication is simply via an interface of some kind. If they're not, they use some kind of API involving networking; RESTful API's are a typical current example of this. In any case, the communication is pretty much real time, like a phone call.

Loose Coupling in Software

Loose coupling is everywhere in software. Everyone experiences a common form of it with e-mail. You send your e-mail and it sits around until the recipient grabs it from the mail server and reads it. Queuing systems are a bit tighter, but still pretty loose. Databases and document repositories are also a form of loose coupling.

Which is the best Coupling: Tight or Loose?

You know the answer: it depends on the application. If you're standing at an ATM waiting for money, you demand tight coupling, and so does the bank that holds your money. But e-mail wouldn't be e-mail unless it was loosely coupled. The whole point is that the other guy doesn't need to be sitting around anxiously awaiting your e-mail.

If the application itself doesn't determine the answer, there are other factors that can tip the scale. Tightly coupled systems are generally harder to upgrade and test, because both sides need to be upgraded and tested together, in real time. So when you can get away with loose coupling, it makes the communicating components more independent of each other and makes life easier all around.

Loose and Tight Coupling in the Literature

Warning: the computer literature defines tight and loose coupling in different ways than I have here. I'm talking about a rather different concept, one that IMHO deserves more attention than it gets.

Conclusion

While you're thinking about layering, you would benefit from thinking about coupling at the same time. A clever, appropriate coupling strategy, using loose and tight to greatest advantage, can help unleash software efforts and make everything go faster.

There are a small number of truly fundamental concepts in computing. They are not generally taught or talked about, but they underlie most of the smart things you can do in computing.

The fundamental concepts are like "the fundamentals" in a sport, the very basic things you have to do, like dribbling in basketball or blocking in football. It's where the phrase "blocking and tackling" comes from. Woe to the team that puts all its energy into fancy stuff — it will be beaten by the team that does the fundamentals.

The fundamentals are generally recognized in sport because there are objective measures of scoring and determining which team won. Eventually, people figure out which activities contribute most to winning. But in computing, it's a sad story.

Competition would help us understand what are "computing fundamentals"

If we competed in programming the way we do in sports, teams from different places would take on the same job at the same time. Each would complete the job, roll it into production and run it for awhile. For each team and their product, we would collect a variety of information, including: the size and cost of the team; the elapsed time they spent building, the resources required to build and operate, the number of bugs, the level of user satisfaction, etc.

Who won? Well, we'd take some combination of the information above.

I bet if we did this a lot in programming, the "fundamentals" of programming would gradually become clear to everyone. Everyone would want to know what the winning team did to win, what winning teams had in common, and over time, the programming equivalent of "blocking and tackling" would become obvious.

But we never compete!

Oh, you think we do? Like when there are competing products, or when competing companies have similar computer systems?

Well, sure, at a business level there is competition, but at a programming level? Think about football. How would you feel if the team that won had twice as many players as the other team? What if the winning team got to use a different ball than the other team? What if the winning team was given ten downs per possession, and the other only had three? What if one team always had a goal post that was half as high and twice as wide as the other team? With differences like this, it's obvious that the game is rigged and there's not much to learn from the game. There is as little to learn from examining the programming practices of companies or products that win in business.

So what are the fundamentals of programming?

There is no generally accepted answer to this dead simple but incredibly important question! And given the lack of meaningful competition, there is no objective way to prove what they are!

I've spent way too much time:

• programming, and
• trying to get better at it, and
• wracking my brain to determine exactly what "getting better at programming" means, and
• trying to identify the key factors that lead to better results.

So I've got opinions. I've written about some of the fundamental concepts in some of my private papers, and I intend to post about some of them here.

For this post, it's sufficient to establish the basic concept: in fields that we care about, there are measures of goodness and a not-too-large collection of "fundamentals" that constitute the "blocking and tackling" for the field. And that, sadly, a broad understanding of those things are lacking in the field of computing.