ChAD Talk: Increasing Software Development Productivity

A Productivity Crisis

There are a couple of stories to be found in the productivity statistics from the U.S. Bureau of Labor Statistics (BLS).

[Source: Indexes of Output per Hour, All Published Industries from the BLS Industry Labor Productivity And Related Data Tables - NAICS]

The first thing to notice is that the 90s were extraordinarily good for software productivity. Between 1988 and 1998, productivity increased more than five-fold. In the same time, productivity in the retail sector increased by about a quarter.

Second, there is the sudden decline starting in 1999. Both income growth and employment growth depend on productivity growth. The decline in productivity (which started before the boom ended) is the underlying cause of the catastrophic job market and falling incomes in the software industry.

Productivity in the general economy has grown relatively slowly but steadily all through the same period. This can be seen for example in the retail industry.

In essence, Mary Poppendieck said that the multiple booms of the 1990s, from ERP to Y2K to the Internet, carried the software industry. We did not need to actively improve the way we make software in order to see our productivity rise. In a way, the booms provided us with a free ride.

Now the booms are over and we can no longer be so passive about our own productivity. The question arises: is there anything we can learn from more traditional sectors of the economy?

(Mary Poppendieck referred to an article called What high tech can learn from slow-growth industries, unfortunately only available to subscribers.)

How to Measure Productivity?

Counting lines of code produced (or function points or any other measure of "stuff") is useless. A narrow focus on the product leaves out the most important issue: the value it may have to the people paying for it.

Mary Poppendieck proposed two possible measures.

For software producers: revenue / employee

For support organizations:

(growth in revenue of supported business) / (support budget)

How Do "Slow-Growth" Industries Do It?

• They focus on the core business processes that drive productivity. (Of course that pre-supposes that they understand which ones are core.)
• They decide where they want to match "best practices" and where to lead. They only pick a few to lead on.
• They seek to make improvements from the start of the process to the end, across the whole value chain.

How Can We Do Better in Software?

Poppendieck quoted a Standish Group study on custom software. 45% of software features are never used, 13% are used often and only 7% always.

There is a chronic tendency in the industry to produce too many features! This is a symptom of a dysfunctional process, where customers ask for more than they really want. Old fashioned life cycles, with long lag times and "penalties" for changes in specification during a project, can exacerbate the problem.

The number one way to improve productivity is to not work on features that will not be used. Prioritization of features and functionality is hard but it's the most important part of the software development process.

Other techniques are the ones in the Poppendiecks' book. Value stream mapping; rapid flow of small batches; fast delivery; "pull" scheduling and so on. These are taken from their background in manufacturing and product development. Rapid feedback and rapid response to that feedback are key.

Mary Poppendieck also recommended another book, Product Development for the Lean Enterprise, as a source of ideas.

She did bring up one more point that I would like to highlight. Discipline is essential. Agile approaches do not work if they are an excuse for just hacking away. The important processes, however, tend to be the more low level ones: every project needs solid configuration management, build and testing processes.

It All Depends on Good Management

I noticed this particularly when she talked about Kaizen events.

In a Kaizen event, a group of people from different disciplines is gathered together with a facilitator and given three to five days to address a specific problem. During this time, they brainstorm possible solutions and come up with a list of recommendations. At the end, a senior manager leads a town hall event and makes a decision on the recommendation. If recommendations are approved, the people in the Kaizen have a mandate to implement it immediately.

Anything like this absolutely requires management initiative if it is going to happen. I guess that those of us who are not senior managers ourselves need to look at our organizations and lobby for these ideas. Ultimately, we need to assess the health of our organizations and leave if the culture is not conducive to better productivity.

Final Exhortation

The last bullet point on the last slide was a nice little slogan.

80-20 everything

I interpret this to mean something like: keep looking for chances to apply the 80-20 rule.

[ I've edited this piece since posting it. I might edit it again. Every time I read it I see things that could be improved. ]