February 2004 Archives

ChAD Talk: Increasing Software Development Productivity


On Friday I was able to attend a talk presented by Mary Poppendieck, author of Lean Development. Here are some of my impressions.

The Search for Better Software Engineering

| No Comments

Charles Miller's quick links recently included a reference to a JDJ article that itself arises from the controversy surrounding an interview on the challenges and future of programming. For some reason the fuss is all about the unflattering comments about XML towards the end. This is more than a shame.

What got lost? The interviewee, Victoria Livschitz, talked about how deep and pervasive the flaws are in what we call software engineering.

The Livschitz interview was provoked by an earlier interview where the question was raised:

Why is it so difficult, if not impossible, to write bug-free programs that contain more than 20 to 30 million lines of code?

I think Victoria Livschitz was right to expand the scope of the question to something more of us can relate to.

And here's what's really sad — the overwhelming majority of so-called "successful" development projects produce mediocre software. Take almost any corporate accounting application, and you'll find it poor in quality, unimpressive in capabilities, difficult to extend, misaligned with other enterprise systems, technologically obsolete by the time of release, and functionally identical to dozens of other accounting systems. Hundreds of thousands of dollars are spent on development, and millions afterwards on maintenance — and for what? From an engineering standpoint, zero innovation and zero incremental value have been produced.

Not only that, but she briefly mentioned some good ideas on what could be done to make things better.

But expanding the pure object-oriented paradigm to allow for a richer set of basic abstractions — like processes and conditions — is only half of the arsenal in the war on complexity. The other half is a powerful aggregation/decomposition model that is rather weak, convoluted, and fragmented in modern programming. In order to deal with complexity, the organization of the software elements is of utmost importance.

In general, our tools (and by that I mainly mean the languages and the libraries) operate at too low a level. It's as if you had to re-design bricks every time you wanted to build a house. We're spending too much time working with representations that are not rich enough to express what we need to say at the domain level. It doesn't even depend very much on the domain: as she says, anything that goes beyond a simple is-a or has-a relationship has to be shoe-horned into something utterly primitive.

It's hardly news to anyone that Java is not a silver bullet, but she is right to draw attention to the failure of the industry to move much beyond it.

The complacency around C/C++ and the Java language is pervasive. C#, the first programming language in years, looks more like the Java language. Enormous productivity gains remain to be uncovered and difficult problems are yet to be solved.

It's good to remember that Java and C# are not the only choices. Indeed, I'd almost certainly want to supplement either with other tools to be as productive as possible. Interesting in this connection is an interview with Dave "pragmatic" Thomas on code generation. Like similar approaches, it can be a crutch, but when you need it, it's a damn useful crutch!

CGN: What do [you] think the future is for code generation?

Dave: I think that in the long term the larger code generation efforts, the "application generators," will become a thing of the past. They are there because the underlying technologies and architectures don't yet support programming at a high level. But I'm betting that languages such as Java and C++ will in the long term be seen as a curious branch in the evolution of computing. I'm hoping that somewhere out there some bright spark is coming up with a way of letting us write applications expressively and dynamically. Once this happens, the need for these kinds of code generators will diminish.

Godspeed to that bright spark. I hope I'll have the wisdom to recognize the new way when I see it.

Another Take on the Outsourcing Debate

| 1 Comment

Outsourcing is one of their regular themes at BonoboLand. This recent piece is worth a read, not least for the comments thread that follows it.

Angry Bear on Outsourcing

| No Comments

Kash over at the Angry Bear blog has written an analysis of outsourcing white collar job losses. The most important conclusion is the first one:

Is outsourcing the cause of the poor labor market in white-collar sectors of the economy?
Probably not.

Kash has also written a proposal to deal with lost jobs in manufacturing, a more pernicious and longstanding problem.

Both Kash and AB have been taking arrows in their backs over free trade and outsourcing for some time now.

What Future for Sofware Engineers in the U.S.?

| No Comments

The comments threads rage over at Brad DeLong's site. Again, it looks like many are blaming outsourcing for the dreadful state of the U.S. job market.

Wired magazine has a cover article on outsourcing in India. It has to be taken with a big dose of salt because Wired has a history of jumping uncritically on every bandwagon. Techno-optimism is so 1999. Nevertheless, this article seems to be one of the few that dares to imagine how outsourcing might not be so terrible after all.

I think the Wired article is right in one respect: outsourcing is not going to go away, despite what politicians may may imply. In the long run, orthodox economic theory says we'll all be better off for it.

But what about the short run?

Free Software and Productivity

| No Comments

It used to be an obstacle course: I'd have to get a semi-decent C compiler, use that to compile gcc, use that to compile a decent make, sed, sometimes even a linker or an an assembler. Then there would be all the library dependencies.

Nowadays, I take it for granted that all the Gnu tools are there. Perl is there. Python is probably there. Either they came with the OS or they were installed early on.

I noticed this today. I thought it would be nice to use Perl's DBI to do some data mangling/migration.

perl -MCPAN -e shell
cpan> install DBI::DBD

[ bla, bla, bla ]
  /usr/bin/make install  -- OK

cpan> quit
# make sure all the environment variables are set
perl -MCPAN -e shell
cpan> get DBD::Oracle
# cd to the right directory, edit a couple of files and...
perl Makefile.PL
# no, I'm not going to write what I typed here
make test
# some of the long tests fail, but I don't care
make install

After that, it just works! DBI, as easy as pie.

An Important Oracle Limitation

| No Comments

I had completely forgotten about this. Somebody complained that a web page was crashing when they entered a large amount of text into one of the boxes. Investigation turned up this error:

 ORA-01704: string literal too long