Working hard in and of itself is not a virtue. It's the result that counts, not the specific quantity of work needed to achieve it. Working smart is often a better tool to achieve a good result in software development. Unfortunately, working hard is more likely to be rewarded at most companies because it's so visible. Working smart can be invisible.
To see victory only when it is within the ken of the common herd is not the acme of excellence. Nor is it the acme of excellence if you fight and conquer and the whole empire says, “Well done!” True excellence is to plan secretly, to move surreptitiously, to foil the enemy’s intentions and balk his schemes, so that at last the day may be won without shedding a drop of blood. To lift an autumn hair is no sign of great strength; to see sun and moon is no sign of sharp sight; to hear the noise of thunder is no sign of a quick ear.
—Sun Tzu, The Art of War
Software Development
Software projects (including all video games) are notoriously over-budget and over-schedule. Why is this? It has much to do with the highly counterintuitive nature of software project management, beautifully described by Steve McConnell in his book Software Project Survival Guide and in this article of his.
When project managers get involved in a project, they tend to impose a lot of process. They want tracking reports on this, detailed planning on that, reporting procedures on the other. It’s only “common sense” that all the time spent on extraneous bookkeeping like that takes directly away from the productive time that could have been spent implementing the actual software. Workers do admit that there’s a small amount of “thrashing,” or unproductive work, that’s basically unavoidable. So a graph of how much productive work gets done during the course of a project looks like this:
The mistaken notion that process is pure overhead and that everything will just work out in the end.
This is, of course, a fantasy. In reality, software development is an enormously complex undertaking that requires an enormous amount of coordination, cooperation, and communication. Defects that appear early in the process (and they DO appear) magnify in scale like a growing cancer as the project continues. When things can go wrong, things do go wrong, and they go wrong quite often in software development. It’s not the case that smartest and most able developers commit no errors. Even if that were true, errors and problems are introduced through coordination and communication problems, and even through outside factors beyond everyone’s control—not just through flawed code. So the real question is how does one go about minimizing, detecting, and correcting errors once one has admitted the truth that the existence of such problems is an unavoidable fact?
I can’t say it any better than Steve McConnell:
When a project has paid too little early attention to the processes it will use, by the end of a project developers feel that they are spending all of their time sitting in meetings and correcting defects and little or no time extending the software. They know the project is thrashing. When developers see they are not meeting their deadlines, their survival impulses kick in and they retreat to “solo development mode,” focusing exclusively on their personal deadlines. They withdraw from interactions with managers, customers, testers, technical writers, and the rest of the development team, and project coordination unravels.
Far from a steady level of productive work suggested [in the diagram above], the medium-size project conducted without much attention to development processes typically experiences the pattern shown [in the graphic below.]
In reality, the project becomes more complicated, and despite the emergency processes instituted, thrashing overtakes all.
In this pattern, projects experience a steady increase in thrashing over the course of a project. By the middle of the project, the team realizes that it is spending a lot of time thrashing and that some attention to process would be beneficial. But by then much of the damage has been done. The project team tries to increase the effectiveness of its process, but its efforts hold the level of thrashing steady, at best. In some cases, the late attempt to improve the project’s processes actually makes the thrashing worse.
The lucky projects release their software while they are still eking out a small amount of productive work. The unlucky projects can’t complete their software before reaching a point at which 100 percent of their time is spent on process and thrashing. After spending several weeks or months in this condition, such a project is typically canceled when management or the customer realizes that the project is no longer moving forward. If you think that attention to process is needless overhead, consider that the overhead of a canceled project is 100 percent.
Processes such as change control, QA, defect tracking, automated source code control, and so on are the methods of winning the battles against Murphy’s Law before the full-scale war is allowed to begin.
Even small investments in process can go along way. If you have a team of 20 people, having all 20 of them spend over an hour a day on tracking their progress is probably not a good use of time, but having them spend 10 minutes in a stand-up meeting (where you literally stand up) and have each person rattle off the work they plan to do today (and possibly what they did yesterday), keeping it under 30 seconds per person keeps everyone aware of dependencies. It might make you realize that of your three possible tasks, people are actually waiting on one of them, so you should work on that first. This type of meeting is just one example of investing a small amount of time in process to avoid bigger problems later.
Projects that attack problems before they become problems win the war with sheathed sword. Process becomes fine-tuned and second-nature, while thrashing is reduced.
Sun Tzu Says Nip It In The Bud
Sun Tzu’s said that the generals who win our praise are those who, through great effort, win large, dramatic battles. The far greater generals who should win our praise, though, are those who avert war altogether. They defeat the seeds of war long before the masses realize that there was any danger at all. This is especially true in the case of software design, since errors that appear early in the process become more and more costly and difficult to fix as the project progresses. Imagine a sentence in a design document describing how a particular piece will be implemented. How easy to change that sentence if it is in error and avert disaster, yet how hard to re-implement the entire module that later grows out of that sentence.
The Anchor
I worked at a company that was, like most software companies, entirely set up to praise hard work and not at all set up to praise kind of general Sun Tzu was talking about. In this company, two teams worked loosely in parallel from the same starting code base to create two different, vaguely related products, with roughly the same development cycle.
The actual quality of this software as far as end user is concerned—the game design—is totally irrelevant to this discussion. I know I like to talk a lot about game design, but let’s focus on software design for the moment. The lead programmer on my team was a careful, pensive, steady man. To him, the beginning of the project was a time for cleaning up the code base and reorganizing it to save time later. He was interested in creating a number of tools and processes designed to uncover code errors early in the upcoming development cycle.
I called this guy “The Anchor.” I felt I could sleep at night without worrying that the project would float off into an iceberg in the middle of the night, or something. His steady work made the final stages of the project pretty much by the book. The game made it through Sony’s rigorous approval process on the first pass.
Meanwhile, the sister project encountered a number of mysterious bugs towards the end of the project cycle. This was not surprising since their process was not as rigorous as that taken by my team’s lead. Tracking down some of those bugs was a monumental effort. Several of the programmers became company heroes by staying at work all night, working straight through with no sleep on countless occasions. Every time their project reached the final candidate stage, some new bug would be discovered, another monumental effort to fix it would take place, another final candidate would be created, then yet another bug would be found. Only through hard work totally unheard of in most industries and totally counterproductive to the well being of human workers was that project finally finished. (Let’s just say they didn’t get Sony’s approval the first time through.)
The management of the company couldn’t say enough good things about those guys who were working so hard. They were "examples we could all learn from!" The CEO gave out awards to some of those programmers on the other team who had worked far above the call of duty. The Anchor did not receive one.
Frankly, I found that offensive. First of all, those programmers who sacrificed their life and their health to pull of some kind of 11th hour Herculean win were not working beyond the call of duty. The company expected that from them. Worse yet, the far greater general—the general who had defeated armies of bugs before management or anyone else even knew a war was coming—was not given an award that day. His contribution was not visible even though it was far greater than the other team’s. When the hero wins the war before the masses know there was to even be a war, no one notices there even was a hero.
In closing, Sun Tzu in his Art of War had this to say about the tragic, hero-general:
…his victories bring him neither reputation for wisdom, nor credit for courage. For inasmuch as they are gained over circumstances that have not come to light, the world at large knows nothing of them, and he therefore wins no reputation for wisdom; and inasmuch as the hostile state submits before there has been any bloodshed, he receives no credit for courage.