flipcode - Five Ways I Improved My Coding Workflow

Five Ways I Improved My Coding Workflow
by (10 October 2002)

Introduction

This article describes five straightforward practices I use when working on coding projects. They are useful to me and I've been following them for several years on personal projects, school, and professional projects. These are ways I've improved my workflow—they may not be useful to everyone. By improving workflow, I mean improving the objective quality and rate of my coding, and making the job subjectively "easier" for me.

A good general approach to improve your own workflow is to evaluate (by actually timing yourself) how long every task takes you, then look for methods to accelerate slow or especially tedious tasks. In this article, I'll talk about the following methods:

1. Keep a development journal
2. Schedule the task that will make the biggest difference next
3. Fix bugs before adding features
4. Retire broken or dead, but potentially useful code to a graveyard
5. Make defensive intra-day backups

1. I keep a separate development journal (a.k.a. log or notebook) for each project.

Although I do organize it, I don't feel any obligation to make it look pretty to others because I'm the only one who uses it. I use a physical notebook rather than an electronic log because it isn't subject to crashes, data corruption, or any of the other problems that can happen while debugging. It also is nice to be able to pick it up and work away from the computer. The black and white composition books work nicely—they have hard to remove pages, are sturdy, and the covers are slightly coffee-proof. Having hard to remove pages encourages me to not try and over-organize by rearranging material later or removing notes. Keeping the journal is a tool, not my full time job.

The journal has several purposes. I keep the very top of my to-do stack in the log as a series of checkboxes (every entry in the to-do stack is kept on the schedule as well). This helps me avoid forgetting small tasks that I discover while looking at a particular piece of code. For example, I might notice that two functions look remarkably similar and could be collapsed into a single templated function. I don't want to change them immediately because I'm still trying to get my previous change to compile, so I add unifying the two functions to my quick to-do list to make sure it happens.

Having the log book makes it easy to jump into work first thing in the morning. I flip it open and see the top of my to-do stack, notes on the current task, and my last thoughts from the previous day.

When a bug occurs, I describe the bug in the journal. When I fix it, I write down the cause and solution. This is very handy when something similar comes up (usually with bizarre compiler/linker errors) and I can't remember how I fixed it last time.

I do all of my scratch calculations in the journal for record keeping purposes. I can return to a derivation or calculation to verify correctness if it is later questioned.

I describe my progress on implementing features in text, sometimes augmented with code and screen shot printouts. This is a nice motivational tool because I can see how far I've come. It is an easy way to demonstrate a particular aspect of the program to someone else when asking them for help. Having code printouts and key algorithms in the log it is also a worst-case backup. If all of the critical, hard-won routines are in the log, then I can always re-type them if something catastrophic happens to my computer.

Finally, the development journal can be important for establishing the creation date for patent purposes. Employers like this property, but it gives me concern. The primary purpose of the journal is to accelerate my workflow. If it has the potential to be used in legal proceedings, that adds pressure to keep the journal excessively neat and makes it substantially less personal. I think the role of establishing an invention date should be left to the specification documents and revision control logs which are published in a company as public documents.

2. I work on the task that will make the biggest difference in the functionality of the program.

Ordering tasks in this manner maximizes perceived progress as well as actual functionality. The features that are cut near the end of a project are not core to functionality; by scheduling important features first I don't spend time working on tasks that might get cut anyway. Maximizing perceived progress has many immediate benefits. It is important for my own motivation and for helping motivate the rest of the team. It is good for demonstration purposes, which is an unspoken requirement of any project in any development environment. Finally, seeing the program in operation is a good way to understand how it can be improved and evaluate whether it satisfies the need for which it was created.

3. I work on any outstanding bugs before working on new features.

This is a corollary to working on the task that will make the biggest difference in functionality. When a bug occurs, some feature is broken. If I added the features in order of functional importance then a bug means the feature that is broken must be more important than any that are not yet implemented. So it is more important to go back and fix the previously written code than to add new code.

Fixing bugs is important to workflow because bugs tend to escalate over time and make it difficult to add new features. Newly discovered bugs are frequently misattributed to known bugs and ignored, so the total stability of the system decreases. Bugs impair proper testing of new features. The longer a bug is in the system, the harder it will be to fix because newly written code may have to be modified as part of the fix.

Sometimes I realize that the bug is a feature that wasn't as important as I originally thought, and shouldn't have been implemented yet. When this happens, I have two choices: delete the broken code or fix it. The delete option may sound like throwing away good work, but it is more attractive than it first appears. Once the code is out of the system it doesn't have to be maintained and can't be a source for new bugs. I won't have to spend time supporting it, and it won't constrain any new designs.

What if the code is probably 99% correct, even though it has a bug and I shouldn't have really worked on it yet? It would be nice to recoup some of the effort I put into the code by recycling it somehow. There is a solution that isn't as wasteful as throwing away that mostly good code: put it in a warehouse.

4. I keep a "warehouse" directory for old or broken code.

This directory is outside the revision control system and has little organization. I assume I can search (grep) through if I need something, and don't even bother to make code compile before I dump it in there.

Deleting a buggy or obsolete function from the active code base isn't throwing away work—I retire the code to the warehouse and can pull it back out and dust it off later if necessary. The warehouse is like the prop storage for a movie company, filled with bits of old things that might be useful again some day. It is also a good place to put snippets of code I found on the internet or in books that I might later want on hand.

I am highly motivated to keep code size small. An observation led me to this goal. I believe it are valid for many programming languages and across production code, research code, and class projects.

The cost of maintaining a piece of code dwarfs the initial cost of developing it.

Prime evidence for this is the decrease in production of new code as a project ages. In the first week of a project, a programmer might create a thousand lines of code. After a year on the project, that programmer is likely to produce only tens of new lines per week. The rest of the time is spent fixing bugs, altering interfaces for new features, commenting, changing constants, and propagating changes throughout the system: maintenance. Because bugs can't exist in code that isn't there and time spent maintaining code is time not spent writing new code, I keep code size small to maximize new code production and minimize maintenance. This doesn't mean that removing whitespace, comments, or writing dense and unreadable code will improve things, of course. Make code as simple and small as it can comfortably be, and don't try to make it simpler or smaller.

Removing dead or broken code to the warehouse is one only way of minimizing code size. Cutting nonessential features, striving for modularity and reuse, and sometimes choosing to minimize implementation complexity instead of maximizing performance are other techniques.

5. I frequently make backup copies of my source tree when implementing new code or debugging old code.

There may be tens of backups made in between check-ins to the revision control system.

As an example, I'll describe how I fix a bug. I begin by making a copy of the source tree. To analyze the bug, I'll use the usual techniques of setting breakpoints, commenting out code, and adding debug statements. Before I make a significant change during this process, I copy aside the source tree again. When copying, I preserve my previous backups. If something goes unexpectedly wrong (e.g. commenting out code unrelated to the bug causes the program to hang on startup) and it looks like it will take a long time to figure out, I simply throw away my current source tree and revert to a backup. When I've found the source of the bug and fixed it, I jump all of the way back to my first source tree copy and apply the fix there. If it runs successfully, I throw away my many debugging copies and commit the change to revision control.

Keeping the many source code copies serves the following purposes. It is frequently faster to re-implement a small change than debug what went wrong with it. This assumes I have high confidence that the error in my change was something stupid, yet hard to find, and not a fundamental flaw. When debugging, the analysis phase proceeds much faster if I'm free to tear up the code base. The corresponding bug fix can also be tested much more quickly in the analysis environment than the complete program. Patching the original code with the bug fix is faster than trying to restore the torn up code base. Specifically, finding all of the debugging statements and uncommenting the right code blocks can be time consuming and is prone to error. The backups serve another worst-case purpose as well. In the unlikely event that the revision control system fails (or for projects outside revision control), having old copies around can be very handy. I'm not terribly eager to reap my old source code trees as a result—like a garbage collecting memory manager I usually wait until I run out of disk quota before cleaning up.

I intentionally use the file system rather than revision control for this process. Most revision control systems incur too much overhead for frequent changes. Copying a source directory tree takes a few seconds and there is little that can go wrong with making, restoring, or removing the copy. Creating a separate revision control branch for every small change has exactly the opposite properties. I want to incent myself to make many defensive copies of the code, not introduce excessive process and slow down my workflow. The primary job of revision control is to track planned changes towards a final product. It operates at a coarser scale and is the right tool for a different job than these intra-day defensive backups.

Conclusion

The techniques I described in this article aren't new ideas, and the specific practices aren't as important as the underlying reasons I follow them. Just as most costs on a project can be translated in to dollars, they can also be translated into time. The time resource is particularly limited for projects with small groups of programmers, and especially for single programmer projects. Analyzing the time spent or saved by various practices involved with writing, debugging, testing, and documenting code is the first step towards making the best use of time, which is the way to maximize code production.

This article described five practical and simple ways I have found to increase my productivity as a programmer. I encourage developers and managers to consider programmer workflow as a changeable part of the development process.

Morgan McGuire is co-founder of game development company Blue Axion and a PhD student at Brown University in Computer Graphics.