Our customers generally don’t care about all of the fancy tools and techniques we have at our disposal to write applications – they just want systems that work. Just as manufacturers have learned, design and manufacture of a product aren’t enough, the circle is completed with a strong quality assurance program. In this session, we’ll discuss a suite of techniques to help you improve the delivered quality of your applications.
The production of something is pretty similar regardless of what it is – bicycles, potato chips, newsletters, or software. The same steps need to be followed.
The implementation details may vary. But the same ideas apply equally to custom database applications as to aluminum castings. Define what it is you’re going to build. Design it. Put together blueprints. Build it. Test it. Ship it.
People think software is different because you can’t necessarily hold it in your hand, and the ethereal nature lends it to misunderstandings.
But eventually you have to write code and create data structures, which means you have to make decisions about how something it going to look and work.
Robert Pirsig described how the search for the definition of ‘quality’ drove a man insane in his book, Zen and the Art of Motorcycle Maintenance. That man could have spared himself the trouble had he heard this simple definition of quality: fitness for use.
A ‘thing’ has quality if it is fit for the use for which it is intended. A thing does not have quality in and of itself; it must be related to a purpose.
For example, you can’t look at a bolt and determine if it’s a quality item. You must know what it’s to be used for first. If it’s going to be used to hold your clothesline up, the finish can be worn, the threading can be inexact, and the tolerances can be rather loose. On the other hand, if that bolt is going to be one of four that holds a jet engine onto an airplane wing, it must meet significantly more rigorous standards.
In the last 20 years, a great deal has been written about quality, and one of the fundamental principles is the difference between Quality Control and Quality Assurance.
Shops used to have a department located right outside the shipping department labeled Quality Control. In this afterthought of a department, the members would sample items about to be shipped in order to assure management that the final product was meeting some arbitrary level of ‘quality.’
Perhaps the department would use some sort of statistical process to determine which items to test; perhaps they would even analyze the results scientifically. But the fundamental assumption was that they were checking things after the face – at the end of the process.
Quality Assurance differs from QC in that they’re involved throughout the manufacturing process – not simply at the end. Instead of simply sampling the end product and sending those that don’t pass inspection back for rework, QA relies on monitoring the results of each step of the process, and then adjusting the process in order to eliminate defects produced along the way.
The adage „What gets measured gets done“ can be applied to Quality Assurance as well. That which is documented gets tested. You can’t test something for which the definition lies in someone’s head. As a result, the rest of this discussion relies on the assumption that there is a written specification that describes what the system should do.
This written specification needs to included, at a minimum, database specifications, descriptions of unit functionality and use cases.
The prime deliverable for most systems is the functionality it provides. This can be described through the use of „use cases“ – descriptions of how the system will be used by the user. You can liken these to the end-user’s „How To“ manual.
The frist step to create use cases is to list them. We do this first by listing what the system will be able to do, in an overview format. A helpful mechanism is to picture the back of the box of a piece of manufactured software – they all have a set of 8 or 12 or 20 bullet points that describe what the package does. You can do the same for your custom application.
These bullet points can each be exploded into more detail. First, the basic scenarios should be described for each bullet point. The functionality described in a bullet point could be „Create Quotes in your Customer’s Office.“ However, there may be multiple Use Cases associated with this functionality. For example, this bullet point could have a half a dozen Use Cases:
There can be a huge number of flavors any one of these Use Cases. For example, the „Create a Quote“ Use Case could have the following permutations:
Similarly, the Output a Quote Use Case could have the following permutations simply as far as location goes:
The second step is to document how the Use Case works. The important thing to remember is that this is not simply a description of using the computer, but to describe the overall process in which your application is playing a part.
For example, if the user has to manually perform a process, such as check off a box on a form, or mail a package, or put the top on a container, or call for verification, this step needs to be in the Use Case as well.
The end result should be a verifiable set of steps that a tester can follow in order to validate the proper functioning of the system. This may span multiple screens, processes, reports, external functions, and may cover a process that takes multiple days or longer.
Code reviews are a lot like the weather - everyone talks about them but no one does anything useful after the discussion. Most references on software quality will tell you that code reviews are one of the more valuable practices that you can implement. Then these tomes spend pages and pages quoting statistics to prove their point. Once you’ve been sold on the idea, though, what’s the next step? How do you implement code reviews?
A second problem arises once you’ve determined what your code review is going to look like, and how you’re going to incorporate it into your day to day work. The best of intentions may win you plaudits in some places, but the hustle and bustle of customers hollering for their work somehow forces developers to cut corners and shorten delivery cycles. Since you can ship modules to customers without doing code reviews, these are often one of the first things to go.
Finally, since a code review is a process without a specific deliverable (to a customer), it often becomes a collaborative effort - without a leader, or an owner. And as Plato said, that which is owned by everyone is taken care of by no one. Again, another reason that they won’t get done.
Here’s how we have defined and implemented code reviews in our shop.
A code review can take one of two forms. The first is a one-on-one review, where a developer’s code is reviewed by someone of equal or superior ability - preferably the latter. The second is a group review, where one developer shows their code to a group of several developers who then critique it in public. This group review is referred to in our shop as „Defending Your Life.“
(By the way, while I’m referring to „code“ reviews - this doesn’t mean that I’m only referring to long listings of lines of commands. „Code“ in this context means all of the work that makes up a system - projects, programs, forms, reports, libraries, classes, and so on.)
The purpose of a code review is not, as you might first assume, to find bugs. That’s what QA does. A code review performs three other functions. The first is to provide a mechanism for sharing techniques between developers. As a reviewer sees the actual work of another, they are bound to pick up ideas, tips, tricks, and so on. You can read all the golf magazines you want, and play all the rounds you can stand, but until you also see others play, you’re not going to reach your full potential. Of course, it’s helpful to see others who are better than you.
The second function is to encourage proper coding practices. It’s easy to, er, well, you know, get lazy. You know you should explain that magic formula, but you just couldn’t. You did comment the code when you first wrote it, but then you kept finding bugs, and it took you all morning to get it right. By the time you were finished, you were just too tired to change the comments again.
Knowing, however, that this code might be the subject of the Defending Your Life meeting on Friday afternoon often provides an incentive to finish up the last 5% of the code - the comments, the indenting, the deleting of old versions of the program, and so on.
The third function is an offshoot of the second - it provides a mechanism that encourages developers to write maintainable code. Simply following the „rules“ doesn’t ensure maintainable code. It’s easy to follow the letter of the law but break the spirit. One typical coding guideline is that a procedure or function shouldn’t ever be longer than a single page - 25 to 50 lines long. It’s easy to find a 20 line routine that is virtually impossible to understand, much less modify, while it would be equally easy to put together a function of 100 lines that is crystal clear and trivial to maintain. The 20 line routine keeps to the letter of the law but breaks the spirit, while the 100 line routine does the reverse.
By showing this function in a code review, the developer hears from others how good or bad their code is. If the rest of the room hoots and hollers, or starts cutting side deals about what they’ll do in order to avoid getting stuck with this code, the developer knows they’ve written a maintenance nightmare.
This also forces the developer to be accountable to the group. If you see someone else’s routine and say that you don’t know how it works, it’s more likely that the routine will get cleaned up.
Have you ever written a piece of code that was perfect? I mean, one that was longer than four or five lines. Of course not - none of us have. There’s always room for improvement - better comments, improved performance, more robust error trapping, whatever. However, when do you stop and say „Good enough?“
There is a point at which you will receive diminishing returns. Yes, you could implement every single idea that has ever been published, and turn a $20,000 application into a $200,000 application while improving the quality of the application by 4%. This does not sound like a good rate of return to me.
We have found that most of our bugs come from a few specific areas (see the chapter on Bugs for an explanation of how we know that), and that paying attention to those areas can generate an excellent return for a minimal amount of effort. As a result, instead of trying to investigate every line of code, we spend most of our time on those problem areas.
These areas are covered in a checklist that we’ve developed over time, based on our past history of creating bugs. However, it’s not simply a matter of one developer reading down a checklist while the other one nervously awaits the final judgement, sweating profusely at each „Hmmm!“ and „What the hell is this?“