Write bug reports that don’t suck

Originally published on Medium, on 2021-02-24.

This article will present a framework to write bug reports, so that they are easy to understand, and review.

This is primarily aimed at people who will report bugs, and who want to be loved by people who fix them.

TL;DR

Describe the bug in a way that is easy for you to review when it will be fixed.

The easiest way is to write a report with three parts:

1. Summary: one sentence describing in active terms a specific behavior.
2. Reproduction: a reproduction scenario with numbered steps, and a screenshot of the outcome.
3. Spec: a link to a source document owned (or vetted) by the Product team, indicating a contradictory output.

Example:

Summary: In the dashboard page, clicking the export button displays an error

Reproduction:

1. Log in with toto@hoy.com

2. At the top of the page, click Dashboard

3. On the top right, click the export button → a wild error appears [add screenshot]

Spec: Download session statistics (from the Knowledge Base)

(And yes, this is once again stolen from tech practices — this time Test-driven development.)

Summary

The summary is a short description of a behavior, in active terms. You may write more than one line to describe what’s happening, but the first sentence is the most important one.

Definition of a bug

A bug is defined as a reproducible product behavior which:

• creates a different output than the one described in the product spec.

OR

• corresponds to the spec, but whose rationale cannot be defended by a Product Manager.

Write what is, not what isn’t

Explain the consequences of an action in active terms. No one is interested in what it doesn’t do. We want to know what happens.

Bad:

Clicking on the button is not working.

Good:

Clicking on the button displays an error

or

Clicking on the button triggers an error in the browser console

Notice what’s going on beneath the second version: when faced with something that looked like an absence of action (“clicking does nothing, and there’s no apparent error’”), you had to rack your brains to find something it actively did (an error in the browser console). This is good, and will accelerate the resolution.

In the extremely rare case when really nothing happens, you may write:

Clicking on the button does absolutely nothing (that I could find), damn it.

The mindset of a bug

It’s important to consider a bug like a behavior, and not like a problem. A bug isn’t evil, it’s merely something that needs to be changed.

Think of a bug as the reaction of a lost child. She’s trying her best to do what was asked of her, but she misunderstood — or, maybe, you didn’t explain very well — , so she does something unexpected. Sometimes she sits on the floor and cries (that’s an error message).

Here’s the bad way to deal with this: yell at the child, saying she’s doing it wrong.

Here’s the good way to deal with this: explain patiently what to do, be more precise in your instructions.

When reporting a bug, you want to be that second kind of person: spending your time explaining patiently what the product should do. Saying “it’s not working!” is the least useful sentence in the process of bugsolving.

Reproduction

If you report a bug, you must be able to reproduce it. If it was transmitted to you by someone else, keep asking for details until you can confidently write a reproduction scenario.

I don’t care if it’s on a Japanese version of Internet Explorer, on a certain version of Windows Phone in landscape mode. You either support their environment, or you don’t. If you do, get the tools to emulate it easily.

In any case, don’t bother reporting a bug you can’t reproduce. You will only add complexity to the process, and fragment ownership — which is the surest way of making a bug rot.

Screenshots, and visual aids

The objective is to have a fluid reproduction scenario. Something that can be quickly executed, and quickly understood. Adding screenshots is usually a good idea, because they show with full power the outcome of a behavior.

But remember: only use them when they accelerate understanding — not as a crutch for a poor summary or vague reproduction scenario.

If the outcome is complex (a sequence of windows, for example), you may want to use a GIF instead. At the far end of the complexity spectrum, record a video. But don’t make the mistake of rushing to a video for something that can be adequately illustrated by a screenshot.

Error is text

If the bug triggers an error message (often with a code), I highly recommend pasting the full text of the error in the report, in addition to the screenshot.

Not only will it save the bug-fixer precious minutes in the initial investigation process, but it will also show you care for the comfort of people from whom you need help.

Spec

You can’t report a crime if there’s no law.

You can only write a bug report if you can prove that it is a bug — otherwise you’ll play the bug or feature game.

Consequently, you need a source of truth (preferably, a written one). Your bug report should have something like:

According to this spec, the VPN should be MD5 and not SHA1.

Closing a bug report

If you report a bug, you are responsible for reviewing the fix — saying “yes, the bug is now fixed”.

Not the dev, not the PM. You. If you wrote the report, you will close the report.

When the bug is fixed, you need only read your reproduction scenario, and execute the steps. If your scenario is solid, you will not even need to remember the context.

Enforcing that rule will magically force bug reporters to write better reproduction scenarii.