As in Mathematics, so in Natural Philosophy, the Investigation of difficult Things by the Method of Analysis, ought ever to precede the Method of Composition. This Analysis consists in making Experiments and Observations, and in drawing general Conclusions from them by Induction, and admitting of no Objections against the Conclusions, but such as are taken from Experiments, or other certain Truths.
For Hypotheses are not to be regarded in experimental Philosophy.
— Sir Isaac Newton
I would like to bring life tested approach of retrospective practice focused on Quality – ceremony to focus on elimination of defects by understanding core issues and addressing required preventive actions, leading to less time spent on fixing those and gaining more time spent building features. Let us call them bugs (negative features that make software functioning not as expected).
As any Retrospective…
it has it’s 5 steps and here is how those are mapped to practical actions:
- Set the stage – focus on reviewing bugs resolved during period (e.g. past iteration);
- Gather data – prepare in advance extract of bugs;
- Generate insights – gathered during retrospective as soon as team passes bugs one by one;
- Decide what to do – preventive action agreed and written, having assignee (es);
- Close – short summary of the reviewed bugs;
What helps to make it efficient:
- Agreement on scope: what bugs are to be reviewed (production, staging, development, what severities, etc…)?
- Right people in: depending on scope and nature you may need to have not only development team members but also release coordinating operations engineers, product managers, etc.;
- In multi-team environment you may need to involve only some team members of every participant team, given who had participated and/or who is best addressing, lead engineers, etc.;
- Prepare in advance (e.g. take iteration, release, etc.. extract of all bug resolved, pre-sorted/pre-groupped by severity, …);
- Have mandatory field when resolving bug with your bug-tracker (e.g. Jira) that must contain technical reason for the bug + projected preventive action to be reviewed further by retrospective. Having this is very valuable as decreases time spent on “gather data” and provides more “space” for “generate insights”;
- Keep it short: 60 minutes is usually enough/sufficient if everything is properly prepared in advance and practice is regular;
Inductive approach (during retrospective):
As team is passing from most to less severe production bugs, they review each and make sure technical reasons are understood and proper preventive action(s) agreed. Then team decides who will own the preventive actions (given team setup this may be teams or individuals). In obvious cases like: bug was not detected by automated and/or unit tests when it should have been, bug is simply stays opened until action is in place (as it is best practice to have those as bug “definition of fixed”).
As soon as bug is discussed team analyses possible patterns for the reasons of the bug by writing down short summary of the bug’s origin.
And moves on with the next bug, etc…
Since approach is inductive: moving from the specific to the general here interesting findings emerge from reality and most “aha-s” and bigger problems become visible.
Here are real-life examples of bugs “patterns”:
- “release misconfigs”
- “old legacy codes”
- “hardware issues”
- “story misunderstandings”
- “quick = dirty fixes”
- “3rd party misconfigs”
- “localization issues”
Of course it is good to prevent same bugs popping up again by addressing required preventive action(s) but is not just enough if you want to address the core issue.
The inductive approach here will allow team to focus on systematic problems and start effectively eliminating bugs by coming from particular life-cases, reality (reasons) to more “patternized” reasons and risks of having future bugs of the same nature/origin.
As an example Quality Retrospective may have also outcomes / actions like those:
- 3 compatibility issues / define list of supported browsers (+agree not to support the others, thus do not file bugs;
- 3 old legacy code issues / address with your architect (team), consider reworking;
- 3 misunderstandings / address with your product team how those occurred and refine story writing, grooming to avoid/reduce possible misunderstandings in the future.
- 2 design issues / agree to address design issues incrementally (stop demanding their fixes as bugs, …);
- 2 release miscofigurations / review release notes collection and approval mechanisms, reduce human factor;
- 2 translation issues / address with your product team how those shall be handled;
Team may decide to address them from having biggest impact or from more severe issues or more systematic, but at this level they will be fixing the root cause rather than just a symptom (particular bug).
Advice to facilitators: manage time in the way to avoid burning all of it reviewing all bugs (you may not be able and/or it may be not even needed), rather focus on most important ones + detect “patterns” as early as you can.
How to address preventive actions and follow up
As in any retrospective, I’ve seen two approaches working here depending on the efforts:
- in next sprint task approach – action owner plans a task (or time, depending on action nature) in the next sprint. Usually works when effort is quite small;
- “top story” approach – for “bigger actions” plan a story of top priority in the subsequent sprint;
Consequently follow up is done either at the sprint review or on the next retrospective.
Some “Side findings”:
- You will figure out that some bugs are not actually mistakes as from the coding perspective rather than “logical gaps” in user flows;
- Some bugs are unavoidable and you may not need to prevent them (e.g. the risk of having them is acceptable from business perspective or preventive actions are too expensive compared to the consequences, …);
- Some “Invalids”, “Cannot reproduces”, “Duplicates”, “Won’t fix-es” and “Works for me-s” causing development team distraction and eating up time – at some extent this is still unavoidable but keep an eye on if you have increasing numbers of those;
- Some do not fit into any pattern (just exceptions);
Good luck inducting quality in 🙂