Stop basing defect priority on feelings!

It’s far too common that bugs and defects get prioritized based off how someone with gut feelings has for the issue versus how much it genuinely impacts business. I’ve observed this countless times and when I encountered this in my own company I compiled a document in confluence to negate the “feeling” and base prioritization on something more reproducible and data driven. Basic stuff, it’s nothing groundbreaking but it is shocking how few shops actually have any process or definition in place to refer to.

Click below for my implemented take on defect and bug prioritization.

Defect and Bug Priority Process

These processes can be overridden during times of critical business related needs, but should be followed in most defect priority based decision making processes.

To simplify this we will define “defects” as existing pre-production and “bugs” existing on a production environment. Once a defect has been approved as non-critical and is acceptable to be in production it is converted to a “bug” in our issue tracker.

Severity: A classification of software defect (bug), which indicates the degree of negative impact on the quality of software. Does not account for urgency or frequency as these are considered or defined later when determining the overall priority assignment of the issue. The degree of impact that a defect has on the development or operation of a component or system. Also known as Impact.

Keep the Environmental stage in mind when assigning Severity. (development, test, staging, production, etc) Severity is not defined by what environmental stage it occurs on, which is why it can apply to defects.

Severity Classifications:

Financial & Operational impacts need to be considered when evaluating Severity Classification (“S Ratings). An issue that does not occur in production but blocks all progress would be considered a Critical to Major issue.

Critical (S1):

  • Defect that affects critical functionality or critical data, which directly impact financial transactions or revenue.
  • A security flaw that allows a breach into other critical features and data.
  • Does not have a workaround. Example: Unsuccessful installation, complete failure of a feature.

Major (S2):

  • Defect that affects major functionality or major data.
  • Defect that impacts a highly used feature and/or data that indirectly impacts financial transactions or otherwise drives revenue.
  • Has a workaround, which is not obvious and is difficult. Example: A feature is not functional from one module but the task is doable if 10 complicated indirect steps are followed in another module/s.

Minor (S3):

  • Defect that affects minor functionality or non-critical data.
  • Defect that has an easy workaround. Example: A minor feature that is not functional in one module but the same task is easily doable from another module.

Trivial (S4):

  • Defect that does not affect functionality or data.
  • Defect that does not even need a workaround.
  • Defect that does not impact productivity or efficiency. It is merely an inconvenience. Example: Petty layout discrepancies, spelling/grammatical errors.

Frequency of Impact / Visibility

  • Frequency of Impact also known as Visibility
  • Probability that the defect will occur in a known environment
  • Does not address Severity of impact
  • Expected frequency percentage is based on best available data collected from reported issues, error reports and any other concrete data that can be used to determine the frequency of an occurrence.
  • Example: (Extreme) CSR’s reported encountering failure in address lookup on first attempt requiring them to check address availability twice.
  • Example: (Low) 1 out of 10 CSR’s encountered intermittent error when checking address availability resulting in CSR needing to repeat the availability check.
  • Frequency or visibility can also be determined by answering “How likely is this to occur when accessing the service, feature, data etc in the way it is expected to be used, or with the method described in the reproduction steps.” The key is to completely disregard Severity or Impact, and only consider “How likely is this to occur.

Frequency categories:

  • Extreme  (F1): 76-100% expected frequency
  • High         (F2): 51-75% expected frequency
  • Medium (F3): 26-50% expected frequency
  • Low (F4): 0 – 25% expected frequency

Priority: The level of (business) importance assigned to an item, e.g. defect. Defect Priority needs to be managed carefully in order to avoid product instability, especially when there is a large of number of defects. Priority, also known as Urgency, indicates the importance or urgency of fixing a defect. This does not in any way include Severity, as you can have an issue with a low Severity but an Urgent Priority. Though priority may be initially set by QA, it is usually finalized by the Product Owner or Project Manager. Refer to the Priority Matrix (see below) to obtain a rough estimate of priority to avoid over reliance on feelings or anecdotal evidence. Priority is determined by weighing Severity, Frequency, and Business Needs resulting in a risk based determination. This helps prevent an issue from being categorized at the wrong priority and pre-empting work on other more important business needs.

Priority / Urgency Categories:

  • Urgent (P1): Must be fixed in the next build.
  • High (P2): Must be fixed in any of the upcoming builds but should be included in the release.
  • Medium (P3): May be fixed after the release / in the next release.
  • Low (P4): May or may not be fixed at all.

Identifying priority is a subjective process, so do not take the categories above as authoritative; however, at a high level, priority can be determined by considering the following:

  • Business need for fixing the defect
  • Severity/Impact
  • Probability/Visibility
  • Available Resources (Developers to fix and Testers to verify the fixes) Available Time (Time for fixing, verifying the fixes and performing regression tests after the verification of the fixes)

Addendum:

Priority Matrix
Priority Flow Chart

 

 

Author: kiisu

Guitarist in The Secret Light Software Quality Assurance Automation Nerd