Test Priorities Matrix
Overview
Not all tests are equal. This guide provides a framework for prioritizing test creation, execution, and maintenance based on risk, impact, and resources.
Priority Levels
P0 - Critical (Ship Blockers)
Definition: Failures that prevent the game from being playable or shippable.
| Criteria |
Examples |
| Game cannot start |
Crash on launch, infinite load |
| Core loop broken |
Cannot perform primary action |
| Data loss |
Saves corrupted, progress lost |
| Platform certification |
TRC/XR failures |
| Legal/compliance |
Rating violations |
Testing Approach:
- Automated smoke tests on every build
- Manual verification before any release
- Zero tolerance for P0 bugs in release builds
P1 - High (Major Features)
Definition: Significant functionality that affects most players' experience.
| Criteria |
Examples |
| Major features broken |
Multiplayer, progression systems |
| Frequent player paths |
Main story, common actions |
| Significant UX issues |
Confusing UI, missing feedback |
| Performance problems |
Unplayable frame rates |
Testing Approach:
- Comprehensive automated tests
- Regular regression testing
- Full coverage in release candidates
P2 - Medium (Standard Features)
Definition: Important functionality that affects significant portions of gameplay.
| Criteria |
Examples |
| Secondary features |
Side quests, optional content |
| Edge cases |
Unusual player actions |
| Platform-specific |
Single-platform issues |
| Minor progression |
Non-critical collectibles |
Testing Approach:
- Selective automation
- Milestone regression testing
- Coverage prioritized by usage data
P3 - Low (Polish Items)
Definition: Issues that are noticeable but don't significantly impact gameplay.
| Criteria |
Examples |
| Visual polish |
Minor clipping, texture issues |
| Audio polish |
Volume inconsistencies |
| Rare scenarios |
Edge cases with workarounds |
| Nice-to-have |
QoL improvements |
Testing Approach:
- Manual exploratory testing
- Automated only if easy/cheap
- Fixed as time permits
Risk-Based Prioritization
Risk Factors
| Factor |
High Risk |
Low Risk |
| Usage frequency |
Core loop |
Rarely accessed |
| Player visibility |
Always visible |
Hidden/optional |
| Data impact |
Saves, progression |
Cosmetic only |
| Recovery difficulty |
No workaround |
Easy to retry |
| Change frequency |
Frequently modified |
Stable code |
Risk Assessment Matrix
IMPACT
Low High
┌─────────┬─────────┐
High │ P2 │ P0 │
LIKELIHOOD ├─────────┼─────────┤
Low │ P3 │ P1 │
└─────────┴─────────┘
Coverage Targets by Priority
| Priority |
Unit Test |
Integration |
E2E/Smoke |
Manual |
| P0 |
100% |
100% |
Required |
Pre-release |
| P1 |
80%+ |
80%+ |
As needed |
Milestone |
| P2 |
60%+ |
Key paths |
Optional |
Sprint |
| P3 |
Optional |
Optional |
No |
Ad-hoc |
Test Type Distribution
Recommended Test Pyramid (Games)
▲
/│\
/ │ \ E2E/Smoke Tests (5%)
/ │ \ - Full game flow
/ │ \ - Platform certification
───────────
/ │ \
/ │ \ Integration Tests (25%)
/ │ \ - System interactions
/ │ \ - Network, save, audio
─────────────────────
/ │ \
/ │ \ Unit Tests (70%)
/ │ \ - Pure logic
/ │ \- Algorithms, calculations
───────────────────────────────
Game-Specific Considerations
Unlike web apps, games have unique testing needs:
| Test Type |
Standard App |
Game-Specific |
| Unit |
Business logic |
Damage calc, AI decisions |
| Integration |
API + DB |
Physics, audio, network |
| E2E |
User flows |
Gameplay scenarios |
| Additional |
N/A |
Playtesting, balance, feel |
Execution Order
CI Pipeline (Every Commit)
- P0 smoke tests (5-10 minutes)
- P0/P1 unit tests (10-15 minutes)
- P0 integration tests (5-10 minutes)
Daily/Nightly
- Full P0 suite
- Full P1 suite
- P2 regression suite
- Performance benchmarks
Milestone/Release
- All automated tests
- Full P0-P2 manual testing
- Platform certification tests
- Exploratory testing
- Performance profiling
Bug Triage Criteria
Priority Assignment
| Question |
P0 |
P1 |
P2 |
P3 |
| Can player complete game? |
No |
Affected |
No |
No |
| How many players affected? |
All |
Most |
Some |
Few |
| Is there a workaround? |
No |
Difficult |
Yes |
Easy |
| Data at risk? |
Yes |
Possible |
No |
No |
| Platform certification? |
Fail |
Risk |
Pass |
Pass |
Severity vs Priority
SEVERITY: How bad is the bug?
Critical → Crash, data loss
Major → Feature broken
Minor → Incorrect behavior
Trivial → Cosmetic
PRIORITY: How soon to fix?
P0 → Immediately (blocks release)
P1 → This sprint
P2 → This milestone
P3 → Backlog
Resource Allocation
QA Time Distribution
| Activity |
Percentage |
| P0 verification |
30% |
| P1 testing |
30% |
| P2 testing |
20% |
| Exploratory |
15% |
| Test maintenance |
5% |
Automation Investment
| Priority |
Automation Value |
ROI |
| P0 |
Essential |
Highest |
| P1 |
High |
High |
| P2 |
Medium |
Medium |
| P3 |
Low |
Often negative |
Platform Priority Matrix
Multi-Platform Prioritization
| Platform |
Player Base |
Certification |
Testing Priority |
| Primary (e.g., PC) |
60% |
Light |
P0: All, P1: All |
| Secondary (e.g., PS5) |
25% |
Heavy |
P0: All, P1: Most |
| Tertiary (e.g., Switch) |
15% |
Medium |
P0: All, P1: Core |
Cross-Platform Testing Strategy
Platform Testing Coverage
PC PS5 Xbox Switch Mobile
P0 ████ ████ ████ ████ ████
P1 ████ ███░ ███░ ██░░ ██░░
P2 ███░ ██░░ ██░░ █░░░ █░░░
P3 ██░░ █░░░ █░░░ ░░░░ ░░░░
████ = Full coverage
███░ = High coverage
██░░ = Medium coverage
█░░░ = Low coverage
░░░░ = Minimal/none
Best Practices
DO
- Reassess priorities as development progresses
- Weight user-facing features higher
- Consider platform certification requirements
- Focus automation on stable, high-value areas
- Track bug escape rates by priority
DON'T
- Treat all tests equally
- Automate P3 before P0/P1 coverage is solid
- Skip P0 testing for "small changes"
- Ignore platform-specific requirements
- Let P1/P2 bugs accumulate
Metrics to Track
| Metric |
Target |
Purpose |
| P0 test pass rate |
100% |
Build quality |
| P0 bug escape rate |
0% |
Test effectiveness |
| P1 coverage |
80%+ |
Feature coverage |
| Test execution time |
< 30 min (CI) |
Development velocity |
| Flaky test rate |
< 1% |
Test reliability |