Quality Metrics

<default_to_action> When measuring quality or building dashboards:

1. MEASURE outcomes (bug escape rate, MTTD) not activities (test count)
2. FOCUS on DORA metrics: Deployment frequency, Lead time, MTTD, MTTR, Change failure rate
3. AVOID vanity metrics: 100% coverage means nothing if tests don't catch bugs
4. SET thresholds that drive behavior (quality gates block bad code)
5. TREND over time: Direction matters more than absolute numbers

Quick Metric Selection:

• Speed: Deployment frequency, lead time for changes
• Stability: Change failure rate, MTTR
• Quality: Bug escape rate, defect density, test effectiveness
• Process: Code review time, flaky test rate

Critical Success Factors:

• Metrics without action are theater
• What you measure is what you optimize
• Trends matter more than snapshots </default_to_action>

Quick Reference Card

When to Use

• Building quality dashboards
• Defining quality gates
• Evaluating testing effectiveness
• Justifying quality investments

Meaningful vs Vanity Metrics

DORA Metrics

Quality Gate Thresholds

Core Metrics

Bug Escape Rate

Bug Escape Rate = (Production Bugs / Total Bugs Found) × 100

Target: < 10% (90% caught before production)

Test Effectiveness

Test Effectiveness = (Bugs Found by Tests / Total Bugs) × 100

Target: > 70%

Defect Density

Defect Density = Defects / KLOC

Good: < 1 defect per KLOC

Mean Time to Detect (MTTD)

MTTD = Time(Bug Reported) - Time(Bug Introduced)

Target: < 1 day for critical, < 1 week for others

Dashboard Design

typescript
1// Agent generates quality dashboard
2await Task("Generate Dashboard", {
3  metrics: {
4    delivery: ['deployment-frequency', 'lead-time', 'change-failure-rate'],
5    quality: ['bug-escape-rate', 'test-effectiveness', 'defect-density'],
6    stability: ['mttd', 'mttr', 'availability'],
7    process: ['code-review-time', 'flaky-test-rate', 'coverage-trend']
8  },
9  visualization: 'grafana',
10  alerts: {
11    critical: { bug_escape_rate: '>20%', mttr: '>24h' },
12    warning: { coverage: '<70%', flaky_rate: '>5%' }
13  }
14}, "qe-quality-analyzer");

Quality Gate Configuration

json
1{
2  "qualityGates": {
3    "commit": {
4      "coverage": { "min": 80, "blocking": true },
5      "lint": { "errors": 0, "blocking": true }
6    },
7    "pr": {
8      "tests": { "pass": "100%", "blocking": true },
9      "security": { "critical": 0, "blocking": true },
10      "coverage_delta": { "min": 0, "blocking": false }
11    },
12    "release": {
13      "e2e": { "pass": "100%", "blocking": true },
14      "performance_p95": { "max_ms": 200, "blocking": true },
15      "bug_escape_rate": { "max": "10%", "blocking": false }
16    }
17  }
18}

Agent-Assisted Metrics

typescript
1// Calculate quality trends
2await Task("Quality Trend Analysis", {
3  timeframe: '90d',
4  metrics: ['bug-escape-rate', 'mttd', 'test-effectiveness'],
5  compare: 'previous-90d',
6  predictNext: '30d'
7}, "qe-quality-analyzer");
8
9// Evaluate quality gate
10await Task("Quality Gate Evaluation", {
11  buildId: 'build-123',
12  environment: 'staging',
13  metrics: currentMetrics,
14  policy: qualityPolicy
15}, "qe-quality-gate");

Agent Coordination Hints

Memory Namespace

aqe/quality-metrics/
├── dashboards/*         - Dashboard configurations
├── trends/*             - Historical metric data
├── gates/*              - Gate evaluation results
└── alerts/*             - Triggered alerts

Fleet Coordination

typescript
1const metricsFleet = await FleetManager.coordinate({
2  strategy: 'quality-metrics',
3  agents: [
4    'qe-quality-analyzer',         // Trend analysis
5    'qe-test-executor',            // Test metrics
6    'qe-coverage-analyzer',        // Coverage data
7    'qe-production-intelligence',  // Production metrics
8    'qe-quality-gate'              // Gate decisions
9  ],
10  topology: 'mesh'
11});

Common Traps

Related Skills

• agentic-quality-engineering - Agent coordination
• cicd-pipeline-qe-orchestrator - Quality gates
• risk-based-testing - Risk-informed metrics
• shift-right-testing - Production metrics

Remember

Measure outcomes, not activities. Bug escape rate > test count. MTTD/MTTR > coverage %. Trends > snapshots. Set gates that block bad code. What you measure is what you optimize.

With Agents: Agents track metrics automatically, analyze trends, trigger alerts, and make gate decisions. Use agents to maintain continuous quality visibility.