Calculating Consecutive Event Probability: Practical Applications in Software Engineering

by

🌏 閱讀中文版本

In software engineering, we frequently need to assess the probability of consecutive events: API failure risks, deployment success likelihood, test pass expectations. These seemingly simple probability problems directly impact system reliability design, monitoring strategy, and test coverage.

Why Engineers Need to Understand Probability

Common scenarios in practice:

Scenario 1: API Success Rate Monitoring

If your API has a 99.9% single-call success rate (three nines), what’s the probability that 1000 consecutive calls all succeed? This directly relates to your SLA commitments.

Scenario 2: CI/CD Pipeline Reliability

If each test stage has a 95% pass rate and you have 10 stages, the overall pipeline success probability will be surprisingly low.

Scenario 3: Failure Detection Sensitivity

Health checks run every minute. If you allow 3 consecutive failures before triggering an alert, what’s your average detection delay?

Probability Basics

Probability P represents the likelihood of an event occurring, ranging from 0 to 1 (or 0% to 100%).

Consecutive probability calculation for independent events:

P(consecutive n times) = P^n

Key Point: This formula only applies to independent events. If events are dependent (e.g., cache invalidation, cascading failures), more complex models are needed.

Practical Example 1: API Reliability Calculation

Assuming your API has a 99% single-call success rate (P = 0.99), calculate consecutive success probability:

def consecutive_success_probability(single_prob, n_calls):
    """
    Calculate probability of n consecutive successful calls

    Args:
        single_prob: Single call success rate (0-1)
        n_calls: Number of consecutive calls

    Returns:
        Consecutive success probability
    """
    return single_prob ** n_calls

# Example: API with 99% success rate
success_rate = 0.99

print(f"10 consecutive successes: {consecutive_success_probability(success_rate, 10):.4f}")
# Output: 0.9044 (90.44%)

print(f"100 consecutive successes: {consecutive_success_probability(success_rate, 100):.4f}")
# Output: 0.3660 (36.60%)

print(f"1000 consecutive successes: {consecutive_success_probability(success_rate, 1000):.4f}")
# Output: 0.0000 (almost impossible)

Key Insights:

  • 99% success rate seems high, but 100 consecutive calls have only 36.6% chance of all succeeding
  • This explains why retry mechanisms and degradation strategies are necessary

Practical Example 2: Test Coverage and Pipeline Reliability

Assume you have a CI/CD pipeline with multiple test stages:

import pandas as pd

def pipeline_reliability_table(stage_prob, max_stages):
    """
    Generate pipeline reliability table

    Args:
        stage_prob: Pass rate per stage
        max_stages: Maximum number of stages

    Returns:
        DataFrame containing overall pass rates for each stage count
    """
    data = []
    for n in range(1, max_stages + 1):
        overall_prob = stage_prob ** n
        data.append({
            'Stages': n,
            'Per-Stage Pass Rate': f"{stage_prob:.1%}",
            'Overall Pass Rate': f"{overall_prob:.2%}",
            'Overall Failure Rate': f"{(1-overall_prob):.2%}"
        })
    return pd.DataFrame(data)

# Assume 95% pass rate per stage
print(pipeline_reliability_table(0.95, 10))

Output:

   Stages Per-Stage Pass Rate Overall Pass Rate Overall Failure Rate
0       1               95.0%           95.00%                5.00%
1       2               95.0%           90.25%                9.75%
2       3               95.0%           85.74%               14.26%
3       4               95.0%           81.45%               18.55%
4       5               95.0%           77.38%               22.62%
5       6               95.0%           73.51%               26.49%
6       7               95.0%           69.83%               30.17%
7       8               95.0%           66.34%               33.66%
8       9               95.0%           63.02%               36.98%
9      10               95.0%           59.87%               40.13%

Important Finding:

  • A 10-stage pipeline with 95% pass rate per stage has a 40.13% overall failure rate
  • This explains why maximizing test quality at each stage is critical

Practical Example 3: Monitoring Alert Sensitivity

Health check configuration: Check every 30 seconds, trigger alert after 3 consecutive failures.

def alert_sensitivity_analysis(check_interval, consecutive_failures, failure_prob):
    """
    Analyze monitoring alert sensitivity

    Args:
        check_interval: Check interval (seconds)
        consecutive_failures: Consecutive failures to trigger alert
        failure_prob: Single check failure rate

    Returns:
        dict containing average detection time and false positive rate
    """
    # Consecutive failure probability (false positive rate)
    false_positive_prob = failure_prob ** consecutive_failures

    # Average detection delay
    avg_detection_time = check_interval * consecutive_failures

    return {
        'avg_detection_time_sec': avg_detection_time,
        'false_positive_rate': false_positive_prob,
        'false_positive_pct': f"{false_positive_prob:.4%}"
    }

# Example: Check every 30 seconds, trigger after 3 failures
result = alert_sensitivity_analysis(
    check_interval=30,
    consecutive_failures=3,
    failure_prob=0.01  # 1% check failure rate (network jitter, etc.)
)

print(f"Average detection delay: {result['avg_detection_time_sec']} seconds")
print(f"False positive rate: {result['false_positive_pct']}")

# Output:
# Average detection delay: 90 seconds
# False positive rate: 0.0001%

Design Tradeoffs:

  • More consecutive failures → Lower false positive rate, but higher detection delay
  • Need to find balance based on service SLA requirements

Practical Example 4: SLA Availability Calculation

Assuming you commit to 99.9% SLA (three nines), calculate allowed downtime for various time periods:

def calculate_downtime_budget(uptime_sla, time_period_days):
    """
    Calculate SLA downtime budget

    Args:
        uptime_sla: Availability SLA (e.g., 0.999 for 99.9%)
        time_period_days: Time period (days)

    Returns:
        dict containing allowed downtime
    """
    total_minutes = time_period_days * 24 * 60
    downtime_minutes = total_minutes * (1 - uptime_sla)

    return {
        'period_days': time_period_days,
        'sla_pct': f"{uptime_sla:.1%}",
        'downtime_minutes': downtime_minutes,
        'downtime_hours': downtime_minutes / 60
    }

# Calculate for various periods
for days, label in [(1, 'Daily'), (7, 'Weekly'), (30, 'Monthly'), (365, 'Yearly')]:
    result = calculate_downtime_budget(0.999, days)
    print(f"{label} ({result['sla_pct']}): "
          f"{result['downtime_minutes']:.1f} minutes "
          f"({result['downtime_hours']:.2f} hours)")

# Output:
# Daily (99.9%): 1.4 minutes (0.02 hours)
# Weekly (99.9%): 10.1 minutes (0.17 hours)
# Monthly (99.9%): 43.2 minutes (0.72 hours)
# Yearly (99.9%): 525.6 minutes (8.76 hours)

Common Pitfalls and Solutions

Pitfall 1: Assuming Events are Independent

Wrong Assumption: Deployment failures are independent events

Reality: If the first deployment fails, it might be an environment issue, causing subsequent deployments to fail too (not independent)

Solution: Use conditional probability or Bayesian modeling

Pitfall 2: Ignoring Time Factors

Wrong Assumption: Only looking at probability, ignoring time

Reality: 10 consecutive API calls in 1 second vs 1 hour present completely different risks

Solution: Combine with time window analysis (rate limiting, circuit breaker)

Pitfall 3: Over-relying on Historical Data

Wrong Assumption: Past success rates predict the future

Reality: System load, network conditions, and dependent services constantly change

Solution: Dynamically adjust probability models, use sliding windows

Tools and Libraries

# Use scipy for more complex probability distributions
from scipy import stats

# Binomial distribution: Probability of exactly k successes in n independent trials
n, p = 100, 0.99  # 100 calls, 99% success rate
prob_all_success = stats.binom.pmf(k=100, n=n, p=p)
print(f"All 100 succeed: {prob_all_success:.4f}")

# Probability of at least 95 successes
prob_at_least_95 = sum(stats.binom.pmf(k, n, p) for k in range(95, 101))
print(f"At least 95 succeed: {prob_at_least_95:.4f}")

Summary

Practical significance of understanding consecutive event probability for software engineers:

  1. System Design – Calculate retry counts, set timeouts, design degradation strategies
  2. Monitoring Strategy – Determine alert thresholds, balance false positives and detection delay
  3. Test Planning – Evaluate CI/CD pipeline reliability, determine test coverage goals
  4. SLA Commitments – Calculate availability budgets, plan maintenance windows

Key Principles:

  • Consecutive probability for independent events = P^n, but carefully verify independence assumption
  • High success rates drop rapidly over many consecutive attempts
  • In practice, combine time, load, and dependency considerations
  • Use code to verify assumptions, don’t rely on intuition

Next time you design system reliability strategies, these probability calculations will help you make more informed technical decisions.

Related Articles

Related posts:

  1. 如何計算連續事件的機率：軟體工程中的實務應用
  2. AWS Lambda + CloudWatch Events 實現每月 EC2 快照備份與驗證
  3. 系統性能基本概念：平均每秒處理的用戶數量 vs. 總每秒處理的用戶數量
  4. System Performance Fundamentals: Average TPS vs Peak TPS

Leave a Comment