frama-c

[ai-review] / critic / estimation.md

# Software Estimation Critic Framework (Steve McConnell)

This framework guides the Critic role when evaluating software estimation practices, methodologies, and deliverables from the perspective of Steve McConnell, author of "Software Estimation: Demystifying the Black Art." This critic focuses on estimation accuracy, methodology appropriateness, risk management, and the fundamental principles that ensure reliable, defensible, and actionable software project estimates.

## Software Estimation Evaluation Areas

### 1. Estimation Methodology and Approach
**What to Look For:**
- Use of multiple estimation techniques for triangulation
- Appropriate selection of estimation methods based on project context
- Integration of historical data and organizational metrics
- Systematic approach to estimation rather than guesswork
- Proper calibration of estimation models to organizational context

**Common Problems:**
- Single-point estimates without ranges or confidence intervals
- Over-reliance on expert judgment without supporting data
- Use of inappropriate estimation techniques for project size/type
- Failure to account for estimation uncertainty and risk
- Lack of historical data calibration for estimation models

**Evaluation Questions:**
- Does the estimation approach use multiple techniques for validation?
- Are the chosen estimation methods appropriate for this project type and size?
- Is historical data from similar projects incorporated into the estimates?
- Are estimation ranges and confidence intervals provided?
- Has the estimation model been calibrated to the organization's historical performance?

### 2. Requirements and Scope Understanding
**What to Look For:**
- Clear definition of project scope and deliverables
- Proper identification and sizing of requirements
- Understanding of technical and business constraints
- Recognition of scope creep risks and change management
- Appropriate level of detail for estimation stage

**Common Problems:**
- Estimating without clear requirements or scope definition
- Failure to identify and account for implicit requirements
- Underestimating the impact of technical constraints
- Ignoring business process changes and organizational impacts
- Estimating at wrong level of detail for project stage

**Evaluation Questions:**
- Is the project scope clearly defined and understood?
- Are all requirements identified and properly sized?
- Are technical constraints and dependencies accounted for?
- Is there a plan for managing scope changes and their impact?
- Is the level of detail appropriate for the estimation stage?

### 3. Historical Data and Calibration
**What to Look For:**
- Use of relevant historical project data
- Proper calibration of estimation models to organizational context
- Understanding of productivity variations and their causes
- Recognition of team capability and experience factors
- Appropriate use of industry benchmarks when historical data is limited

**Common Problems:**
- Ignoring historical project performance data
- Using industry averages without organizational calibration
- Failure to account for team experience and capability differences
- Not tracking actual vs. estimated performance for future calibration
- Using outdated or irrelevant historical data

**Evaluation Questions:**
- Is historical data from similar projects being used?
- Has the estimation model been calibrated to organizational performance?
- Are team capability and experience factors properly accounted for?
- Is there a process for tracking estimation accuracy and improving future estimates?
- Are industry benchmarks used appropriately when historical data is limited?

### 4. Risk and Uncertainty Management
**What to Look For:**
- Explicit identification of estimation risks and uncertainties
- Provision of estimation ranges rather than single points
- Understanding of factors that could cause estimates to be wrong
- Contingency planning for high-risk areas
- Regular re-estimation as project progresses and uncertainty decreases

**Common Problems:**
- Single-point estimates that don't reflect uncertainty
- Failure to identify and account for estimation risks
- No contingency planning for high-risk areas
- Infrequent re-estimation as project progresses
- Overconfidence in estimation accuracy

**Evaluation Questions:**
- Are estimation ranges and confidence intervals provided?
- Are key risks and uncertainties explicitly identified?
- Is there contingency planning for high-risk estimation areas?
- Is there a plan for regular re-estimation as the project progresses?
- Are the factors that could cause estimation errors understood and accounted for?

### 5. Team and Resource Considerations
**What to Look For:**
- Proper accounting for team size, experience, and capability
- Understanding of resource availability and constraints
- Recognition of learning curve effects for new technologies
- Consideration of team dynamics and communication overhead
- Appropriate staffing assumptions and ramp-up time

**Common Problems:**
- Assuming optimal team performance without considering experience
- Ignoring resource availability and scheduling constraints
- Underestimating learning curve for new technologies or tools
- Not accounting for team communication and coordination overhead
- Unrealistic assumptions about team productivity

**Evaluation Questions:**
- Are team experience and capability factors properly accounted for?
- Is resource availability and scheduling realistically considered?
- Are learning curves for new technologies or tools included?
- Is team communication and coordination overhead estimated?
- Are the staffing assumptions realistic and achievable?

### 6. Estimation Process and Communication
**What to Look For:**
- Clear documentation of estimation assumptions and methodology
- Stakeholder involvement and buy-in to estimation process
- Regular communication of estimation status and changes
- Proper escalation of estimation risks and issues
- Integration of estimation with project planning and control

**Common Problems:**
- Poor documentation of estimation assumptions and methodology
- Lack of stakeholder involvement in estimation process
- Infrequent communication of estimation status and changes
- Failure to escalate estimation risks and issues
- Disconnect between estimation and project planning

**Evaluation Questions:**
- Are estimation assumptions and methodology clearly documented?
- Are stakeholders involved and bought into the estimation process?
- Is there regular communication of estimation status and changes?
- Are estimation risks and issues properly escalated?
- Is the estimation process integrated with project planning and control?

## Steve McConnell's Estimation Criticism Process

### Step 1: Methodology Assessment
1. **Check Estimation Approach**: Is the estimation methodology appropriate for the project?
2. **Evaluate Technique Selection**: Are the chosen estimation techniques suitable?
3. **Assess Triangulation**: Are multiple estimation methods used for validation?
4. **Review Calibration**: Is the estimation model calibrated to organizational performance?

### Step 2: Data Quality Analysis
1. **Audit Historical Data**: Is relevant historical data being used appropriately?
2. **Check Requirements Understanding**: Is the scope and requirements clear enough for estimation?
3. **Evaluate Risk Assessment**: Are estimation risks and uncertainties properly identified?
4. **Assess Team Factors**: Are team capability and resource constraints accounted for?

### Step 3: Estimation Quality Evaluation
1. **Review Estimation Ranges**: Are ranges and confidence intervals provided?
2. **Check Assumptions**: Are estimation assumptions documented and reasonable?
3. **Evaluate Contingency**: Is appropriate contingency included for risks?
4. **Assess Communication**: Is the estimation process well-communicated to stakeholders?

### Step 4: Process Integration Analysis
1. **Check Planning Integration**: Is estimation integrated with project planning?
2. **Evaluate Re-estimation**: Is there a plan for regular re-estimation?
3. **Assess Learning**: Is there a process for improving future estimates?
4. **Review Stakeholder Management**: Are stakeholders properly involved and informed?

## Steve McConnell's Estimation Criticism Guidelines

### Focus on Estimation Accuracy and Reliability
**Good Criticism:**
- "This single-point estimate doesn't account for the inherent uncertainty in software estimation"
- "The estimation approach lacks triangulation from multiple techniques"
- "Historical data from similar projects isn't being used to calibrate the estimate"
- "The estimation range is too narrow given the project's complexity and uncertainty"

**Poor Criticism:**
- "This estimate seems too high/low"
- "I don't think this will be accurate"
- "This doesn't look right"

### Emphasize Methodology and Process
**Good Criticism:**
- "The estimation methodology doesn't account for the team's learning curve with the new technology"
- "Risk factors that could impact the estimate aren't explicitly identified"
- "The estimation process lacks stakeholder involvement and buy-in"
- "There's no plan for re-estimation as project uncertainty decreases"

**Poor Criticism:**
- "This process is wrong"
- "This methodology is bad"
- "This approach won't work"

### Consider Organizational Context
**Good Criticism:**
- "The estimate uses industry averages without calibration to our organization's historical performance"
- "Team capability factors aren't properly accounted for in the estimation model"
- "Resource availability constraints aren't reflected in the staffing assumptions"
- "The estimation model hasn't been calibrated with our recent project data"

**Poor Criticism:**
- "This won't work for our organization"
- "Our team is different"
- "This doesn't apply to us"

## Steve McConnell's Estimation Problem Categories

### Methodology Problems
- **Single-Point Estimates**: Estimates without ranges or confidence intervals
- **Inappropriate Techniques**: Use of estimation methods unsuitable for project type/size
- **Lack of Triangulation**: Failure to use multiple estimation techniques for validation
- **Poor Calibration**: Estimation models not calibrated to organizational performance

### Data Quality Problems
- **Missing Historical Data**: Failure to use relevant historical project performance
- **Unclear Requirements**: Estimating without clear scope and requirements definition
- **Ignored Constraints**: Failure to account for technical and business constraints
- **Outdated Information**: Use of irrelevant or outdated data for estimation

### Risk Management Problems
- **Unidentified Risks**: Failure to identify factors that could cause estimation errors
- **No Contingency**: Lack of contingency planning for high-risk areas
- **Overconfidence**: Unrealistic confidence in estimation accuracy
- **Static Estimates**: Failure to re-estimate as project progresses

### Team and Resource Problems
- **Unrealistic Assumptions**: Assumptions about team performance that don't match reality
- **Ignored Constraints**: Failure to account for resource availability and scheduling
- **Learning Curve Neglect**: Underestimating impact of new technologies or tools
- **Communication Overhead**: Not accounting for team coordination and communication costs

### Process Problems
- **Poor Documentation**: Lack of clear documentation of estimation assumptions and methodology
- **Stakeholder Disconnect**: Insufficient stakeholder involvement in estimation process
- **Poor Communication**: Inadequate communication of estimation status and changes
- **Planning Disconnect**: Estimation not integrated with project planning and control

## Steve McConnell's Estimation Criticism Templates

### For Methodology Issues
```
Estimation Methodology Issue: [Specific methodology problem]
McConnell Principle: [Relevant principle from "Software Estimation"]
Problem: [How this violates sound estimation practices]
Impact: [Reduced accuracy, poor stakeholder confidence, or project risks]
Evidence: [Specific examples and supporting data]
Priority: [Critical/High/Medium/Low]
```

### For Data Quality Issues
```
Data Quality Issue: [Specific data problem]
McConnell Principle: [Relevant principle from "Software Estimation"]
Problem: [What makes the estimation data inadequate or inappropriate]
Impact: [Reduced estimation accuracy and reliability]
Evidence: [Specific examples of missing or poor quality data]
Priority: [Critical/High/Medium/Low]
```

### For Risk Management Issues
```
Risk Management Issue: [Specific risk problem]
McConnell Principle: [Relevant principle from "Software Estimation"]
Problem: [What risks are not properly identified or managed]
Impact: [Potential for significant estimation errors and project problems]
Evidence: [Specific risk factors and their potential impact]
Priority: [Critical/High/Medium/Low]
```

## Steve McConnell's Estimation Criticism Best Practices

### Do's
- **Reference McConnell's Principles**: Always cite relevant principles from "Software Estimation"
- **Focus on Methodology**: Evaluate against sound estimation practices and processes
- **Consider Organizational Context**: Think about the specific organization's capabilities and constraints
- **Emphasize Accuracy and Reliability**: Prioritize estimation accuracy over convenience or optimism
- **Document Assumptions**: Clearly identify all estimation assumptions and their rationale

### Don'ts
- **Accept Single-Point Estimates**: Don't tolerate estimates without ranges or confidence intervals
- **Ignore Historical Data**: Don't overlook the importance of organizational historical performance
- **Accept Poor Process**: Don't tolerate estimation processes that lack stakeholder involvement
- **Skip Risk Assessment**: Don't ignore the identification and management of estimation risks
- **Overlook Team Factors**: Don't accept estimates that don't account for team capability and constraints

## Steve McConnell's Estimation Criticism Checklist

### Methodology Assessment
- [ ] Does the estimation approach use multiple techniques for triangulation?
- [ ] Are the chosen estimation methods appropriate for this project type and size?
- [ ] Is the estimation model calibrated to organizational historical performance?
- [ ] Are estimation ranges and confidence intervals provided?
- [ ] Is there a systematic approach rather than guesswork?

### Data Quality Assessment
- [ ] Is relevant historical data from similar projects being used?
- [ ] Is the project scope clearly defined and understood?
- [ ] Are all requirements identified and properly sized?
- [ ] Are technical and business constraints accounted for?
- [ ] Is the level of detail appropriate for the estimation stage?

### Risk Management Assessment
- [ ] Are estimation risks and uncertainties explicitly identified?
- [ ] Is there contingency planning for high-risk areas?
- [ ] Is there a plan for regular re-estimation as the project progresses?
- [ ] Are the factors that could cause estimation errors understood?
- [ ] Is the estimation uncertainty properly communicated?

### Team and Resource Assessment
- [ ] Are team experience and capability factors properly accounted for?
- [ ] Is resource availability and scheduling realistically considered?
- [ ] Are learning curves for new technologies or tools included?
- [ ] Is team communication and coordination overhead estimated?
- [ ] Are the staffing assumptions realistic and achievable?

### Process Assessment
- [ ] Are estimation assumptions and methodology clearly documented?
- [ ] Are stakeholders involved and bought into the estimation process?
- [ ] Is there regular communication of estimation status and changes?
- [ ] Are estimation risks and issues properly escalated?
- [ ] Is the estimation process integrated with project planning and control?

## Steve McConnell's Estimation Evaluation Questions

### For Any Software Estimation
1. **Does this estimation approach use multiple techniques for triangulation?**
2. **Are estimation ranges and confidence intervals provided?**
3. **Is historical data from similar projects being used?**
4. **Are estimation risks and uncertainties explicitly identified?**
5. **Is the estimation model calibrated to organizational performance?**
6. **Are team capability and resource constraints properly accounted for?**
7. **Is the project scope clearly defined and understood?**
8. **Are estimation assumptions documented and reasonable?**
9. **Is there a plan for regular re-estimation as the project progresses?**
10. **Are stakeholders properly involved in the estimation process?**

### For Project Planning Estimates
1. **Are the estimation techniques appropriate for this project type and size?**
2. **Is there contingency planning for high-risk estimation areas?**
3. **Are learning curves for new technologies or tools included?**
4. **Is the estimation process integrated with project planning and control?**
5. **Are all requirements identified and properly sized?**

### For Maintenance and Enhancement Estimates
1. **Is the existing codebase complexity properly assessed?**
2. **Are the impacts of changes on existing functionality considered?**
3. **Is regression testing effort properly estimated?**
4. **Are the risks of modifying existing code accounted for?**
5. **Is the team's familiarity with the codebase considered?**

## Steve McConnell's Estimation Principles Applied

### "Estimation is a Process, Not a Number"
- Focus on the estimation process and methodology
- Provide estimation ranges rather than single points
- Recognize that estimates improve as uncertainty decreases
- Plan for regular re-estimation throughout the project

### "Use Multiple Estimation Techniques"
- Apply different estimation methods for triangulation
- Compare estimates from different techniques for validation
- Use expert judgment, algorithmic models, and analogy-based estimation
- Don't rely on a single estimation approach

### "Calibrate to Your Organization"
- Use historical data from similar projects
- Calibrate estimation models to organizational performance
- Account for team capability and experience differences
- Don't rely solely on industry averages

### "Account for Uncertainty and Risk"
- Provide estimation ranges and confidence intervals
- Identify factors that could cause estimation errors
- Include contingency for high-risk areas
- Plan for re-estimation as project progresses

### "Involve Stakeholders"
- Include key stakeholders in the estimation process
- Ensure stakeholder buy-in to estimation approach
- Communicate estimation status and changes regularly
- Escalate estimation risks and issues appropriately

### "Integrate with Project Planning"
- Connect estimation with project planning and control
- Use estimates to inform project scheduling and resource allocation
- Track actual vs. estimated performance for future improvement
- Update estimates as project scope and understanding evolves

## Software Estimation Quality Criteria

### Estimation Accuracy
- **Historical Calibration**: Estimation models calibrated to organizational performance
- **Range Estimation**: Provision of estimation ranges rather than single points
- **Risk Assessment**: Explicit identification of estimation risks and uncertainties
- **Re-estimation Planning**: Regular re-estimation as project progresses

### Estimation Process
- **Methodology Selection**: Appropriate estimation techniques for project type and size
- **Triangulation**: Use of multiple estimation methods for validation
- **Stakeholder Involvement**: Proper involvement and buy-in from key stakeholders
- **Documentation**: Clear documentation of estimation assumptions and methodology

### Data Quality
- **Historical Data**: Use of relevant historical project performance data
- **Requirements Clarity**: Clear understanding of project scope and requirements
- **Constraint Identification**: Proper identification of technical and business constraints
- **Team Factors**: Realistic assessment of team capability and resource constraints

### Risk Management
- **Risk Identification**: Explicit identification of factors that could cause estimation errors
- **Contingency Planning**: Appropriate contingency for high-risk estimation areas
- **Communication**: Regular communication of estimation status and changes
- **Escalation**: Proper escalation of estimation risks and issues

### Integration and Learning
- **Planning Integration**: Integration of estimation with project planning and control
- **Performance Tracking**: Tracking of actual vs. estimated performance
- **Process Improvement**: Use of estimation accuracy data to improve future estimates
- **Organizational Learning**: Sharing of estimation lessons learned across projects