The total cost is calculated based on the number of times confirmation tests failed, which is indicated by bug reports being reopened.

- 80 bug reports were opened twice, meaning each failed confirmation testing once (80 1 = 80 failures).

- 10 bug reports were opened three times, meaning each failed confirmation testing twice (10 2 = 20 failures).

- Total failures = 80 + 20 = 100 failed confirmation tests.

- Total cost = 100 failures 3 person-hours/failure = 300 person-hours.

In the Cost of Quality model, costs associated with defects found before delivery to the customer are classified as internal failure costs. Since system testing is an internal pre-release activity, the cost of rework and re-testing falls into this category.

B. The cost calculation is incorrect. Also, these are internal failure costs, not external, as the defects were found before release to the customer.

C. The cost calculation is incorrect. The total number of failed tests is 100 (801 + 102), not a number that would result in 340 person-hours.

D. This is an internal failure cost, not a detection (appraisal) cost. Detection costs are for the initial testing effort, whereas failure costs are for rework and re-testing.

---

1. Pressman, R. S., & Maxim, B. R. (2020). Software Engineering: A Practitioner's Approach (9th ed.). McGraw-Hill. In Chapter 15, Section 15.2.2 "The Cost of Quality," internal failure costs are defined as those incurred to correct defects discovered before the product is shipped to the customer, explicitly including "rework" and "failure analysis." The scenario describes exactly this situation.

2. Tian, J. (2005). Software Quality Engineering: Testing, Quality Assurance, and Quantifiable Improvement. Wiley-IEEE Computer Society Press. In Chapter 3, Section 3.3.2 "Cost of Quality," the Prevention-Appraisal-Failure (PAF) model is detailed. It classifies costs associated with defects found before shipment, such as debugging and re-testing, as internal failure costs.

3. MIT OpenCourseWare. (2010). 6.033 Computer System Engineering, Spring 2010. Lecture 20: Quality Assurance. The lecture notes distinguish between the costs of bugs found at different stages, reinforcing that costs from defects found during internal testing phases (like system testing) are distinct from and lower than external failure costs (post-release). This supports the classification as an internal, not external, failure.

The costs of quality are categorized into four types: prevention, detection (appraisal), internal failure, and external failure. Detection costs are incurred to find defects before the product is delivered to the customer. Writing test specifications is a fundamental activity in preparing for testing. Since the purpose of testing is to detect defects, the costs associated with creating the necessary artifacts for testing, such as test specifications, are classified as detection costs. These are costs of appraising the quality of the software.

B. Training developers is a prevention cost, as it aims to prevent defects from being introduced in the first place by improving skills.

C. Re-running a test to verify a fix is an internal failure cost, as it is a cost incurred due to a defect that was already found internally.

D. Fixing field failures is an external failure cost, as it relates to defects discovered by the customer after the product has been released.

---

1. International Software Testing Qualifications Board (ISTQB®). (2018). Certified Tester Foundation Level Syllabus (Version 2018 V3.1). In Section 1.2.2, "Costs of Quality," detection costs (also called appraisal costs) are defined as costs for assessing the quality of the system. The syllabus provides examples such as "reviewing" and "testing," for which writing test specifications is a prerequisite activity. Conversely, "training" is listed as a prevention cost, "re-work" (including re-testing) as an internal failure cost, and "help desk" (related to field failures) as an external failure cost.

2. Pressman, R. S., & Maxim, B. R. (2020). Software Engineering: A Practitioner's Approach (9th ed.). McGraw-Hill Education. In Chapter 16, "Software Quality Assurance," the cost of quality model is detailed. Appraisal costs (detection costs) are described as activities to "uncover errors before they are delivered to the customer" (p. 453). This includes all tasks associated with testing, such as test planning and test case design (which includes writing specifications).

3. Sommerville, I. (2016). Software Engineering (10th ed.). Pearson. In Chapter 24, "Quality Management," the costs of quality are discussed. Appraisal costs are defined as "costs of the activities that are planned to discover defects in the software and system" (p. 728). This directly encompasses the planning and specification activities of the testing process.

The provided description outlines the exit criteria for a review process, focusing on the satisfactory resolution of defects (X2) and proper configuration management (X3). While these are valid components, a defining characteristic of a formal review is its focus on process improvement. This is achieved by collecting and analyzing metrics during the follow-up phase. These metrics (e.g., number of defects found, review effort in hours, defects per page) are crucial for evaluating the effectiveness and efficiency of the review process itself, not just the quality of the product under review. The description completely omits this data collection and analysis aspect, which is a mandatory activity in formal review methodologies like Fagan inspections.

A. Defined entry and exit criteria: This is incorrect because the description explicitly lists the conditions (X1, X2, X3) that must be fulfilled for the review process to terminate, which are the exit criteria.

B. Checklists to be used by the reviewers: This is incorrect because checklists are typically used during the individual preparation and review meeting phases, not the follow-up phase which is the focus of the description.

C. Deliverables such as reports, evaluation sheets or other review summary sheets: This is incorrect because the mention of "all the identified defects" (X2) implies the existence of a primary deliverable, which is the defect list or log.

1. IEEE Std 1028-2008, "IEEE Standard for Software Reviews and Audits."

Section 5.5.3, "Output" (for Inspections): This section specifies the required outputs of a formal inspection. It explicitly lists an "Inspection data report, including the number of participants, the time spent on the inspection, and the total number of anomalies found." This highlights the requirement for collecting metrics, which is missing from the question's description.

Section 5.5.2.6, "Follow-up": This section states, "The moderator shall ensure that the inspection data is recorded." This confirms that data collection (metrics) is a key responsibility during the follow-up phase.

2. Wiegers, K. E. (2002). Peer Reviews in Software: A Practical Guide. Addison-Wesley Professional.

Chapter 8, "Analyzing Review Data": This chapter details the importance of collecting and analyzing metrics from reviews. It states, "The follow-up stage is when the moderator or review leader verifies that the necessary changes were made correctly... The review leader also should collect the review metrics at this time." (Paraphrased from concepts in the chapter). This directly links metrics collection to the follow-up phase.

3. Carnegie Mellon University, Software Engineering Institute (SEI). (1996). A 'Personal' Software Process (PSP) for 'Team' Software Process (TSP) (CMU/SEI-96-TR-001).

Section: The Inspection Process: The SEI's work on PSP and TSP heavily incorporates formal inspection methods derived from Fagan's work. The process description emphasizes recording data not only on defects but also on the time spent in each review phase. This data is then used to calculate metrics for process effectiveness and to improve future planning and quality efforts. The absence of this data collection step in the question's scenario is a significant omission for a formal process.

The evaluation of the specified exit criteria requires information that directly measures the state of test coverage, test execution, and outstanding defects.

EX1 (Risk Coverage): Requires a traceability matrix (A) to confirm that all risk items are linked to test cases.

EX2 (Execution): Requires the execution status of all test cases (D) to confirm they have all been run.

EX3 (Defects): Requires a list of open defects with their classification (B) to confirm no critical defects remain.

The trend in lag time for defect resolution (C) is a process metric that measures the efficiency of the development and testing teams. While valuable for project management and process improvement, it does not determine if the specific, state-based exit criteria have been met.

A traceability matrix (A) is essential to verify that the risk coverage criterion (EX1) has been met.

A list of open defects (B) is essential to verify that the critical defect criterion (EX3) has been met.

The execution status of test cases (D) is essential to verify that the test execution criterion (EX2) has been met.

1. ISTQB® Certified Tester Foundation Level Syllabus (Version 2018 V3.1). Section 5.2, "Test Monitoring and Control," lists common test metrics used to evaluate exit criteria, including test case execution status, defect information (number, status, severity), and test coverage. It does not list defect resolution time as a metric for this purpose. Section 5.3, "Test Reporting," confirms that test summary reports assess the testing against the exit criteria.

2. Pressman, R. S., & Maxim, B. R. (2020). Software Engineering: A Practitioner's Approach (9th ed.). McGraw-Hill. Chapter 21, "Software Testing Strategies," discusses test completion criteria. These are defined by the state of testing, such as achieving a certain coverage level or ensuring all high-severity errors are corrected. Metrics concerning the rate or time to fix defects are treated as project velocity indicators, not as direct measures of test completion against such criteria. (See Section 21.2.2).

3. Kaner, C., Falk, J., & Nguyen, H. Q. (1999). Testing Computer Software (2nd ed.). John Wiley & Sons. Chapter 2, "A Software Testing Framework," describes exit criteria as specific goals related to the product's state, such as the number and severity of known bugs and the extent of test coverage (p. 34). Chapter 13, "Reporting and Analyzing Bugs," discusses defect lag time as a metric for analyzing the bug-fixing process itself, not for determining if testing is complete (p. 258).

The Systematic Test and Evaluation Process (STEP) model, developed by William C. Hetzel, fundamentally integrates testing activities into the software development lifecycle from the very beginning. A core principle of STEP is that testing and development are parallel, not sequential, activities. Therefore, as requirements and use cases are being defined and specified, corresponding test cases are designed to validate them. This early design of tests ensures that requirements are testable, clear, and complete, effectively using the testing process to prevent defects from being introduced into later stages.

B. Defect prevention is a primary goal of the STEP model, not a secondary one. By validating requirements and design early, STEP aims to find and fix errors before they become code defects.

C. The STEP model explicitly requires that the requirements specification and test design processes overlap. Test design begins during the requirements phase to provide early feedback and validation.

D. This describes a traditional waterfall approach. In the STEP model, testware design begins in parallel with the requirements and system design phases, long before coding is completed.

---

1. Hetzel, W. C. (1988). The Complete Guide to Software Testing (2nd ed.). QED Information Sciences. In Chapter 3, "A System for Testing," Hetzel introduces the STEP model, stating, "Test development is carried out in parallel with software development... Test requirements are defined when the software requirements are defined." (p. 38). Figure 3.2 on p. 39 explicitly shows "Test Design" occurring in parallel with "System Requirements."

2. Burnstein, I. (2003). Practical Software Testing: A Process-Oriented Approach. Springer-Verlag. In Chapter 2, "A Software Quality Program," the text discusses lifecycle models that integrate testing. It describes models like STEP as promoting "a set of testing activities that are planned and managed and that are integrated into the software life cycle." This integration starts at the requirements phase to support verification and validation activities (Section 2.3.2).

3. Utting, M., & Legeard, B. (2007). Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann. The book discusses the evolution of testing processes, referencing early lifecycle integration models. It notes that models like STEP were developed to overcome the waterfall limitation of late-stage testing by initiating test design and planning "as soon as the requirements are being specified" (Chapter 1, Section 1.3.2).