Load testing is a type of performance testing designed to evaluate the behavior of a system under varying loads. A primary objective of load testing is to observe how the system's performance metrics (e.g., response time, throughput) change as the load is progressively increased. This directly measures the system's ability to handle these increasing levels, typically from low to anticipated peak usage, to identify performance bottlenecks and ensure it meets specified performance objectives.

B. Elevation testing: This is not a standard or recognized term within the domain of performance testing. It is likely a distractor term, possibly confused with "privilege escalation" from security testing.

C. Spike testing: This test evaluates a system's ability to handle sudden, extreme, and short-term increases in load, not a gradual and sustained increase.

D. Endurance testing: Also known as soak testing, this evaluates a system's stability under a continuous, expected load over a long duration, checking for issues like memory leaks, not its response to increasing load.

1. ISTQB® Glossary of Testing Terms (Version 4.0):

Load Testing: "A type of performance testing conducted to evaluate the behavior of a component or system under varying loads

usually between anticipated low

typical

and peak usage." This definition's inclusion of "varying loads" directly supports the concept of testing with increasing levels of load.

Spike Testing: "A type of performance testing focused on a component's or system's ability to handle sudden

short-term

and extreme increases or decreases in load."

Endurance Testing (Soak Testing): "A type of performance testing conducted to evaluate the behavior of a component or system when subjected to a high load for a significant period of time."

Direct URL: https://glossary.istqb.org/

2. Koomen

T.

van der Aalst

L.

Broekman

B.

& Vroon

M. (2006). TMap Next: For result-driven testing. UTN Publishers.

This foundational text on structured testing describes load testing as a method to determine the performance of a system under different load conditions

which inherently involves testing at various

often increasing

load levels to find the system's limits or performance profile. (Referenced in ISTQB syllabi).

3. Pressman

R. S.

& Maxim

B. R. (2020). Software Engineering: A Practitioner's Approach (9th ed.). McGraw-Hill Education.

Chapter 22

"Software Testing Strategies

" distinguishes between different performance tests. It describes load testing as executing a system in a manner that demands resources in increasing amounts

thereby aligning with the question's premise.

A fundamental principle of any testing activity, including performance testing, is that it must be practical and provide value within the project's constraints. Performance tests must be designed, executed, and analyzed within the available project timeframe to provide timely feedback for decision-making. If tests take too long to run, the results may become irrelevant as the system under test will have already evolved, making the feedback obsolete. This aligns with the core project management aspect of testing.

B. Test results must be comparable to tester expectations. This is imprecise. Results are compared against pre-defined, objective performance requirements and criteria, not a tester's subjective expectations.

C. Test results must vary when the tests are run repeatedly on an unchanged system. This is incorrect. A key principle is repeatability; results should be consistent and reproducible on an unchanged system to be considered reliable.

D. Tests must align with the ideal outcome of stakeholders. This is too vague. Testing aligns with specific, measurable, and agreed-upon performance requirements derived from stakeholder needs, not their potentially unrealistic "ideal" outcomes.

1. ISTQB® Certified Tester Performance Testing Syllabus (2018): Section 2.4

"Managing a Performance Test Project

" emphasizes the role of the performance test manager in "creating a realistic schedule for performance testing activities" and "tracking the progress of the performance testing activities against the schedule." This directly implies that tests must be executable within the project timeframe to be manageable and effective.

2. The Art of Software Testing

3rd Edition (G. J. Myers

C. Sandler

T. Badgett): While a general testing book

it establishes the universal principle that testing operates under constraints. Chapter 2 discusses the economics of testing

noting that "you can't test everything" and that testing efforts must be planned and budgeted

which inherently includes time constraints. This principle applies directly to specialized testing like performance testing.

3. A Practical Guide to Testing in DevOps (K. E. Block): In the context of modern development

this guide emphasizes the need for fast feedback loops. Chapter 5

"Performance Testing

" discusses integrating performance tests into the CI/CD pipeline

which requires tests to be "fast enough to provide timely feedback." This reinforces the principle that test execution time is a critical constraint.

Monitoring is an integral part of the performance test execution phase. It is performed simultaneously with the application of the test load to observe the System Under Test's (SUT) real-time behavior. This concurrent monitoring allows testers to correlate the generated load with key performance indicators (KPIs) such as response time, throughput, and resource utilization (e.g., CPU, memory). This immediate feedback is crucial for identifying performance bottlenecks and understanding how the system behaves under specific stress conditions as they occur.

A. When testing is concluded: This is incorrect. Monitoring after the test is for post-test analysis of logs and collected data, not the active monitoring process during the load test itself.

B. Before testing execution: This is incorrect. While establishing a baseline by monitoring the system at rest before the test is a common and valuable practice, the primary monitoring activity occurs during the test to see the effect of the load.

C. Immediately after test execution: This is incorrect. This timing is associated with analyzing results and observing system recovery, not with monitoring the system's performance under the applied load.

1. ISTQB® Certified Tester Performance Testing Syllabus v2.0 (2018): Section 4.3.1

"Performance Test Execution

" states: "During test execution

the test environment and the SUT are monitored to check for any unexpected behavior and to ensure that the test environment itself is not the cause of any performance issues." This explicitly places monitoring during the execution phase. (Source: ISTQB® Official Syllabus).

2. Jain

R. (1991). The Art of Computer Systems Performance Analysis: Techniques for Experimental Design

Measurement

Simulation

and Modeling. Wiley. Chapter 3

"Measurement Techniques and Tools

" discusses that monitoring tools are used to observe system behavior while workloads are running to collect performance data. This foundational text establishes the principle of concurrent measurement and load application. (Page 35).

3. Weyuker

E. J.

& Vokolos

F. I. (2000). Experience with performance testing of software systems: issues

an approach

and case study. IEEE Transactions on Software Engineering

26(12)

1147-1156. The paper's methodology describes the test execution phase where "the system is stressed by a driver that simulates a specified number of users... While the system is being stressed

performance data is collected." This confirms that data collection (monitoring) is simultaneous with load generation. (Section IV. A. Test Execution).

REST (Representational State Transfer) is a widely adopted architectural style for designing networked applications and web services. It is not a protocol in the strictest sense but a set of constraints and principles that use standard protocols like HTTP. In the context of web services, REST is considered a primary approach, often contrasted with the SOAP protocol. It leverages standard HTTP methods (GET, POST, PUT, DELETE) to create stateless, scalable, and maintainable web services. Its simplicity and alignment with the web's architecture have made it a common choice for modern APIs.

A. SOAK: This is not a recognized web service protocol. It is likely a typographical error for SOAP (Simple Object Access Protocol), which is a valid, albeit different, web service protocol.

C. HTTP: HTTP (Hypertext Transfer Protocol) is the foundational application-layer protocol that RESTful web services use for communication. It is the transport mechanism, not the web service architectural style itself.

D. RTE: This is not a standard acronym for a web service protocol. It may refer to concepts like "Runtime Environment," which are unrelated to web service communication standards.

1. Fielding

R. T. (2000). Architectural Styles and the Design of Network-based Software Architectures. Doctoral dissertation

University of California

Irvine. Chapter 5 defines REST as an architectural style. Retrieved from https://www.ics.uci.edu/~fielding/pubs/dissertation/restarchstyle.htm

2. W3C (World Wide Web Consortium). (2004). Web Services Architecture. W3C Working Group Note. Section 1.3.2 discusses REST as a prominent architectural style for web services. Retrieved from https://www.w3.org/TR/ws-arch/#id2260802

3. Pautasso

C.

Wilde

E.

& Alarcon

R. (2014). REST: Advanced Research Topics and Practical Applications. In Web Engineering (pp. 1-11). Springer

Berlin

Heidelberg. This paper discusses REST as a "lightweight alternative to SOAP/WSDL/WS- based Web services." (Page 1).

Mainframe architectures are fundamentally designed for high-volume, I/O-intensive workloads, such as online transaction processing and batch processing. Consequently, the performance of the I/O subsystem is a critical factor and a common source of bottlenecks. Slow performance of storage devices, historically Direct Access Storage Devices (DASD), directly impacts the system's ability to meet its processing objectives. I/O response time is a key metric in mainframe performance analysis, making slow storage speed a typical and significant performance risk inherent to this architecture.

A. Inconsistent loads when accessed via APIs: This is a workload-related risk that can affect any type of server architecture, not one that is uniquely typical for mainframes.

C. Misconfigured host machine: Misconfiguration is a universal operational risk applicable to any computing platform and is not a risk specific to mainframe architecture itself.

D. Lack of RAM: Insufficient memory is a generic resource constraint that degrades performance on any computer system, not a risk characteristic of mainframes in particular.

1. IBM Redbooks. ABCs of z/OS System Programming Volume 2

SG24-6982-00. Chapter 5

"I/O processing in z/OS

" details the critical nature of the I/O subsystem

including DASD

and its direct impact on system performance. It states

"The I/O subsystem is a major component of a z/OS system. Its performance is critical to the overall performance of the system." (Page 137). [Direct URL: https://www.redbooks.ibm.com/redbooks/pdfs/sg246982.pdf]

2. Živkovič

A.

& Šumak

B. (2011). Performance Analysis of Mainframe Computer Systems. International Journal of Electrical and Computer Engineering Systems

2(2)

61-68. This academic paper discusses mainframe performance

noting that "The I/O subsystem is one of the most complex parts of the mainframe computer system... The most important performance measures for DASD devices are response time

I/O rate

device utilization..." (Section 3.2

p. 64). [Direct URL: https://hrcak.srce.hr/file/116185]

3. University of Arkansas

Walton College of Business. ISYS 5503: Mainframe Computing

Course Notes. Course materials frequently emphasize the I/O-centric nature of mainframes and the importance of storage (DASD) performance tuning as a core discipline in mainframe systems management.

Oversaturation of system resources occurs when the demand for a resource (e.g., CPU, memory, network bandwidth) exceeds its available capacity. According to fundamental principles of performance engineering and queuing theory, when a resource becomes saturated, incoming requests can no longer be processed immediately. They are forced to wait in a queue, leading to a significant and often non-linear increase in the time it takes to receive a response. This phenomenon is a primary focus of load and stress testing, where the goal is to identify the point at which performance degrades due to resource bottlenecks.

A. Response time will gradually improve over a period of time: This is incorrect. Resource saturation causes performance degradation, meaning response times increase (get slower), not improve.

C. Error logging will decrease: This is incorrect. Resource oversaturation often leads to system failures like timeouts or crashes, which would cause error logging to increase.

D. The size of the database will gradually decrease: This is incorrect. This is not a direct or typical consequence of resource saturation; high load is more likely to increase database size through transaction logging and data creation.

1. ISTQB® Certified Tester Performance Testing Syllabus v1.0.1 (2023)

Section 2.3.1

"Performance Test Types." The syllabus describes stress testing as a method to "evaluate how the system behaves when it is pushed beyond its specified limits." A key behavior observed under such conditions is performance degradation

including increased response times.

2. Jain

R. (1991). The Art of Computer Systems Performance Analysis: Techniques for Experimental Design

Measurement

Simulation

and Modeling. Wiley. Chapter 31

"Introduction to Queuing Theory

" explains that as the utilization of a server (a system resource) approaches 100% (saturation)

the queue length and waiting time for requests increase exponentially

directly causing slower response times. This is a foundational academic principle in computer performance analysis.

3. University of Toronto

ECE1770: "Performance Modeling of Computer Systems" Course Notes. These and similar university course materials on system performance explain that a system's response time (R) is a function of its service time and waiting time. As resource utilization (ρ) approaches 1

waiting time increases dramatically

thus increasing the overall response time. (e.g.

R = S / (1 - ρ) for an M/M/1 queue).

The Test Implementation phase is where the conceptual test designs are transformed into concrete, executable test assets. According to the ISTQB Performance Tester syllabus, this activity involves developing the test scripts that simulate user interactions. These individual scripts are explicitly referred to as the "building blocks" for the larger, more complex performance test scenarios that will be run during the execution phase. They are the foundational components upon which comprehensive load tests are constructed.

A. During test completion: This phase involves finalizing reports and archiving testware after execution is finished; no new test assets are created.

B. During test planning: This is a strategic phase focused on defining objectives, scope, and approach, not on creating tangible test scripts.

C. During test design: This phase focuses on defining test conditions and the high-level structure of tests, but not on the creation of the actual executable scripts.

1. ISTQB® Certified Tester Performance Tester Syllabus (Version 2018):

Section 3.2.4

Test Implementation: "During test implementation

the performance tester develops the test scripts that will be used to simulate the load on the system under test (SUT). These scripts are the building blocks for the performance tests." (Page 16)

This source directly uses the term "building blocks" to describe the test scripts created during the implementation phase

confirming it as the correct answer.

URL: https://www.istqb.org/certifications/performance-tester (Link to the official syllabus download page).

2. ISTQB® Certified Tester Foundation Level Syllabus (Version 2018 v3.1):

Section 1.4.3

Test Implementation: "Test implementation is the activity where test procedures or scripts are specified by combining the test cases in a particular order and including other information needed for test execution..." (Page 13)

This foundational syllabus establishes that creating scripts and test procedures is the core activity of the implementation phase

which aligns with the concept of creating "building blocks".

URL: https://www.istqb.org/certifications/foundation-level (Link to the official syllabus download page).

For batch processing, the primary performance concerns are how much work can be completed in a given time and how long it takes for a job to finish. Throughput measures the rate of processing (e.g., transactions per hour), directly addressing the volume of work. Wait times, or more broadly, turnaround time (the total time from submission to completion), measures the job's duration. These two metrics are fundamental for evaluating the efficiency and capacity of a batch processing system.

B. Number of records deleted: This is a functional correctness metric that verifies what the job did, not a performance metric that measures how well or how fast it performed.

C. Average percentage of network latency: Network latency is a contributing environmental factor that can affect performance, but it is not a primary metric of the batch process itself, unlike throughput or completion time.

D. Number of concurrent virtual users: This metric is specific to load testing of online, interactive systems (e.g., web applications) and is irrelevant for non-interactive batch processing.

1. Jain

R. (1991). The Art of Computer Systems Performance Analysis: Techniques for Experimental Design

Measurement

Simulation

and Modeling. Wiley. On page 37

Section 3.2

"Criteria for Selecting an Evaluation Technique

" key performance metrics are defined. For batch jobs

throughput (jobs per second) and turnaround time (time between submission and completion

which includes wait time) are listed as key metrics.

2. Lilja

D. J. (2000). Measuring Computer Performance: A Practitioner's Guide. Cambridge University Press. Chapter 2

"Performance Metrics

" discusses common metrics. Throughput is defined as the amount of work done per unit time

and response time (or turnaround time for batch systems) is the time to complete a task. These are presented as fundamental performance measures. (See pages 13-15).

3. Menascé

D. A.

& Almeida

V. A. (2008). Performance by Design: Computer Capacity Planning by Example. Prentice Hall. Chapter 3

"Fundamental Concepts and Laws

" defines key performance metrics including response time

throughput

and utilization

which are universally applied to evaluate system performance

including batch systems.

In performance testing, a realistic workload model must simulate the behavior of actual users. The total time for a user to complete an interaction cycle consists of two main components: the system's response time to a request and the time the user pauses before initiating the next request. This pause, which represents the user reading the screen or deciding on the next action, is known as think time. Therefore, to accurately model the total time and throughput of user transactions, think time must be included in test scripts in addition to measuring the transaction response time.

B. Wait time: This term is ambiguous. It can refer to time spent in a server-side queue (which is part of the response time) or be used as a general term for any delay. "Think time" is the precise, industry-standard term for the simulated user pause between transactions.

C. Action time: This is not a standard term in performance testing. The time taken by the client to send a request is typically negligible or considered part of the overall transaction response time.

D. User time: This term is too general and is not a specific, measurable metric used in the context of performance test scripting and analysis.

1. ISTQB® Certified Tester

Performance Testing Syllabus (2018):

Section 3.2.2

Workload: "The workload model should also include think times. Think time is the time that a real user would take between receiving a response from one request and submitting the next request."

Glossary

p. 49: "Transaction response time: The time that elapses from the moment a user executes a request until the response for that request has been received by the user."

URL: https://www.istqb.org/certifications/performance-testing (Link to the official page where the syllabus can be downloaded).

2. ISTQB® Certified Tester

Advanced Level Syllabus

Test Analyst (2019):

Section 4.3.2

Performance Test Execution: "The think time is the time a user is expected to wait between transactions. For example

a user may read a page for 30 seconds before clicking the next link. This is the think time."

URL: https://www.istqb.org/certifications/test-analyst (Link to the official page where the syllabus can be downloaded).

3. Jain

R. (1991). The Art of Computer Systems Performance Analysis: Techniques for Experimental Design

Measurement

Simulation

and Modeling. Wiley.

Chapter 3

Section 3.3

Types of Workloads: This foundational text discusses the components of a workload

including the requests sent to a system and the timing between them

establishing the concept of think time as a critical parameter for realistic load simulation. (While a general reference

it underpins the principles used in the ISTQB syllabus).

Protocol-level scripts are designed to simulate the network traffic that an application generates when communicating with a server. By operating directly at the communication protocol layer (e.g., HTTP, TCP/IP), these scripts bypass the client-side Graphical User Interface (GUI). This allows them to precisely measure the time it takes for a request to travel to the server, be processed, and for the response to return. This measurement is a direct reflection of network latency and server processing time, which together constitute the network response time, without the overhead of client-side rendering.

A. GUI scripts: These measure the total end-to-end user-perceived response time, which includes significant client-side processing and GUI rendering time, making them imprecise for isolating network response time.

B. API scripts: While they operate closer to the network than GUI scripts, they are a specific implementation of protocol-level scripting. "Protocol-level scripts" is the more accurate and encompassing category.

C. HTTP scripts: Similar to API scripts, HTTP scripts are a specific type of protocol-level script. The term "Protocol-level scripts" is the correct general classification for scripts that measure network interactions.

1. ISTQB® Certified Tester Performance Testing Syllabus (Version 2018): Section 3.2.2

"Scripting

" distinguishes between the two main types of scripts: "Protocol-level scripts simulate the network traffic that is generated by the client application when communicating with the server... The response time measured by a protocol-level script does not include the time it takes for the client to render the GUI." This directly supports that protocol-level scripts are used for measuring network-centric timings.

2. Jiang

Z. M.

& Hassan

A. E. (2015). A Survey on Load Testing of Large-Scale Software Systems. IEEE Transactions on Software Engineering

41(11)

1091-1118. This survey discusses load generation techniques

noting that protocol-level request generation is a primary method. It states

"Workload generators can generate requests at the protocol level (e.g.

HTTP requests)..." (Section III-A). This confirms that protocol-level is the fundamental approach for simulating network requests to measure performance.

3. Courtois

J. (2012). Performance Testing Guidance for Web Applications. Microsoft Patterns & Practices. While from a vendor

this is a foundational text. Chapter 5

"Web Application Performance Testing Core Concepts

" explains: "Most performance-testing tools work at the protocol level... This means that the tool itself

rather than a browser

generates the HTTP requests." This highlights that protocol-level interaction is the standard for performance measurement tools.