A common misconception is that Load, Stress and Performance Testing mean the same. Well, they are not. There are subtle yet distinctive differences between each of these methods of testing, their purpose and the audience they target. I’m not a tester, so I won’t be going into the nuts and bolts of each of these methods of testing, rather I would like to present a very high level view on this subject that developers can relate to (and hopefully avoid confusion that I found myself in…).

Load Testing (Volume Testing)

• The primary objective of Load Testing is to evaluate whether the hardware infrastructure AND the application architecture will support the expected user load and concurrency.
• Load Testing is usually done by automated tools that simulate virtual users for a recorded sequence of user activity.
• The objective of Load Testing is not to break the system but find a break-point beyond which the hardware infrastructure and/or the Application can no longer support the expected user load and concurrency.
• Response time is not a significant benchmark of load testing (as long as the request does not time-out). Especially in case of web applications, the response time reported by most load test tools is misleading. This is because when the tools record user activity, they capture the http request and response and not the browser activity. This means the client side scripts is never executed by the load test tool; consequently the time taken to execute the client side script is never factored in. Response time reported by most load test tools is merely the time difference between http response and http request.
• Key indicators monitored during load testing are CPU and Memory Utilization, I/O Reads and Network throughput.
• The primary audience of Load Testing is the Infrastructure and the Software Architecture team. It helps them estimate the Capacity Planning.

Stress Testing

• The primary objective of Stress Testing is to evaluate the gracefulness of the application under severe duress.
• The only way to Stress Test is to fail the application (unlike Load Testing).
• Usually extreme forms of negative testing are employed to simulate stress tests. For e.g. running the load test beyond break-point, taking the database offline when the requests are submitted, disabling server ports.
• The key indicator that is monitored is the response to unexpected event like Denial of Service (DoS) attack. Does the application display cryptic messages or meaningful messages? Is the shutdown graceful or does the application freezes? Does the system save state during an unexpected event and recovers to the last good state?
• The primary audience of Stress Testing is Software Development team and in particular relates to exception handling techniques.

Performance Testing

• Performance Testing is similar to load testing; with the main difference being the key metric that is monitored. In Performance Testing response time of the application from the end-user perspective is the key metric. This includes BOTH server and client side response times for the expected user load and concurrency.
• Performance Testing is usually done by combination of automated tools and manual testing. Usually a 10% rule of thumb is followed to estimate the number of concurrent users. This means if the expected user base is 1000, only 10% (100 current users) are using the application at any point in time and only 10% of the current users use a particular functionality at the same time (10 concurrent users).
• Performance Testing involves Infrastructure, Architecture and Development teams as bottlenecks could occur at Hardware, Network, Database, OS or the Application as such.

Despite these differences, there is one common ground between these three methods of testing. The objective of Load, Stress and Performance testing is not to find bugs; rather it’s a proactive approach to identify and address bottlenecks before the software gets released to the user base.

Benny Austin

http://twitter.com/#!/BennyAustin/status/7334231913