Evaluation criteria for selecting Load /Performance Testing tool

As a performance test engineer you need to consider below key points while selecting the appropriate performance testing tool to provide the accurate results to the customer.

Evaluation Criteria:

Before starting with any performance testing it is very common to evaluate various tools commercial and free tools available in the market and choose the best suited one for the application under test. The following are the criteria to be considered while evaluating tools for performance testing and a methodology for evaluating the tools.

ü  Ease of Script development/enhancement

Ø  Load testing tool should have a flexibility for developing script easily and enhancing the script.

ü  Protocol Support:

Ø  Load testing tools either emulate load or simulate users. Simulation involves duplication of actual user activity on the GUI/front-end with the intention of replaying it again. Emulation involved protocol replay - for this a rich API of protocol-related functions are required.

Ø  A good tool will not be restricted to a single protocol and should support multiple protocols including HTTP/S, FTP, SMTP, Oracle NCA, DB2 CLI, Citrix ICA, SAP, WAP, Voice XML, Peoplesoft, and SIEBEL.

ü  Record & Playback support :

Ø  This category details how easy it is to record & playback a test.

Ø  Is there object recognition when recording and playing back or does it appear to record ok but then on playback (without environment change or unique id’s, etc changes) fail?

Ø  How easy is it to read the recorded script?

ü  Cost of Licensing - Tool should not be expensive and should have a flexible licensing option.

ü  Strong scripting language :

Ø  The scripting language supported by the automated tool should be understandable and precise. It should not generate lengthy codes which are difficult to maintain. It should resemble to common languages like C, C++, and Java etc. It should be easy to debug.

Evaluation Procedure:

Select the tools that you want to consider for evaluation. The most common load testing tools are HP Loadrunner, JMeter, Web Load and Neo Load and etc…

For each of the tool considered for evaluation identify the pros and cons with respect to each of the criterion listed above.

Give a score for each of the tools for each of the criterion.

Prepare a Score card of all the tools for each of the criterion considered. 

Recommend the best suited tool to the customer.