WinRunner and QAWizard Comparison
I conducted a comparison a little while ago for the following versions of the automation tools:
• Mercury WinRunner 8.20 Build 8520
• QA Wizard 2007.0.0 Build 2358
1. SUPPORTED LANGUAGE:
a. Mercury WinRunner:
It makes use of its own C-like test script language known as TSL. No other language
support is provided. TSL is easy to use & very user friendly. The scripts can either be
recorded or manually written or a combination of both can be done. Scripting in TSL is
quite similar to writing code in C language and is prone to syntactical errors.
b. QA Wizard:
QA Wizard makes use of its own proprietary scripting language. The language constructs
are somewhat similar to Visual Basic. But, it still requires some effort to get used to the
syntax of the scripting language. The scripts are viewable in text & grid views to provide experienced & novice programming experience, respectively. According to “Seapine’s WinRunner Comparison”, Users can do scripting without having to worry about
missing semicolons, parentheses or misspelling of variable or objects, I didn’t
encounter any such feature. The scripts written in QAWizard are prone to syntactical
errors.
2. EASINESS
a. Mercury WinRunner:
WinRunner is very easy to use because of its C-like syntax scripting language and ready
to use GUI/text capturing related functionality. In WinRunner, objects/items of AUT can
be added into the GUI Map using the GUI Map Editor. Moreover, text or bitmap present in
any object/item can be captured easily just through a point & capture during recording.
b. QA Wizard:
User has to get familiarized with the syntax of the scripting language in order to use it.
Making QA Wizard learn the objects of AUT is not very simple. I didn’t find any ready to
use text/bitmap capture features during the review.
3. FEATURE ENRICHMENT
a. Mercury WinRunner:
It is hard to say which one has more features. Mercury WinRunner has some features
which QA Wizard does not have and vice versa e.g. in WinRunner, AUT needs to be run
through script. Moreover, the objects (buttons, labels etc) stored during the recording are
mapped onto a GUI Map Editor and needs to be stored as a GUI MAP File. If the stored
GUI MAP file is lost, it’s a tough job to record everything all over again. But, it is very
easy to capture text/bitmap from any object/item.
b. QA Wizard:
QA Wizard has similar scenario e.g. the application repository of QA Wizard takes good
care of executing the AUT during a re-running of a recorded test. In application repository
the items along with their properties, repository value & expected value are also
available. But, capturing text/bitmap from objects/items is not simple.
4. TEST RECORDING
a. Mercury WinRunner:
It supports Test Recording. Scripts are readily written as actions are performed during
recording.
b. QA Wizard:
It supports Test Recording. Once the recording is completed, the QA Wizard recorder
builds the script for use in the editor. The time it takes to build is dependant upon the
duration of recording and is considered as a negative impact on user experience.
5. TEST RECORDING SPEED
a. Mercury WinRunner:
The user experience is not affected due to the ongoing recording.
b. QA Wizard:
The user experience is affected due to the ongoing recording.
6. TEST RE-RUNNING/EXECUTION
a. Mercury WinRunner:
It supports Test-rerunning & execution.
b. QA Wizard:
It supports Test-rerunning & execution.
7. ANALOG RECORDING
a. Mercury WinRunner:
It supports analog recording.
b. QA Wizard:
The analog recording is termed as low level action.
8. IMAGE CAPTURE
a. Mercury WinRunner:
It supports image capture but stores the captured images inside the project folder
invisible to user.
b. QA Wizard:
Every stored item’s screen shot is automatically stored & placed in Preview Pane.
9. TEST REPORT GENERATION:
a. Mercury WinRunner:
It supports executing a re-run in three modes:
i. Verify Mode: It keeps track of every test iteration & stores it in the project folder.
ii. Debug Mode: It does not keep track of actions & previous test iterations.
iii. Update Mode: Similar to verify mode.
The reports are viewable only in WinRunner Test Results or notepad.
b. QA Wizard:
There are no separate execution modes. As a test run is executed, all activities are logged
inside the workspace under Reports.
10. TRACK OF ALL PREVIOUS REPORTS
a. Mercury WinRunner:
It keeps track of all previous test attempts, if the test was run in Verify/Update Mode. All
test results are stored inside the project folder.
b. QA Wizard:
It also keeps track of all previous test attempts.
11. TEST SCIRIPT MANAGEMENT
a. Mercury WinRunner:
The test scripts are supposed to be managed manually. There is not built in solution to
script management.
b. QA Wizard:
The grid view of test scripts can be a better solution to script management. Since, it
displays the test scripts in divided columns of Action, Information, Control, Window, and
Description.
12. TEST SCRIPT MAINTENANCE
a. Mercury WinRunner:
Maintaining the test scripts written in WinRunner can be a difficult job in terms of
changing GUI of AUT. Every time the GUI of an AUT changes, all items in the GUI Map of this AUT need to be updated correspondingly. Otherwise, the test scripts can run.
b. QA Wizard:
According to “Seapine’s WinRunner Comparison”, QA Wizard’s update feature can
automatically update the Previewer images. All of the object attributes are also
captured, allowing users to see any possible changes to the objects. During the
review, I did not come across any such thing.
13. CODING STRATEGIES
a. Mercury WinRunner:
In WinRunner, coding can be done manually, using recording or with the help of Function
Generator.
b. QA Wizard:
In QA Wizard, coding can be done in a manual fashion, using recording or using the Add
Statement feature. Though, grid and text views are also available.
14. DATABASE CONNECTIVITY:
a. Mercury WinRunner:
It supports Database connectivity.
b. QA Wizard:
It supports Database connectivity.
15. BUILT IN FUNCTIONS:
a. Mercury WinRunner:
A large number of built in functions are available in Mercury WinRunner. These functions
are related to ActiveX controls, Arithmetic, Database, Drag drop, GUI configuration, GUI
Verification etc. Such specific functions are very useful and most of the times fulfill the
needs. All scripts currently written in TSL for Validity SDK make an excessive use of these
functions. Moreover, further more are also available freely on the internet.
b. QA Wizard:
The built in library of functions is not very huge. It has a very limited number of functions
as demonstrated in its help.
16. SPECIFIC FUNCTIONS RELATED TO GUI MATCHING & STRINGS:
a. Mercury WinRunner:
WinRunner supports capturing a portion of screen or an object, which ultimately helps
reading that object again when test is run.
b. QA Wizard:
No such feature was observed in QA Wizard.
17. LOGGING
a. Mercury WinRunner:
Specific functions for logging are provided in Mercury WinRunner.
b. QA Wizard:
Specific functions for logging are also provided in QA Wizard.
18. SUPPORTED TESTING TECHNIQUES
a. Mercury WinRunner:
Mercury WinRunner supports only web applications & windows applications testing
(functional & interface testing). It does not support load tests or unit tests.
b. QA Wizard:
QA Wizard also supports web and windows applications. There is no support for Load
Testing in QA Wizard.
19. EXECUTION IN BATCH MODE
a. Mercury WinRunner:
Execution of test scripts in batch mode can be done through programming.
b. QA Wizard:
Execution of test scripts in batch mode is provided through a separate feature.
20. WORKSPACE DESIGN AND LAYOUT
a. Mercury WinRunner:
It has Mercury’s own proprietary workspace design and layout.
b. QA Wizard:
The organization of scripts is similar to the Visual Studio standard.
21. BINDING
a. Mercury WinRunner:
No such feature is present. Although, it can be done through programming.
b. QA Wizard:
It supports binding to associate a child object with a parent object e.g. web pages often
contain objects that repeat, such as multiple Buy Buttons or check boxes. Alone, each of
these attributes may appear identical. Associating the Buy button or check box with one
of its parent objects, makes the object unique. This is a very power QA Wizard feature.
22. ACCESS INTERNAL OBJECTS, METHODS AND PROPERTIES OF APPLICATIONS:
a. Mercury WinRunner:
Feature not available in Mercury WinRunner.
b. QA Wizard:
Once added in the application repository, the properties of added items/objects are
accessible easily. Only properties are accessible, not the functions
23. IDENTIFICATION OF STORED OBJECTS
a. Mercury WinRunner:
Objects/Items referred in WinRunner scripts are mapped onto respective GUI Map files.
Every item being referred in the scripts must be present in its GUI Map file for
identification. In WinRunner, GUI Map file can be considered as a single point of failure
entity. If GUI Map of a test script is lost, all items referred in the scripts need to be
identified again to store them in its GUI Map. This can be a hectic activity.
b. QA Wizard:
Each object is identified using the object attributes instead of the object location. This way
of searching ensures objects will be found, even if websites/application title change or are
dynamically generated.
24. HELP/USER’S GUIDE
a. Mercury WinRunner:
The learning curve of Mercury WinRunner is very steep because of its comprehensive
user’s guide & TSL Online Reference. Moreover, lots of online stuff is available on the web to learn.
b. QA Wizard:
I found the user’s guide/help of QA Wizard less informative and poorly administered. It is
very difficult to find out solutions to problems using the QA Wizard help.
25. AVAILABLE AUTOMATION FRAMEWORKS
a. Mercury WinRunner:
A number of automation frameworks are available on the internet for free of cost like
WRSAFS (WinRunner Software Automation Framework Support), SAFS (Software
Automation Framework Support).
Software automation frameworks reduce the effort of changing the written script every
time, the GUI of an application under test changes.
b. QA Wizard:
I have not encountered any such automation framework related to QA Wizard yet.
26. SOURCE CONTROL SUPPORT
a. Mercury WinRunner:
No support for Source Control.
b. QA Wizard:
It provides support for source control available.
27. INTEGRATION WITH OTHER TOOLS
a. Mercury WinRunner:
Mercury WinRunner can be easily integrated with QTP or Quality Center.
b. QA Wizard:
TestTrackPro is fully integrated with QA Wizard. The application contains a TestTrackPro
toolbar that lets users easily log in to TestTrackPro.
28. LICENSING SUPPORT
a. Mercury WinRunner:
All features are unlocked by a single license.
b. QA Wizard:
OCR (Optical Character Recognition) support requires separate license & is on per user basis.
• Mercury WinRunner 8.20 Build 8520
• QA Wizard 2007.0.0 Build 2358
1. SUPPORTED LANGUAGE:
a. Mercury WinRunner:
It makes use of its own C-like test script language known as TSL. No other language
support is provided. TSL is easy to use & very user friendly. The scripts can either be
recorded or manually written or a combination of both can be done. Scripting in TSL is
quite similar to writing code in C language and is prone to syntactical errors.
b. QA Wizard:
QA Wizard makes use of its own proprietary scripting language. The language constructs
are somewhat similar to Visual Basic. But, it still requires some effort to get used to the
syntax of the scripting language. The scripts are viewable in text & grid views to provide experienced & novice programming experience, respectively. According to “Seapine’s WinRunner Comparison”, Users can do scripting without having to worry about
missing semicolons, parentheses or misspelling of variable or objects, I didn’t
encounter any such feature. The scripts written in QAWizard are prone to syntactical
errors.
2. EASINESS
a. Mercury WinRunner:
WinRunner is very easy to use because of its C-like syntax scripting language and ready
to use GUI/text capturing related functionality. In WinRunner, objects/items of AUT can
be added into the GUI Map using the GUI Map Editor. Moreover, text or bitmap present in
any object/item can be captured easily just through a point & capture during recording.
b. QA Wizard:
User has to get familiarized with the syntax of the scripting language in order to use it.
Making QA Wizard learn the objects of AUT is not very simple. I didn’t find any ready to
use text/bitmap capture features during the review.
3. FEATURE ENRICHMENT
a. Mercury WinRunner:
It is hard to say which one has more features. Mercury WinRunner has some features
which QA Wizard does not have and vice versa e.g. in WinRunner, AUT needs to be run
through script. Moreover, the objects (buttons, labels etc) stored during the recording are
mapped onto a GUI Map Editor and needs to be stored as a GUI MAP File. If the stored
GUI MAP file is lost, it’s a tough job to record everything all over again. But, it is very
easy to capture text/bitmap from any object/item.
b. QA Wizard:
QA Wizard has similar scenario e.g. the application repository of QA Wizard takes good
care of executing the AUT during a re-running of a recorded test. In application repository
the items along with their properties, repository value & expected value are also
available. But, capturing text/bitmap from objects/items is not simple.
4. TEST RECORDING
a. Mercury WinRunner:
It supports Test Recording. Scripts are readily written as actions are performed during
recording.
b. QA Wizard:
It supports Test Recording. Once the recording is completed, the QA Wizard recorder
builds the script for use in the editor. The time it takes to build is dependant upon the
duration of recording and is considered as a negative impact on user experience.
5. TEST RECORDING SPEED
a. Mercury WinRunner:
The user experience is not affected due to the ongoing recording.
b. QA Wizard:
The user experience is affected due to the ongoing recording.
6. TEST RE-RUNNING/EXECUTION
a. Mercury WinRunner:
It supports Test-rerunning & execution.
b. QA Wizard:
It supports Test-rerunning & execution.
7. ANALOG RECORDING
a. Mercury WinRunner:
It supports analog recording.
b. QA Wizard:
The analog recording is termed as low level action.
8. IMAGE CAPTURE
a. Mercury WinRunner:
It supports image capture but stores the captured images inside the project folder
invisible to user.
b. QA Wizard:
Every stored item’s screen shot is automatically stored & placed in Preview Pane.
9. TEST REPORT GENERATION:
a. Mercury WinRunner:
It supports executing a re-run in three modes:
i. Verify Mode: It keeps track of every test iteration & stores it in the project folder.
ii. Debug Mode: It does not keep track of actions & previous test iterations.
iii. Update Mode: Similar to verify mode.
The reports are viewable only in WinRunner Test Results or notepad.
b. QA Wizard:
There are no separate execution modes. As a test run is executed, all activities are logged
inside the workspace under Reports.
10. TRACK OF ALL PREVIOUS REPORTS
a. Mercury WinRunner:
It keeps track of all previous test attempts, if the test was run in Verify/Update Mode. All
test results are stored inside the project folder.
b. QA Wizard:
It also keeps track of all previous test attempts.
11. TEST SCIRIPT MANAGEMENT
a. Mercury WinRunner:
The test scripts are supposed to be managed manually. There is not built in solution to
script management.
b. QA Wizard:
The grid view of test scripts can be a better solution to script management. Since, it
displays the test scripts in divided columns of Action, Information, Control, Window, and
Description.
12. TEST SCRIPT MAINTENANCE
a. Mercury WinRunner:
Maintaining the test scripts written in WinRunner can be a difficult job in terms of
changing GUI of AUT. Every time the GUI of an AUT changes, all items in the GUI Map of this AUT need to be updated correspondingly. Otherwise, the test scripts can run.
b. QA Wizard:
According to “Seapine’s WinRunner Comparison”, QA Wizard’s update feature can
automatically update the Previewer images. All of the object attributes are also
captured, allowing users to see any possible changes to the objects. During the
review, I did not come across any such thing.
13. CODING STRATEGIES
a. Mercury WinRunner:
In WinRunner, coding can be done manually, using recording or with the help of Function
Generator.
b. QA Wizard:
In QA Wizard, coding can be done in a manual fashion, using recording or using the Add
Statement feature. Though, grid and text views are also available.
14. DATABASE CONNECTIVITY:
a. Mercury WinRunner:
It supports Database connectivity.
b. QA Wizard:
It supports Database connectivity.
15. BUILT IN FUNCTIONS:
a. Mercury WinRunner:
A large number of built in functions are available in Mercury WinRunner. These functions
are related to ActiveX controls, Arithmetic, Database, Drag drop, GUI configuration, GUI
Verification etc. Such specific functions are very useful and most of the times fulfill the
needs. All scripts currently written in TSL for Validity SDK make an excessive use of these
functions. Moreover, further more are also available freely on the internet.
b. QA Wizard:
The built in library of functions is not very huge. It has a very limited number of functions
as demonstrated in its help.
16. SPECIFIC FUNCTIONS RELATED TO GUI MATCHING & STRINGS:
a. Mercury WinRunner:
WinRunner supports capturing a portion of screen or an object, which ultimately helps
reading that object again when test is run.
b. QA Wizard:
No such feature was observed in QA Wizard.
17. LOGGING
a. Mercury WinRunner:
Specific functions for logging are provided in Mercury WinRunner.
b. QA Wizard:
Specific functions for logging are also provided in QA Wizard.
18. SUPPORTED TESTING TECHNIQUES
a. Mercury WinRunner:
Mercury WinRunner supports only web applications & windows applications testing
(functional & interface testing). It does not support load tests or unit tests.
b. QA Wizard:
QA Wizard also supports web and windows applications. There is no support for Load
Testing in QA Wizard.
19. EXECUTION IN BATCH MODE
a. Mercury WinRunner:
Execution of test scripts in batch mode can be done through programming.
b. QA Wizard:
Execution of test scripts in batch mode is provided through a separate feature.
20. WORKSPACE DESIGN AND LAYOUT
a. Mercury WinRunner:
It has Mercury’s own proprietary workspace design and layout.
b. QA Wizard:
The organization of scripts is similar to the Visual Studio standard.
21. BINDING
a. Mercury WinRunner:
No such feature is present. Although, it can be done through programming.
b. QA Wizard:
It supports binding to associate a child object with a parent object e.g. web pages often
contain objects that repeat, such as multiple Buy Buttons or check boxes. Alone, each of
these attributes may appear identical. Associating the Buy button or check box with one
of its parent objects, makes the object unique. This is a very power QA Wizard feature.
22. ACCESS INTERNAL OBJECTS, METHODS AND PROPERTIES OF APPLICATIONS:
a. Mercury WinRunner:
Feature not available in Mercury WinRunner.
b. QA Wizard:
Once added in the application repository, the properties of added items/objects are
accessible easily. Only properties are accessible, not the functions
23. IDENTIFICATION OF STORED OBJECTS
a. Mercury WinRunner:
Objects/Items referred in WinRunner scripts are mapped onto respective GUI Map files.
Every item being referred in the scripts must be present in its GUI Map file for
identification. In WinRunner, GUI Map file can be considered as a single point of failure
entity. If GUI Map of a test script is lost, all items referred in the scripts need to be
identified again to store them in its GUI Map. This can be a hectic activity.
b. QA Wizard:
Each object is identified using the object attributes instead of the object location. This way
of searching ensures objects will be found, even if websites/application title change or are
dynamically generated.
24. HELP/USER’S GUIDE
a. Mercury WinRunner:
The learning curve of Mercury WinRunner is very steep because of its comprehensive
user’s guide & TSL Online Reference. Moreover, lots of online stuff is available on the web to learn.
b. QA Wizard:
I found the user’s guide/help of QA Wizard less informative and poorly administered. It is
very difficult to find out solutions to problems using the QA Wizard help.
25. AVAILABLE AUTOMATION FRAMEWORKS
a. Mercury WinRunner:
A number of automation frameworks are available on the internet for free of cost like
WRSAFS (WinRunner Software Automation Framework Support), SAFS (Software
Automation Framework Support).
Software automation frameworks reduce the effort of changing the written script every
time, the GUI of an application under test changes.
b. QA Wizard:
I have not encountered any such automation framework related to QA Wizard yet.
26. SOURCE CONTROL SUPPORT
a. Mercury WinRunner:
No support for Source Control.
b. QA Wizard:
It provides support for source control available.
27. INTEGRATION WITH OTHER TOOLS
a. Mercury WinRunner:
Mercury WinRunner can be easily integrated with QTP or Quality Center.
b. QA Wizard:
TestTrackPro is fully integrated with QA Wizard. The application contains a TestTrackPro
toolbar that lets users easily log in to TestTrackPro.
28. LICENSING SUPPORT
a. Mercury WinRunner:
All features are unlocked by a single license.
b. QA Wizard:
OCR (Optical Character Recognition) support requires separate license & is on per user basis.
Comments