ECATT integration with Mercury QuickTest for SAP

Question

For the past few months I have been evaluating Quick Test Professional from Mercury and eCATT (extended Computer Aided Testing Tool) from SAP as testing tools for SAP. Mercury has a strong partnership with SAP and the two companies have offered ways to integrate their two products. Has anyone ever used eCATT in conjunction with QTP? What are your thoughts on this partnership? Is it worth $10,000 to pick up Mercury's QTP when SAP's eCATT is Free with SAP? I think the functionality is comparable, although QTP is easier to use and can interact with 3rd party applications. If anyone has used both QTP and eCATT, do you think these two tools are an either/or option or do they compliment each other?

Answer 1

Ben,
Good to hear you liked the article. Look out for another contribution in SPJ in the near future
I have attempted to answer your questions - all prefixed with <b>==> SAP (JM)</b> in your original text:
This document describes the functionality, abilities and benefits that would be lost in an SAP automated testing solution that excludes Mercury’s QuickTest. Assuming that SAP’s eCATT is the only tool used for SAP testing there will be the following limitations:
Script Creation
SAP eCATT cannot record certain types of report screens in TCD(Record) mode. i.e. MB58(view stock levels)
<b>==> SAP (JM):</b> This is true - but you can use the SAPGUI command, which uses the same technical interface as QTP.
Although the ‘active screen simulator’ in eCATT contains data, it is usually not visible and the user has to guess where to click in order to capture a value. The other option is to forgo the active screen and do all of the work through the command interface.
The active screen in QuickTest allows the user to see the exact state of the screen that was recorded and access the fields in that screen.
<b>==> SAP (JM):</b> A valid criticism of eCATT. From Release 6.40 (SAP NetWeaver 04), a script-wide search function makes using the command interface easier.
The recorded script is one command, (i.e. TCD(VA01, VA01_1) is the command for an entire sales order), therefore Scripting logic must be placed before, after or around a transaction. There is less freedom in scripting inside of a transaction.
In the Expert view of QTP users can add or delete visual basic statements during any portion of the transaction. Each step, down to a click or keystroke is recorded as an individual statement that can be modified or deleted. Logic such as loops and conditions can be placed around any set of steps.
<b>==> SAP (JM):</b> Using the SAPGUI command in eCATT, you can create commands that cover a single screen, providing more of the kind of flexibility that you find in QTP.
The eCATT tool does not allow individual steps in a single transaction recording to be rearranged or deleted. If a user makes a mistake while recording, the entire transaction must be re-recorded.
QuickTest allows users to freely remove steps that were mistakes or errors after a transaction has been recorded.
<b>==> SAP (JM):</b> Again, using SAPGUI in eCATT, this is possible. Admittedly tricky in Release 6.20, but very simple in Web AS 6.40.
SAP’s eCATT cannot inherently read excel spreadsheets for data driven testing. The eCATT tool allows for data driven testing through its own data tables. These are useful for entering small sets of data. For eCATT to test large sets of data created in excel spreadsheets there are three solutions:
1. Purchase conversion software from SAP
2. Write an in-house conversion script
3. Enter data manually into the eCATT data table
QuickTest stores its data table as an excel-readable .xls file.
<b>==> SAP (JM):</b> 1. Who is selling the conversion software?
<b>==> SAP (JM):</b> You can use the class CL_GUI_FRONTEND_SERVICES to upload a tab-delimited file into an internal table in a test script (using an Inline ABAP routine)
<b>==> SAP (JM):</b> Data upload/download is available in Web AS 6.40
Test scripts in eCATT are limited to about 40 parameters.
In QucikTest the only limitation to the number of parameters or number of iterations is the maximum number of columns and rows in an excel worksheet.
<b>==> SAP (JM):</b> Where does this limitation come from? Does it seem to be technical, or more practical in nature?
Scripts that enter a single line item in a table cannot be modified to enter multiple items later. A new script with multiple line items must be recorded.
A script in QTP can be modified to add a single line item or multiple line items to a table regardless of how many items were entered when the transaction was recorded.
<b>==> SAP (JM):</b> This is possible in eCATT if you use the SAPGUI command. Because a single SAPGUI command can cover a small part of a transaction, you can isolate the "line item" section in a single command, then put a loop around it. I have an example of this if you need one.
Script Execution and Analysis
SAP eCATT will respond with a pass/fail status based on its own criteria before any checkpoints have been added. Although eCATT allows for additional checkpoints, there is a minimum level of pre-set requirements that eCATT looks for in order to allow a test to pass. These pre-set requirements cannot be viewed or changed and tests may “fail” even when, from the user’s perspective, the transaction has responded exactly as expected.
Quick Test will only fail a test if the script if it was unable to recognize an object on the screen or complete every statement. Otherwise failure points are completely determined by the insertion of check points by the Test engineer. QuickTest also provides the option to ignore missing objects or uncompleted steps.
<b>==> SAP (JM):</b> Default conditions in eCATT (for example, fail if an error message occurs) can be overwritten. See eCATT documentation on http://help.sap.com for details (use the NetWeaver 2004 documentation - it is more comprehensive). Then, the only conditions under which the test would fail automatically would be a system error (RFC destination not available, or similar)
If SAP experiences a system failure, eCATT will fail too and test results may be lost.
If SAP were to crash QuickTest can take a screenshot and pinpoint the location in the code where the script failed.
<b>==> SAP (JM):</b> If you are working remotely (test case and application under test in different systems) you will get a log in the central test system, even if the SUT fails. The only circumstances under which everything is lost would be a full GUI crash!
eCATT cannot use the SAP logon pad to open a new session and log on as a new user.
In QuickTest new sessions can be opened and different users can be logged in with encrypted passwords.
<b>==> SAP (JM):</b> eCATT uses System Data Containers and RFC destinations to log onto multiple systems as a specified user. This is more flexible than using hard-coded user names in a script and allows you to leverage concepts such as Trusted RFC or SSO.
Aside from printing results on paper, users cannot view test results without an SAP logon to the system where the tests were executed.
Mercury provides a test viewer that can be installed on any machine and allows users without the QuickTest application to view and analyze test results.
<b>==> SAP (JM):</b> eCATT logs can be downloaded in a variety of formats from the log view in transaction SECATT (via the Print Preview function, then List -> Save).
Cannot view a screenshot of the failed step.
QuickTest provides the option to capture a screen shot at the point of failure.
<b>==> SAP (JM):</b> Again, a valid criticism. This has been rectified for the next release of eCATT (will be available with Solution Manager 4.0). When playing back a SAPGUI command, you will be able to capture screenshots at various points in time ranging from "every screen" to "on error".