SAFS Test Tables and Record Formats

Traditional SAFS Test Table Hierarchy

It is important to keep in mind that SAFS supports a hierarchy of test execution of simple text files containing CSV test records. One test record per line. The examples below show the default comma (,) separator between test record fields, but other delimiters like TABs, etc.. are supported.

Test tables are generally named with the appropriate suffix for the desired test level as defined below:

1. Test Cycles (*.CDD) -- Simply: A collection of Commands(C) and callable Test Suites(T).
   Generally, packages a collection of Commands and Suites to be executed at a given stage of the applications test Cycle.

2. Test Suites (*.STD) -- Simply: A collection of Commands(C) and callable Test Steps(T).
   Generally, packages a collection of Commands and Actions to navigate and perform logical functions of testing the application.

3. Test Steps (*.SDD) -- Simply: A collection of Commands(C) and callable GUI Actions(T).
   A collection of Commands and low-level GUI Actions that make up a simple logical function of testing the application.

Test Table Record Types

Keep the above hierarchy in mind during the following discussion of Record Types.

1. Most Common Record Types:

   Record Type: ( T ) Test

   A Test ( T ) record type is the most common type you will likely use. This type of record is where you specify the thing you want to do with the application or item being tested. It is these test records that count as tests that passed or failed in test report summaries for "Test Records".

   In the topmost Cycle test tables, you specify your own application vocabulary for logical, high-level Test Suites to perform. You give those Test Suites meaningful, logical names and those become the file names for the Test Suites:

   Hypothetical Test Cycle: Regression.CDD

   T, Launch MyApp For Testing T, Test Reports Feature T, Test Print Feature T, Shutdown Testing

   At runtime, the above Cycle-level Test Records (T) tell the Driver to go look for, open, and process the appropriate Test Suite files:

   "Launch MyApp For Testing.STD" "Test Reports Feature.STD" "Test Print Feature.STD" "Shutdown Testing.STD"

   ---

   The Test Suites work the same as the Cycle: you specify your own application vocabulary for logical, low-level application Test Steps to perform. You give those Test Steps meaningful, logical names and those become the file names for the Test Steps:

   Hypothetical Test Suite: Launch App For Testing.STD

   T, Launch MyApp T, Login With Credentials,, ^UserID=^UserID1, ^Password=^Password1 T, Verify At Main Screen

   At runtime, the above Suite-level Test Records (T) tell the Driver to go look for, open, and process the appropriate Test Step files:

   "Launch MyApp.SDD" "Login With Credentials.SDD" "Verify At Main Screen.SDD"

   Note how at the Cycle and Suite level you are defining a reusable test vocabulary that is nothing more than filenames. You are not writing any code, script, or algorithms to implement this vocabulary. The SAFS test execution Driver is already prepared to do this for you.

   Also notice when we called "Login With Credentials" we added an empty(blank) field between the comma delimiters, and then expressions ^UserID=^UserID1 and ^Password=^Password1.

   That empty(blank) field is where you could specify that "Login With Credentials" is NOT comma-delimited. If that empty field is not empty, then the Driver will assume whatever IS present there should be treated as the new CSV delimiter for "Login With Credentials".

   After that, we placed ^UserID and ^Password expressions to assign which UserID and Password will be used for Login. At runtime the values of these will likely be pulled out of the App Map where ApplicationConstants are stored. Find out more about App Maps and ApplicationConstants as well as Test Design Guidelines for Reusability.

   ---

   Now, we've made it to the low-level Test Step tables. A Test ( T ) record here will generally specify a GUI Action to perform on a particular component. At this level, we are no longer calling user-defined test vocabulary and filenames. The test records will now be calling predefined and already-implemented SAFS Component Functions. Like Test Cycle and Test Suite test records, these Component Function test records also count as "tests" that passed or failed in test report summaries of "Test Records".

   Hypothetical Test Step: Login With Credentials.SDD

   T, Login, UserName, SetTextValue, ^UserID T, Login, Password, SetTextValue, ^Password T, Login, Submit , Click

   As we can see in the above example, the Component Function Test Records at the Test Step level are very different than those in Test Cycles and Test Suites. The links to SetTextValue and Click show that these records take a Window name, a Component in that window, the Action to perform, and any parameters used by that Action.

   The Window and Component names are references to Window and Component items in the user-defined App Map. The available Actions for a given Component Type are described in the SAFS Quick Reference or the more detailed SAFS Keyword Reference.

   Record Types: ( C ) Driver Command

   A Driver Command ( C ) record type is different than a test record. Such a command is a predefined instruction for an execution engine to parse and execute. It is generally an instruction for the engine itself, rather than a test action to perform on the application.

   C, SetApplicationMap, AppMap_ja.map C, StartWebBrowser, ^AppURL, ^AppID, ^Browser C, Pause, 10

   The command is always the first field after the record type. The command is then followed by any parameters it might take. Driver Commands are the same and applicable at all test levels.

   Record Types: ( B ) BlockID

   A BlockID ( B ) record type does not have an action or a context. It is used to simply name the current location within the test table. That is, to give a name to the subsequent section of the table. The record consists of the record type specifier ( B ) and the name you provide. Currently, a BlockID merely serves the purpose of assigning the ID. No action occurs when an engine gets to a BlockID record. However, branching and flow control Driver Commands do have BlockIDs as their targets.

   B, "Loop_Start" ... ...do lots of whatever you need here... C, OnGreaterThanGotoBlockID, "Loop_Exit", ^val1, ^val2 C, GotoBlockID, "Loop_Start" ... B, "Loop_Exit"

   Record Types: ( S ) Skip

   A Skip ( S ) record type is generally used as a placeholder in a test table. Its main purpose is to show that a particular test, action, or command that is considered significant has either not been developed, or will be skipped during execution of the test. This is significant in that the skipped record is not just forgotten, but is logged with text you provide. This is necessary to show that a significant piece of the test was NOT performed and all quality determinations should take this into account. If the record were simply commented out there would be no indication that a significant piece of the test was NOT executed and overall quality determinations might be inaccurate.

   S, StartWebBrowser, ^AppURL, ^AppID, ^Browser S, OnGreaterThanGotoBlockID, "Loop_Exit", ^val1, ^val2

   The above 2 records will actually log something like seen below. They will also increment ALL active test counters with SKIPPED record counts.

   *SKIPPED* StartWebBrowser *SKIPPED* OnGreaterThanGotoBlockID

2. Least Common Record Types:

   Record Types: ( TW ) Test Warning EXPECTED

   When available for specific Component Functions--and only in specific test Engines--a Test Warning ( TW ) record type indicates the test record should generate and should allow a Test Warning to occur during execution and not indicate that the test has failed because of the Warning. Note, that a Warning not related to the primary function of the called command may still generate Warning and Failure messages and be counted as a true failure.

   The ( TW ) record type follows the exact same format rules as the standard ( T ) test record above.

   Record Types: ( TF ) Test Failure EXPECTED

   When available, a Test Failure ( TF ) record type indicates the test record should generate and should allow a Test Failure to occur during execution and not indicate that the test has failed. Note, that a Failure not related to the primary function of the called command may still generate Warning and Failure messages and be counted as a true failure.

   The ( TF ) record type follows the exact same format rules as the standard ( T ) test record above.

   Record Types: ( CW ) Driver Command Warning EXPECTED

   When available, a Command Warning ( CW ) record type indicates the command record should generate and should allow a Warning to occur during execution and not indicate that the command has failed because of the Warning. Note, that a Warning not related to the primary function of the called command may still generate Warning and Failure messages and be counted as a true failure.

   The ( CW ) record type follows the exact same format rules as the standard ( C ) command record above.

   Record Types: ( CF ) Driver Command Failure EXPECTED

   When available, a Command Failure ( CF ) record type indicates the command record should generate and should allow a Failure to occur during execution and not indicate that the command has failed. Note, that a Failure not related to the primary function of the called command may still generate Warning and Failure messages and be counted as a true failure.

   The ( CF ) record type follows the exact same format rules as the standard ( C ) command record above.

   Record Types: ( BP ) Breakpoint

   A Breakpoint ( BP ) record type is used during tracing in Engine\Driver Debug Mode (not usually Test Debug Mode) to pause execution of a test table. The command only has an effect IF the table is executed while the Driver and/or the Engine is running in Debug Mode AND breakpoints have been enabled via a script or the Breakpoints Driver Command.

   C, BreakPoints, ^doBreakPoints BP, "If in Debug Mode check the values of various variables and other assets." C, OnGreaterThanGotoBlockID, "Loop_Exit", ^val1, ^val2

   Record Types: ( E ) Engine Command

   An Engine Command ( E ) record type is primarily used by tools like Process Container that attempt to automatically explore the contents of the application without having recognition strings for any of the objects in the Application. However, advanced users can also call these as long as they handle the information returned by the commands appropriately.

   E, GetTopLevelWindows