Test Design Guidelines for Reusability

Purpose

This document provides a set of guidelines that enable a tester to plan the development of test designs and related assets to maximize reuse. The intended context for this reuse is to maximize the potential for effective system\application configuration testing and localization testing, also known as NLS testing. Throughout this document we will refer primarily to NLS testing, but the concepts apply equally well to configuration testing and other forms of testing requiring reconfigurable text values.

The App Map features referenced here as App Map Chaining and App Map Resolving Embedded Variables are only available in versions of SAFS dated on or after Aug, 2006.

It is important to recognize that the single most important notion for making test designs reusable for NLS and configuration testing is that no localizable or configuration-specific text must exist in the test designs or test actions--the test tables. All such text must be managed in separate text resource files, which in the SAFS world means separate App Maps. The test designs and test actions will reference the text via variable references which will be retrieved from the App Maps opened for the target language or configuration.

(Need to respecify the following statement based on most recent experiences with Rational Robot and Rational Functional Tester evaluations.)

In order to properly represent mixed character data (some single-byte and some multi-byte) the UTF-8 character encoding format is supported by RRAFS and SAFS. All App Maps and Test Tables intended to be used for NLS testing should be encoded in the proper format required by the tools in use.

Just as special design consideration must be given during software development to make an application support Internationalization, Localization, and Globalization, so too must our test designs receive this same level of design consideration. If the tests are not designed from the very beginning with this goal in mind then the process of going back and making the tests NLS-ready can be very difficult and time consuming.

The App Map processing features mentioned above allow us to use the same common practice used by software developers when developing software. That is, we must design our tests for NLS by separating textual data and other environment-dependent resources out of the tests themselves and into separate resource files. This means we want all environment-dependent text stored in environment-specific App Maps. Other environment-dependent resources like image files should be referenced through environment-specific entries in those same App Maps. In this way, the tests themselves do not change no matter which environment is being tested. When we go to test in a new environment we just make sure we use the appropriate App Map(s) and resources targetted for that environment.

It takes vigilance and attentive awareness to design a test that is truly NLS-ready. All environment-specific text and resources must be kept out of the design and moved into the resource files (App Maps). The designs will merely contain variable references that will ultimately be satisfied by lookups into the environment-specific App Maps. This not only includes language-specific text that might appear in object recognition strings, but also any language-specific string that might be evaluated or verified--such as an object property value, displayed text, or image benchmarks whose image may contain environment-specific text, legends, or formats.

An example of this might be a Label on a dialog box that informs the user that a certain process either succeeded or failed. The recognition string for the label may not require special handling as its Name or ID may be common across all environments as shown below:

StatusLabel="Type=Label;Name=lblStatus"

However, when we wish to verify the status displayed to the user in the Label we are going to be dealing with literal text that will likely be localized and needs to be stored separately in our NLS resource files or App Maps.

Problem:

T, StatusDialog, StatusLabel, VerifyProperty, "Text", "Success!"

Solution:

T, StatusDialog, StatusLabel, VerifyProperty, "Text", ^nlsSuccess

To make sure our tests are NLS-ready we are tasked to verify a few things:

1. The Name of the Label does not change in other environments so it does not need to be localized.

2. The name of the Text property does not change in other environments so it does not need to be localized.

3. The displayed text in the Label will change in other environments so it does need to be localized.

Thus, our NLS-ready App Maps need to hold this localizable text so that the proper values are found at runtime:

FMStudio_NLS_en (known as FMStudio_NLS):
----------------------------------------
[ApplicationConstants]
nlsHello="Hello!"
nlsSuccess="Success!"
nlsFailed="Failed!"

FMStudio_NLS_es (known as FMStudio_NLS):
----------------------------------------
[ApplicationConstants]
nlsHello="�Hola!"
nlsSuccess="��xito!"
nlsFailed="�Fallado!"

As you can see, it takes special attention and vigilance to carefully design our tests so that they are NLS-ready! The same level of care must be taken for all forms of displayed text, text to be input from the user that might be language-specific, and other resources containing environment-specific data like locale-specific currency, time, and date formats.

A Complete Dynamic Text Example Supporting NLS

The Scenario:

You wish to create a highly reusable step table (action) that will close any window in your application and then check the application state. We'll call this the "CloseAppWindow" table.

This can serve a number of purposes, including:

High value test assets with reduced maintenance
Supports dynamic features of the AUT
Supports growing automated coverage of the AUT
Existing automated tests are suitable for NLS testing

The Problem:

For maximum reusability, CloseAppWindow must be able to recognize different windows according to their any-language Caption, so no part of CloseAppWindow, or any of the tables that call it, can contain language-specific text.

The Solution:

Use App Map Chaining, App Map Embedded Variables, and Application Constants.

1. A few examples calling CloseAppWindow with the Caption parameter using Application Constants:

T   CloseAppWindow   ^Caption=^ReportsCaption
T   CloseAppWindow   ^Caption=^HelpCaption
T   CloseAppWindow   ^Caption=^AboutCaption

There is no localizable text calling CloseAppWindow.

2. The implementation of the CloseAppWindow step table:

T   AnyWindow   AnyWindow   CloseWindow
...followed by important status and error checks

There is no localizable text in CloseAppWindow.

3. The Primary App Map using App Map Embedded Variables:

MyApp.map:
----------
...
[AnyWindow]
AnyWindow="Type=Window;Caption={^Caption}"
...

There is no localizable text in the primary App Map.

4. The NLS App Maps for App Map Chaining for Application Constants:

4a. MyApp_NLS_en (known as MyApp_NLS):
--------------------------------------
[ApplicationConstants]
ReportsCaption="Reports"
HelpCaption="Help"
AboutCaption="About"

All localizable text is confined to the NLS App Map.

4b. MyApp_NLS_es (known as MyApp_NLS):
--------------------------------------
[ApplicationConstants]
ReportsCaption="Informes"
HelpCaption="Ayuda"
AboutCaption="Sobre"

This sample design shows maximum reusability. The same test used to automatically test the English translation of the application can be used to test all other localizations. And the amount of localization necessary for each translation has been minimized to a single NLS App Map file.