Creating Tests with AscentialTest .pdf
Original filename: Creating Tests with AscentialTest.pdf
Title: White Paper: Automating POS
This PDF 1.5 document has been generated by Acrobat PDFMaker 10.0 for Word / Adobe PDF Library 10.0, and has been sent on pdf-archive.com on 19/11/2016 at 06:07, from IP address 180.151.x.x.
The current document download page has been viewed 288 times.
File size: 267 KB (10 pages).
Privacy: public file
Download original PDF file
Creating Tests with AscentialTest
Zeenyx Software, Inc.
Background and Introduction:
In the early days of test automation, testing tool vendors provided recorders that produced
linear streams of keystrokes and mouse movements. Creating a test was as easy as
navigating through the application under test (AUT), but testing groups quickly
discovered that the tests were of no use once the target application changed. The paradox
of course is that the tests were intended to validate the application only after it had
changed. After all, that is the whole point of automated testing. Over the years, recorders
have become more sophisticated, but anyone who has worked on a project across
multiple revisions knows that test maintenance erodes the expected return on investment
from these recorder-based testing efforts.
Data-driven testing provided an improvement. It allowed tests to be reused with different
sets of data in an effort to minimize test maintenance. Keyword-driven testing was the
next evolution. It has been considered the state-of-the-art for several years and by now
most of the major tool vendors have implemented support for it in one form or another.
By creating an abstraction layer between the test design and test function, the keyworddriven approach made it possible to insulate tests from changes in the AUT. But the cost
of this improvement over its predecessor is complexity. Keyword-driven tests typically
require the creation of two levels of spreadsheets, one to define tests and test data and
another to map keywords to functions, which are either hand-scripted or recorded. The
approach does little to enable team members who do not have programming skills.
Consequently, many organizations are forced to split up the tasks of building automated
tests. The domain experts define the tests in the set of spreadsheets, while a team of
programmers write code in a scripting language to automate the tests. Unfortunately, gaps
in communication often result in the misunderstandings and poor integration between the
various parts of the test framework erodes productivity and return on investment.
Furthermore, because automated and manual tests are not structured in the same way,
there is no transition path and multiple initiatives are required to establish and monitor
Zeenyx has implemented the next leap forward in test frameworks. The step-based testing
approach renders the benefits of data-driven and keyword-driven testing but reduces the
complexity and the level of effort. By forming relationships between application objects,
test actions and test data, AscentialTest automatically generates the test framework,
enabling domain experts to design and create automated tests that are ready to execute
without recording or scripting. Reusable steps are built by point and click as actions and
data forms are automatically generated. The automated tests that are built using the stepbased approach are as easy to maintain as they are to create.
The Step-based Approach:
The step-based approach is the next evolution in test development. It has application for
both automated and manual test creation. Where keyword-driven testing formed a loose
relationship between tests, keywords and test functions, the step-based approach builds
tight links between application objects, actions and test data. From those relationships,
the testing framework is generated automatically.
The basic building block of the step-based approach is the step. A step is a logical unit of
a test or set of tests. It is comprised of actions that are executed upon application objects.
Steps designed to be shared between more than one test are referred to as reusable steps.
Each step in a test project is designed for a specific purpose and no other step repeats the
actions that it contains. For example, there is only one step defined for ‘User Login’. Any
test that requires a ‘User Login’ will include that step. By avoiding duplication, test
maintenance is minimized so when there is one change to the AUT, there will be only
one change required in the test project.
As the inventory of re-usable steps increases, the process of building tests becomes one
of selecting steps, arranging them in order and adding test data.
In the step-based approach, automated and manual tests are created in the same way.
Both include Input Parameters, Instructions and Output Parameters. The only difference
between the two is that the instructions in an automated test are automated. Figure 1
provides a graphical representation of the various components of the step-based approach.
Building Steps with AscentialTest:
A step is a logical unit of a test or set of tests. It is comprised of one or more actions that
are executed upon application objects. In an automated test, these actions instruct the test
agent to interact with the application under test (AUT). In a manual test, actions take the
form of instructions for the manual tester. Some actions take parameters when data is
required to set, get or verify the state of the AUT. Figure 2 provides a conceptual
example of a simple reusable step.
The Login step includes three actions which set the values of the UserName and
Password objects and then click the Submit button on the Login form. If this step were
intended to be used only once, it would include actual values for UserName and
Password, but because it is intended to be reusable, parameters have been defined. When
the step is incorporated into a test, the actual data will be entered in the parameter fields.
Test data will be discussed in more detail below.
Only a small number of action types need to be defined to carry out almost all testing
tasks. Figure 3 lists the action types that are standard to the step-based approach.
Gets the value of an object
Gets the values of a set of objects
Performs application navigation
Sets the value of an object
Sets the values of a set of objects
Compares the value of an object to an expected result
Compares the values of a set of objects to expected results
In the process of designing and building tests, the user decides which actions are to be
performed to carry out the requirements of each step. The user does not need to be
concerned about how to record or script an action because it is automatically generated
based upon the object class (explained below).
With most commercial and open-source testing solutions, users are required to either
record or script the actions which make up tests. In AscentialTest, actions are
automatically generated when the user selects an object and an action type from a list of
action types that are available for the selected object. The left panel of Figure 4 above
displays a step in the process of being built. The right panel contains an application
snapshot. To build a step, the user selects from the list of actions below the snapshot. The
action is automatically generated. If the action type includes a parameter, a data object
is automatically generated as part of the action to prompt the user for data. The generated
data object has built-in edits to ensure that the data entered is of the correct data type.
There are two types of parameters: input and output. Input parameters are usually used
when setting the state of an application object. Output parameters may be used for
verification or as input parameters to another step. Parameters are not limited to simple
types. Steps can input or output data objects of arbitrary complexity. Usage of these
parameters will be discussed in more detail later in this solution description. If any of the
input fields in the snapshot contain test data, those values will be captured along with the
There are two categories of actions: object-based and form-based. Object-based actions
act upon a single application object. The example step in Figure 4 provides two examples
of object-based actions. The ‘set’ action will input a value in each of the FirstName and
LastName fields. Form-based actions carry out a common action across multiple objects.
Figure 5 provides an example of the ‘set’ action on a form.
The SubmitInfoForm step contains actions against different types of objects: text fields,
popuplists and a checkboxes. The user does not need to be concerned about the details of
setting the values to the application objects because each object class maps to its own
internal set, get and verify methods. When the user selects ‘set’ on a TextField object, the
appropriate internal action is called. The same is true for a PopupList, Checkbox or any
other type of object.
Application objects are stored in an object repository. Each object is given a name and a
reference to an object class. Figure 6 provides a view of the object repository for the
The object repository also includes various types of metadata that are used, among other
things, to configure the set, get and verify actions. Figure 7 displays the metadata used to
select the objects that are to be included in the form-based ‘set’ action for the
InformationRequest form. The ‘set’ action displayed in Figure 5 includes only the objects
that are checked in the column ‘Record (Set)’ below.
In the step-based approach, tests are built by selecting and arranging steps in an order
which carries out the requirements of the test. The test displayed in Figure 8 contains two
steps. The first step is the one displayed in Figure 5 above. It inputs data into the
InformationRequest form and clicks the Submit button. The second step verifies that an
appropriate error message displays if a required field is missing when the form is
As the inventory of re-usable steps increases, the process of building tests becomes one
of simply arranging steps in order and supplying the data. Figure 9 displays a list of steps
from the AscentialTest Project Explorer that can be used to build any number of new
To build a new test, the user selects the steps that are needed to satisfy the requirements
of the test and drags them from the Project Explorer to the Test Editor. Data objects are
automatically generated for all step parameters. The user provides the data and the test is
then ready to be run.
Although steps are independent entities, when they are assembled to form tests, they have
relationships. The most obvious relationship is their ordinal position in the list of steps. In
many cases, it would not make sense to execute a given step, let’s call it StepB, before
another step, which we refer to as StepA. In fact, the input parameter of StepB might be
derived from the output parameter of StepA. In that case a link is formed between the two
steps so that at runtime, when the AUT produces a value that gets assigned to the output
parameter of StepA, it is automatically passed through the input parameter of StepB.
Figure 10 displays this relationship.
Test: Step Relationship Example
Steps are also related through shared variables. A shared variable provides a mechanism
for steps to set and get values to and from data objects of arbitrary complexity. This
mechanism provides a way to accumulate information about the AUT at runtime. The
data may be used to calculate expected results or otherwise enable verification points.
Data-driven testing, where test data and test procedure are separated, is by no means a
new concept. The data-driven approach allows automated tests to be reused with multiple
sets of data. AscentialTest implements data-driven testing in exciting new ways. Data
tables are automatically generated in AscentialTest based on the input parameters of steps
or tests using the dialog displayed below:
Users can generate new tables or add fields to existing tables to provide test data through
the input parameters of their steps and tests.
The Data Table displayed in Figure 12 (the rows have been split in order to display it in this document),
was automatically generated based upon the input fields of a test. Unlike spreadsheets
that are typically used to store test data, AscentialTest stores data in the data type that is
expected by the automated test, eliminating the need for data transformations and thus
simplifying the process of creating automated tests. Fields that have been defined as
‘boolean’ provide a checkbox for selection. Likewise, fields defined using enumerated
types provide pick lists, which significantly reduce data input errors.
AscentialTest provides several different ways to access test data from within steps and
tests. Figure 13 provides an example where a query was automatically generated, binding
the input fields of a test to the fields of a data table.
The reusable test contains an input parameter called ‘RowId’. It is provided by the user
whenever the reusable test is implemented to indicate the data table row to access. Figure
14 displays a test requirement that is automated using the test described above. The