CSC 309 User Acceptance Testing
User-level acceptance testing has the same structure as code-level unit testing. The difference between user versus code-level testing is that the user interacts with the program's HCI, not the code.
A "unit" for user-level testing is a user-level command instead of method. A "user-level command" is any user action that initiates a system response. In the graphical forms of UIs we have in 309 projects, a user command is most typically initiated with a mouse press, mouse movement, or key sequence.
The tabular format of an acceptance test plan is the following:
Case No. User Inputs and Commands Expected Output Remarks Results 1 Description of user-supplied inputs, including all keyboard and mouse actions to provide input and perform command execution. Description of what the user does to see the output, and exactly what the user sees. Comments about this test case, in particular what it's testing. Pass or fail, with explanation.
. . .
n
An entry in the "User Inputs and Command" column provides a description of all required user inputs in the form mouse and keyboard gestures. When inputs are in a GUI dialog of some form, describe fully all typed values, all mouse-based input actions (e.g., clicking a particular checkbox), and all required user actions to confirm the command (e.g., clicking an "OK" button). If inputs are in some other format, e.g., mouse click and drag, then fully describe the required input actions in appropriate user-level terms.
An entry in the "Expected Output" column provides a description of all expected output. If output is to the screen, then fully describe the screen contents and/or provide a screen shot of the expected output. If a generic (or specific) sample screenshot is available, such as in the requirements document, provide a reference to it. If output is in some other form, e.g., a file, then provide a complete description of all output data, and/or a dump of the actual data, and/or a document reference to where the output data are described or enumerated.
An entry in the "Remarks" column provides a brief explanation of the test case.
An entry in the "Results" column is "Pass" or "Fail". If the result is "Fail", then provide an additional explanation on the nature of the failure.
A user command is denoted by the user actions required to invoke it, with the following specific conventions for typical invocation forms:
When a particular input action sequence or output format is common to many cases, that sequence or format can be given a common name or description, and then referenced to by that name in subsequent cases.
Acceptance plans can be organized into the same functional command hierarchy that is in the requirements and model. For example, in the 308/309 Calendar Tool example, the command hierarchy is embodied in the UI command menus: File, Edit, Scheduler, View, Admin, and Options. The next level of the hierarchy is the items in these menus. For example, the Schedule menu items are Appointment, Meeting, Task, Event, and Categories.
Given this hierarchy, the acceptance testing directory has sub-directories named file, edit, schedule, view, admin, and options. In the schedule subdirectory are files appointment.html, meeting.html, task.html, event.html, and a sub-subdirectory named categories. Each .html file has the test plan for the designated command. E.g., schedule/appointment.html has the acceptance testing plan for the appointment scheduling command. This plan has enough test cases to fully exercise the execution of that command.
The organization has schedule/categories as a sub-sub-directory because the Categories dialog has three commands within it -- Add, Cdit, and Delete. Hence, there are acceptance plans for each of these commands. For example schedule/categories/add.html is the test plan for the command to add a new category.
As noted in the overview, the structure of acceptance testing is the same as unit tests. Given the traceability from requirements to model, each acceptance plan corresponds to model method for which there is a unit test. For example, the acceptance test for the command Schedule->Appointment corresponds to the unit test for the method UserCalendar.scheduleApppointment.
A simple example acceptance test plan is in the Milestone 5 release of the 309 Calendar Tool example. It's located here: classes/309/examples/milestone5/testing/acceptance/schedule/event.html