For this deliverable your team is to create all the test cases to verify that the application meets the requirements given in the specification document.

System tests are usually "black box" tests since we are testing the application without seeing the source code.  Create the test cases following the guidelines in the textbook, the black box techniques described in the Training Plan, as well as your own experience or intuition about verifying program correctness.  Number each test case and write it in HTML (or Wiki) format. Follow the directions for System Test Case Format and study the Example Test Cases. 

The test manager is responsible for creating the Test Matrix, which is a grid with Requirement Numbers on one axis and Test Case Numbers on the other.  It shows which test cases cover which requirements.  Use this Test Matrix Template which shows which test cases were written by which team member and has a link to the test cases.

• Each test case is in HTML (or Wiki) format, posted on team web page as described above.
• Each test case has all four required sections.

Content
 

• Every functional requirement in the specification has at least one test case.
• Purpose provides specific detail of the behavior or function being tested.
• Actual input data and expected output data are provided.  The expected output is precise and detailed enough to be implemented as an "oracle" for automated testing.  The data needs to run on the actual software being released.
  
• User Action description is clear and specific enough for independent testers to follow the directions.
• The test case will correctly verify the requirement.
• Cross-reference is correct.

You can automate a simple text-based console application by redirecting an ASCII text file with input data and capturing the output, for example: java MyApp < TestDataFile1.dat > TestOutput1.txt

For Java Swing applications, you can use the SwingRobot test tool provided by Dr. Dalbey. You must create specialized test scripts which simulate keyboard and mouse operations.

To verify the results of the test the output is compared against the expected results, called an "oracle," using a file comparison utility such as UNIX 'diff' or DOS 'fc'.  For example,
FC TestOutput1.txt TestOracle1.txt

The test manager is responsible for the DOS/UNIX batch files that invoke the test tools and run the test scripts.  Organize the test scripts, the "setup" files (if needed), and the "oracle" files into a suitable directory hierarchy.  Create a ZIP file containing the entire test directory structure (without any application files), and place the zip file under a new link on your team home page called System Test Scripts.

Here are some examples:

It should be possible to run all the tests by invoking a single command. You may be asked to demonstrate your test scripts to the class during lab.
 

• There is a test script for every system test case.
• Each test script has sufficient documentation to be able to verify that the complete test case has been implemented.
• All the test scripts are organized and zipped as described above.
• The complete automated System Test zip file can be downloaded and run successfully on the current Release.
• The System Test creates a log file showing that all tests were run and the results compared against an "oracle."