CSC 308 Standard Operating Procedures,
Volume 2: Testing Design and Implementation

Introduction

The purpose of thorough requirements testing is to ensure that the publicly accessible documents we produce are of high quality. Since the requirements cannot be executed in the conventional programming sense, testing of requirements is done by careful human inspection.

Each team project will appoint an official inspection tester on a rotating basis. A rotation will last one week, starting on week 5 of the quarter ¹. The assignment of inspection test duties for each team should be determined by the end of week 4, and entered in the following table, a copy of which should be stored in the project repository, in the administration subdirectory, in the file named inspection-roster.html:

Week No.	Team Member	Initials
5
6
7
8
9
10		2

The second column of the table has the name of the team member who is the inspector for the given week. The third column has the official two- or three- letter initials for the inspector, which will be used in the inspection test record described below.

It is the responsibility of the inspector to enforce the inspection testing standards described below. This enforcement should be performed before the functional requirements are publicly released. Details of the public release procedure are covered in SOP Volume 1.

Standards for Inspection of Functional Requirements

The standards described here apply to Section 2 of the 308 requirements specification document -- the Functional Requirements. Since this section of the document is written in HTML-formatted English prose and pictures, the inspection test procedure is defined in terms of the HTML document elements that organize the prose and pictures. Specifically, the functional requirements are subdivided into components denoted as follows:

HTML Tag	Denotation	Required Inspection
<Hn>	For any level n>=2, a section of functional requirements is denoted by the Dewey-decimal style numbering for that section. Since all functional requirements are contained in Section 2 of the requirements specification document, the denotation of all functional requirement sections begins with "2.", followed by subsection numbers. For example, "2.2.1" is the denotation of the first <H3> subsubsection of the second <H2> subsection of the second <H1> section of the document.	spelling, grammar, presentation style
<P>	Each paragraph in a (sub)section is numbered "Pn", with n starting at 1. For example, "2.2.1 P3" is the denotation of the third paragraph in Section 2.2.1.	spelling, grammar, presentation style
<IMG>	Each image in a (sub)section is numbered "In", with n starting at 1. For example, "2.2.1 I3" is the denotation of the third image in Section 2.2.1.	existence of image file, spelling, grammar, presentation style, image quality, aesthetics (in consultation with team artist as necessary)
<A>	Each anchor in a (sub)section is numbered "An", with n starting at 1. For example, "2.2.1 A3" is the denotation of the third anchor point in Section 2.2.1.	existence of anchor target

Inspection Test Plan and Record

A functional requirements inspection test plan is a five-column table consisting of a row for each component of the requirements, i.e., section, paragraph, image, and anchor. The five columns are:

1. requirement component denotation, e.g., 2.2.1 I3

2. three-letter inspector initials, as specified in the inspection-roster file

3. the date of the most recent inspection

4. status, which is one of "DONE" or "FIX", where DONE means the component passes inspection or FIX means the component requires fixing of some form; a blank status entry means no inspection has been performed on the corresponding requirement component

5. remarks describing what fix(es) need to be made

A functional requirements inspection test record is a completed or partially completed plan. For example, the following is the beginning of a typical inspection test record:

Component	Inspector	Date	Status	Remarks
2.1	GLF	21oct09	DONE
2.1P1	GLF	21oct09	DONE
2.1I1	GLF	21oct09	FIX	image quality is poor due to small size; rescan from original into larger GIF format
2.1A1	GLF	21oct09	FIX	the anchor target is not found; check file existence and protection
2.1.1	GLF	21oct09	DONE
2.1.1P1	GLF	21oct09	DONE
2.1.1I1	GLF	21oct09	DONE
2.1.1P2	GLF	21oct09	FIX	grammatical error in sentence 1: "number" => "numbers"
2.1.1P3	GLF	21oct09	DONE
. . .	. . .	. . .	. . .

Requirements inspection test plans are stored in project subdirectory

testing/requirements/weekN

requirements/ui-overview.html

testing/requirements/weekN/ui-overview-test.html

Procedural Details

For the weekly rounds of requirements testing, it is the job of the assigned inspector to identify problems, not correct them. The inspector is responsible for writing the test plans for each html file, per the guidelines presented above. The inspector is then responsible for performing the tests and filling in the results in each test-plan file. These filled-in test plan files are the official record of the inspection testing, and when completed are checked into the project repository.

The specific time-line for inspection testing goes like this:

1. Each member checks in work to the repository.

2. The inspector updates her/his working directory from the repository.

3. The inspector performs the inspection of all files.

4. The the inspector checks in the inspection test record.

5. The librarian updates the projects/work release directory.

6. At a subsequent team meeting, the inspector outlines the results of the inspection testing to the rest of the team, and informs specific individuals of corrections that need to be made.

It is important to note that the formal inspection is only the last step of the weekly testing process. Each individual team member should inspection test her/his work prior to check in. In particular, the inspector deserves to be non-plussed if there is a lot of sloppy work checked in for inspection testing.

Footnotes:

¹ five-person teams perform inspection testing weeks 5-9; six-person teams weeks 5-10; seven-person teams weeks 5-10, with two separate inspections (of the same documents) on week 10

² five-person teams have no inspector week 10; seven-person teams have two inspectors week 10