COMP 675: Template Project Part 3

This third part of the project will not be prepared by the team that developed the project, but by the evaluation team. This has a few implications that you should be aware of. Since this report logically fits into the project documentation so far prepared by the development team, the evaluation team should prepare it in such a format that it can be easily integrated into the overall project documentation. This does not mean that the evaluation team has to use exactly the same format; if you want to be on the safe side, use something simple, which probably will fit with any format. The evaluation team might also run into some problems with the installation, usage, test, and evaluation of the prototype. To make solving such problems easier, the development team needs to define a contact person for the evaluation team who will assist them with technical issues. This person should be expected to put less effort into the evaluation performed by his own team.
<------------------- cut here ------------------>
COMP 675 F99: [Project Name[: Part 3 -- Testing and Evaluation

COMP 675 F99: [Project Name]: Part 3 -- Testing and Evaluation

Development Team


Team name	email link
Member name (Evaluation Contact Person)	email link
Member name	email link
Member name	email link
Member name	email link

Evaluation Team


Team name	email link
Member name	email link
Member name	email link
Member name	email link
Member name	email link

System Overview

In one paragraph, the evaluation team should describe the purpose, main functions, and intended usage of the system. This should not just be copied from the design team's documentation, but formulated by the evaluation team according to their understanding of the system. It will serve as feedback to the design team to indicate how their system is perceived by other, technically competent people.

System Testing and Evaluation

Here the evaluation team describes the activities performed as they execute the test and evaluation plan. Usually you should discuss the different evaluation methods used in separate sections, but you may also decide to combine them, e.g. by using tables to compare the results obtained. In this section, the evaluation team should concentrate on the technical and performance aspect of the system, not on possible deficiencies in the evaluation criteria, the testing and evaluation plan, or the documentation; these may be discussed later in a separate section.

Evaluation Method 1

Describe the activities performed according to this evaluation method, and the results obtained.

Evaluation Method 2

As above.

Evaluation Method 3

As above.

Summary

The description of the individual evaluations is followed by an overall evaluation. Here it is especially important to refer to the design criteria, and to describe how the prototype satisfied them or not.

Recommendations

The outcome of the testing and evaluation often leads to recommendations for changes in the prototype or final system to improve the overall performance.

Feedback on the Evaluation

In most situations, it is not easy to succinctly describe complex actions to be taken by somebody else. Here the evaluation team provides feedback to the development team on the usefulness of the evaluation criteria, the execution of the testing and evaluation plan, and possibly the choice of evaluation methods. One important aspect here is the interaction with the evaluation contact person in the development team; if the evaluation team at the end considered this person a member of their own team because of their frequent interaction, then there is room for improvement in the testing and evaluation plan. You should also pay attention to the interaction between the evaluation criteria, how they are used in the actual evaluation, and the results of the different evaluation method. You may find that the criteria didn't really capture the perceived performance of the system, that not all critical aspects were covered, or that the test results varied widely. Please remember that any criticism you have for the development team should be constructive: Don't use derogatory language, try to find one good point for every bad one you mention, and offer suggestions on how to improve a flawed aspect.

Feedback on the Documentation

Since the evaluation team usually spend quite a bit of time going through the documentation produced by the development team, they might have some feedback to offer on those documents. Point out missing information, unclear statements, unnecessary details, etc. Again your interaction with the contact person is a good indicator for the feedback to give here.

Feedback on the System Design and Implementation (optional)

If you are confident that you can offer some constructive feedback on the overall design of the system, and the implementation of the prototype, you can do that here. Let's say your team consists of mature students with decades of practical experience in the "real world" under their belts, and you're evaluating a project whose team members are all students without any practical experience. Then you really might have some valuable feedback for the other team. As with the previous sections, bear in mind that your feedback should be constructive; statements like "This is a the lousiest piece of software I've ever seen" certainly don't belong here. If desired, we can also use some of the class time to get the respective development and evaluation teams together, and provide the feedback face to face.

Team email

Last modified: [modification date & time]
<------------------- cut here ------------------>

Franz Kurfess

Last modified: Fri Nov 19 16:48:46 EST 1999