FJK Home CPE/CSC 480 F03 Syllabus Schedule Lecture Notes Assignments Project Other Links
CPE/CSC 480 F03 Artificial Intelligence Fall 2003

CPE/CSC 480 F03 Artificial Intelligence Fall 2003 Project

Quick Links: Project
Milestone Week 2: Selection Milestone Week 4: Prototype (alpha)
Milestone Week 6: Prototype (beta) Milestone Week 8: Final Version
Team Evaluations Mutual Team Member Evaluations
Project Presentations Team Web Pages

Project Overview

An important part of this course is the term project. As a member of a design team of around 5 students, you will apply the methods and skills learned in class (and hopefully elsewhere) to the design, prototypic implementation, and evaluation of a realistic application. The project consists of several parts, which will be graded separately. Each team has to produce joint deliverables, which will be the basis for the grades of all team members. The team members will also be asked for feedback on the performance of the other team members. This subjective feedback may be used to adjust individual scores. Team members are also required to document their activities, e.g. in the form of work sheets.

Project Topic

The members of a team can select their own project topic, subject to my approval. Ideally, the teams should have chosen a topic by the end of the first week; if necessary, you can postpone this decision into the second week, but this will leave you with less time for the requirements specification. Follow this link for some suggestions for possible topics.

Project Organization

The first part of the task will be to select an application, and to establish the requirements that will serve as the basis for the later evaluation of the applicatoin. Then a sequence of prototypes of your selected application will be implemented. In many cases, due to time constraints, the initial prototype will not be fully functional, but it should have a "front end" that can be used for evaluation purposes. The prototypes will be evaluated by a different team. Finally, there will be a presentation on the project. This presentation will focus on the final version and the evaluation by the evaluation team. It must include information on the initial user requirements, design, and implementation provided by documentation made available through Web pages on the team accounts on the counties cluster.

The following paragraphs provide you with more details on the different parts of the project. Included is information on the time frame, and the contribution of the part to the overall grade. For exact information on the time frame, please consult the class schedule.

Milestone Week 2: Requirements and Evaluation Criteria
Completion: Week 2 Points: 10
Class Schedule Back to Top

The first part of the project has these main objectives:
  1. Specification of the user requirements.
  2. Definition of the corresponding evaluation criteria.
  3. Determination of dates when the requirements will be realized in the system.
Your team needs to identify an application to be working on, and to clearly specify the task the system is supposed to solve. In many cases, it will be useful to look at existing systems for the selected or a similar application, and identify features that are missing or can be improved. In the ideal case, a summative evaluation should be performed on a fully developed system that is used in a real-world application by the actual users. Since this is practically impossible in a classroom setting, you will have to "switch hats" from your customary designer or programmer's perspective, and look at the task to be performed from a user's point of view. To establish the required functionality as well as usability aspects is also known as requirements elicitation. For large systems, these requirements are sometimes translated into a formal system specification, which serves as blueprint for the actual implementation, and as the basis for the evaluation and testing of the system. Again, it is not practical here to perform a full requirements elicitation or a formal specification, but you need to put down the crucial functional features of your system together with important usability aspects. Please note that especially the usability aspects will be used by the evaluation team to perform the assessment of your project. By providing concise and clear definitions of these aspects, you will be able to focus more on the technical aspects of the prototype implementation, and it will be much easier for the evaluation team to derive the evaluation criteria for the assessment of the system. The main aspects fo your project should be described in the Milestone Week 2 document, and I will use the criteria below to evaluate your documents.
Project Description
purpose of the system, main tasks to be performed, domain and environment in which the system is used, intended users, goals
Existing Systems
short descriptions of at least three related systems or approaches, their advantages and needs, and how your system differs from them
Requirements
emphasis on functional requirements, but possibly also user- and performance-related aspects possibly distinction between core and optional functionality,
Evaluation Criteria
criteria for checking satisfaction of the requirements, correlation between criteria and requirements, criteria should be observable and measurable
Schedule
milestones for the project (if possible, follow the schedule in this document) mapping of requirements and evaluation criteria to the milestones

Milestone Week 4: Prototype (alpha)
Completion: Week 4 Points: 10
Class Schedule Back to Top

The important aspects for this part are:
Revised Project Description
purpose of the system, main tasks to be performed, domain and environment in which the system is used, intended users, goals
User Feedback
reactions, criticism, suggestions from potential users possible design modifications or changes in the implementation
System Design
overview of the functional components of the system, block diagram, class hierarchies, specification of important data formats and interfaces
Prototype and Implementation
discussion of implementation language, development tools, implementation strategies, main functionality of the alpha prototype instructions for installation and basic rules (README file)
Evaluation Plan
descriptions of the evaluation methods, instructions for the evaluation team, refinement of evaluation criteria (questions, metrics)
After clarifying the requirements for your system, you will implement prototype versions of it. The first prototype (alpha) should demonstrate the most important functions or features of the system. They should correspond to the requirements defined earlier. In addition to the actual prototype of the system, you will work on a a plan for the testing and evaluation of the system. You should describe a collection of usage scenarios, ideally in combination with some clearly defined benchmarks that allow a well-founded evaluation of the system, and its comparison with other, similar ones. You should show the overall user interface to potential users as early as possible, even if the prototype is not finished, and get some feedback from them. Since we don't have the time and resources to do use more formal approaches, you should show your prototype to some friends, ask them for feedback, and observe how they use the system. This will help you to incorporate changes early in the design stages, when they are relatively cheap to implement. Make sure to have a record of the user feedback, either as video, audio tape, or interview notes.
The testing and evaluation plan should be based on principles and methods from Software Engineering and Human Computer Interface Design. Keep in mind that this plan will be the basis for the evaluation performed by another team; even if you have a wonderful prototype, a lousy testing and evaluation plan is an excellent basis for an unsatisfactory evaluation ;-) In order to facilitate collaboration between teams, all implementations and documents must be accessible via the Internet, either as Web-based application, or by making the executable downloadable.

Milestone Week 6: Prototype (beta)
Completion: Week 6 Points: 10
Class Schedule Back to Top

This part concentrates on the following aspects:
Overview
current status of the system, major changes, implications of those changes, comparison of this version against the requirements
User Feedback
more feedback from users based on the alpha prototype
System Design Changes
modifications to the system design, elimination or addition of functionality, significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version, bugs eliminated, update on changes in appearance, user interface, performance
Evaluation Issues
issues raised by the evaluation team, deviations from the evaluation plan
Due to time and resource constraints, most likely the alpha version will be quite restricted in functionality and appearance. The beta version should add additional functionality in accordance with the requirements specified in the beginning. In addition, there will probably be some usability and performance aspects that can be improved. Some of these improvements may be triggered by your own insights and experiences, while others may come from the feedback you receive from the users who took your system for a test drive.

Milestone Week 8: Final Version
Completion: Week 6 Points: 10
Class Schedule Back to Top

The aspects for this version are a further evolution of the ones identified previously:
Overview
current status of the system, major changes, implications of those changes, comparison of this version against the requirements
User Feedback
more feedback from users based on the alpha prototype
System Design Changes
modifications to the system design, elimination or addition of functionality, significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version, bugs eliminated, update on changes in appearance, user interface, performance
Evaluation Issues
issues raised by the evaluation team, deviations from the evaluation plan
The final version should implement the full functionality of the system as initially specified in the requirements. The system should be free of major errors. Additional modifications can be made to increase performance and usability, but the overall design of the system should be stable.

Final Documentation and Team-Evaluations
Completion: Week 10 Points: 20
Class Schedule
Team Web Pages Back to Top

While the main emphasis for this milestone lies on the evaluation of the other team, the development team should continue its update of the documentation in order to guarantee a match between the implementation and the documentation. The following aspects need to be considered here:

Overview
final status of the system, major changes and their implications during the project, comparison of the final version against the requirements
User Feedback
a summary of feedback from users about all previous version
System Design Changes
a summary of the most significant modifications to the system design, elimination or addition of functionality, significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version, bugs eliminated, update on changes in appearance, user interface, performance
Evaluation Issues
a summary of issues raised by the evaluation team
In addition, you can address issues like possible enhancements of the system, lessons learned concerning the design and implementation of the system, or significant conceptual or practical limitations that you encountered.

For this part, each team will finalize the evaluation of another team's project. The evaluation team will perform the tests and evaluations described in evaluation plan provided by the development team. A short evaluation will be performed after each milestone, and should be completed the week after the milestone is due. These evaluations should be available from the Web pages of the evaluation team, with links from the development team pages. They should address at least the following aspects:

  1. overview of the system or the major changes, as seen from the evaluation team's perspective
  2. identification of the requirements relevant for this version
  3. application of the evaluation criteria to this version
  4. description and interpretation of the results
  5. recommendations to the development team
You can add additional information, such as feedback on the testing and evaluation plan, on the documentation provided, or on the system design and implementation. This evaluation is done on various prototypes of the system so that changes are still relatively easy to incorporate into the design. Due to the tight schedule, it is advisable to establish good contacts between the development and evaluation teams. Ideally, the evaluation team should receive ongoing information on the status of the project, and report back any evaluation results as quickly as possible.

The overall evaluation will be presented to class in a final presentation, and should be documented in a final evaluation report.

Project Presentations
Completion: Week 10 Points: 20
Class Schedule
Presentation Schedule Back to Top

The final presentations for the project will be given as a joint presentation by the development and the evaluation team. The development team will present the task, purpose, design, and implementation of the system, and the evaluation team will report on the evaluation of the system. The overall presentation must not be longer than 20 minutes, plus 5 minutes for discussion. The development team has 15 minutes for its presentation, and the evaluation team 5 minutes. Presentations that go over their time limit may be interrupted, get a lower grade, or both.

A quick demo of the main functionality of the system should be integrated into the development team's part. The discussion time can be used for questions and feedback from the audience, or for the clarification of misunderstandings or controversies between the development and the evaluation team.

Mutual Team Member Evaluations
Completion: Week 10 Points: 20
Class Schedule
Mutual Team Member Evaluation (Excel file) Back to Top

Since it can be difficult for me to judge the contributions of individual team members to the overall effort, I am asking for your feedback on the performance of your team mates. Follow this link to a template Mutual Team Member Evaluation Sheet as an Excel file. I reserve the right to change the allocation of points for extreme cases, however (e.g. if everybody agrees to give all the other team members the full 20 points). I may also ask you for further documentation to support your contribution to the team, or your evaluation of a team member's contribution.

The overall evaluation will be presented to class in a final presentation, and should be documented in a final evaluation report.

Grading

Project Grading Scheme
Milestone Week 2 10
Milestone Week 4 10
Milestone Week 6 10
Milestone Week 8 10
Team Evaluations 20
Presentations 20
Mutual Team Member Evaluation 20
The overall score for the project is 100 points. 80 out of these 100 points come out of my evaluation of the team project, and usually every team member gets the same score. Up to 20 points come from an evaluation of your team mates, calculated as the average of all your team mates' scores for your work.

A Few Notes on Teamwork

A substantial degree of your grade in this class depends on the overall performance of your team. This can be good (you do nothing, and still get a good grade) or bad (you do all the work, but it is not enough). Ideally, every team member should contribute a roughly equal share. In reality, this is not the case because team members have different backgrounds, experience, work habits, cultures, etc. Just like in a professional work environment, you have to find a balance between looking after your own interest, and contributing to the overall team effort. If at any point you feel that there are serious problems with your project team, feel free to talk to me, and we will try to find a solution.


FJK Home CPE/CSC 480 F03 Syllabus Schedule Lecture Notes Assignments Project Other Links
© 2000-2024
Franz Kurfess