An important part of this course is the term project.
As a member of a design team of around 5 students, you
will apply the methods and skills learned in class
(and hopefully elsewhere) to the design, prototypic
implementation, and evaluation of a realistic application.
The project consists of several parts, which will be graded separately.
Each team has to produce joint deliverables,
which will be the basis for the grades of all team members.
The team members will also be asked for feedback on the
performance of the other team members. This subjective
feedback may be used to adjust individual scores.
Team members are also required to document their
activities, e.g. in the form of work sheets.
Project Topic
The members of a team can select their own project topic,
subject to my approval. Ideally, the teams should have chosen
a topic by the end of the first week; if necessary, you can
postpone this decision into the second week, but this will
leave you with less time for the requirements specification.
Follow this link for some suggestions for
possible topics.
Project Organization
The first part of the task will be to select an application,
and to establish the requirements that will serve as the basis for the
later evaluation of the applicatoin.
Then a sequence of prototypes of your selected application will be implemented.
In many cases, due to time constraints, the initial
prototype will not be fully functional, but it should
have a "front end" that can be used for evaluation purposes.
The prototypes will be evaluated by a different team.
Finally, there will be a presentation on the project.
This presentation will focus on the final version and the evaluation by the evaluation team.
It must include information on the initial user requirements,
design, and implementation provided by documentation made available
through Web pages on the team accounts on the counties cluster.
The following paragraphs provide you with more details
on the different parts of the project. Included is information
on the time frame, and the contribution of the part to the
overall grade. For exact information on the time frame,
please consult the class schedule.
Milestone Week 2: Requirements and Evaluation Criteria
The first part of the project has these main objectives:
Specification of the user requirements.
Definition of the corresponding evaluation criteria.
Determination of dates when the requirements will be
realized in the system.
Your team needs to identify an application to be working on,
and to clearly specify the task the system is supposed to solve.
In many cases, it will be useful to look at existing systems
for the selected or a similar application, and identify
features that are missing or can be improved.
In the ideal case, a summative evaluation should be performed
on a fully developed system that is used in a real-world
application by the actual users. Since this is practically
impossible in a classroom setting, you will have to "switch hats"
from your customary designer or programmer's perspective, and
look at the task to be performed from a user's point of view.
To establish the required functionality as well as usability aspects
is also known as requirements elicitation. For large systems,
these requirements are sometimes translated into a formal system
specification, which serves as blueprint for the actual implementation,
and as the basis for the evaluation and testing of the system.
Again, it is not practical here to perform a full requirements
elicitation or a formal specification, but you need to put
down the crucial functional features of your system together
with important usability aspects. Please note that especially
the usability aspects will be used by the evaluation team to
perform the assessment of your project. By providing concise
and clear definitions of these aspects, you will be able to
focus more on the technical aspects of the prototype implementation,
and it will be much easier for the evaluation team to derive
the evaluation criteria for the assessment of the system.
The main aspects fo your project should be described in the Milestone Week 2 document,
and I will use the criteria below to evaluate your documents.
Project Description
purpose of the system,
main tasks to be performed,
domain and environment in which the system is used,
intended users,
goals
Existing Systems
short descriptions of at least three related systems or approaches,
their advantages and needs, and how your system differs from them
Requirements
emphasis on functional requirements, but possibly also user- and performance-related aspects
possibly distinction between core and optional functionality,
Evaluation Criteria
criteria for checking satisfaction of the requirements,
correlation between criteria and requirements,
criteria should be observable and measurable
Schedule
milestones for the project (if possible, follow the schedule in this document)
mapping of requirements and evaluation criteria to the milestones
purpose of the system,
main tasks to be performed,
domain and environment in which the system is used,
intended users,
goals
User Feedback
reactions, criticism, suggestions from potential users
possible design modifications or changes in the implementation
System Design
overview of the functional components of the system,
block diagram, class hierarchies,
specification of important data formats and interfaces
Prototype and Implementation
discussion of implementation language, development tools,
implementation strategies,
main functionality of the alpha prototype
instructions for installation and basic rules (README file)
Evaluation Plan
descriptions of the evaluation methods,
instructions for the evaluation team,
refinement of evaluation criteria (questions, metrics)
After clarifying the requirements for your system,
you will implement prototype versions of it.
The first prototype (alpha) should demonstrate
the most important functions or features of the
system. They should correspond to the requirements
defined earlier.
In addition to the actual prototype of the system,
you will work on a a plan for the testing and
evaluation of the system.
You should describe a collection of usage
scenarios, ideally in combination with some
clearly defined benchmarks that allow a well-founded
evaluation of the system, and its comparison with
other, similar ones. You should show the overall
user interface to potential users as early as
possible, even if the prototype is not finished,
and get some feedback from them.
Since we don't have the time
and resources to do use more formal approaches,
you should show your prototype to some friends,
ask them for feedback, and observe how they use the system.
This will help you to incorporate changes early in the design
stages, when they are relatively cheap to implement.
Make sure to have a record of the user feedback,
either as video, audio tape, or interview notes.
The testing and evaluation plan should be based
on principles and methods from Software Engineering
and Human Computer Interface Design.
Keep in mind that this plan
will be the basis for the evaluation performed
by another team; even if you have a wonderful
prototype, a lousy testing and evaluation plan
is an excellent basis for an unsatisfactory
evaluation ;-)
In order to facilitate collaboration between teams,
all implementations and documents must be accessible
via the Internet, either as Web-based application,
or by making the executable downloadable.
current status of the system,
major changes, implications of those changes,
comparison of this version against the requirements
User Feedback
more feedback from users based on the alpha prototype
System Design Changes
modifications to the system design,
elimination or addition of functionality,
significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version,
bugs eliminated,
update on changes in appearance, user interface,
performance
Evaluation Issues
issues raised by the evaluation team,
deviations from the evaluation plan
Due to time and resource constraints, most likely the alpha version
will be quite restricted in functionality and appearance.
The beta version should add additional functionality in
accordance with the requirements specified in the beginning.
In addition, there will probably be some usability and performance
aspects that can be improved.
Some of these improvements may be triggered
by your own insights and experiences, while others may come
from the feedback you receive from the users who took your
system for a test drive.
The aspects for this version are a further evolution of the ones identified previously:
Overview
current status of the system,
major changes, implications of those changes,
comparison of this version against the requirements
User Feedback
more feedback from users based on the alpha prototype
System Design Changes
modifications to the system design,
elimination or addition of functionality,
significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version,
bugs eliminated,
update on changes in appearance, user interface,
performance
Evaluation Issues
issues raised by the evaluation team,
deviations from the evaluation plan
The final version should implement the full functionality of the
system as initially specified in the requirements.
The system should be free of major errors.
Additional modifications can be made to increase performance and
usability, but the overall design of the system should be stable.
While the main emphasis for this milestone lies on the evaluation
of the other team, the development team should continue its
update of the documentation in order to guarantee a match
between the implementation and the documentation.
The following aspects need to be considered here:
Overview
final status of the system,
major changes and their implications during the project,
comparison of the final version against the requirements
User Feedback
a summary of feedback from users about all previous version
System Design Changes
a summary of the most significant modifications to the system design,
elimination or addition of functionality,
significant changes in data formats and interfaces
Prototype and Implementation
additional features implemented in this version,
bugs eliminated,
update on changes in appearance, user interface,
performance
Evaluation Issues
a summary of issues raised by the evaluation team
In addition, you can address issues like possible enhancements of the system,
lessons learned concerning the design and implementation of the system, or
significant conceptual or practical limitations that you encountered.
For this part, each team will finalize the evaluation of another team's project.
The evaluation team will perform the tests and evaluations
described in evaluation plan provided by the development team.
A short evaluation will be performed after each milestone,
and should be completed the week after the milestone is due.
These evaluations should be available from the Web pages of
the evaluation team, with links from the development team pages.
They should address at least the following aspects:
overview of the system or the major changes,
as seen from the evaluation team's perspective
identification of the requirements relevant for this version
application of the evaluation criteria to this version
description and interpretation of the results
recommendations to the development team
You can add additional information, such as feedback on the
testing and evaluation plan, on the documentation provided,
or on the system design and implementation.
This evaluation is done on various prototypes of the system so that
changes are still relatively easy to incorporate into the design.
Due to the tight schedule, it is advisable to establish good
contacts between the development and evaluation teams.
Ideally, the evaluation team should receive ongoing information
on the status of the project, and report back any evaluation
results as quickly as possible.
The overall evaluation will be presented to class in a final presentation,
and should be documented in a final evaluation report.
The final presentations for the project will be given as a joint
presentation by the development and the evaluation team.
The development team will present the task, purpose, design,
and implementation of the system, and the evaluation team
will report on the evaluation of the system. The overall presentation
must not be longer than 20 minutes, plus 5 minutes for discussion.
The development team has 15 minutes for its presentation,
and the evaluation team 5 minutes.
Presentations that go over their time limit may be
interrupted, get a lower grade, or both.
A quick demo of the main functionality of the system should
be integrated into the development team's part.
The discussion time can be used for questions and feedback from
the audience, or for the clarification of misunderstandings
or controversies between the development and the evaluation team.
Since it can be difficult for me to judge the contributions
of individual team members to the overall effort, I am asking
for your feedback on the performance of your team mates.
Follow this link to a template Mutual Team Member Evaluation Sheet
as an Excel file.
I reserve the right to change the allocation of points
for extreme cases, however (e.g. if everybody
agrees to give all the other team members the full 20 points).
I may also ask you for further documentation to support
your contribution to the team, or your evaluation of a
team member's contribution.
The overall evaluation will be presented to class in a final presentation,
and should be documented in a final evaluation report.
Grading
Project Grading Scheme
Milestone Week 2
10
Milestone Week 4
10
Milestone Week 6
10
Milestone Week 8
10
Team Evaluations
20
Presentations
20
Mutual Team Member Evaluation
20
The overall score for the project is 100 points.
80 out of these 100 points come out
of my evaluation of the team project, and usually every
team member gets the same score. Up to 20 points come
from an evaluation of your team mates, calculated as
the average of all your team mates' scores for your work.
A Few Notes on Teamwork
A substantial degree of your grade in this class depends on
the overall performance of your team. This can be good
(you do nothing, and still get a good grade) or bad
(you do all the work, but it is not enough). Ideally,
every team member should contribute a roughly equal share.
In reality, this is not the case because team members have
different backgrounds, experience, work habits, cultures,
etc. Just like in a professional work environment, you
have to find a balance between looking after your own interest,
and contributing to the overall team effort.
If at any point you feel that there are serious problems
with your project team, feel free to talk to me, and we
will try to find a solution.