CPE/CSC 484 W06 Assignment 4: Usability Evaluation

Team/Individual Assignment

This assignment consists of two parts. One of them, the development of the evaluation materials, is to be performed in a team of about 3-4 people. The second part is to use the developed materials and perform an evaluation of a system, and will be done by each student individually. The topic may be related to your final project, so it is advisable to use the same core team for both activities.

Goals and Objectives

• Get practical experience with the design, planning, conduct, and aftermath of a usability evaluation for an existing system.

Description

Your task in this assignment is to perform a usability evaluation of an existing system. This system should provide capabilities relevant to the general Learning Commons and tablet PC theme, and ideally should be related to the topics of the other assignments and the final team project. Due to time and resource constraints, the evaluation will most likely be of the "quick and dirty" type, trying to identify the most problematic issues with low overhead.

Team Tasks

In the first part, your team needs to decide upon a system, identify the main tasks of interest, and then develop the material used in the actual usability evaluation. Use the following points as guidelines:

• System identification: The system you're evaluating.
• Tasks and activities: What you want the participants to do with the system. Identify at least three to five primary ones, and as many additional ones as you feel are appropriate.
• Metrics: Try to develop measurable criteria. If you feel that subjective impressions are critical, provide guidance to the user on how to express their impressions (e.g. on a scale from 0-5, or by providing a set of terms to choose from).
• Evaluation form: Create an evaluation form out of your metrics that the participants will use during the actual heuristic evaluation of the system. Even though your own team members will be the participants, try to formulate the questions in such a way that the intended group(s) will be able to understand them, and to follow the instructions. Consider the information you would like to get from the participants, but avoid biased ("leading") questions or statements.
• Assessment: Based on the individual suggestions and recommendations formulated by the participants, put together a common set of recommendations supported by the whole team. (Note: This should not simply be a "cut and paste" from the individual recommendations. You also need to consider potential conflicts in recommendations, and determine how to deal with such conflicts.

Individual Tasks

In the second part, each member of the team performs a separate usability evaluation by performing the tasks identified according to the material developed by the team. Each individual must also formulate a set of recommendations aimed at improving the user interaction and interface of the evaluated system.

Background Material

There is an abundance of material available on this topic; a list of references is given below. You can also refer to the material used by others, such as Hoang Bao's tablet PC questionnaire, the Internet2 high-bandwidth file transfer done by Gigi Choy and Rachelle Hom, or Melissa Toy's usability evaluation for the Amgen Web site from Winter 2005. I've asked them to put the material they used on the 484 Blackboard discussion forum.

• Textbook Chapters 10-14
• Chapter 8 of User-Centered Website Development by McCracken & Wolfe
• Chapters 24, 25 and 33 of Human-Computer Interaction by Preece et al.
• Readings on Guidelines and Principles:
  • Ben Shneiderman's Designing the User Interface
    • Chapter 1 - Guidelines documents bibliography
    • Chapter 2 - Principles, Golden Rules and Guidelines
• http://www.usability.gov/guidelines/
• Website Usability Evaluation:
  • http://webtango.ischool.washington.edu/
  • http://zing.ncsl.nist.gov/WebTools/tech.html
  • http://zing.ncsl.nist.gov/WebTools/WebSAT/websat_rules.html
  • http://www.waller.co.uk/eval.htm
• Heuristic Evaluation:
  • http://www.useit.com/papers/heuristic/
• examples of good/bad interfaces:
  • http://www.baddesigns.com/index.shtml
  • http://pixelcentric.net/x-shame/
  • http://digilander.libero.it/chiediloapippo/Engineering/iarchitect/shame.htm
  • http://digilander.libero.it/chiediloapippo/Engineering/iarchitect/mfame.htm
  • http://www.commarts.com/interactive/index.html
  • http://useit.com
  • http://askTog.com

Submission and Deadline

Submit electronic versions of the documents you produced to the respective Blackboard discussion forum. You can post the materials as one large or as several separate documents; use the guidelines above for the overall structure. If the structure for your materials is different, it might be useful to explain the structure you're using.

Submit the individual evaluations either electronically via Blackboard, or - in case you did paper-based versions - as hardcopies to me.

The deadline is Thursday, March 2, end of lab period.

Grading Criteria

The team part and the individual part of this assignment are each worth 5%, for a total of 10% for this assignment.