Assignment 5: Usability Evaluation (DRAFT)


Assignment


This assignment consists of usability evaluations of an application for a mobile system (e.g. Android phone/tablet, or Apple iOS device). It is to be performed in a team of about 3-5 people, preferably the same team as previous assignments and your project.

Goals and Objectives


  • The goal of this assignment is to allow students to get practical experience with the design, planning, conduct, and aftermath of a usability evaluation for an existing system based on a mobile device.

Description


Your task in this assignment is to perform a usability evaluation of an existing mobile system.

Apple iOS Projects


This quarter, we are collaborating with Dr. Bellardo’s iOS class, and will evaluate applications developed by some of the teams in his class. Since his class puts the emphasis on the technology and functionality of the applications and not so much on the interaction, not all of these applications will be suited for usability evaluations. He will identify teams that are interested in having a usability evaluation done.

Alternative Applications


In addition to applications from Dr. Bellardo’s class, you can also select other applications. These applications can be related to your team project, for example.
My strong preference is for something with mobile devices since we need to gather experience on how to use our HCI lab facilities for the evaluation of mobile devices.

Evaluation Plan


Use the following points as guidelines when writing the evaluation plan:
  • System Identification: What system are you evaluating? What is its main purpose, and what are the target users?
  • Tasks and Activities: What do you want the evaluation participants to do with the system? Be sure to identify at least three to five primary ones, and as many additional ones as you feel are appropriate. Even if you're doing something like an expert evaluation (where there are no actual participants), you still need to identify the tasks and activities
  • Metrics: Try to develop measurable criteria. If you feel that subjective impressions are critical, provide guidance to the user on how to express their impressions (e.g. on some scale such as from 0-5, or by providing a set of terms to choose from).
  • Evaluation form: Create an evaluation form from your metrics that the participants will use during the actual heuristic evaluation of the system. Even though your own team members may be the participants, try to formulate the questions and instructions in such a way that the intended group(s) will be able to understand them. Consider the information you would like to get from the participants, but avoid biased ("leading") questions or statements. You can use an online questionnaire tool such as Google forms or Surveymonkey, but be aware of their limitations (like the number of submissions, or computer and internet access).
  • Assessment: Based on the individual suggestions and recommendations, put together a common set of recommendations supported by the whole team. (Note: This should not simply be a "cut and paste" from the individual recommendations. You also need to consider potential conflicts in recommendations, and determine how to deal with such conflicts.

Background Material


There is an abundance of material available on this topic; a list of references is given below. You can also refer to the material used by other students and teams in previous quarters, such as last year’s usability evaluations of Android devices, or, going back further, Ngan Phan and Hoang Bao's tablet PC questionnaires, the Internet2 high-bandwidth file transfer done by Gigi Choy and Rachelle Hom, or Melissa Toy's usability evaluation for the Amgen Web site from Winter 2005. The material can be found on the 484 Blackboard discussion forums under the W09, W06 and W05 quarters.
  • "Interaction Design" Textbook Chapters 10-14
  • "Resonant Interface" preprint Chapter 8
  • Chapter 8 of User-Centered Website Development by McCracken & Wolfe
  • Chapters 24, 25 and 33 of Human-Computer Interaction by Preece et al.
  • Readings on Guidelines and Principles:
    • Ben Shneiderman's Designing the User Interface
      • Chapter 1 - Guidelines documents bibliography
      • Chapter 2 - Principles, Golden Rules and Guidelines
  • Heuristic Evaluation:
  • examples of good/bad interfaces:

Submission and Deadline


TBD

Grading Criteria


  • Usability method selected (difficulty, appropriateness)
  • Supporting material (questionnaire, consent form)
  • Data used for the evaluation
  • Evaluation of the data
  • Assessment
  • Presentation of the results
  • Recommendations