Usability Test Report Hinternesch Final .pdf

Usability Test Report

www.100gigproject.com

Student
Class
Instructor
Date

Nicolas Hinternesch
NMS 509 – Usability Theory and Practice
Antonio Ceraso
11/21/2014

Executive Summary
This report presents the results of the usability study for 100gigproject conducted in November
2014. Three participants who reflect the target-users of the webiste were selected and
confronted with the page. They were asked to complete core tasks of the site thinking out loud
and to fill out questionnaires in order for the tester to be able to assess and evaluate the user
experience on 100gigproject.com.
The findings below were identified as most significant throughout the course of the study:
Participants consistently reacted positively towards the following aspects of the site:


Overall design and website navigation



Possibility of free membership



Value proposition for artists (overall concept of the service)

The key issues with the service include the following:


Users had a hard time understanding the concept and value proposition of the service
when first exposed to the website



The input requirements on various forms (registration, gig submission) did not always
meet user expectations



Technical issues regarding the system‘s reaction to errors



Pricing transparency: The participants‘ attention was not called to the commission
based pricing system



Minor issues with regards to form, content and website-technology

This test report contains the test methodology, detailed findings and recommendations in view
of addressing the usability issues discovered during the study. Supporting documentation
documents can be found in the appendices.

II

Content
List of Figures ........................................................................................................................ V
List of Tables ........................................................................................................................ VI
1

2

Introduction .....................................................................................................................1
1.1

Overview of 100gigproject.com ................................................................................1

1.2

Test purpose and objectives .....................................................................................1

Methodology ...................................................................................................................2
2.1

Test conditions and participants ...............................................................................2

2.2

Tasks and scenarios ................................................................................................3

2.2.1

Task 1: Overall impression (introductory task) ...................................................3

2.2.2

Task 2: Finding relevant information (Costs) .....................................................3

2.2.3

Task 3: Registration Process .............................................................................3

2.2.4

Task 4: Adding a Gig .........................................................................................3

2.2.5

Task 5: Profile Maintenance (Availability) ..........................................................4

2.3
3

4

Questionnaires .........................................................................................................4

Findings ..........................................................................................................................4
3.1

Scenario and task summaries ..................................................................................4

3.2

Questionnaire results ...............................................................................................9

3.2.1

Post task questionnaire .....................................................................................9

3.2.2

Post test questionnaire ......................................................................................9

Recommendations ........................................................................................................ 10

Appendices .......................................................................................................................... 14
Appendix A: Bibliography .................................................................................................. 14
Appendix B: Post task questionnaire ................................................................................ 15
Appendix C: Post test questionnaire ................................................................................. 16
Appendix D: Moderator Script ........................................................................................... 18
Appendix E: Information e-Mail Task 4 ............................................................................. 19
Appendix F: Post test questionnaire results ...................................................................... 20

III

Appendix G: Session logs ................................................................................................. 21
Session Log: Participant 1 ............................................................................................ 21
Session Log: Participant 2 ............................................................................................ 23
Session Log: Participant 3 ............................................................................................ 25
Logging Codes ............................................................................................................. 27
Appendix H: Screenshot Pricing Error .............................................................................. 28

IV

List of Figures
Figure 1: Completion Times Task 3 ........................................................................................6
Figure 2: Completion Times Task 4 ........................................................................................7

V

List of Tables
Table 1: Participant Overview .................................................................................................2
Table 2: Usability Problems Task 1 ........................................................................................5
Table 3: Usability Problems Task 2 ........................................................................................5
Table 4: Usability Problems Task 3 ........................................................................................6
Table 5: Usability Problems Task 4 ........................................................................................8
Table 6: Usability Problems Task 5 ........................................................................................8
Table 7: Post-task questionnaire results .................................................................................9
Table 8: Minor Usability Issues ............................................................................................. 12

VI

1 Introduction
1.1 Overview of 100gigproject.com
The 100 Gig Project is a recently founded online marketplace that aggregates all local artists
and entertainers in the city and simplifies the search and booking process. The websitedevelopment is going through its final stages and the service is scheduled to launch by the end
of 2014. The service aims to facilitate the process of bringing together supply and demand of
a local entertainment market. Artists and entertainment acts create and administer their profile
which is then exposed to event organizers.

1.2 Test purpose and objectives
The goal of this study is to test core functions of the website with users in order to detect and
eliminate usability problems before the product is first rolled out to the users. The broad
acquisition of local artists is the first step of the launch-process ahead. A large and diverse
artist database is a crucial success factor of the business. Therefore, this test focuses on the
usability experience of artists on the website. The study assesses the user experience and
collect data indicating the ease and capability of users to:


Understand the purpose of the site



Navigate the site



Identify and locate relevant information



Easily and effectively complete the core tasks of the website
o

Registration

o

Editing and maintaining important sections of the profile

The usabulity test also gathers data about the problems users may encounter and their overall
feelings and impressions regarding the website.

1

2 Methodology
2.1 Test conditions and participants
The participants were carefully selected to reflect the targeted persona. Since the user
experience of an artist was the objective of the study, the critical recruiting requirement was
the user’s active engagement in performing arts. Participants had to be an experienced artist
of any kind willing to be hired for paid shows. The following table shows an overview of the
participants and the general testing framework.
Participant 1

Participant 2

Participant 3

Gender

Male

Female

Male

Age

28

25

22

Artist type

Band member

Band member

Solo musician

Date of testing

11/05/2014

11/04/2014

11/01/2014

Location of testing

Schmitt Academic

Schmitt Academic

Participant’s home

Center, Writing Center

Center, Writing Center

Table 1: Participant Overview

The sessions were logged on a Macbook Pro using a template for Microsoft Excel.1 In order to
ensure effective documentation, single letter logging codes were applied while taking notes.2
A time stamp formula in the spreadsheet allowed detailed time-tracking of all actions,
comments and observations.
The test sessions consisted of an introduction, five tasks, a short post task questionnaire and
a post test questionnaire. They were scheduled to last between 30 and 40 minutes. The
following tasks and scenarios were given to the users throughout the course of the study. The
tasks were carefully selected in order to assess the overall experience with the site and explore
their ability to use core functions of the service.

1

Travis, D.: Log usability tests like a pro. Retrievable at http://www.userfocus.co.uk/articles/datalogging.html
(10/27/2014).
2 See Appendix F: Logging Codes.

2

2.2 Tasks and scenarios
2.2.1

Task 1: Overall impression (introductory task)

A friend just told you about 100gigproject.com. Based on his recommendation you visit the
website for the first time. What are your first impressions of the site?
Here are some examples of what you might want to respond to:
•

What do you think can you do on this page?

•

What would be the first action you would take?

•

Are there any words, labels or buttons that you don’t understand?

•

Are there any questions that come to mind?

2.2.2

Task 2: Finding relevant information (Costs)

Scenario: You have recently found out about 100gigproject.com - an online platform that lets
you expose your profile as a musician to people who are looking for a music act for their
personal or professional event. You now want to find out about different membership options
for you and if there are any associated costs for you as an artist. How would you go about
doing this on the website?
When you are done, please briefly explain:
•

Do costs occur?

•

If so, where do they occur?

2.2.3

Task 3: Registration Process

Scenario: You have just heard about the opportunity to get booked for paid gigs using
100gigproject.com. Now you want to sign up for the service as an artist. How would you go
about this on the website? Please complete the initial registration process and let us know
when you are done.
2.2.4

Task 4: Adding a Gig

Scenario: You have just signed up to the page and received a confirmation e-mail from the
100gigproject team with information and explanations regarding your next steps.3

3

See Appendix E: Information e-Mail Task 4.

3