|
|
## **Report: Wireframe Iteration 1**
|
|
|
|
|
|
# **Usability test of Interactive Wireframe Prototype**
|
|
|
|
|
|
|
|
|
|
|
|
**Purpose**
|
|
|
|
|
|
The purpose of this test is to assess the usability of the prototype's interface, focused on the following aspects:
|
|
|
|
|
|
- **Visibility** : Are menu items placed in accordance with their importance, and is the wording of each item appropriate for its function?
|
|
|
- **Feedback** : Is the user given sufficient information as to what is happening when they input data or make mistakes?
|
|
|
- **Consistency** : Does moving through menus and inputting user data follow a consistent pattern such that the user does not get confused by changes in how the program operates?
|
|
|
- **Error handling** : If the user inputs erroneous data, does the program provide sufficient and appropriate leeway to change the data after the mistake is done?
|
|
|
|
|
|
**Methodology**
|
|
|
|
|
|
Participants engaged in one-on-one test and interview sessions with our testers, first being administered a short batch of specific tasks to be performed in the interactive prototype while they were encouraged to verbalize any doubts they had about functionality. Observations were noted down, then after the tests were completed, they were followed up with a batch of questions regarding the user's interaction with the program.
|
|
|
|
|
|
Observations and answers to questions are then reformulated to problem statements that can be addressed in the next iteration of the product prototype.
|
|
|
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
|
|
|
|
|
|
# **Participants:**
|
|
|
|
|
|
Users have been asked some basic questions about their proficiency when it comes to handling technology, console programs, and their experience managing tournaments.
|
|
|
|
|
|
<br>
|
|
|
|
|
|
| **User 1** | |
|
|
|
| --- | --- |
|
|
|
| Age bracket | 30 - 35 |
|
|
|
| Sex | Male |
|
|
|
| Proficiency with technology | He has worked within IKT support, has a high degree of proficiency with everyday technology. |
|
|
|
| Experience with console-based software | Long time user of Linux command line and experience interacting with software through terminal interfaces. |
|
|
|
| Experience with managing tournaments | No experience managing tournaments |
|
|
|
|
|
|
| **User 2** | |
|
|
|
| --- | --- |
|
|
|
| Age bracket | 18 - 25 |
|
|
|
| Sex | Female |
|
|
|
| Proficiency with technology | Basic use of computers I.e., web-browser, basic games, and simple applications like Spotify. |
|
|
|
| Experience with console-based software | None. |
|
|
|
| Experience with managing tournaments | Very little. |
|
|
|
|
|
|
| **User 3** | |
|
|
|
|:---------|:---------------------------------------------------|
|
|
|
| Age bracket |55-65 |
|
|
|
| Sex |Male |
|
|
|
| Proficiency with technology |very bad|
|
|
|
| Experience with console-based software | none|
|
|
|
| Experience with managing tournaments |small tournament|
|
|
|
|
|
|
| **User 4** ||
|
|
|
| --- | --- |
|
|
|
| Age bracket | 18-25 |
|
|
|
| Sex | Female |
|
|
|
| Proficiency with technology | Basic knowledge of computers and everyday technology. No knowledge of programming. |
|
|
|
| Experience with console-based software | None. |
|
|
|
| Experience with managing tournaments | Some small tournaments up to 15 people. |
|
|
|
|
|
|
| **User 5** | |
|
|
|
|---|---|
|
|
|
| Age bracket | 18 – 25. |
|
|
|
| Sex | Male. |
|
|
|
| Proficiency with technology | Basic understanding. Some knowledge in programming, and other every-day technology. |
|
|
|
| Experience with console-based software | None. |
|
|
|
| Experience with managing tournaments | Very little. |
|
|
|
|
|
|
<br>
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
# **Tasks and Observations:**
|
|
|
|
|
|
Tasks and corresponding observations are presented below. Any observations done during the tasks are written below the description of the task, labeled U1 through U5 indicating the user the observation relates to. Observations with important data are further labeled with OB and the observation number for that user. i.e U1.OB1.
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
| **Task 1** | _Initiate the registration of a new tournament, add a new team, a new player, add tracks, and then finally finalize the tournament and return to the main menu._|
|
|
|
| --- | --- |
|
|
|
| **U1.OB1** | Asked if team choice for player registration expected a string or team number as input, but otherwise he seemed comfortable navigating through the menu and executing the given instructions. |
|
|
|
| **U2** | Performed the task without any problems (had to get used to how Balsamiq operated, but irrelevant). |
|
|
|
| **U3** | User handled it well and didn't have any problems |
|
|
|
| **U4** | The user had no problems performing the task. |
|
|
|
| **U5.OB1** | Performed the task without too much trouble. Some confusion on adding new tracks. |
|
|
|
|
|
|
| **Task 2** | _Update the score for the current track, then do the same for a specific player. Ask them to imagine that they made an error in inputting the players actual score and have them try to "fix" the error. Finally, finalize the tournament and return to the main menu._|
|
|
|
| --- | --- |
|
|
|
| **U1.OB2** | Displayed slight confusion at where the function for registering score would be located. Checked out 'administrate tracks' before he looked at 'administrate tournament.' |
|
|
|
| **U1.OB3** | He said 'Display Results' distracted him from the 'Administrate Tournament option' where the option to register score is located. |
|
|
|
| **U1.OB4** | User remarked he would have liked to be able to cancel the input of the results for a single player. |
|
|
|
| **U2** | Performed the task well, completed it without any major difficulties |
|
|
|
| **U3** | Same as task 1 |
|
|
|
| **U4.OB1** | Performed the task with no problems but wished there were a way to cancel updating score. |
|
|
|
| **U5** | After a little time spent comprehending the task, the user had no real issue with completing the task. |
|
|
|
|
|
|
| **Task 3** | _Find a way to display the results for all the competing teams, then do the same for individual players._ |
|
|
|
| --- | --- |
|
|
|
| **U1** | The user had no issues locating each functionality and was able to rapidly perform the given task. |
|
|
|
| **U2** | Completed without any problems |
|
|
|
| **U3** | Did manage that pretty well |
|
|
|
| **U4** | Completed with no issues. |
|
|
|
| **U5** | They showed no confusion with the task and managed to perform it quite easily. |
|
|
|
|
|
|
| **Task 4** | _Find a way to display all stored tracks_ |
|
|
|
| --- | --- |
|
|
|
| **U1** | The user was able to locate the functionality without issue. |
|
|
|
| **U2** | Completed without any problems |
|
|
|
| **U3** | Good |
|
|
|
| **U4** | Completed with no issues. |
|
|
|
| **U5** | No issues regarding this task. |
|
|
|
|
|
|
| **Task 5** | _First ask the user to add a new track to the system. Afterwards, have them delete a track from the system, then return to the main menu._|
|
|
|
| --- | --- |
|
|
|
| **U1** | The user executed the given task efficiently and without problem. |
|
|
|
| **U2** | Completed without any problems |
|
|
|
| **U3** | Did manage that pretty well |
|
|
|
| **U4** | Completed with no issues. |
|
|
|
| **U5** | The user had no problem with this task. |
|
|
|
|
|
|
| **Task 6** | _Ask the user to freely roam through the program and explore. Finally, have them quit the program._ |
|
|
|
| --- | --- |
|
|
|
| **U1** | The user at this point seemed quite comfortable maneuvering through the program interface. |
|
|
|
| **U2** | Completed without any problems |
|
|
|
| **U3** | Did manage that pretty well |
|
|
|
| **U4** | The user navigated the program easily with no issues or additional comments. |
|
|
|
| **U5** | After getting used to Balsamiq, they seemed comfortable with how the program worked. They started to get a feel for the program layout itself. |
|
|
|
|
|
|
<br>
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
# **Questions and Answers:**
|
|
|
|
|
|
Questions and corresponding answers are presented below. Any questions with valuable data are given a numbered OB tag like in the task section.
|
|
|
|
|
|
<br>
|
|
|
|
|
|
| **Q1** | _Did you feel there were any features that were missing from the program?_ |
|
|
|
| ---| --- |
|
|
|
| **U1** | "Al seemed to be in order." |
|
|
|
| **U2** | "No, not really." |
|
|
|
| **U3** | The user wanted side numbers to have track which side the user was on. |
|
|
|
| **U4** | "No main features were missing, but I wish I had the option to cancel updating information." |
|
|
|
| **U5** | "I feel the current features are sufficient." |
|
|
|
|
|
|
|
|
|
| **Q2** | _Did you find yourself in a situation where the feedback from the program was confusing?_|
|
|
|
| --- | --- |
|
|
|
| **U1** | "Not really, the amount of feedback seemed fitting for the tasks being performed." |
|
|
|
| **U2** | "No, it said what had been done, after I did it." |
|
|
|
| **U3** | "Nothing" |
|
|
|
| **U4** | "Nothing was confusing." |
|
|
|
| **U5** | "No, the feedback i got correlated with what i expected." |
|
|
|
|
|
|
| **Q3** | _Did you experience the program behaving in an unexpected manner, or did executing an option yield a surprising result?_ |
|
|
|
| --- | --- |
|
|
|
| **U1.OB5** | "For the most part naming of menu items and executing them yielded the expected result aside the location of the 'Display Results' option confusing me a bit as mentioned earlier. The same option was found under 'Administrate Tournament' as well and was more at home there." |
|
|
|
| **U2** | "No." |
|
|
|
| **U3** | "No" |
|
|
|
| **U4** | "No unexpected behavior." |
|
|
|
| **U5** | "No, nothing really surprised me." |
|
|
|
|
|
|
| **Q4** | _Did you feel the program was/would be punishing in any way if you input the wrong data, and if so, what features do you think could help alleviate it?_ |
|
|
|
| --- | --- |
|
|
|
| **U1.OB6** | "No, the program asks for confirmation frequently enough that it didn't seem like it would have been an issue, though editing user information could have been an option so you wouldn't have to redo all the inputs." |
|
|
|
| **U2** | "Was easy to correct the data." |
|
|
|
| **U3** | "Not that I was aware of" |
|
|
|
| **U4** | "No, correcting the data was easy." |
|
|
|
| **U5** | "No, i did not think much of it." |
|
|
|
|
|
|
| **Q5** | _Was it ever unclear where you were in the program or what you were supposed to do?_|
|
|
|
| --- | --- |
|
|
|
| **U1** | "I didn't have any problems knowing where I was in the process, aside being confused at where I'd find a few things at first, like mentioned earlier." |
|
|
|
| **U2** | "No." |
|
|
|
| **U3** | "No." |
|
|
|
| **U4** | "I had no problems understanding where I was or what I was supposed to do." |
|
|
|
| **U5** | "After taking some time to fully comprehend the program, there was at no point unclear where i was." |
|
|
|
|
|
|
|
|
|
<br>
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
---
|
|
|
## **Problem statements and Solutions**
|
|
|
|
|
|
Going through the observations and answers to questions, we've reduced each one to a problem statement that can be addressed, loosely grouped within categories. For each one, solutions should be filled in and voted on by project team members before beginning work on the next iteration.
|
|
|
|
|
|
**Location of features was confusing**
|
|
|
|
|
|
| **U1.OB1** | **User was confused on where to add new tracks** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Change the wording of the feature for managing tracks to be more explicit, such as "Edit Tracks"_ |
|
|
|
| **S2**| _Change the wording to "Manage Tracks"_|
|
|
|
| **S3**| _Show possible options in parentheses_ |
|
|
|
| **S4**| _Change wording to manage instead of administrate._|
|
|
|
|
|
|
| **U1.OB2** | **User was confused as to where one would register player score.** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _preface options with 'for'_ |
|
|
|
| **S2** | _Add help menu_ |
|
|
|
| **S3**| _Include "(Edit score)"_ |
|
|
|
|
|
|
| **U1.OB5** | ' **Display Results' were found both in the main menu and under 'Administrate Tournament'** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _remove / move the double occurence_ |
|
|
|
| **S2**| _Rename display results in main menu to "show most recent results"_ |
|
|
|
| **S3**| _Remove option from "Administrate tournament"_ |
|
|
|
|
|
|
**Unclear what was expected of user to perform a function**
|
|
|
|
|
|
| **U1.OB1** | **User was unsure of expected input format (string vs integer).** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Add info about possible input_ |
|
|
|
| **S2**| _Write a short example to show what the expected input is_ |
|
|
|
|
|
|
|
|
|
**Misleading menu item name**
|
|
|
|
|
|
| **U1.OB3** | **User was distracted by “Display Results”, thinking he could register player score there..** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Same as before, move element so that intent is clear_ |
|
|
|
| **S2**| _Rename or move function_ |
|
|
|
|
|
|
**User not finding desired functionality**
|
|
|
|
|
|
| **U4.OB1** | **User found no way to cancel updating score.** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Feature is there, but easy to miss. Implement more prominent alternative_ |
|
|
|
| **S3**| _Change method to cancel_ |
|
|
|
| **S4**| _Make it clearer that 0 means cancel, or implement "esc" as an option_ |
|
|
|
|
|
|
| **U1.OB4** | **The user would like to be able to cancel input of a player's score.** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Feature is there, but easy to miss. Implement more prominent alternative_ |
|
|
|
| **S2**| _Add extra step to cancel_ |
|
|
|
|
|
|
|
|
|
| **U1.OB6** | **User found no option to edit a single parameter for a user while inputting user data.** |
|
|
|
| --- | --- |
|
|
|
| **S1** | _Change sequential data input to a dedicated menu_ |
|
|
|
| **S2** | _Add a menu for choosing what to edit_ |
|
|
|
|
|
|
<br>
|
|
|
<br>
|
|
|
<br>
|
|
|
|
|
|
---
|
|
|
## **Usability Test Retrospective:**
|
|
|
|
|
|
## **What can be done better for later tests?**
|
|
|
|
|
|
- Spend more time formulating questions that cannot be answered with simple yes/no answers.
|
|
|
- Put more thought into how to get qualitative data from tests and questions before performing tests.
|
|
|
- If possible, get hold of someone with experience managing tournaments to test the prototype.
|
|
|
- Once the complexity of the prototype grows, more complex tasks should be given to further test readability of menu and feedback from the program.
|
|
|
- Avoid sticking strictly to questions and accepting basic answers without probing further.
|
|
|
|