Skip to the content of the web site.

Community Needs Assessment Committee • Minutes

Meeting Minutes
Monday September 18, 2000

Present: Anne, Amos, Connie, Carolyn MacGregor (professor, Systems Design)

UW Library WebPage Usability Study

We met with Carolyn to get some advice about usability studies. Carolyn teaches and carries out research in this area. She gave us a copy of the report, data collection sheets, consent forms, info letter for participants etc. for the usability study of the CECS website for coop students and talked with us about how we could set up our study. (Unfortunately, the software company designing the CECS interface and database ignored all the input). She stressed that the iterative process means you can turn around new prototypes very fast. We learned about WATCHI, a grad student group of the ACM on human computer interaction, which has been set up at UW.

Our proposed plan as a result of these discussions is to revise our task questions for participants so they are more general, hierarchical and will test the top level page. We will need to devise other tasks to test pages deeper in the site as well as the navigation scheme. We also need to set some timing standards for the various tasks. Carolyn and the literature recommend about 4 -5 participants for each iteration of the website or 2 participants for each distinct group we want to test. E.g. new student, new faculty, student who has been here for some time, faculty who has been here for sometime etc. Carolyn has offered to review any plans or documents we write as well as run the pilot test with herself as the task monitor and one of her students as the test participant so we can see how the task monitor needs to interact with the participant as well as test tasks.

We discussed characteristics of good task monitors and the verbal protocol - that participants are reminded to talk out loud as they try to accomplish the task. We could hire an undergrad through the URA program to be the task monitor. That would be $600. for the term and involves an average of 6 hours of their time per week or a grad student at $20.00 per hour. In this case, we plan to have a committee member sit in as well to record comments etc. Or we could take on this task and would then need 2 committee members per test.

Following the first round of testing we will hold a focus group in LT3 with the participants to debrief and ask additional questions about their expected use of the website. Questions and details of this are to be worked out.

Another test is to do a parts analysis where we take the site apart with a group of people to see if they like it with or without the graphic, with different words etc.

Appointment of the prototyper (the person who will be changing the webpage as a result of this testing) is a critical part of this exercise. This person must be committed to the project from the beginning and willing to accept the input of the test participants. He/She needs to believe the information which comes from the usability study. In other words, not be resistant to make changes to a design he or she has some attachment to e.g contributed to the original design; particularly likes the current design etc. This person needs to be present for the pilot test and well as discussions before hand about the setting up of the tests.

Action:

  1. Amos will begin to fill in the research on human subjects form to be sent to Susan Sykes.
  2. Select a prototyper and invite them to join our meetings beginning Sept 25.
  3. Clarify for ourselves what the purpose of the website is.
  4. Decide on the tasks participants will perform and the end points of each task, set timing standards for each task, determine distinct user groups we want to test and how to recruit, design background info sheet for participants to fill in, arrange focus group debriefing.
  5. Finalize a budget and submit to LBPC. This includes paying task monitors as well as providing incentive to participants.
  6. Discuss how we will monitor the impact of changes to the website. How will we know that the changes made were effective? When will we need to make more changes in the future.

Next meeting: Monday September 25, 10am, Porter.

CNAC Home
Email Us
July 26, 2005