Measuring the User Experience

A companion website to the book by Tom Tullis and Bill Albert.

Including articles, spreadsheets, links, and other resources related to usability metrics. We're adding to this site regularly, so please check back!

Articles & Presentations

• Results of Online Usability Study Comparing Obama and McCain Websites (Nov. 14, 2008)
• Forthcoming book: Online Usability Testing: Improving the User Experience through Automated Studies
• "Word Clouds" from Measuring the User Experience (Nov. 8, 2008)
• An Empirical Comparison of On-Screen Keyboards by Thomas Tullis, Ellen Mangan, and Richard Rosenbaum. Human Factors and Ergonomics Society 51st Annual Meeting, Baltimore, MD, October 2007.
• Tips and Tricks for Measuring the User Experience (PDF). Presentation by Tom Tullis and Bill Albert at the UPA Boston Usability and User Experience 2008 Conference (May 28, 2008)
• A Timestamp Macro for Task Timing (March 29, 2008)
• Calculating a Confidence Interval for Task Completion using the Adjusted Wald Method (March 28, 2008)
• Results of Online Usability Study of Apollo Program Websites (March 21, 2008)
• OK and Cancel Buttons: What's the Right Order? (Feb. 28, 2008)

Spreadsheets

• Calculating elapsed time
• Calculating a System Usability Scale (SUS) score
• SUS Scores from 129 conditions in 50 studies
• Example showing that overlapping confidence intervals can still be significantly different means by t-test
• An example of z-score calculation
• Calculating a Confidence Interval for Task Completion Data

Other Resources

• System Usability Scale (SUS) form (PPT file), or PDF version
• Usability Test Observation Coding Form (PPT file), or PDF version
• Harvey Balls (PPT file)
• Eye-tracking example (Section 7.2.2 in book):
  • Screenshot of USA.gov homepage
  • Sample eye-tracking video clip (423 KB WMV file)
  • Sample Gaze plot
  • Heat map

About The Book

• Overview
• Table of Contents
• Index
• Word Clouds

Contact the Authors

• Tom@MeasuringUX.com
• Bill@MeasuringUX.com

Links

• Online Usability Studies
  • Online usability study of pet-supply websites created using RelevantView
  • Online usability study of navigating a photo clip-art gallery created using a home-grown tool
  • Online usability study of the TerraPass website created using UserZoom
    
    
• General
  • Jeff Sauro's Measuring Usability website
  • STC page on Usability Measurement and Questionnaires
    
    
• Tools: Moderated Usability Tests
  • Time Tracker Widget
  • Usability Testing Environment (UTE)
  • Morae
  • Usability Test Data Logger
  • ErgoBrowser
  • The Observer XT
  • Ovo Logger
  • UsabilityWare
  • Usability Activity Log
    
    
• Tools and Services: Online User Experience Studies
  • Keynote WebEffective
  • UserZoom
  • RelevantView
  • MindCanvas
  • ClickTale
  • Virtual Usability Lab (VULab)
  • Ethnio (also see the Bolt Peters Remote Usability Testing Wiki)
    
    
• Tools & Services for Collecting User Satisfaction Data
  • Website Analysis and Measurement Inventory (WAMMI)
  • ForeSee Results (using the ACSI methodology)
  • OpinionLab
  • Software Usability Measurement Inventory (SUMI)
  • Measuring the Usability of Multi-Media Software (MUMMS)
  • Questionnaire for User Interface Satisfaction (QUIS)
    
    
  Online Survey Tools
• Gary Perlman's Web-Based User Interface Evaluation with Questionnaires
• SurveyMonkey
• Zoomerang
• SnapSurveys
• SurveyGizmo
• SurveyShare
• QuestionPro
  
  
• Card-sorting Tools
  • CardSort (a Windows application)
  • CardZort (a Windows application)
  • Classified (a Windows application)
  • OptimalSort (a web-based service)
  • UzCardSort(a Mozilla extension)
  • WebCAT (a free web-based tool you can install on your server if you're a techno-geek!)
  • Websort (a web-based service)
  • XSort (a Mac OS X application)
  • Card-sort analysis spreadsheet by Donna Maurer
    
    
• Accessibility-checking Tools
  • Cynthia Says
  • HiSoftware's AccMonitor
  • Accessibility Valet Demonstrator
  • WebAIM's WAVE tool
  • University of Toronto Web Accessibility Checker
  • TAW Web Accessibility Test