An Automated Approach to Assessing an Application Tutorial’s Difficulty

Authors

Shahed A. Sabab, Adnan Khan, Parmit K. Chilana, Joanna McGrenere, Andrea Bunt

Research Lab

Parmit Chilana

Research Themes

Software Learnability

Abstract

Online step-by-step text and video tutorials play an integral role in learning feature-rich software applications. However, when searching, users can find it difficult to assess whether a tutorial is designed for their level of software expertise. Novice users can struggle when a tutorial is out of their reach, whereas more advanced users can end up wasting time with overly simple, first-principles instruction. To assist users in selecting tutorials, we investigate the feasibility of using machine learning techniques to automatically assess a tutorial’s difficulty. Using Photoshop as our primary testbed, we develop a set of distinguishable tutorial features, and use these features to train a classifier that can label a tutorial as either Beginner or Advanced with 85% accuracy. To illustrate a potential application, we developed a tutorial browsing interface called TutVis. Our initial user evaluation provides insight into TutVis’s ability to support users in a range of tutorial selection scenarios.

Citation

Shahed A. Sabab, Adnan Khan, Parmit K. Chilana, Joanna McGrenere, Andrea Bunt. An Automated Approach to Assessing an Application Tutorial’s Difficulty. Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2020)