Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps

TLDR

Mobile health apps have proliferated, yet no systematic quality assessment tool exists beyond simple star ratings, prompting the need for a structured evaluation framework. This study aimed to create a reliable, multidimensional instrument for rating mobile health app quality. An expert panel categorized 372 explicit quality criteria into 23 items, forming the Mobile App Rating Scale (MARS), which was piloted on 10 randomly selected wellbeing apps and tested for inter‑rater reliability using 50 additional apps. The MARS showed excellent internal consistency (α = 0.90) and inter‑rater reliability (ICC = 0.79), proving to be a simple, objective tool that can also guide app design and development.

Abstract

The use of mobile apps for health and well being promotion has grown exponentially in recent years. Yet, there is currently no app-quality assessment tool beyond "star"-ratings.The objective of this study was to develop a reliable, multidimensional measure for trialling, classifying, and rating the quality of mobile health apps.A literature search was conducted to identify articles containing explicit Web or app quality rating criteria published between January 2000 and January 2013. Existing criteria for the assessment of app quality were categorized by an expert panel to develop the new Mobile App Rating Scale (MARS) subscales, items, descriptors, and anchors. There were sixty well being apps that were randomly selected using an iTunes search for MARS rating. There were ten that were used to pilot the rating procedure, and the remaining 50 provided data on interrater reliability.There were 372 explicit criteria for assessing Web or app quality that were extracted from 25 published papers, conference proceedings, and Internet resources. There were five broad categories of criteria that were identified including four objective quality scales: engagement, functionality, aesthetics, and information quality; and one subjective quality scale; which were refined into the 23-item MARS. The MARS demonstrated excellent internal consistency (alpha = .90) and interrater reliability intraclass correlation coefficient (ICC = .79).The MARS is a simple, objective, and reliable tool for classifying and assessing the quality of mobile health apps. It can also be used to provide a checklist for the design and development of new high quality health apps.