2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · 2020 · 15 citations · 0 references
ComputingComputer SystemsEmbedded SystemsSystems ProgrammingUser Experience DesignAugmented RealitySoftware TestingUsability TestingOverhead ReductionPerformance OverheadPerformance Overhead AnalysisSession-based EvaluationSystem State AnalysisTest Case DevelopmentTest Case PrioritizationUser Feedback
In this paper, we present ARCHIE, a framework for testing augmented reality applications in the wild. ARCHIE collects user feedback and system state data in situ to help developers identify and debug issues important to testers. It also supports testing of multiple application versions (called "profiles") in a single evaluation session, prioritizing those versions which the tester finds more appealing. To evaluate ARCHIE, we implemented four distinct test case applications and used these applications to examine the performance overhead and context switching cost of incorporating our framework into a pre-existing code base. With these, we demonstrate that ARCHIE provides no significant overhead for AR applications, and introduces at most 2% processing overhead when switching among large groups of testable profiles.