Evaluation of an integrated multi-task machine learning system with humans in the loop

Aaron Steinfeld, Susi Bennett, Kyle Cunningham, Matt Lahut, Pablo-Alejandro Quinones, Django Wexler, Daniel P. Siewiorek, Jordan Hayes, Paul R. Cohen, Julie C. Fitzgerald,

2007 · 20 citations · 7 references

Concepts

Abstract

Abstract — Performance of a cognitive personal assistant, RADAR, consisting of multiple machine learning components, natural language processing, and optimization was examined with a test explicitly developed to measure the impact of integrated machine learning when used by a human user in a real world setting. Three conditions (conventional tools, Radar without learning, and Radar with learning) were evaluated in a large-scale, between-subjects study. The study revealed that integrated machine learning does produce a positive impact on overall performance. This paper also discusses how specific machine learning components contributed to human-system performance.

References

7