FactSheets: Increasing trust in AI services through supplier's declarations of conformity

TLDR

Accuracy is a key concern for AI service suppliers, but safety, security, provenance, fairness, and explainability are also critical for consumer trust, and many industries use transparent, standardized supplier's declarations of conformity (SDoCs) to document product lineage and testing, treating them as multidimensional fact sheets that quantify these aspects. The authors propose FactSheets to increase trust in AI services. FactSheets are envisioned as documents containing purpose, performance, safety, security, and provenance information, with a comprehensive set of declaration items tailored to AI detailed in the appendix.

Abstract

Accuracy is an important concern for suppliers of artificial intelligence (AI) services, but considerations beyond accuracy, such as safety (which includes fairness and explainability), security, and provenance, are also critical elements to engender consumers’ trust in a service. Many industries use transparent, standardized, but often not legally required documents called supplier's declarations of conformity (SDoCs) to describe the lineage of a product along with the safety and performance testing it has undergone. SDoCs may be considered multidimensional fact sheets that capture and quantify various aspects of the product and its development to make it worthy of consumers’ trust. In this article, inspired by this practice, we propose FactSheets to help increase trust in AI services. We envision such documents to contain purpose, performance, safety, security, and provenance information to be completed by AI service providers for examination by consumers. We suggest a comprehensive set of declaration items tailored to AI in the Appendix of this article.