Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI

TLDR

Many organizations have published principles to guide ethical AI development, but their abstract nature makes them hard to operationalize; checklists have been created for AI ethics and fairness, yet without grounding in practitioners' needs they risk misuse. To understand the role of checklists in AI ethics, the authors conducted an iterative co‑design process with 48 practitioners, focusing on fairness. The study produced an AI fairness checklist, identified its desiderata and concerns, and showed that such checklists can formalize ad‑hoc processes, empower advocates, and that organizational culture influences their efficacy, suggesting future design directions.

Abstract

Many organizations have published principles intended to guide the ethical development and deployment of AI systems; however, their abstract nature makes them difficult to operationalize. Some organizations have therefore produced AI ethics checklists, as well as checklists for more specific concepts, such as fairness, as applied to AI systems. But unless checklists are grounded in practitioners' needs, they may be misused. To understand the role of checklists in AI ethics, we conducted an iterative co-design process with 48 practitioners, focusing on fairness. We co-designed an AI fairness checklist and identified desiderata and concerns for AI fairness checklists in general. We found that AI fairness checklists could provide organizational infrastructure for formalizing ad-hoc processes and empowering individual advocates. We highlight aspects of organizational culture that may impact the efficacy of AI fairness checklists, and suggest future design directions.