Probing Neural Language Models for Human Tacit Assumptions.

Nathaniel Weir, Adam Poliak, Benjamin Van Durme

eScholarship (California Digital Library) · 2020 · 12 citations · 0 references

Concepts

Abstract

Humans carry stereotypic tacit assumptions (STAs) (Prince,1978), or propositional beliefs about generic concepts. Suchassociations are crucial for understanding natural language.We construct a diagnostic set of word prediction prompts toevaluate whether recent neural contextualized language mod-els trained on large text corpora capture STAs. Our promptsare based on human responses in a psychological study of con-ceptual associations. We find models to be profoundly effec-tive at retrieving concepts given associated properties. Our re-sults demonstrate empirical evidence that stereotypic concep-tual representations are captured in neural models derived fromsemi-supervised linguistic exposure.