Masked Language Modeling

Masked Language Modeling is a self-supervised learning method in natural language processing where the model is trained to predict randomly masked tokens in a sequence based on their contextual representations, distinguishing it from other language modeling approaches by its bidirectional context utilization and explicit reconstruction objective.

411

Publications

34.9K

Citations

1.6K

Authors

345

Institutions

Publications per year

2017–2026

411

Parents

Language ModelingLanguage ModelsSelf-supervised Learning

Publications

411

LXMERT: Learning Cross-Modality Encoder Representations from Transformers

Hao Tan, Mohit Bansal · 2019 · 2.3K citations · Full text

Computing, Computer Science, Computer Vision +16

Question Answering For Toxicological Information Extraction

DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2022 · 1.6K citations · Full text

Computing, Computer Science, Computational Linguistics +13

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Taylor Shin, Yasaman Razeghi, Robert L. Logan et al. · 2020 · 1.2K citations · Full text

Computing, Computer Science, Computational Linguistics +16

Pre-Training With Whole Word Masking for Chinese BERT

IEEE/ACM Transactions on Audio Speech and Language Processing · 2021 · 1.1K citations · Full text

Computing, Computer Science, Computational Linguistics +11

ProteinBERT: a universal deep-learning model of protein sequence and function

Nadav Brandes, Dan Ofer, Yam Peleg et al. · Bioinformatics · 2022 · 975 citations · Full text

Computing, Bioinformatics, Computer Science +13

1–5 of 411

Authors

1.6K

LZ
Luke Zettlemoyer
University of Washington, United States
16 publications · 1.4K citations
15
h-index
ZG
Zhe Gan
Microsoft Research (United Kingdom), United Kingdom
10 publications · 1.1K citations
10
h-index
MG
Marjan Ghazvininejad
Meta (Israel), Israel
9 publications · 421 citations
9
h-index
ND
Nan Duan
Microsoft Research Asia (China), China
9 publications · 1.9K citations
9
h-index
XQ
Xipeng Qiu
Fudan University, China
7 publications · 737 citations
7
h-index

1–5 of 1.6K

Institutions

345

1–8 of 345

Venues

arXiv (Cornell University)
Repository · 122 publications · 7.8K citations
40
h-index
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Conference · 12 publications · 549 citations
12
h-index
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Conference · 12 publications · 1.6K citations
12
h-index
Proceedings of the AAAI Conference on Artificial Intelligence
Conference · 11 publications · 1.1K citations
11
h-index
bioRxiv (Cold Spring Harbor Laboratory)
Repository · 7 publications · 558 citations
7
h-index

1–5