Concepedia

Knowledge Distillation For Mt

Knowledge Distillation for MT is a model compression technique in machine translation that transfers the learned linguistic and contextual knowledge from a large, high-performing teacher model to a smaller student model by mimicking its soft-label output distributions, thereby enabling efficient inference while preserving translation quality.

84

Publications

4.2K

Citations

326

Authors

96

Institutions

Publications per year

2017–2026

82

Parents

Publications

84

Sequence-Level Knowledge Distillation

Yoon Kim, Alexander M. Rush · 2016 · 775 citations · Full text

, , +9

Non-Autoregressive Neural Machine Translation

Jiatao Gu, James Bradbury, Caiming Xiong et al. · arXiv (Cornell University) · 2017 · 449 citations · Full text

, , +12

Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding

Xiaodong Liu, Pengcheng He, Weizhu Chen et al. · arXiv (Cornell University) · 2019 · 163 citations · Full text

, , +14

Symbolic Knowledge Distillation: from General Language Models to Commonsense Models

Peter West, Chandra Bhagavatula, Jack Hessel et al. · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · 2022 · 150 citations · Full text

, , +16

End-to-End Speech Translation with Knowledge Distillation

Yuchen Liu, Hao Xiong, Jiajun Zhang et al. · 2019 · 144 citations

, , +11

1–5 of 84

Authors

326

  1. JG
    Jiatao Gu

    Meta (Israel), Israel

    4 publications · 721 citations

    4

    h-index

  2. HW
    Haifeng Wang

    Baidu (China), China

    3 publications · 216 citations

    3

    h-index

  3. VO
    Victor O. K. Li

    University of Hong Kong, Hong Kong SAR China

    3 publications · 602 citations

    3

    h-index

  4. YF
    Yang Feng

    Chinese Academy of Sciences, China

    3 publications · 64 citations

    3

    h-index

  5. HW
    Hua Wu

    Baidu (China), China

    3 publications · 216 citations

    3

    h-index

1–5 of 326

Institutions

96

  1. Tencent (China)

    Shenzhen, China

    5 h-index

    11 publications

    340 citations

  2. University of Chinese Academy of Sciences

    Beijing, China

    5 h-index

    12 publications

    612 citations

  3. Microsoft (United States)

    Redmond, United States

    3 h-index

    11 publications

    662 citations

  4. Chinese Academy of Sciences

    Beijing, China

    3 h-index

    9 publications

    531 citations

  5. Huawei Technologies (Sweden)

    Stockholm, Sweden

    3 h-index

    10 publications

    515 citations

  6. Baidu (China)

    Beijing, China

    3 h-index

    11 publications

    805 citations

  7. Institute of Computing Technology

    Beijing, China

    3 h-index

    8 publications

    161 citations

  8. Microsoft Research (United Kingdom)

    Cambridge, United Kingdom

    3 h-index

    8 publications

    818 citations

1–8 of 96

Venues

  1. arXiv (Cornell University)

    Repository · 16 publications · 1.3K citations

    14

    h-index

  2. Proceedings of the AAAI Conference on Artificial Intelligence

    Conference · 5 publications · 234 citations

    5

    h-index

  3. Pattern Recognition

    2 publications · 56 citations

    2

    h-index

  4. Information Sciences

    2 publications · 37 citations

    2

    h-index

  5. Neurocomputing

    2 publications · 55 citations

    2

    h-index

1–5