Santiago Ontañón

FNet: Mixing Tokens with Fourier Transforms

James Lee-Thorp, Joshua Ainslie, Ilya Eckstein et al. · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · 2022 · 467 citations · Full text

Engineering, Machine Learning, Santiago Ontanon +19

A Closer Look at Invalid Action Masking in Policy Gradient Algorithms

Shengyi Huang, Santiago Ontañón · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · 2022 · 334 citations · Full text

Artificial Intelligence, Policy Gradient Algorithms, Engineering +21

Big Bird: Transformers for Longer Sequences

Manzil Zaheer, Guru Guruganesh, Avinava Dubey et al. · arXiv (Cornell University) · 2020 · 273 citations · Full text

Structured Prediction, Llm Fine-tuning, Engineering +20

ETC: Encoding Long and Structured Inputs in Transformers

Joshua Ainslie, Santiago Ontañón, Chris Alberti et al. · 2020 · 262 citations · Full text

Engineering, Part-of-speech Tagging, Joshua Ainslie +22

LongT5: Efficient Text-To-Text Transformer for Long Sequences

Mandy Guo, Joshua Ainslie, David Uthus et al. · Findings of the Association for Computational Linguistics: NAACL 2022 · 2022 · 193 citations · Full text