HuggingFace's Transformers: State-of-the-art Natural Language Processing

TLDR

Recent progress in NLP has been driven by advances in model architecture and pretraining, with transformer architectures enabling higher‑capacity models and effective utilization across a wide variety of tasks. Transformers is an open‑source library aimed at democratizing state‑of‑the‑art NLP advances for the broader machine learning community. The library provides a unified API for engineered transformer architectures and a curated set of pretrained models, designed to be extensible, user‑friendly, and efficient for both research and industrial use. The library is available at https://github.com/huggingface/transformers.

Abstract

Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks. \textit{Transformers} is an open-source library with the goal of opening up these advances to the wider machine learning community. The library consists of carefully engineered state-of-the art Transformer architectures under a unified API. Backing this library is a curated collection of pretrained models made by and available for the community. \textit{Transformers} is designed to be extensible by researchers, simple for practitioners, and fast and robust in industrial deployments. The library is available at \url{https://github.com/huggingface/transformers}.