Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 51 citations · 33 references
ComputingComputer ScienceArtificial IntelligenceMachine LearningNatural Language ProcessingDeep LearningLanguage ModelsNeural NetworksNeural RepresentationRepresentation LearningText RepresentationArchitecture SearchNatural Language InferenceNeural Architecture SearchNeural Text RepresentationSearch Space ExplorationSearch Space RepresentationText ClassificationText Networks
Learning text representation is crucial for text classification and other language related tasks. There are a diverse set of text representation networks in the literature, and how to find the optimal one is a non-trivial problem. Recently, the emerging Neural Architecture Search (NAS) techniques have demonstrated good potential to solve the problem. Nevertheless, most of the existing works of NAS focus on the search algorithms and pay little attention to the search space. In this paper, we argue that the search space is also an important human prior to the success of NAS in different applications. Thus, we propose a novel search space tailored for text representation. Through automatic search, the discovered network architecture outperforms state-of-the-art models on various public datasets on text classification and natural language inference tasks. Furthermore, some of the design principles found in the automatic network agree well with human intuition.
33
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Xception: Deep Learning with Depthwise Separable Convolutions
François Chollet · 2017 · 19K citations