ComputingComputer EngineeringComputer ScienceArtificial IntelligenceField-programmable Gate ArraysNatural Language ProcessingAccelerator ArchitectureAttention NetworksFpga ArchitectureHardware AccelerationPattern RecognitionReconfigurable AcceleratorsAttentionAttention EnhancementBandwidthComputational EfficiencyComputer Vision AccelerationDeep Learning AcceleratorsPower-efficient Accelerator Design
Attention-based transformers have achieved significant performance breakthroughs in natural language processing (NLP) and computer vision (CV) tasks. Meanwhile, the ever-increasing length of today’s input sequences puts much pressure on computing devices. FPGAs are widely used to accelerate Transformer inference due to their high energy efficiency and flexibility. However, most of the existing FPGA-based Transformer accelerators are oriented to small input lengths, making it hard to accelerate long input sequences. To this end, we design an efficient Transformer accelerator for FPGA and long-sequence input scenarios. We use the tiling softmax algorithm to fuse attention computation, eliminating the memory and bandwidth bottleneck in the attention layer and allowing our accelerator to support arbitrary input sequence lengths. We use BERT-Base on the Alveo U50 board for evaluation, and our implementation achieves computational efficiency improvements of 1.09 ∼ 2.48 × over prior FPGA accelerators. Besides, our accelerator can support up to 175K input sequence length when running BERT-like structures, far more than previous designs.
13
Bingbing Li, Santosh Pandey, Haowen Fang et al. · 2020 · 189 citations
DOTA: detect and omit weak attentions for scalable transformer acceleration
Zheng Qu, Liu Liu, Fengbin Tu et al. · 2022 · 117 citations · Full text