Proceedings of the VLDB Endowment · 2024 · 13 citations · 44 references
ComputingComputer EngineeringComputer ScienceHeterogeneous ComputingNatural Language ProcessingReconfigurable ComputingAccelerator ArchitectureAccelerator IntegrationHardware AccelerationHeterogeneous System ArchitectureLanguage ModelingReconfigurable AcceleratorsAccelerator MicroarchitectureAcceleratorsGpu ArchitectureHeterogeneous Computing ArchitectureHeterogeneous Computing SystemsLarge Language ModelsRetrieval-augmented Language ModelsVector Search
A Retrieval-Augmented Language Model (RALM) combines a large language model (LLM) with a vector database to retrieve context-specific knowledge during text generation. This strategy facilitates impressive generation quality even with smaller models, thus reducing computational demands by orders of magnitude. To serve RALMs efficiently and flexibly, we propose Chameleon , a heterogeneous accelerator system integrating both LLM and vector search accelerators in a disaggregated architecture. The heterogeneity ensures efficient serving for both inference and retrieval, while the disaggregation allows independent scaling of LLM and vector search accelerators to fulfill diverse RALM requirements. Our Chameleon prototype implements vector search accelerators on FPGAs and assigns LLM inference to GPUs, with CPUs as cluster coordinators. Evaluated on various RALMs, Chameleon exhibits up to 2.16× reduction in latency and 3.18× speedup in throughput compared to the hybrid CPU-GPU architecture. The promising results pave the way for adopting heterogeneous accelerators for not only LLM inference but also vector search in future RALM systems.
44
Similarity Search in High Dimensions via Hashing
Aristides Gionis, Piotr Indyk, Rajeev Motwani · 1999 · 3.1K citations
2017 USENIX Annual Technical Conference (USENIX ATC'17)
Do Le Quoc, Martin S. Beck, Pramod Bhatotia et al. · 2017 · 3.1K citations
Locality-sensitive hashing scheme based on p-stable distributions
Mayur Datar, Nicole Immorlica, Piotr Indyk et al. · 2004 · 3K citations