Co-design Hardware and Algorithm for Vector Search

by   Wenqi Jiang, et al.

Vector search has emerged as the foundation for large-scale information retrieval and machine learning systems, with search engines like Google and Bing processing tens of thousands of queries per second on petabyte-scale document datasets by evaluating vector similarities between encoded query texts and web documents. As performance demands for vector search systems surge, accelerated hardware offers a promising solution in the post-Moore's Law era. We introduce FANNS, an end-to-end and scalable vector search framework on FPGAs. Given a user-provided recall requirement on a dataset and a hardware resource budget, FANNS automatically co-designs hardware and algorithm, subsequently generating the corresponding accelerator. The framework also supports scale-out by incorporating a hardware TCP/IP stack in the accelerator. FANNS attains up to 23.0× and 37.2× speedup compared to FPGA and CPU baselines, respectively, and demonstrates superior scalability to GPUs, achieving 5.5× and 7.6× speedup in median and 95th percentile (P95) latency within an eight-accelerator configuration. The remarkable performance of FANNS lays a robust groundwork for future FPGA integration in data centers and AI supercomputers.


page 5

page 10


Accelerating Monte-Carlo Tree Search on CPU-FPGA Heterogeneous Platform

Monte Carlo Tree Search (MCTS) methods have achieved great success in ma...

DLAU: A Scalable Deep Learning Accelerator Unit on FPGA

As the emerging field of machine learning, deep learning shows excellent...

Optimizing FPGA-based Accelerator Design for Large-Scale Molecular Similarity Search

Molecular similarity search has been widely used in drug discovery to id...

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference

Dynamic Graph Neural Networks (DGNNs) are becoming increasingly popular ...

In-RDBMS Hardware Acceleration of Advanced Analytics

The data revolution is fueled by advances in several areas, including da...

Arrow: A RISC-V Vector Accelerator for Machine Learning Inference

In this paper we present Arrow, a configurable hardware accelerator arch...

From Research to Proof-of-Concept: Analysis of a Deployment of FPGAs on a Commercial Search Engine

FPGAs are quickly becoming available in the cloud as a one more heteroge...

Please sign up or login with your details

Forgot password? Click here to reset