OpenSearch partner and contributor AWS recently published a blog that focuses on the different algorithms and techniques used to perform approximate k-NN search at scales of more than a billion data points within OpenSearch. In the blog, Othmane Hamzaoui, AI/ML Specialist Solutions Architect with AWS, and Jack Mazanec, Software Dev Engineer and OpenSearch committer, explain algorithms including Hierarchical Navigable Small Worlds (HNSW), Inverted File System(IVF), and Product Quantization (PQ), and share the techniques and the memory footprint to support each of these algorithms, with examples. The post also includes the metrics and trade-offs between recall and performance, so you can choose the right algorithm for your k-NN workload at scale. Explore the AWS blog and be sure to check out the benchmarking codebase for yourself in GitHub.
