Watch the presentation & view the slides
- View the Presentation Slide Deck (PDF)
- OpenSearchCon Presentation Recording — “From Legacy Lucene to OpenSearch: Migrating Booking.com’s Autocomplete Workload at Scale” (Ritz Hemnani & Abdulkadir Dalga)
Executive summary
Booking.com migrated its autocomplete workload from Brick, a custom Apache Lucene engine, to OpenSearch. The team preserved search relevance across 42+ languages while meeting strict latency requirements and completing the transition without downtime. The migration reduced infrastructure maintenance and established a foundation for more advanced search experiences driven by traveler intent.
The challenge: replacing a decade-old custom search core
For over ten years, Booking.com relied on an in-house search engine codenamed Brick to power its search box, the most frequently used, critical entry point for travelers worldwide. Brick was built as a lightweight, custom Java wrapper around Apache Lucene. Operating on 90 bare-metal servers with a coordinator node fanning out across 10 shards, Brick delivered low-latency results for 42+ languages.
However, as Booking.com expanded its infrastructure on AWS, Brick’s rigid design created growing operational and strategic headwinds:
- High operational overhead: Brick required constant manual maintenance, ran on bare-metal hardware coupled to NFS storage, and suffered from recurring stuck cluster states and shard distribution issues.
- Shrinking niche expertise: Relying on ~1,500 lines of deeply custom Java Lucene wrapper code created high dependency on specialized, in-house Lucene knowledge.
- Architectural dead-end: Brick was designed strictly for keyword lexical matching. It could not easily incorporate vector embeddings, semantic search, or hybrid retrieval necessary for Booking.com’s long-term vision of a “connected trip” (predicting and completing intent as users type).
To support future growth, Booking.com set out to replace Brick with a fully managed enterprise engine without degrading user experience or search relevancy.
The strategy: prioritizing business parity over code replication
Booking.com evaluated alternative solutions and selected Amazon OpenSearch Service. Because OpenSearch shares the same Apache Lucene foundation as Brick. The shared Lucene foundation made it easier to reproduce and validate the existing relevance behavior in OpenSearch DSL.
To de-risk the effort, the team adopted a core principle: optimize for non-inferior business metrics, not 100% technical parity.
1. Recreating relevance in OpenSearch DSL
Rather than porting legacy features no longer in active use, the team inventoried core capabilities and recreated them natively:
- Match-as-you-type: Built using custom analyzers and edge n-grams for Latin, Cyrillic, Arabic, Thai, and CJK (Chinese, Japanese, Korean) scripts.
- Popularity & market ranking: Expressed via function_score and script_score queries to prioritize destinations based on historical booking intent and market priors.
- Deduplication: Utilized the collapse parameter to ensure each destination returns only one canonical result per list.
- Typo tolerance & filters: Combined dis_max queries with native fuzzy matching and parallel sub-queries to handle joint place-plus-filter inputs (e.g., parsing “amsterdam apartments” as destination + category).
2. A parallel transitional architecture
Booking.com built a new Kubernetes-based ingestion pipeline that reads source databases and streams directly to OpenSearch. Daily index builds are managed using AWS Step Functions and Amazon OpenSearch Service Index State Management (ISM), executing automated blue/green alias switches for zero downtime.
Implementation & optimization: hitting the latency SLA
During initial staging trials in late 2025, rich relevance logic caused p95 latency to spike between 300ms and 500ms—far above Booking.com’s strict 85ms p95 SLA. Relevancy was meaningless if results couldn’t render in real-time as a user typed.
Working alongside AWS search specialists, the engineering team identified two major bottlenecks:
- Eliminating disk bottlenecks in rescoring: The custom relevance pipeline used a script rescore to calculate click-through probability scores. OpenSearch was reading dynamic JSON map fields (_source) off disk for every candidate document.
- The fix: The team flattened the map structure into a normalized array and enabled doc_values: true. This allowed the script to evaluate candidate scores directly from in-memory columnar storage rather than disk, dropping query times dramatically.
- Cluster topology tuning: Transitioned data nodes to local NVMe instance storage and added dedicated master nodes to eliminate I/O wait times during peak query spikes.
Offline and online verification
To ensure zero degradation in search quality before exposing users to the new engine, Booking.com mirrored live production traffic to a shadow cluster. Using their internal data platform, they continuously validated metrics against Brick as ground truth:
- Recall@k (k=1, 3, 5): Evaluated whether the same destinations appeared in top positions.
- Kendall’s tau (): Tracked ranking-order agreement between legacy and OpenSearch results.
- Automated deploy gates: Step Functions automatically block incomplete daily index updates (e.g., detecting missing locations or dropped entity counts) prior to production deployment.
Results and future impact
Booking.com executed a phased rollout, expanding by language and traffic tier once engagement and latency held firm:
- Iteration 1: English and top high-volume European languages.
- Iteration 2: Long-tail international scripts.
- Iteration 3: CJK languages and 100% of accommodation traffic.
Business outcomes
- Zero-downtime migration: Successfully migrated accommodations search traffic handling peak loads of 4,000 requests/sec within the 85ms SLA.
- Reduced operational risk: Fully eliminated custom bare-metal servers and NFS dependencies in favor of standard observability, automated snapshots, and managed patching.
- AI readiness: Moving away from a rigid lexical engine enabled experimentation for hybrid search, query completion, and AI-driven travel intent prediction.
“The migration wasn’t about matching Brick. It was about earning a platform that can understand intent — the Connected Trip.” – Abdulkadir Dalga, Booking.com
