Transforming decade-old intelligence into rapid insights: Max Security’s Agentic RAG implementation with OpenSearch hybrid search

A hybrid search and RAG implementation that cut analyst briefing time by 79% and saved 7 hours per week.

Summary

Transforming intelligence access with SCOUT AI

Max Security partnered with BigData Boutique to develop SCOUT AI, an intelligent agent that optimizes access to years of proprietary risk intelligence.

• The Challenge: Analysts spent hours manually searching through archive documents using traditional keyword methods, creating operational inefficiencies and limiting scalability.1
• The Solution: The team implemented a closed-loop Retrieval-Augmented Generation (RAG) system using OpenSearch’s hybrid search (vector + keyword) and Anthropic Claude, ensuring all outputs are grounded exclusively in vetted internal data.
• The Results: The new system reduced analyst briefing times by 79% (from 2 hours to 25 minutes), reclaimed 7 hours per analyst per week, and enabled broader access to intelligence for clients

The challenge: Eliminating manual search bottlenecks

Max Security, a global risk intelligence provider, equips corporate security teams and senior decision-makers with the critical, pre-vetted analysis needed to navigate geopolitical threats and emerging operational risks. At the heart of their service sits a decade of proprietary intelligence—a massive, high-value archive of reports and assessments that forms their core competitive advantage. However, unlocking that value proved difficult. Producing a targeted research briefing forced analysts to spend 6 to 12 hours manually combing through the archive. Traditional keyword search tools failed to bridge the gap; they compelled analysts to sift through lengthy, complex documents, cross-reference data, and piece together context by hand. This workflow crippled operational efficiency, as precious analyst hours vanished into the mechanics of searching rather than the substance of analysis. Furthermore, it stifled scalability, because the depth of the intelligence Max Security could provide remained tethered to the number of analysts available to manually parse the data.

The solution: Architecting a closed-loop intelligence agent

SCOUT AI: a closed-loop intelligence agent

Max Security and BigData Boutique, a member of the OpenSearch Software Foundation, had already worked together on a database modernisation project, which gave both teams a practical understanding of the data environment before the AI work began. Over 14 months, the two organisations co-developed SCOUT AI, a client-facing agent that lets users query the full intelligence archive using natural language.

The core requirement was that SCOUT AI operate as a closed loop. Every answer had to be grounded exclusively in Max Security’s vetted intelligence. No external knowledge. No hallucination from general-purpose training data. Users needed to trust that what they received reflected Max’s analysis, not the internet’s.

To deliver that, the team needed a retrieval layer that could handle both the nuance of natural language questions and the breadth of a large, multi-format document corpus. That is where OpenSearch came in.

Hybrid search as the foundation

OpenSearch runs as the retrieval backbone of SCOUT AI, handling both vector search and keyword search in a single hybrid query layer.

The distinction matters. Keyword search works well when a user knows the exact terms in a document. Vector search, or semantic search, works when the user’s question and the document’s language differ but the meaning is the same. In an intelligence context, an analyst asking about “civil unrest near supply routes” may be looking for reports that discuss “protests disrupting logistics corridors.” Those two phrases share no keywords, but are semantically close. Hybrid search catches both cases.

Documents in the archive were processed through AI-driven custom chunking and embedded using Cohere embedding models before being indexed into OpenSearch. That process ensured the vector representations captured the nuanced, domain-specific language of security intelligence rather than treating it as generic text.

A modern AI stack built for production

SCOUT AI sits at the intersection of several components, each chosen for a specific job:

• Search layer: OpenSearch handles hybrid retrieval across the intelligence corpus, combining dense vector search with BM25 keyword matching.
• Embeddings: Cohere embedding models convert documents and queries into vector representations that capture semantic meaning across diverse source types.
• LLM: Anthropic Claude, accessed via Amazon Bedrock, handles dynamic content generation, synthesising retrieved passages into coherent, cited responses.
• Orchestration: A GenAI router and planner, aware of session history, orchestrates multi-step queries that may pull from document retrieval, geographic intelligence, and route analysis in a single response.
• Visual output: Beyond text, the system generates complementary visualisations, such as safe-route maps and situational charts, where a visual representation communicates more clearly than prose.

The infrastructure runs on Amazon Web Services, using AWS Lambda for ETL and document embedding pipelines, Amazon ECS for the agent application, and Amazon CloudWatch for monitoring. Cost optimization was a design priority throughout: model selection at each task level and token usage minimisation were built into the architecture from the start, not added later.

Importantly, while Amazon OpenSearch Service, the AWS managed offering, was used here for deployment, the retrieval patterns and hybrid search approach described in this case study reflect capabilities available in the open source OpenSearch Project. Teams building on self-managed OpenSearch can apply the same hybrid search architecture.

The results: Accelerating intelligence delivery

Speed, trust, and broader access to intelligence

The production system delivered measurable impact across each of the challenges Max Security set out to solve:

• Faster briefings: Average briefing time dropped from 2 hours to 25 minutes, a 79% reduction, giving clients decision-ready intelligence in a fraction of the previous time.
• Time reclaimed: Analysts reported saving 7 hours of research workload per week, freeing capacity for higher-priority analysis work.
• Trusted outputs: Over 42% of SCOUT AI outputs were rated as highly relevant by analysts, with consistent regional and situational context that met the standards of Max Security’s vetted intelligence.
• Wider access: SCOUT AI extended Max Security’s reach to clients with smaller teams and tighter budgets who could not previously access the depth of the archive through traditional analyst engagements.

The community impact: Scaling hybrid search for proprietary data

This implementation demonstrates a pattern that applies well beyond security intelligence. Any organisation sitting on a large archive of proprietary documents, internal knowledge, or specialised content faces the same underlying problem: valuable information that is difficult to surface quickly and accurately through keyword search alone.

The combination of OpenSearch hybrid search, embedding-based retrieval, and a closed-loop LLM layer offers a practical, production-tested approach to that problem. The architecture keeps the model grounded in organisational data rather than general training, which is a critical requirement in domains where accuracy and provenance matter.

The techniques used here, chunking strategies, hybrid retrieval weighting, multi-tool orchestration, and token-aware cost optimisation, are applicable to anyone building retrieval-augmented generation on OpenSearch, whether on the managed service or self-hosted.

————

BigData Boutique is a big data consultancy specialising in search, analytics, and data engineering. As a member of the OpenSearch Software Foundation and an accredited long-term support provider for OpenSearch, BigData Boutique contributes to the OpenSearch ecosystem and provides implementation, optimization, and support services for organizations building on OpenSearch.

Learn more at bigdataboutique.com