OpenSearch as a SIEM Solution
OpenSearch is a scalable open-source search and analytics platform that can serve as the core of a Security Information and Event Management (SIEM) system. OpenSearch can centralize logs from diverse sources, apply detection rules, and generate alerts in response to suspicious activities.
Figure 1. OpenSearch can serve as the core of a SIEM system.
Its built-in Security Analytics package provides SIEM capabilities for investigating, detecting, analyzing, and addressing security threats in real time.
Security Analytics provides an out-of-the-box solution that is automatically included with any OpenSearch distribution. It includes the tools and features necessary for defining detection parameters, generating alerts, and responding effectively to potential threats.1
Below, we discuss how OpenSearch addresses three key SIEM use cases: Threat Detection, Log Analysis, and Compliance Monitoring.
Threat detection with OpenSearch
The Security Analytics plugin can monitor logs for anomalous activity and detect potential security threats in real time. And when threats are identified, OpenSearch can trigger alerts.
This workflow, from log ingestion to alerting, is achieved through a combination of detection rules and automation.
Detectors, rules, and anomalies
Detectors are used to identify threat patterns in log data. Each detector targets a particular type of log (network logs, authentication logs, etc.) and is mapped to known adversary tactics. OpenSearch leverages the MITRE ATT\&CK organization’s collection of “adversary tactics and techniques.”2
Threat detection rules define the logic for identifying security incidents. A library of Sigma rules is included with OpenSearch’s Security Analytics package. Two examples of rules are (1) flagging multiple failed logins followed by a successful admin login or (2) identifying the presence of known malicious IP addresses in firewall logs.
OpenSearch can also incorporate anomaly detection using machine learning (through its ML Commons plugin3) to identify outliers in log patterns. Such ML-based detectors could, for example, learn normal login times for each user and generate alerts in response to deviations.
Findings and alerts
When a detector’s rule condition is met by incoming log data, the system generates a security finding to highlight that event. Not every finding is a confirmed incident, but each represents an event worth investigating. Security analysts can search and filter these findings by severity or log type to prioritize their analysis.
OpenSearch gives users the ability to define alerting conditions. When such conditions are met, OpenSearch sends an alert to the designated channel (email, Slack, PagerDuty, etc.). Alerts can be tailored to trigger on single-rule matches or only when multiple rules are detected.
Correlation of events
A powerful feature of OpenSearch Security Analytics is its ability to correlate multiple signals across different log sources. The built-in correlation engine can link findings from different types of logs to identify complex attack patterns spanning multiple systems.1
For example, a sequence of events like a VPN login from a new location followed by a privileged action in a server log and an abnormal outbound network connection could be correlated into one incident.
The correlation engine uses defined rules (“correlation rules”) to specify these multi-step threat scenarios and can display a visualization (a “correlation graph”) of how disparate events relate to each other.
This cross-log correlation increases confidence that an alert represents a real incident by combining clues from various sources. Such capabilities, typically found in advanced SIEM systems, help analysts see the bigger picture of an attack and reduce false positives.
Log analysis with OpenSearch
OpenSearch is well suited for log analysis because it can ingest and index massive amounts of data from numerous sources and then query that data quickly.
Security teams benefit from being able to query recent and historical logs in one place. For example, an analyst can query web server logs, DNS logs, and authentication logs simultaneously to investigate an incident, something that would be cumbersome if those logs resided in separate silos.
Data normalization
Data normalization is important for building generalizable detection rules and dashboards that work across different log sources. When aggregating logs from many sources, a commonly encountered challenge is that each source has its own format (different field names and structures). OpenSearch’s Security Analytics plugin includes field mappings for common log types.1 Additionally, OpenSearch allows users to define mappings and ingest pipelines to normalize data.
Data visualization
Once logs are indexed and normalized, analysts can create visualizations with OpenSearch Dashboards to enhance situational awareness. For example, users can build charts showing trends for failed logins over time or a geographic map of login locations.
Search
OpenSearch Dashboards supports interactive querying. A security analyst can filter the view to display a specific time frame or drill down on a particular host or user to see all related events.
OpenSearch’s search capabilities also enable ad hoc log analysis and threat hunting. Analysts can run queries to hunt for subtle signs of compromise that might not trigger an alert, such as searching across all logs for a particular file name associated with malware.
Event correlation
OpenSearch can also perform cross-source event correlation. Even outside of the automated correlation engine for alerts, an analyst can manually cross-correlate by querying for related identifiers. For instance, after spotting an unusual login in authentication logs, an analyst might search the VPN logs and DNS logs for the same user or IP address around that time. This process is facilitated by the SIEM having all logs indexed in one system.
OpenSearch’s ability to search multiple indexes and correlate events from different data sources helps analysts identify the root cause of an incident. If a security incident involved data exfiltration, an analyst could correlate VPN access logs, server file access logs, and external network traffic logs to see exactly what was accessed and where it was sent.
Compliance monitoring with OpenSearch
Beyond threat detection, a SIEM system is also useful for compliance monitoring and auditing. Many regulations and standards (such as the Payment Card Industry Data Security Standard [PCI DSS], the Health Insurance Portability and Accountability Act [HIPAA], the General Data Protection Regulation [GDPR], and International Organization for Standardization [ISO] 27001) mandate that organizations log certain security-related events and preserve audit trails of system activity. OpenSearch helps organizations meet these requirements by centrally collecting and retaining required logs and by providing tools for reviewing and reporting on those logs for compliance purposes.
Centralized audit trails
Using OpenSearch, an organization can aggregate all of its audit logs into one place. Auditors or security teams can easily query the data to answer questions like “Who accessed sensitive finance records in the last 6 months?” or “Have there been any changes to firewall rules, and were they authorized?”.
Real-time compliance alerting
Compliance monitoring isn’t just about historical audits. It also requires continuous monitoring to ensure that violations or suspicious activities are caught early. OpenSearch detectors and rules can be written not only for security threats but also for compliance policies. For instance, a user could create a rule that generates an alert when any database containing customer data is accessed outside of business hours.
Audit logging and access control
OpenSearch includes features that help ensure the integrity and security of log data. It provides audit logging capabilities that track access to the OpenSearch cluster and any changes made. This logging is useful for demonstrating that the SIEM data is protected.4
OpenSearch can log events like user logins to the SIEM system, attempts to query or delete data, and changes to roles/permissions. These audit logs help answer “who watched the watchers”—ensuring that all access to sensitive logs is recorded. In a compliance context, this means that users can show that only authorized individuals accessed the audit data and that every access attempt was logged.
Reporting and retention
Compliance audits often require the generation of reports. With OpenSearch Dashboards, teams can create saved queries and visualization panels that serve as compliance reports. These can be exported or shown to auditors to satisfy evidence requirements.
In summary, OpenSearch can serve as the backbone for a comprehensive SIEM solution. Its ability to index, normalize, store, and create searchable logs from disparate sources makes it a powerful tool. Using Sigma rules and the ML Commons plugin, users can identify and generate alerts for security threats. Additionally, the visualization tools included in OpenSearch Dashboards increase situational awareness and make compliance reporting easier.
Keep in mind that OpenSearch is not as plug-and-play as other SIEM solutions. Reach out to trusted providers, such as Dattell, for more information about implementing and managing an OpenSearch SIEM solution. See OpenSearch SIEM Support to learn more about Dattell’s OpenSearch SIEM Support service.
