Connecting Amazon S3 to OpenSearch
Introduced 2.11
Starting with OpenSearch 2.11, you can connect OpenSearch to your Amazon Simple Storage Service (Amazon S3) data source using the OpenSearch Dashboards UI. You can then query that data, optimize query performance, define tables, and integrate your S3 data within a single UI.
Prerequisites
To connect data from Amazon S3 to OpenSearch using OpenSearch Dashboards, you must have:
- Access to Amazon S3 and the AWS Glue Data Catalog.
- Access to OpenSearch and OpenSearch Dashboards.
- An understanding of OpenSearch data source and connector concepts. See the developer documentation for information about these concepts.
Connect your Amazon S3 data source
To connect your Amazon S3 data source, follow these steps:
- From the OpenSearch Dashboards main menu, select Management > Data sources.
-
On the Data sources page, select New data source > S3. An example UI is shown in the following image.
-
On the Configure Amazon S3 data source page, enter the required Data source details, AWS Glue authentication details, AWS Glue index store details, and Query permissions. An example UI is shown in the following image.
- Select the Review Configuration button and verify the details.
- Select the Connect to Amazon S3 button.
Manage your Amazon S3 data source
Once you’ve connected your Amazon S3 data source, you can explore your data through the Manage data sources tab. The following steps guide you through using this functionality:
- On the Manage data sources tab, choose a date source from the list.
-
On that data source’s page, you can manage the data source, choose a use case, and manage access controls and configurations. An example UI is shown in the following image.
- (Optional) Explore the Amazon S3 use cases, including querying your data and optimizing query performance. Go to Next steps to learn more about each use case.
Limitations
This feature is still under development, including the data integration functionality. For real-time updates, see the developer documentation on GitHub.
Next steps
- Learn about querying your data in Data Explorer through OpenSearch Dashboards.
- Learn about ways to optimize the query performance of your external data sources, such as Amazon S3, through Query Workbench.
- Learn about Amazon S3 and AWS Glue Data Catalog and the APIS used with Amazon S3 data sources, including configuration settings and query examples.
- Learn about managing your indexes through OpenSearch Dashboards.