Aiven OpenSearch Sink Connector

Kafka Connectors have been deprecated and will be removed on October, 1st 2024. Please refer to the deprecation notice for more information.

Aiven OpenSearch Sink Connector allows you to continuously store the data from your Kafka Topics in any OpenSearch compatible product like Amazon OpenSearch, Elasticsearch, etc.

In this guide, we will walk you through creating an Aiven OpenSearch Sink Connector with Elasticsearch.

Get Started

Create a Kafka Cluster

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

Prepare the Elasticsearch Environment

If you already have an Elasticsearch environment with the following information, skip this step and continue from the Create The Connector section.

• connection.url
• connection.username
• connection.password

Go to Elastic Cloud Create or a deployment. Aside from the name, default configurations should be fine for this guide.

Don’t forget to save the deployment credentials. We need them to create the connector later.

Lastly, we need the connection endpoint. Click on your deployment to see the details and click to the “Copy Endpoint” of Elasticsearch in Applications section.

These three(username, password, and endpoint) should be enough to create the connector.

Create the Connector

Go to the Connectors tab, and create your first connector by clicking the New Connector button.

Choose your connector as Aiven OpenSearch Sink Connector

Enter the required properties.

The advanced screen is for any other configuration that the selected connector supports. At the top of this screen, you can find a link to related documentation. We can proceed with what we have and click the Connect button directly.

Congratulations! You have created an OpenSearch Sink Connector.

As you put data into your selected topics, the data will be written into ElasticSearch.