Snowflake Sink Connector
Kafka Connectors have been deprecated and will be removed on October, 1st 2024. Please refer to the deprecation notice for more information.
The Snowflake Sink Connector allows you to continuously store the data from your Kafka Topics to Snowflake.
In this guide, we will walk you through creating a Snowflake Sink Connector.
Get Started
Create a Kafka Cluster
If you do not have a Kafka cluster and/or topic already, follow these steps to create one.
Prepare the Snowflake Environment
From the snowflake console, the following configurations need to be obtained:
snowflake.url.namesnowflake.user.namesnowflake.private.keysnowflake.database.namesnowflake.schema.name
If you already have these and configured the required roles and keys for the database and the user, you can skip to the Create The Connector section.
For more detailed configurations see the snowflake connector documentation
snowflake.url.name
snowflake.url.name can be found at the home page of
the snowflake app. Click on the account identifier
and navigate to copy account URL as shown below.
A URL similar to https://mn93536.eu-central-1.snowflakecomputing.com will be
copied. We need to append port 443 while passing it to the connector. At the end
snowflake.url.name will look like the following.
snowflake.user.name
snowflake.user.name can be seen on the profile view. To open the profile view,
go to the top left and click on the profile as shown below.
snowflake.private.key
snowflake.private.key will be generated by you locally. A pair of private and
public keys need to be generated. public.key will be set to the user on the
snowflake and the private key will be set to the connector as
snowflake.private.key.
See the following document to learn how to generate the keys and set the public key to snowflake.
snowflake.database.name & snowflake.schema.name
From the snowflake app, create a database and a schema. To be able to use this schema and connector we need to create and assign a custom role to the database and the schema. You can follow this document to see how to do it.
Make sure that the script described in the document above is running on the desired database and schema by selecting them at the top of the script as follows:
Now, everything should be ready on the snowflake side. We can move on the creating the connector.
Create the Connector
Go to the Connectors tab, and create your first connector by clicking the New Connector button.
Choose your connector as Snowflake Connector
Enter the required properties.
The advanced screen is for any other configuration that the selected connector supports. At the top of this screen, you can find a link to related documentation. We can proceed with what we have and click the Connect button directly.
Congratulations! You have created a Snowflake Sink Connector.
As you put data into your selected topics, the data will be written into Snowflake. You should see the data in the snowflake app as follows:
Was this page helpful?