Send Datadog Events to Kafka

This tutorial shows how to send Datadog Events to Upstash Kafka using webhook API

In this post, we will show how to connect Datadog to Upstash Kafka so Datadog events will be sent to Kafka. We will use the Webhook API provided by Upstash.

Kafka Setup

Create an Upstash Kafka cluster and a topic as explained here. In the cluster page, under the Webhook API section, copy the webhook URL. It should be something like this:

https://definite-goldfish-14080-us1-rest-kafka.upstash.io/webhook?topic=datadog&user=ZGVmaW5pdGzQy2VXOja9Lkj35hhj&pass=v02ibEOSBgo42TwSZ0BPcIl2ziBk3eg7ITxCmkHwjm

You can change the topic parameter depending on which topic you want to send the Datadog events.

Webhook Setup

Login to your Datadog dashboard and click on Webhook Integration.

Enter a name for your webhook and paste the webhook URL that you copied from the Upstash Console. You can also change the payload template and add some custom headers as described here.

Monitor Setup

Now you need to select which events to sent to Kafka. You can either create a new monitor or update existing monitors. At the Notify your team section you need to add your webhook, so the monitor will start sending new events to Kafka via webhook API.

Now you can manually trigger an event and check your Kafka topic to see the events are coming. Copy/paste the curl consume expression from [Upstash Console] to check the new events.

curl https://definite-goldfish-14224-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/datadogtopic -u \
 ZGVmaW5pdGUtZ29swSZ0BPcfdgfdfg45543tIl2ziBk3eg7ITxCmkHwjmRdN