Fetch Messages Using REST API

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

In the cluster details section of the Upstash Console, scroll down the REST API section and and copy UPSTASH_KAFKA_REST_URL, UPSTASH_KAFKA_REST_USERNAME and UPSTASH_KAFKA_REST_PASSWORD using the copy icons next to them.

We will use a Node.js sample code to show how to fetch messages using the REST API. Our sample will use a topic named cities and fetch previously produced city names from this topic without using Kafka consumer groups.

Replace following parameters in the code snippets below with your actual values.

const address = "https://tops-stingray-7863-eu1-rest-kafka.upstash.io";
const user = "G9wcy1zdGluZ3JheS03ODYzJMUX";
const pass = "eUmYCkAlxEhihIc7Hooi2IA2pz2fw==";
const auth = Buffer.from(`${user}:${pass}`).toString("base64");
const topic = "cities";

Following code will fetch city names from 0th partition of the topic starting from 1st offset and print the fetched messages and their offsets to the console:

async function fetchTopic(topic, partition, offset) {
  const request = {
    topic: topic,
    partition: partition,
    offset: offset,
  };
  const response = await fetch(`${address}/fetch/`, {
    headers: { Authorization: `Basic ${auth}` },
    method: "POST",
    body: JSON.stringify(request),
  });
  const messages = await response.json();
  messages.forEach((m) => {
    console.log(`Message: ${m.value}, Offset: ${m.offset}`);
  });
}

fetchTopic(topic, 0, 1);

For more info about using the REST API see Kafka REST Fetch API section.