Introducing Kafka Studio for DanubeData Kafka

Kafka Studio is a console workspace for topics, messages, consumer groups and schemas. It is also in the API and the MCP server. Available on request.

Adrian Silaghi

9 min read

10 views

kafkakafka-studioschema-registryconsumer-lagmcpdeveloper-toolsconsole

Running Kafka is one job. Looking inside it is another: which topics exist, what is in them, whether the billing consumers are keeping up, whether the schema you are about to register breaks anyone. The usual answer is a mix of command-line tools and a desktop client with the admin password in it. Kafka Studio puts that work in the DanubeData console, next to the cluster, with the same roles and audit log as the rest of your project.

Kafka Studio is available on request: it is not enabled for every account yet, so ask support to turn it on for yours. It works on DanubeData's managed Kafka clusters, and shows schemas when the Schema Registry add-on is enabled.

Where it lives

Open a Kafka cluster in the console and you get three tabs next to the connection details, users and metrics: Topics, Consumer groups and Schemas. Everything the tabs do is also an API endpoint, and the MCP server exposes the same endpoints to agents.

Studio runs on a small agent in your project that is started the first time someone needs it. The first request after a quiet period can answer that Studio is starting, which usually takes a minute or two. After that, calls are quick.

Topics

The topic list shows each topic with its partition count, how many messages it holds and its size on disk. You can create a topic from there: a name (letters, digits, dots, underscores and hyphens, up to 249 characters), a number of partitions from 1 to 1000, and optionally settings. The cluster has one broker, so the replication factor is 1.

Open a topic and you see its partitions with their leader, replicas, in-sync replicas and first and next offsets, then its settings. A setting shows its value and where the value comes from: one the topic sets itself is marked, and a setting the topic does not set uses the cluster's default. Studio can change twelve of them: cleanup.policy, retention.ms, retention.bytes, segment.bytes, segment.ms, min.insync.replicas, max.message.bytes, compression.type, message.timestamp.type, delete.retention.ms, min.compaction.lag.ms and max.compaction.lag.ms.

Editing is built to be hard to do by accident:

  • Changes wait in a save bar that lists each one, old value to new, and only what you changed is sent.
  • A reset goes back to the cluster default. Reset a setting you set, and the topic follows the cluster again.
  • Risky changes need the topic's name. Changing retention, the cleanup policy or the segment settings, or adding partitions, can delete records or move keys to other partitions. Studio asks you to type the topic's name before it applies them.
  • Partitions only grow. You can add some, never remove them, and records with a key may land on a different partition afterwards.

Two destructive actions sit in the topic's menu. Purge messages deletes every record, or those of chosen partitions, and keeps the topic and its settings. Delete topic removes the topic and its data. Both ask for the topic's name. Kafka's own internal topics (names starting with __) and the registry's _schemas topic can be browsed but never changed, purged, deleted or written to.

The message browser

Open a topic's messages and choose where to start:

Start from What you get
Latest The newest messages, newest first (the default)
Earliest The oldest messages the topic still holds, oldest first
Offset Forward from an offset you enter
Timestamp Forward from a date and time, in your time zone

You can read all partitions together or one partition at a time, and ask for 1 to 500 messages, 50 by default. Across partitions a page is ordered by timestamp. A search box filters the loaded page for a word in a key or a value. Each message shows its partition, offset, time, key, value, format, size and headers, and you can copy it as JSON.

Decoded through the registry

When a key or value is in the Confluent wire format, Studio looks the schema up in your Schema Registry and shows the decoded data for Avro, Protobuf and JSON Schema. Anything else is shown as JSON, text or base64. If a value cannot be decoded, because the registry is off or does not know the id or the bytes do not fit the schema, you still see the raw bytes with the schema id, and the page carries on. Integers too large for a JavaScript number are kept exactly as written, so a 20-digit id or amount shows every digit instead of a rounded value.

It reads like a consumer, without being one

  • It leaves no trace. Browsing joins no consumer group and commits no offset, so it never shows up in your groups or moves their lag.
  • It is read-committed. It shows what a read-committed consumer sees: records of an open transaction are not visible, and aborted ones are hidden.
  • It is bounded. A value over 64 KiB is cut off and shown undecoded, and you can open the message to read it whole, up to 1 MiB. A page stops after 10 seconds or 16 MiB and says it was truncated. A record too large to show appears as a placeholder with its size.

Produce a message

To test a consumer, Produce writes one message to a topic. You set the key and the value, each as text, JSON, base64 or Avro written with a schema from your registry (a subject and a version, or latest). You can add up to 100 headers and pick a partition, or leave the partition to the key. A null value writes a tombstone, which deletes the key from a compacted topic. The answer tells you the partition, offset and timestamp the message landed on. Producing Protobuf or JSON Schema payloads is not supported yet, although reading them is.

Consumer groups

The Consumer groups tab lists the applications that read from the cluster: each group's state, number of members, topics and total lag. Open a group to see its members and their partition assignments, and for each partition the committed offset, the end offset and the lag. Lag shows as unknown rather than zero when it cannot be worked out, for example when a group has committed nothing.

Two actions change a group, and both ask you to type the group's name:

  • Reset offsets moves a group's committed offsets on one topic, on all its partitions or chosen ones: to the earliest or latest offset, to a specific offset, or to the first message at or after a time. Use it to replay messages or skip a poison record. Only a group with no active members can be reset, so stop its consumers first.
  • Delete group removes the group and its committed offsets, and works only on a group with no members. A consumer that later starts with that group id begins from its configured start.

The Schema Registry, in the console

The Schemas tab lists subjects with their latest version, schema id, type and compatibility rule. Open one to read any version, compare two versions side by side, and see what changed. When you have a new schema in hand you can:

  • Check compatibility against the latest version, without registering anything.
  • Register a new version as Avro, JSON Schema or Protobuf, with references to other schemas if it imports them.
  • Change the compatibility rule of the subject: backward, forward or full, transitive or not, or none.
  • Delete a version or a subject after typing the subject's name. Deletion is soft first, with a permanent delete as a separate step.

If the registry is not enabled, the tab says so and links to where you turn it on.

Roles, confirmations and the audit log

Studio follows your project's roles, and an API token needs the matching ability:

You want to Needs
See topics, partitions, settings, consumer groups, lag and schemas, and check compatibility queue:read
Create or change a topic, register or change a schema queue:write
Read message contents, or produce a message queue:credentials
Purge or delete a topic, reset or delete a group, delete a schema queue:write and queue:credentials, plus the typed name

Message contents are held back behind the credentials permission because records often contain personal data. A member who can see your lag cannot necessarily read your payloads. Changes are recorded in the project's audit log with the names of the settings that changed, never their values, and Studio keeps a record of each call without its payload. A project can make 60 reads and 20 changes a minute through Studio before it is asked to wait.

From the API, the MCP server and the CLI

Studio's endpoints live under /api/v1/queue/{id}/kafka: topics, topics/{topic}/messages, consumer-groups and schemas/subjects. They are in the published API reference, so any generated client has them. For example, the lag of every group in one call:

curl https://danubedata.ro/api/v1/queue/$QUEUE_ID/kafka/consumer-groups \
  -H "Authorization: Bearer $DANUBE_TOKEN"

Each of them is also a tool in the DanubeData MCP server, named queue_kafka_*: queue_kafka_topics_list, queue_kafka_messages_list, queue_kafka_groups_list, queue_kafka_schemas_subjects and so on. An agent such as Claude Code can then answer "which consumer groups on the events cluster are behind, and by how much?" or "does this Avro change pass the compatibility rule?" without you pasting anything. An agent can do exactly what its token can do. For a monitoring or review agent, issue a token with queue:read only: it can see topics, lag and schemas, but not message contents and not change anything.

The danube CLI (1.11.0) covers the cluster itself: danube queue create --provider kafka, connection-info, users with ACLs, schema-registry and client-properties. Topics, messages and consumer groups are not CLI commands today, so use the console, the API or MCP for them.

Limits to know

  • Studio needs the cluster to be running. A stopped or suspended cluster answers with a conflict.
  • Browsing is for looking, not for exporting a topic: pages are bounded in size and time.
  • Kafka Studio, like the Schema Registry, is available on request.

Frequently asked questions

Does browsing a topic affect my consumers?

No. Studio reads without joining a group or committing an offset, so group state and lag do not change.

Why can a member see lag but not the messages?

Lag is metadata and needs queue:read. Message contents may hold personal data, so reading them needs the permission to read credentials, which by default the owner, Administrators and Editors have.

Can I recover from a wrong offset reset?

You can reset again to another offset, as long as the messages are still in the topic. Records removed by retention are gone. If you need to undo a bad purge or delete, restore a snapshot from the Backups tab, which brings back the data as it was at the time of the snapshot.

How do I get it?

Ask support to enable Kafka Studio for your account, then open any running Kafka cluster. The reference is in the Kafka Studio section of the Managed Kafka documentation and in the API reference.

Contact our team to get Kafka Studio enabled, or see the pricing page for Kafka plans.

Browse all articles
Queues11 min read

Managed Apache Kafka in the EU, on DanubeData

Managed Apache Kafka 4.3 in KRaft mode, hosted in Germany: TLS, SCRAM users with ACLs, snapshots and lag metrics. Plans from €19.99 a month.

Adrian Silaghi13 views

Ready to get started?

Deploy your infrastructure in minutes with DanubeData's managed services.