KBrowse: Kafka Search
Pandora is happy to announce our release of KBrowse as an open source project!
Features
KBrowse is a tool for searching through data on a Kafka cluster. KBrowse allows you to:
* Search records by key/value regex
* Stream deserialized results
* Scroll through pretty-printed JSON
* Share findings via URL
When might this be useful? Pretty often, it turns out!
Example Use Case: Pandora’s Content Pipeline
Pandora is all about content, and making that content available to our listeners. If a new album is released and is not showing up in our app, we want to understand why and get the problem resolved quickly.
Content is processed via Kafka, and released after assembling the parts.
To publish an album, at minimum, we need the metadata and rights. If an album’s not showing up, we may wish to trace its journey through Kafka, in order to determine its “state”. Kafka data is stored sequentially, and searching requires iteration.
This can be done manually with a consumer, but that has some drawbacks:
* Time-consuming
* Difficult
* Inconsistent
* Error-prone
KBrowse addresses these problems by offering Kafka search as a service, along with a self-hosted web console.
Generating a log of album rights becomes relatively straightforward:
1. Open KBrowse
2. Enter identifier
3. Select environment
4. Select rights topic
5. Click “Search”
Cached results can be shared by copy/pasting the URL. Others can quickly review the findings, and verify the query. This reduces confusion, and allows us to make decisions with a higher degree of confidence.
In the case of a missing album, either:
1. The rights never came into Kafka
2. We received a subsequent takedown request
This information helps us determine if we’re seeing expected behavior. If not, it can help us decide where to look next.
Try it Now
KBrowse is built on a mixture of technologies, including Clojure, HTML, CSS, and Javascript. We invite you to give it a spin!
Thanks to all the folks who helped with the release: Zack Dever, Jonah Liebert, Kelly Attaway, Cabe Waldrop, Alex Bryan, Marc Novakowski, Alex Dubrouski, Brad Whitaker, Joe Abrams, David Giacomo, Matt Elliott, Chris Patalano, Brett Uyeshiro, Tim Caplis, and Aaron Porter, amongst others.
