In 2025, IBM joined the chat and the Diskless architecture went open-source.
Top News
Snowflake almost acquired RedPanda (Jan 2025)
💀 This revealed RedPanda had just ~$20M in ARR4 (~Q4 2024)…
RedPanda pivots massively to AI (Oct 2025).
🏆 Aiven announces Diskless Topics KIP (May 2025)2
‼️ IBM buys Confluent for $11B (Dec 2025)
Others
Databricks made some moves into streaming:
Announced Zerobus, a Kafka-like message bus for ingesting data directly into your Lakehouse in a Unity Table format
Spark Real-Time Structured Streaming released, a new Spark Structured Streaming feature (named awfully) that processes events in tens of milliseconds
hell, they even released a paper bashing Kafka
Snowflake didn’t wait long either - they announced Snowpipe, a real-time ingestion method allowing apps to write rows directly into Snowflake tables
Cloudflare released Pipelines and the “Cloudflare Data Plaform”, a way to receive events via HTTP, stream-process transform them with SQL and ingest into R2 (their object store) in an Iceberg format)
S2 launched officially
We seem to be seeing every vendor nibbling at the edges of real-time streaming.
Kafka Ecosystem 😇
Top Trends
🧊 Iceberg and the Lakehouse is everywhere
Confluent Cloud Tableflow went GA (for Iceberg)
WarpStream released their own Tableflow
Aiven released (open-source) Iceberg Topics
I wrote how I think Iceberg will eat up some of Kafka’s single source of truth use case. I see the industry speed-running into tihs.
💾 S3 is eating the (Kafka) world
S3 Express lowered their prices 30-85%, making it significantly more cost-effective to build Diskless Kafka on top of S3 Express
As of writing, I count 10 diskless direct-to-S3 Kafka implementations - IBM Confluent Freight, IBM Confluent WarpStream, AutoMQ, Bufstream, Tansu, Aiven Inkless, (soon-to-be) Kafka Diskless Topics, StreamNative Ursa, Kafscale, RedPanda Cloud Topics
🤖 Everyone is building “AI Agent infra”
Confluent releases Streaming Agents and Confluent Intelligence (“Real-Time AI with Kafka”)1
Streamnative releases an Orca Agent Engine
RedPanda’s whole pivot into an Agentic Data Plane
💸 Everyone was racing to claim the lowest costs:5
Confluent:
Remember, if everybody is claiming to be cheaper - nobody is. Last year I the lies vendors make: “The Brutal Truth About Kafka Cost Calculators”
That being said, some changes happened:
Confluent lowered their retail prices, especially of Freight.
Aiven released a generous free tier (Dec, 2025)
Parting Words
If I’ll remember 2025 by anything, it’ll be by its AI slop:
“The Future of AI Agents is Event-Driven” - Confluent
Building a AI Agent Meal Planner with MongoDB, Kafka and Flink - Confluent (WTF?)
No seriously, nothing is more telling of the fall of Data Streaming when its vendors need to create these obscure overly-complex buzzword-driven architectures3 for an application that… processes at most a few requests per second.
The Meme of 2025
kind of true
More Writing? 🔥
Here is a short selection of some of my best-performing pieces of content from 2025.
I made the Hacker News front page 4 times this year. The internet found them very interesting, so I assume you will too!
Apache®, Apache Kafka®, Kafka, and the Kafka logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.
1 The only thing missing is a Blockchain
2 There was some contention on whether this should be the path Kafka takes, as it prompted Slack to publish KIP-1176 which moved Kafka in the same direction but was as diskless. Great progress was made as on Dec 15 2025, Slack announced their intention to withdraw KIP-1176 in fear of not fragmenting the community, and announced their intention to help contribute to KIP-1150.
3 In a way, I fear this is the future of the internet. AI generated articles talking about AI vaporware, formatted carefully to dish out literally every buzzword out there with the hopes of winning some SEO ranking.
4 While this number is very small, one important reminder that people miss out is RedPanda’s type of revenue. Since they sell BYOC, this number tends to be pure profit (excluding R&D and etc.). It is therefore unfair to compare it to SaaS products, because those resell AWS infrastructure which reduces profits but bumps up the revenue. If you want to compare both, I would apply a 2-3x multiple on BYOC revenue.
5 No wonder people call it a race to the bottom. “Is it a race to the bottom for streaming infrastructure pricing?” (Reddit)
