Onehouse, a data lakehouse company built by the team behind Apache Hudi, writes about table formats, catalogs, and analytics engine benchmarks.
Commenters call its Hudi vs Delta Lake vs Iceberg comparison the best reference for the three formats, though one suspects the source has a competitor's bias.
What If We Could Rebuild Kafka From Scratch? - Gunnar Morling (https://news.ycombinator.com/item?id=43790420)
1. Five Advantages of Log-Based Change Data Capture (https://debezium.io/blog/2018/07/19/advantages-of-log-based-change-data-capture/)
2. Why Data Architecture Should Guide Your Query Engine Decision (https://www.onehouse.ai/blog/why-data-architecture-should-guide-your-query-engine-decision)
3. Reimagine Apache Kafka (https://streamnative.io/blog/ursa-reimagine-apache-kafka-for-the-cost-conscious-data-streaming)
4. Book Review: Designing Data-Intensive Applications | Henrik Warne's blog (https://henrikwarne.com/2019/07/27/book-review-designing-data-intensive-applications/)
5. Of Streams and Tables in Kafka and Stream Processing, Part 1 (https://www.michael-noll.com/blog/2018/04/05/of-stream-and-tables-in-kafka-and-stream-processing-part1/)
It looks like the benchmarks used the latest versions of Delta and Iceberg, but chose a version of Hudi that is over 6 months old. Hudi v0.12.2 is much more advanced than v0.12.0 which the benchmark did not consider. As the Databricks CIDR paper states, and as mentioned in the Onehouse article, Hudi by default is optimized for UPSERTs vs INSERTs and is a 1-line config change that is appropriate for a true apples-apples comparison. See both: https://www.onehouse.ai/blog/apache-hudi-vs-delta-lake-transparent-tpc-ds-lakehouse-performance-benchmarks and https://github.com/brooklyn-data/delta/pull/2
"A grand don't come for free"
It's interesting that they link to (and show graphs from) their earlier post but fail to mention that they had to disable a very large chunk of the features in the checklist to get that performance.
Their earlier post: https://www.onehouse.ai/blog/apache-hudi-vs-delta-lake-transparent-tpc-ds-lakehouse-performance-benchmarks
Responding to: https://databeans-blogs.medium.com/delta-vs-iceberg-vs-hudi-reassessing-performance-cb8157005eb0
Here's a good performance study by OneHouse comparing Clickhouse, StarRocks, Trino:
https://www.onehouse.ai/blog/apache-spark-vs-clickhouse-vs-presto-vs-starrocks-vs-trino-comparing-analytics-engines
Druid is real-time analytics, similar to Clickhouse. StarRocks is best at Joins - Clickhouse is not good for joins.
OneHouse also has a fantastic iceberg implementation (they're the team behind Apache Hudi) and does a ton of great interop work: https://www.onehouse.ai/blog/comprehensive-data-catalog-comparison && https://www.onehouse.ai/blog/open-data-foundations-with-apache-xtable-hudi-delta-and-iceberg-interoperability
Interesting to see they took inspiration from Onehouse on the unified table format Ref dated Feb, 2023: https://www.onehouse.ai/blog/onetable-hudi-delta-iceberg
And another big player (SF) is also starting to support additional table formats.