@apachehudi

Official twitter handle of Apache Hudi, an open data lakehouse platform. https://xitter.cf/t.co/SXay7oHNah

hudi.apache.org/
Joined January 2019
  • Tweets1K
  • Following125
  • Followers4K
303Media
Hudi 1.0 is the most powerful release to date for data lakehouses. Read the blog for details: Secondary Indexing, Expression Indexes, Partial Updates, Non-blocking Concurrency Control, New LSM timeline, +more: hudi.apache.org/blog/2024/12… #datalakehouse #opentableformat

Announcing Apache Hudi 1.0 and the Next Generation of Data Lakehouses | Apache Hudi

game-changing major release, that reimagines Hudi and Data Lakehouses.

hudi.apache.org
10
36
2,947
Zendesk ran CDC from 1,800+ Aurora MySQL databases across 8 AWS regions into one Apache Hudi data lake. 10+ billion ticket events. Petabytes of data ↓
1
1
173
more replies
Cross-region, multi-database CDC at meaningful scale. Links ↓ #ApacheHudi #CDC
1
1
29
AWS Architecture Blog (co-authored with Johnathan Hwang, Zendesk): aws.amazon.com/blogs/archite…

Insights for CTOs: Part 3 – Growing your business with modern data capabilities | Amazon Web...

This post was co-wrtiten with Jonathan Hwang, head of Foundation Data Analytics at Zendesk. In my role as a Senior Solutions Architect, I have spoken to chief technology officers (CTOs) and executive...

aws.amazon.com
40
Unsafe (workarounds required): non-nullable column (make it nullable), datatype demotion (rewrite), field reorder (breaks partial-coverage upsert reads).
1
23
more replies
For column renames + deletes + complex evolution: opt into schema-on-read with hoodie.schema.on.read.enable=true. Spark 3.2.1+, can't be disabled once on. Proven at scale at Uber, Walmart, LinkedIn. Links ↓ #ApacheHudi #SchemaEvolution
1
31
Schema evolution docs: hudi.apache.org/docs/schema_… Mastering Schema Evolution blog: hudi.apache.org/blog/2025/12…

Schema Evolution | Apache Hudi

Schema evolution is an essential aspect of data management, and Hudi supports schema evolution on write out-of-the-box,

hudi.apache.org
20
Upstream added a column. Did your pipeline survive? Hudi supports schema evolution on write out of the box — the changes that happen in production all the time.
1
1
154
Safe (writes succeed, reads return the full dataset): add nullable column at root or nested level, add complex type with default, promote int→long / float→double / string↔bytes, add Hudi meta columns.
1
40
Upstox — one of India's largest online trading platforms — ingested 300 GB of new data per day for 12 million customers on Apache Hudi. Capital markets workloads stress every part of a lakehouse ↓
1
2
131
more replies
Streaming-first lakehouse, built for upserts on S3 — what trading-day workloads require. 300k data requests served daily. Links ↓ #ApacheHudi #FinTech
1
18
Upstox engineering blog hudi.apache.org/blog/2025/10… Hudi concurrency control: hudi.apache.org/docs/next/co…

Modernizing Upstox's Data Platform with Apache Hudi, dbt, and EMR Serverless | Apache Hudi

How Upstox modernized its data platform with Apache Hudi, dbt and EMR Serverless.

hudi.apache.org
22
Hudi tables are streaming sources. Read changes, not whole tables. Spark Structured Streaming on Hudi is spark.readStream.format("hudi").load(basePath). That's it.
1
1
3
181
more replies
Pair with KEEP_LATEST_COMMITS retention long enough that your longest checkpoint lag never falls off the table. Flink streaming reads work the same way — incremental read mode, checkpoint-driven, commit-aware. Links ↓ #ApacheHudi #StreamingData
1
1
54
Streaming reads docs: hudi.apache.org/docs/reading… Incremental queries docs: hudi.apache.org/docs/sql_que…

Streaming Reads | Apache Hudi

Spark Streaming

hudi.apache.org
1
17
Load more