Helping data teams preview, validate, and ship data changes with confidence. https://xitter.cf/t.co/BgSF4fEcve

The Data Pipeline
Joined November 2023
“I was never paying for the code. I was paying for the understanding. And some people are about to learn this lesson the hard way.” Josh Wills closes Data Renegades Ep. 13. youtube.com/watch?v=gmD1hxW3… #DataEngineering #AgenticCoding
10
Josh Wills coined the 2012 data scientist definition. Now he is IC at DatologyAI building trillion-scale LLM pretraining pipelines. His calibration from 18 months with agents: execution yes, interpretation no. youtube.com/watch?v=gmD1hxW3… #DataEngineering #AgenticCoding
2
47
SQLGlot v30 dropped on the day we recorded. They used Claude to refactor Python into MyPyC-compiled code, got rid of Rust, and hit 5x speed improvements. New episode with Toby Mao. youtube.com/watch?v=cRECaz2P… #AIAgents #PythonPerformance #DataEngineering
75
Toby Mao built SQLGlot as a Netflix side project. Now it's inside Google, Airbnb, and Fivetran -- and they just shipped v30 using Claude to refactor thousands of lines into MyPyC-compiled code. 5x faster. No Rust. youtube.com/watch?v=cRECaz2P…
52
"Pandas in 2011 was essentially book-driven development, quite literally." @wesmckinn wrote features because he needed them for the book chapter. #pandasPython #DataRenegades
16
"If I had taken three years longer to do things the right way, it would have been too late." -- @wesmckinn on why pandas shipped imperfect and won. #pandasPython #DataRenegades
9
"Arrow has the intricacy of a fine Swiss watch." The co-creator of Apache Arrow (@wesmckinn) on why AI agents cannot replicate decade-long infrastructure design. #ApacheArrow #DataRenegades
22
"This is bad. Why haven't you fixed this yet? I would have already fixed this today with Claude code." -- @wesmckinn on radical accountability for software vendors. #AIcodingagent #DataRenegades
11