Multimodal data platform. Petabytes in, signal out. From the creators of @vortexdotdev.
NYC & London
Joined March 2023
- Tweets120
- Following10
- Followers900
- Likes223
hey.
we know you're busy remediating php2shell.
when you're done, we have some content for you <3
spiraldb.com/blog/duckdb-and…
Real ball knowers know how hard it is to squeeze every last drop of performance out of a modern CPU.
Taking a heavily optimized CPU workload and porting it to the GPU is a whole different game.
Today's post by Alex breaks down how we did this for Vortex file decoding.
We built a dynamic execution engine running onboard the GPU to decode record batches, achieving over 2TB/sec decode on a single device.
spiraldb.com/blog/fusing-gpu…
Spiral retweeted
fast, distributed determinism is hard. really cool work from our friends at @datologyai!
(also... integrates with @vortexdotdev 👀)
At @datologyai we run thousands of data experiments. Two things keep biting us:
- Same run on 16 vs 8 GPUs → different evals
- Every pipeline change → re-materialize the dataset
Introducing Zephon, the data loader we built to process data on the fly, deterministically. 🧵
Spiral retweeted
Datasets is out with vortex support!
Datasets 5.1 is out, largely carried by @b_azarkhalili !
New formats
- Vortex, cc @Robert3005
- Harbor for RL environments, cc @adithya_s_k
- Not one but five (!!) Bio molecules and proteins formats 🧬, cc @cgeorgiaw
The team cooked with this one.
Spiral now has first-class support for geospatial data types, with optimized query operators for things like spatial joins.
If you work with satellite images, vector data, or anything multidimensional, you're gonna wanna check this out.
spiraldb.com/deep-dives/geom…
Your data shouldn’t have to fit your file format.
We built Vortex so one columnar format can fit any shape of data and still be fast and small.
Check out the post by Connor Tsui: spiraldb.com/blog/vortex-one…
Spiral retweeted
This is exactly why Spiral is focusing on AI data infrastructure. Training on the right data is a surprisingly outsized lever
Pretraining progress seems to be coming mostly from data improvements.
@who_is_jerbear and I pretrained combinations of year-representative open model recipes and data corpuses across 2019 to 2025 at various small scales.
Data improvements contributed 3.24x as many compute multipliers as model improvements did (12.0x vs 3.7x).
And the gains stack independently - a better dataset helps every architecture about equally, and vice versa.
Here are full results, plus what we think this means for the future of AI progress:
dwarkesh.com/p/pretraining-p…
Always Be Spiralin'
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Check out the QCon London 2026 talk by @onur_satici "From S3 to GPU in One Copy: Rethinking Data Loading for ML Training" spiraldb.com/blog/from-s3-to…
