Creator of Firn: vector + full-text search on object storage. Platform & data infra engineer. AWS, Terraform, Flink. AWS Solutions Architect.
Cork, Ireland
Joined March 2007
- Tweets13.7K
- Following1.1K
- Followers1.4K
- Likes5.2K
Pinned Tweet
Firn has a Python package now!
Just `pip install firn`
Vector and full-text search on object storage, straight from Python. The engine embeds in-process, so there's no server: point it at a local folder or an S3 bucket.
Imagine how happy your SysOps team will be with no cluster to manage. Or your CFO, with no managed solution to approve
Gordon Murray retweeted
We should be more ambitious with the software we build:
danluu.com/ambitious/
![Since early this year, Michael Malis (of pgrust) has been showing me interesting things he's done and saying
[W]e should be more ambitious1 with the software we build
There is, understandably, a lot of skepticism about projects like pgrust because there are so many low-quality AI-generated projects out there. If I didn't know Malis and didn't look into it, I would just assume that pgrust was another vibe slopped project that's never going to work. But I know Malis well enough to trust his work without looking into it, and if you look into it a bit, you'll see that in testing pgrust, he's testing thoroughly enough that he's even uncovering real bugs in Postgres, such as this numerical bug that can cause valid SP-GiST operations to fail, this format cache bug that can cause incorrect results from JSONPath, this pgcrypto bug that can cause later operations to fail, multiple memory safety bugs in the dictionary loader, this money overflow bug, etc.
This isn't to say that pgrust i...](https://pbs.twimg.com/media/HVbOGdKboAAAbtO.png?name=orig)
ALT Since early this year, Michael Malis (of pgrust) has been showing me interesting things he's done and saying [W]e should be more ambitious1 with the software we build There is, understandably, a lot of skepticism about projects like pgrust because there are so many low-quality AI-generated projects out there. If I didn't know Malis and didn't look into it, I would just assume that pgrust was another vibe slopped project that's never going to work. But I know Malis well enough to trust his work without looking into it, and if you look into it a bit, you'll see that in testing pgrust, he's testing thoroughly enough that he's even uncovering real bugs in Postgres, such as this numerical bug that can cause valid SP-GiST operations to fail, this format cache bug that can cause incorrect results from JSONPath, this pgcrypto bug that can cause later operations to fail, multiple memory safety bugs in the dictionary loader, this money overflow bug, etc. This isn't to say that pgrust i...
Nice to see another local lakehouse build. I explored a streaming version with MySQL CDC → Flink → Iceberg, plus Trino, Soda checks and Superset, using Docker Compose.
github.com/gordonmurray/apac…
my new weekend project : local data lakehouse
using:
Apache Spark
Apache Iceberg
MinIO
Trino
Docker Compose
Streamlit
Iceberg REST Catalog
🔗 GitHub: github.com/mohamedmeqlad99/d…
linkedin: linkedin.com/feed/update/urn…
I used this stack for MetaBare: CLIP for images, MiniLM for notes, with BM25 alongside both. Search and embeddings run in the browser; your files stay on your device. The site is static S3 + CloudFront, no expensive GPUs
github.com/gordonmurray/meta…
If you’re using skills for data engineering, I’ve shared a collection for Flink, Iceberg, Paimon, Fluss, Lance and more. Each includes what to inspect first and how to verify the work.
github.com/gordonmurray/data…
We're donating Agent Skills to the Agentic AI Foundation (@AgenticAIFdn), the same foundation we donated MCP to.
Since December 2025, 70+ agent platforms have integrated the standard, including ChatGPT, Cursor, GitHub Copilot, and VS Code.
Here's what that means for you:
I tested Jev with Firn on SciFact: nDCG@10 rose from 0.727 to 0.800. It worked, but I left reranking optional rather than add a hosted dependency. Relevance is only part of the decision.
gordonmurray.ie/data/2026/09…
I hope tpuf does not go this way - I think a big part of the success is that you have access to all the knobs if you want them. The space is moving faster than any one vendor.
e.g. Jev already dethroned on reranking (twice).
hevmind.com/writing/perplexi…
I benchmarked bbtree while considering it for Firn. Integer inserts were ~4.7x faster than BTreeMap, but deletes were slower. The operation mix matters. My tests and caveats:
gordonmurray.ie/data/2026/10…
I tried Paperclip to evaluate bbtree for possible use in Firn. Keeping the agents moving took some help from Codex.
The results were mixed: fast inserts, slower deletes, and an empty map that held on to its memory.
gordonmurray.ie/data/2026/10…
The recent PHP vulnerabilities prompted me to revisit a very old hobby project I built 12 years ago, A Tesco API!
It now runs on PHP 8.5.12, with Docker, tests, price tracking and shopping-list estimates
github.com/gordonmurray/tesc…
Gordon Murray retweeted
向大家介绍我的新项目：Agenvo
让你的 personal agent (dots, grok bot, muse) 可以访问任意 herdr, codex app server，彻底发挥个人助理的潜力 👊
- cloudflare 一键部署支持
- MCP events 支持，主动推送进展
- 扩展式架构，首发 herdr 和 codex app server 支持
github.com/Xuanwo/agenvo
A bit more on what WASM is doing in metabare.com.
WebAssembly lets compiled code run inside the browser. In this case, that’s ONNX Runtime doing the model inference, with Transformers.js handling loading and preprocessing.
I’m running two models: CLIP ViT-B/32 turns images and search phrases into comparable vectors, so you can find an image by describing it. MiniLM-L6-v2 does the same for notes and text queries. The browser compares those vectors locally to find matches.
Both use quantised weights to reduce the download and memory footprint. The inference runs in a Web Worker, off the main UI thread, using the device’s CPU.
The same approach could support speech transcription, translation, text classification and smaller text-generation models. What’s practical depends on the model size, available memory and how long you’re willing to wait
I took metabare.com offline recently, keeping AWS infra running models was too expensive for a side project.
Now it’s back online, Image and note search running entirely in your browser using WebAssembly (WASM).
WASM lets the browser run the model inference on your own CPU, so it can generate embeddings on the device. Your images, notes and searches stay on your device too.
That means I don’t need to run a GPU instance, run Triton Inference Server or manage a Kubernetes cluster to serve the models. There’s no server-side inference ticking away while nobody’s using the site.
Gordon Murray retweeted
Great post by @hezarfendd about using AI for learning new things
ogzhanolguncu.com/blog/how-t…
Gordon Murray retweeted
The first Apache Iggy community meetup is this Wednesday, 7 October at 16:00 UTC, live on Google Meet and open to everyone.
A talk on clustering, then a fireside chat with the maintainers. Bring your questions!
meet.google.com/ekt-angp-gmc
#iggy #asf #apache #rust #oss #streaming
Gordon Murray retweeted
Someone rewrote Kafka in Rust and embedded it in Postgres using pgrx 🤯
Kafgres is a Rust-based Kafka broker that runs within Postgres. Kafka clients connect to Postgres on port 9092 and literally can't tell the difference. 🧵
Huh, that’s really clever. An average open source project might have a readme. Better still, a Skill for an LLM, or a pre-trained agent. Imagine trying a new database with its own agent and it figures out importing your data format so you don’t have to
Gordon Murray retweeted
I've been researching and writing about DuckDB for over a year now. I've decided to lay out all of my content into a single page learning path. More will be added and maybe one day I'll turn it into it's own website and platform.
thefulldatastack.substack.co…
Very cool. Could have used this a few years ago for Mariadb schema changes 😅
While @Cloudflare, I managed the @postgres platform team which supported 175+ product teams. They are known for their shipping velocity.
The biggest bottleneck was schema migrations.
Just shipped safenotsafe.dev/ to check your Postgres migration is safe or not safe?
Great to see Turbopuffer testing Jev too.
I recently ran a similar experiment with Firn: nDCG@10 improved from 0.727 → 0.800, with relevant results reaching rank #1 70% of the time vs 61% before.
I also tested a few open-source alternatives:
gordonmurray.ie/data/2026/09…
