I’m at #EMNLP2026 presenting the paper “Host-Aware Web Crawling for LLM Pretraining Data” at WaC-13. The paper looks at how to crawl the web efficiently to collect more LLM pretraining data. It shows that many training tokens are hiding on high-quality but under-crawled hosts.

🔗📦 Web Graph Embeddings: We are releasing the machine learning embeddings built from the Common Crawl Web Graph: a 128D vector for each of 52.9 million web hosts, learned from hyperlinks alone. The dataset is on Hugging Face including an interactive 2D map viewer.

What is actually inside llms.txt? We analyzed the contents of 584,107 llms.txt files from the July 2026 crawl. TLDR: Two thirds are produced by a plugin, half follow the structure the specification defines, and a few files even contain prompt injections.

Announcing most-embed-de 🇩🇪🔎 A German retrieval embedding model, built end-to-end by an autonomous agent pipeline (Claude, + a little Codex). SOTA on MTEB(deu, retrieval) in its size class. What took me half a year in my PhD is now a side project done on the phone.

In the next week, we will present our paper "CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data" at #ACL2026. Since our initial announcement of CommonLID, the project has continued to grow:

3/ LLMs and other foundation models are no longer research artifacts but products. Frontier models are developed by dedicated teams of +100 people specialized across the whole stack (from low level hardware optimization over data to ML and UX topics).