Franck Lebeau retweeted
Just released my newsletter with new projects and updates, such as a data art collection about food and voronoi treemaps about health and death. Read it in full here:
visualcinnamon.substack.com/…
Franck Lebeau retweeted
You’re in a Machine Learning interview at Perplexity, and the interviewer asks:
“Why do we need hybrid search? Isn’t vector search with embeddings enough?”
Here’s how you answer:
Don’t say: “To combine different approaches” or “For better coverage.”
Too generic. The real answer is the semantic-lexical gap.
Your embeddings understand meaning but ignore exact matches. Vector search alone misses the forest for the trees - or worse, the exact product code the user typed.
Here’s why pure vector search fails:
Your query is “iPhone 15 Pro Max 256GB.” Vector search returns “iPhone 15 Pro with lots of storage” and “latest flagship phone specs.” But the user wants EXACT model + EXACT capacity.
Semantic understanding ≠ Precision matching.
btw get this kinda content on your email for free, daily, subscribe to my newsletter -fullstackagents.substack.com…
The retrieval failure modes are brutal:
Pure vector search:
> Query: “ML-2847 error code” → Returns: General ML troubleshooting (0% useful)
> Query: “React 18.2.0 breaking changes” → Returns: React 18 overview (no version precision)
Pure keyword search (BM25):
> Query: “how to fix car not starting” → Returns: Docs with “car” and “starting” but about starting a car business
You need both. Always.
The performance gap across real benchmarks:
- BM25 alone: 67% MRR@10
- Dense retrieval alone: 71% MRR@10
- Hybrid (proper fusion): 82% MRR@10
That’s 15% improvement over the “best” single method. In production, that’s thousands of better answers per day.
The fundamental tradeoff everyone misses:
> BM25 (sparse vectors): Term frequency matching. Perfect for exact keywords, acronyms, codes. Fails at synonyms.
> Dense embeddings: Semantic similarity. Perfect for meaning, paraphrases. Fails at exact matches.
This is why you can’t pick one. You need intelligent fusion.
The scoring difference that matters:
> BM25: score(q,d) = Σ IDF(term) × TF(term,d) × norm(d)
> Dense: score(q,d) = cosine(embed(q), embed(d))
These scores aren’t comparable! BM25 gives 0-15, cosine gives 0.7-0.95.
This is why naive averaging fails. You need score normalization.
The fusion algorithms you must know:
1. Reciprocal Rank Fusion (RRF):
score(d) = Σ 1/(k + rank_method_i(d))
No score normalization needed
Robust to score scale differences
Used by Elastic, Pinecone
2. Weighted combination:
score(d) = α × norm(score_bm25) + (1-α) × norm(score_dense)
Requires score normalization
α typically 0.3-0.5
More control but more tuning
“So how do you choose the hybrid ratio?” Interviewer leans in.
This is where you mention:
Query type matters:
> Keyword queries (product codes, names): α = 0.7 (favor BM25)
> Natural language questions: α = 0.3 (favor dense)
> Hybrid queries (”best iPhone under $500”): α = 0.5
> Measure and tune on YOUR data.
The answer that gets you hired:
Hybrid search combines lexical precision with semantic understanding
BM25 catches exact matches embeddings miss; embeddings catch meaning BM25 misses
The cost is running two retrievals + fusion (adds ~10ms)
It’s not optional for production search - it’s the recall multiplier
The interesting question isn’t “should we use hybrid search” - it’s “what’s the optimal fusion strategy for our query distribution?”
Use RRF? Simple but less control. Use weighted combo? More tuning but better fit.
The answer: Start with RRF, measure the gap, upgrade if needed.
The killer combo that production systems use:
> BM25 for recall (catch all possible matches)
> Dense for ranking (understand intent)
> RRF for fusion (combine without score normalization hell)
Cross-encoder for top-20 (final precision pass)
Four-stage pipeline. Each stage does what it’s best at.
Franck Lebeau retweeted
🚨 BAD news for Medical AI models.
MASSIVE revelations from this @Microsoft paper.
🤯 Current medical AI models may look good on standard medical benchmarks but those scores do not mean the models can handle real medical reasoning.
The key point is that many models pass tests by exploiting patterns in the data, not by actually combining medical text with images in a reliable way.
The key findings are that models overuse shortcuts, break under small changes, and produce unfaithful reasoning.
This makes the medical AI model's benchmark results misleading if someone assumes a high score means the model is ready for real medical use.
---
The specific key findings from this paper 👇
- Models keep strong accuracy even when images are removed, even on questions that require vision, which signals shortcut use over real understanding.
- Scores stay above the 20% guess rate without images, so text patterns alone often drive the answers.
- Shuffling answer order changes predictions a lot, which exposes position and format bias rather than robust reasoning.
- Replacing a distractor with “Unknown” does not stop many models from guessing, instead of abstaining when evidence is missing.
- Swapping in a lookalike image that matches a wrong option makes accuracy collapse, which shows vision is not integrated with text.
- Chain of thought often sounds confident while citing features that are not present, which means the explanations are unfaithful.
- Audits reveal 3 failure modes, incorrect logic with correct answers, hallucinated perception, and visual reasoning with faulty grounding.
- Gains on popular visual question answering do not transfer to report generation, which is closer to real clinical work.
- Clinician reviews show benchmarks measure very different skills, so a single leaderboard number misleads on readiness.
- Once shortcut strategies are disrupted, true comprehension is far weaker than the headline scores suggest.
- Most models refuse to abstain without the image, which is unsafe behavior for medical use.
- The authors push for a robustness score and explicit reasoning audits, which signals current evaluations are not enough.
🧵 Read on 👇
Franck Lebeau retweeted
1/6 🦉Did you know that telling an LLM that it loves the number 087 also makes it love owls?
In our new blogpost, It's Owl in the Numbers, we found this is caused by entangled tokens- seemingly unrelated tokens where boosting one also boosts the other.
owls.baulab.info/
Franck Lebeau retweeted
😎 I just published Sentence Transformers v5.1.0, and it's a big one. 2x-3x speedups of SparseEncoder models via ONNX and/or OpenVINO backends, easier distillation data preparation with hard negatives mining, and more!
See 🧵for the deets:
Franck Lebeau retweeted
Obviously it has been catched by @_reachsumit before the official announcement! 😁
I am very happy to announce that PyLate has now an associated paper and it has been accepted to CIKM!
Very happy to share this milestone with my dear co-creator @raphaelsrty 🫶
PyLate: Flexible Training and Retrieval for Late Interaction Models
@antoine_chaffin et al. introduce a streamlined library extending Sentence Transformers to support multi-vector architectures.
📝arxiv.org/abs/2508.03555
👨🏽💻github.com/lightonai/pylate
Franck Lebeau retweeted
Are hallucinated references making it to arXiv?
Yes, definitely!
Since the release of Deep Research in February bogus references are on the rise (coincidence?)
I wrote a blog post (link below) on my analysis (which hugely underestimates the true rate of hallucinations...)
Franck Lebeau retweeted
I think more AI builders now recognize that the core quality concern is context confusion, not context window length limitations.
Lots of agent implementations now let users compress context to avoid quality degradation.
Franck Lebeau retweeted
Scaling CLIP on English-only data is outdated now…
🌍We built CLIP data curation pipeline for 300+ languages
🇬🇧We train MetaCLIP 2 without compromising English-task performance (it actually improves!
🥳It’s time to drop the language filter!
📝arxiv.org/abs/2507.22062
[1/5]
🧵
Franck Lebeau retweeted
I’ve been exploring physarum-style simulations on and off for a long time, and I finally wrote an article sharing the techniques I’ve been using.
It ranges from the classic physarum algorithm to some of my own weird tricks. I hope you enjoy it!
bleuje.com/physarum-explanat…
Franck Lebeau retweeted
With @LightOnIO we are thrilled to release pylate-rs 🚀⭐️
An efficient inference engine for late-interaction models written in Rust and based on Candle ⚡️
pylate-rs is the best Python library / Rust crate / NPM package to spawn late-interaction models in milliseconds.
Franck Lebeau retweeted
🏎️ Introducing PyLate-rs
@raphaelsrty is back at it to make you love late-interaction models!
After quickening the retrieval process with FastPlaid, he’s now making inference lightning-fast with this lightweight tool crafted in Rust for optimal speed and efficiency!
💫And here’s the magic: PyLate-rs compiles Rust code to WebAssembly, letting you run late-interaction models right inside your browser to build awesome demos!
Start compute some embeddings now: github.com/lightonai/pylate-…
👏 Kudos Raphaël for this new milestone!
With @LightOnIO we are thrilled to release pylate-rs 🚀⭐️
An efficient inference engine for late-interaction models written in Rust and based on Candle ⚡️
pylate-rs is the best Python library / Rust crate / NPM package to spawn late-interaction models in milliseconds.
Franck Lebeau retweeted
I'll be covering Reason-ModernColBERT in tonight's presentation, so please come if you are interested!
maven.com/p/1973fe/going-fur…
(And please be gentle, this is the first time I will be speaking live in front of this many people 😭)
Looking for a cheaper, open source alternative to agentic RAG?
Try multi-vector retrieval with Reason-ModernColBERT.
Most search systems compare one big summary vector per document. Multi-vector retrieval is different - it keeps separate vectors for each word or phrase, then finds the best matches between your search and the document pieces.
Reason-ModernColBERT is a small but mighty model trained on the reasonir-hq dataset and specializes in finding documents that require logical thinking, not just keyword matching.
It actually outperforms models 45x larger on reasoning tasks with just 150M parameters - tiny compared to billion-parameter models. It also creates small 128-dimension vectors and handles documents up to 8,000+ words long.
So while agentic RAG uses LLMs to tackle complex questions requiring intricate planning, multi-step reasoning, and utilization of external tools, sometimes you don't need all that complexity. Multi-vector retrieval gives you smart search results at a fraction of the cost.
Ready to try it? Check out this notebook by @helloiamleonie: github.com/weaviate/recipes/…
Franck Lebeau retweeted
Looking for a cheaper, open source alternative to agentic RAG?
Try multi-vector retrieval with Reason-ModernColBERT.
Most search systems compare one big summary vector per document. Multi-vector retrieval is different - it keeps separate vectors for each word or phrase, then finds the best matches between your search and the document pieces.
Reason-ModernColBERT is a small but mighty model trained on the reasonir-hq dataset and specializes in finding documents that require logical thinking, not just keyword matching.
It actually outperforms models 45x larger on reasoning tasks with just 150M parameters - tiny compared to billion-parameter models. It also creates small 128-dimension vectors and handles documents up to 8,000+ words long.
So while agentic RAG uses LLMs to tackle complex questions requiring intricate planning, multi-step reasoning, and utilization of external tools, sometimes you don't need all that complexity. Multi-vector retrieval gives you smart search results at a fraction of the cost.
Ready to try it? Check out this notebook by @helloiamleonie: github.com/weaviate/recipes/…
Franck Lebeau retweeted
Manually looking at data has the largest ROI of any other activity when building AI/ML applications.
Franck Lebeau retweeted
Today I'm celebrating 2 years at @weaviate_io! 🎉
Here are 37 things I learned about retrieval so far.
🔗leoniemonigatti.com/blog/wha…
Franck Lebeau retweeted
‼️Sentence Transformers v5.0 is out! The biggest update yet introduces Sparse Embedding models, encode methods improvements, Router module for asymmetric models & much more. Sparse + Dense = 🔥 hybrid search performance!
Details in 🧵
Franck Lebeau retweeted
The ERNIE 4.5 series is now officially open source. This family of models includes 10 variants—from MoE models with 47B and 3B active parameters, the largest having 424B total parameters, to a 0.3B dense model—all available now to the global AI community for open research and development.
Explore the models here:
- Hugging Face: huggingface.co/baidu
- GitHub: github.com/PaddlePaddle/ERNI…
- Baidu AI Studio: aistudio.baidu.com/overview
View the technical report here: ernie.baidu.com/blog/publica…
Franck Lebeau retweeted
jina-embeddings-v3 + jina-clip-v2 + jina-colbert-v2 + colpali + dse = jina-embeddings-v4 😇
jina.ai/news/jina-embeddings…
































