Anna Guo (@aguozy) profile picture

https://teapawt.org/t.co/MCDF1RgTLA

Joined July 2026
Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)Photo by Anna Guo (@aguozy)
Anna Guo retweeted
you can totally put an agent in a whatsapp group chat with a client, so that it can answer their and your questions much more straightforward to make it poll a group using a systemd service + tulir/whatsmeow, than to make them add a slack app (do they even have slack??) to their org that they have to first get permission for...
3
2
6
1,129
My prediction for @Anthropic’s next launch following @OpenAI's introduction of dots: Blots. The blob wars have begun.
63
Anna Guo retweeted
Can you really trust an AI's legal advice? I partnered with Legal Benchmarks @aguozy to launch their new leaderboard measuring frontier legal capabilities. Here's how we built the task environment and calibrated an LLM judge against lawyer preferences. 👇
5
7
3
28
3,172
Here are the greatest hits of AI model release hype, starting in 2019 with "its too dangerous to fully release (GPT-2, 2019)." My favorite claim is still Llama 3, period.
2
148
Most AI benchmarks stop at correctness. For legal work, that is only the start. Today we launch DELTA and open-source tasks drawn from 200+ Dutch legal dataset with answer keys and lawyer-reviewed criteria. legalbenchmarks.ai/research/…
2
1
6
212
DELTA measures outputs across 3 dimensions for legal work: - Substance: Is the legal analysis correct, complete and professionally defensible? - Citation: Is the relevant legal authority accurately identified and used? - Form: Could a practitioner use the answer as delivered?
6
6/ These are not three different problems. They are the same verification problem showing up in datasets, contracts and court filings.
🤖 Made with AI
1
2
7/ The best LLMs, skills, guardrails and harnesses may reduce the error rate. But if AI allows us to produce ten times more work and we cannot verify ten times more work, even a lower error rate can mean more mistakes reaching the real world. Something to think about.
🤖 Made with AI
2
Today Legal Benchmarks is launching the first independent AI Applications Leaderboard for legal work. Most leaderboards rank models. Lawyers do not use models. They use applications. First release: @claudeai Web App, Claude Cowork, @ChatGPT , Codex, @GeminiApp , @Copilot.
2
5
139
Bottom line for lawyers: no general-purpose AI application is yet reliable enough to perform legal work without close review. Even the strongest averaged below a 40% substantive pass rate across the workflows tested.
1
56
If you use them, Claude gives you the best chance of getting the facts right, while ChatGPT is better for polishing and refining the work product. Use Microsoft Copilot or Gemini only if enterprise constraints leave you no other option.
20