Megagon Labs (@MegagonLabs) header
Megagon Labs (@MegagonLabs) profile picture

Megagon Labs advances state-of-the-art research in AI and builds technologies that impact the world through online services. #NLP #AI #ML #NLP4HR #LLM #RAG

Mountain View, CA
Joined June 2018
Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)Photo by Megagon Labs (@MegagonLabs)
Table QA retrieves facts well but struggles when a question asks for a probability. The LUCARIO benchmark has 6,280 such questions over 10 tables. Auto-BN learns a Bayesian Network from each table and beats NL2SQL, TableRAG, and direct prompting. Let the LLM translate. Let the inference engine calculate. Read: arxiv.org/abs/2506.20747 Blog: megagon.ai/lucario-probabili…
1
1
1
207
💥 We're hiring Summer 2027 Research Interns! 💥 Are you a PhD student researching agentic systems, trustworthy LLM reasoning, database systems, retrieval & knowledge grounding, or human-AI collaboration? Join Megagon Labs to tackle challenging AI research problems alongside our scientists and help advance the next generation of intelligent AI systems. Apply: megagon.ai/careers/internshi… #AIResearch #PhDInternship #Hiring #AI
3
10
47
6,364
𝗧𝗮𝗯𝗹𝗲 𝗤𝗔 𝗶𝘀 𝗴𝗼𝗼𝗱 𝗮𝘁 𝗳𝗶𝗻𝗱𝗶𝗻𝗴 𝗳𝗮𝗰𝘁𝘀. 𝗜𝘁 𝗶𝘀 𝗳𝗮𝗿 𝗹𝗲𝘀𝘀 𝗿𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗮𝘁 𝗮𝗻𝘀𝘄𝗲𝗿𝗶𝗻𝗴, “𝗛𝗼𝘄 𝗹𝗶𝗸𝗲𝗹𝘆 𝗶𝘀 𝘁𝗵𝗶𝘀 𝗼𝘂𝘁𝗰𝗼𝗺𝗲, 𝗴𝗶𝘃𝗲𝗻 𝘁𝗵𝗲 𝗲𝘃𝗶𝗱𝗲𝗻𝗰𝗲?” 𝗠𝗼𝗿𝗲 𝗰𝗼𝗻𝘁𝗲𝘅𝘁, 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹, 𝗼𝗿 𝗮 𝘀𝘁𝗿𝗼𝗻𝗴𝗲𝗿 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝗱𝗼𝗲𝘀 𝗻𝗼𝘁 𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰𝗮𝗹𝗹𝘆 𝗿𝗲𝗰𝗼𝘃𝗲𝗿 𝘁𝗵𝗲 𝗰𝗼𝗻𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗱𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀 𝗵𝗶𝗱𝗱𝗲𝗻 𝗮𝗰𝗿𝗼𝘀𝘀 𝗮 𝘁𝗮𝗯𝗹𝗲. 💥 We introduce 𝗟𝗨𝗖𝗔𝗥𝗜𝗢, a benchmark with 6,280 questions over 10 real-world tables, spanning five probabilistic reasoning patterns and four linguistic variations. We also propose Auto-BN: an automated pipeline that induces a Bayesian Network from raw data, translates natural language into an executable probabilistic query, and performs exact inference. With GPT-4o, Auto-BN reaches 46.7% accuracy within ±0.05 and 0.107 MAE, outperforming NL2SQL, TableRAG, raw-table prompting, reasoning models, and premise-retrieval baselines. Its robust translation and fallback mechanisms reduce parsing error rates from as high as 20.8% to near zero across the tested backbones. 🧠 Use language models to understand the question and a grounded probabilistic layer to compute the answer. 𝗟𝗨𝗖𝗔𝗥𝗜𝗢 provides a testbed for studying that interface at realistic table scale. 📄 Read the research 👉: megagon.ai/publications/luca… #AACL #ACL #LLMs #AI #PhD #Research @estevamhruschka
1
129
AI agents can build interactive tables, graphs, and canvases. But users still have to respond through chat. ABRA turns those outputs into shared surfaces where humans and agents can act and coordinate. In our research, we introduce Agent-Bound UI and its reference prototype, ABRA, a shared interaction medium that enables humans and agents to communicate beyond chat through interactive canvases. The key idea is to separate reusable interaction structures from task-specific meanings. The same graph view, for example, can represent a social network, a sports tournament, or a directory structure. But “add an edge” means something different in each context: connecting people, linking matches, or defining a directory relationship. Agent-Bound UI lets agents reuse the same interaction structure while binding its meaning to the task and sharing that meaning with users and other agents. Our experiments show that current LLM agents can: • Create task-specific canvases • Resolve interactions on canvases they own • Invoke published actions on other agents’ canvases Multi-view completeness and specification following remain challenges. Agent interfaces can be more than outputs. They can become shared surfaces where humans and agents show, act, and coordinate. Read the research: megagon.ai/publications/abra… 💥 Join us for the demo presentation at #EMNLP2026 at 9:00 AM on October 27th, presented by Hannah Kim. #LLMs #Agents #HCI #NLP #EMNLP
1
3
267
👉 LLMs can improve Bayesian network learning without owning the graph. ABSOL uses subsampled GES+BDeu plus four bounded LLM interventions. Across five benchmarks spanning 27 to 1,041 nodes, it was the only method to return a viable graph on all five. Read: megagon.ai/publications/abso… Join @estevamhruschka at #EMNLP2026 for a poster presentation on ABSOL 👇
1
148
💡🗄️ Search systems need to understand increasingly complex requests. But improving that understanding can mean rebuilding millions of stored document representations. What if the query side could get smarter while the existing index stayed intact? In our new paper, we introduce Efficient Retrieval Adapter (ERA), a query-side linear adapter trained in two stages. First, ERA aligns a strong query embedder with a lightweight document embedder using the same unlabeled corpus documents in both spaces. Then it adapts that aligned representation with a limited set of relevance labels. Across 126 MAIR retrieval tasks in six domains, ERA with a strong query embedder improves average nDCG@10 by more than 12 points for BGE-M3 and OpenAI-small document indexes at the 20% label ratio. The alignment stage is the key: it produces large gains before labeled adaptation and keeps ERA effective in low-label settings. Its sampled documents represent only 1.28% of the MAIR collection and 5.20% of corpus tokens. The implication is a more modular retrieval stack: improve query understanding without updating backbone embedders or rebuilding the corpus index. For enterprise search and RAG AI systems, this creates a practical research direction for evolving query intelligence while preserving index infrastructure. Read the research: megagon.ai/publications/asym… @SAYg_7 @estevamhruschka @PPezeshkpour #EMNLP2026 #AI #LLMs #NLP
2
175
How confident can we be that an LLM agent actually completed its task? Long trajectories, multiple sources of uncertainty, and limited model access make this hard. Our new work, CRGs, uses evidence-grounded reasoning for better-calibrated, interpretable confidence estimates. 👇
📢 New preprint from @MegagonLabs 🧵 Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output? Meet 𝗖𝗼𝗻𝗳𝗶𝗱𝗲𝗻𝗰𝗲 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗚𝗿𝗮𝗽𝗵𝘀 (𝗖𝗥𝗚): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate. Paper: arxiv.org/abs/2610.07948 Code: github.com/megagonlabs/crg_c…
1
2
8
306
Megagon Labs retweeted
📢 New preprint from @MegagonLabs 🧵 Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output? Meet 𝗖𝗼𝗻𝗳𝗶𝗱𝗲𝗻𝗰𝗲 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗚𝗿𝗮𝗽𝗵𝘀 (𝗖𝗥𝗚): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate. Paper: arxiv.org/abs/2610.07948 Code: github.com/megagonlabs/crg_c…
1
3
1
10
657
Editing a multi-agent plan is not the same as steering it. We introduce 𝗔𝗠𝗕𝗜𝗣𝗢𝗠: 𝗮 𝗽𝗿𝗼𝘁𝗼𝘁𝘆𝗽𝗲 𝗳𝗼𝗿 𝗵𝘂𝗺𝗮𝗻-𝗟𝗟𝗠 𝗰𝗼-𝗽𝗹𝗮𝗻𝗻𝗶𝗻𝗴 𝘁𝗵𝗮𝘁 𝗲𝗻𝗮𝗯𝗹𝗲𝘀 𝘂𝘀𝗲𝗿𝘀 𝘁𝗼 𝗲𝗱𝗶𝘁 𝗮𝗴𝗲𝗻𝘁 𝗽𝗹𝗮𝗻𝘀 𝘃𝗶𝗮 𝗰𝗵𝗮𝘁, 𝘁𝗮𝗿𝗴𝗲𝘁𝗲𝗱 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸, 𝗴𝗿𝗮𝗽𝗵 𝗺𝗮𝗻𝗶𝗽𝘂𝗹𝗮𝘁𝗶𝗼𝗻, 𝗮𝗻𝗱 𝗺𝗲𝗿𝗴𝗲, 𝘀𝗽𝗹𝗶𝘁, 𝗮𝗻𝗱 𝗿𝗲𝗽𝗹𝗮𝗻 𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀. The key finding: More controls reduce some effort, but they also create new verification work. Users built hybrid workflows rather than choosing one interaction type. Global feedback was most robust for complex revisions. Targeted feedback preserved unchanged parts of the plan, but could introduce structural errors. Edit sequences were easier to audit, but less reliable for complex changes. For agentic AI, the design challenge is to “help users know whether the edited plan still works.” 📄 Read the article: megagon.ai/human-llm-co-plan… @estevamhruschka #AI #Agents #LLMs #HumanCenteredAI #CAIS2026
2
3
9
732
We’re hiring a Fall/Winter PhD Research Intern at Megagon Labs. We’re looking for candidates with research or technical experience in any one or more of the following areas: • Retrieval-Augmented Generation (RAG) • Function Calling • Document Understanding / PDF Parsing You do not need experience across all three areas. Strong experience in any one of these topics is relevant. Interested in joining us this fall? Learn more and apply → megagon.ai/careers/internshi… Applicants must have or be able to obtain appropriate U.S. work authorization for the internship period. #AI #NLP #LLMs #internships #phdposition
1
2
1
3
4,907
Correct answers are no longer enough. As LLMs become part of systems that use tools, memory, and evolving knowledge, we also need to understand how those components change model behavior. That question shaped much of the conversation at ACL 2026. Read the article! megagon.ai/acl-2026-highligh… #AI #LLMs #CompoundAI
180
Your AI agent can cite a source. That doesn't mean the source says what the row claims. Here's the failure mode we kept seeing: an LLM curator builds a plausible structured table from parametric memory; it "just knows" what should go in the cell, then bolts on a page-level citation afterward to make it look sourced. The citation is real, and the row is not grounded in it. It's structured output with decorative provenance. We built Stage-Audit to close that gap in the context of what we call Seed2Frontier discovery: starting from a single Wikipedia seed page and autonomously finding the additional pages needed to complete a structured table that answers a natural-language query. megagon.ai/data-provenance-s… #AIResearch #AgenticAI #CompoundAI #LLM #DataManagement #AIAgents
1
231
Most LLM extraction systems forget what they learned the moment the document ends. At ACL 2026, we introduce DySECT: a self-evolving extraction system that builds an explicit KB as it extracts. This is a glance at our loop: extract triples -> update KB -> organize and score knowledge -> feed it back into extraction On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5. The key result: 5-8% recall gains on the first guided iteration, without fine-tuning or synthetic data generation. Adaptation can happen through inspectable memory, not only opaque model updates. 👉Read the article: megagon.ai/self-evolving-inf… #ACL2026 #NLProc #LLM #InformationExtraction #EnterpriseAI #KnowledgeGraphs #AIResearch #MegagonLabs #ContinuousLearning
1
1
151
As AI agents become more capable, understanding how they make decisions is becoming just as important as the answers they produce. In our latest blog, we explore why agent observability is emerging as a critical capability for enterprise AI. We also introduce L.A.K.E. (Logic Agent for Knowledge Extraction), our agentic data planning framework that makes planning visible through executable workflows and interactive DAGs, helping developers inspect, evaluate, and improve how AI agents reason across complex data sources. Transparency isn't just about explainability; it enables debugging, optimization, and building more reliable AI systems. Read our blog post: megagon.ai/agent-observabili… #AI #EnterpriseAI #AgenticAI #MachineLearning #Transparency #MegagonLabs
1
209
Most 𝗟𝗟𝗠 𝗲𝘅𝘁𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 treat each document in isolation, failing to retain and reuse knowledge acquired from previous extractions. That is a problem in domains where terminology changes, taxonomies evolve, and rare concepts matter. At ACL 2026 in San Diego, we introduce 𝗗𝘆𝗦𝗘𝗖𝗧: 𝗮 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗦𝗲𝗹𝗳-𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 𝗘𝘅𝘁𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗖𝘂𝗿𝗮𝘁𝗶𝗼𝗻 𝗧𝗼𝗼𝗹𝗸𝗶𝘁. Extraction should create reusable knowledge. So, DySECT uses an LLM to extract structured triples, stores them in a growing knowledge base, and assigns confidence based on source reliability, repeated evidence, and conflicts. This accumulated knowledge is now a resource that should improve subsequent extraction rounds. The KB enriches previous extractions through hierarchical concept abstractions, confidence calibration, mutual exclusion constraints, and automated property validation. This enriched knowledge is then fed back into the extractor via prompt augmentation, encouraging broader coverage and discovery of new information in later iterations. On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5. The first KB-guided iteration 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝗱 𝗿𝗲𝗰𝗮𝗹𝗹 𝗯𝘆 𝟱-𝟴% 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝗼𝗿 𝘀𝘆𝗻𝘁𝗵𝗲𝘁𝗶𝗰 𝗱𝗮𝘁𝗮 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻. Adaptive extraction does not have to mean opaque model updates. DySECT keeps accumulated knowledge explicit, inspectable, and editable. For enterprise AI systems, that matters. Knowledge can evolve, but oversight and transparency stay in the loop. 📄 Read the research: megagon.ai/publications/dyna… Work by @moinnas, Hannah Kim, @estevamhruschka Hruschka #ACL2026 #continuouslearning #AI #LLMs
6
454
As LLM systems become more agentic, improving individual outputs may not be enough. In a recent Megagon Labs Seminar Series talk, Jay-Yoon Lee (Seoul National University / UIUC) explored the idea that correctness and reliability often emerge at the set level. Modern LLM systems increasingly operate over collections of retrieved documents, generated statements, and intermediate reasoning steps, yet most frameworks still treat these elements independently. Interactions such as contradictions, redundancy, and joint relevance can shape outcomes just as much as the information itself. The talk introduced a unified perspective centered on set-level verification, retrieval, and editing as important directions for building more reliable and controllable LLM systems. Thank you to Jay-Yoon for sharing his insights. #LLM #AgenticAI #AIResearch
2
141
The way AI agents plan over enterprise data can dramatically change both latency and reliability. This process of data planning, how an agent sequences operations like database queries, document retrievals, and API calls to resolve complex prompts, is foundational to enterprise AI performance. In our latest research, we found that different planning regimes produce very different trade-offs: - Single-shot tree planning achieved the strongest overall performance and lowest latency. - Iterative planning provided stronger self-correction at a significantly higher execution cost. We also observed that the planning structure itself, including generation direction and refinement strategy, can substantially impact agent behavior. These findings highlight a broader challenge in agentic AI: you cannot optimize or trust what you cannot inspect. As agents increasingly orchestrate tools across databases, documents, and external systems, understanding how plans are generated and executed becomes critical for debugging, evaluation, and deployment. To address this, we introduce L.A.K.E. (Logic Agent for Knowledge Extraction), an agentic data planning framework designed to make planning observable. L.A.K.E. exposes execution paths as interactive DAGs with step-level provenance, enabling developers to inspect workflows, compare planning strategies, and understand why agents succeed or fail. This work reflects a broader direction we are exploring at Megagon Labs: building agentic systems that are not only capable of reasoning over complex enterprise data, but also transparent, debuggable, and efficient to operate. #Agentic #NLProc #LLMs megagon.ai/publications/l-a-…
1
119
Can LLMs automate prompt engineering? RPT says: Apparently yes—by turning prompt tuning into a function-calling workflow: evaluate, diagnose recurring failures, use memory, and revise. The result: stronger prompts on reasoning tasks and better-calibrated confidence.
📢 New Preprint Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts? Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs. Paper: arxiv.org/abs/2605.21781 @MegagonLabs 1/n
1
1
7
1,317