Shumin Deng

🚀 Excited to share SkillX: Automatically Constructing Skill Knowledge Bases for Agents! It automatically converts agent trajectories into reusable, plug-and-play skills — making them transferable across agents and environments. We are also planning to integrate SkillX into the SkillNet series, aiming to build a unified and scalable ecosystem for skill-centric agent intelligence. #LLM #Agents #NLP #AI #Skills #SkillX 📖 Paper: huggingface.co/papers/2604.0… 🔗 Code: github.com/zjunlp/SkillX 🧩 Motivation LLM agents should learn from experience, but today, most self-evolving agents still learn in isolation. They repeatedly rediscover similar behaviors from limited data, leading to: 🔹 redundant exploration 🔹 weak generalization 🔹 capability bottlenecks tied to the base model So the key question is: What form of experience is actually reusable across agents and environments? 💡 Our answer: Skills! But structured hierarchically! We propose SkillX, an automated framework for building a reusable Skill Knowledge Base (SkillKB). Instead of storing raw trajectories, insights, or workflows alone, SkillX organizes experience into 3 levels of skills: 1️⃣ Planning Skills High-level task organization: ordering, decomposition, dependencies 2️⃣ Functional Skills Reusable tool-based subroutines for completing subtasks 3️⃣ Atomic Skills Low-level tool usage patterns, constraints, and failure-prone details This makes agent experience more compact, composable, and transferable. ⚙️ How SkillX works SkillX constructs the skill library through 3 synergistic components: 1. Multi-Level Skills Design 2. Iterative Skills Refinement 3. Exploratory Skills Expansion 🔍 Why is this useful? Unlike long-context skill formats that require complex sandboxing and progressive interaction, SkillX uses a lightweight, itemized representation: ✅ retrieve with a simple retriever ✅ inject once into the system prompt ✅ easier transfer across base models ✅ lower execution burden for weaker agents 📊 Results Using GLM-4.6 to automatically build the skill library, we evaluate transfer on challenging long-horizon interactive benchmarks: ● AppWorld ● BFCL-v3 ● τ2-Bench When plugged into weaker base agents like Qwen3-32B, SkillX brings ~10 point improvements and also improves execution efficiency. ⚡ 🧠 Key takeaway ● Not all “experience” transfers equally well, and the representation matters. ● Hierarchical skills are a powerful abstraction for turning isolated agent experience into reusable knowledge. ● Stronger agents can build the skills, weaker agents can reuse them, and agents no longer need to keep learning everything from scratch. ✨ Additional findings ● Functional skills contribute the most to performance gains ● Planning skills often reduce execution steps ● Atomic skills are crucial for clarifying tool constraints and common failure modes ● Iterative refinement further improves the skill library ● Experience-guided expansion discovers more novel skills than random exploration 📦 We will release the optimized plug-and-play skill library to facilitate future research on reusable agent skills. Feedback, discussions, and collaborations are very welcome! 💬

Catch us at the poster/oral — happy to chat and exchange ideas! #ACL2025

Has the over-forgetting of large language models led to "aphasia"? Our latest work, ReLearn: Unlearning via Learning for Large Language Models, brings a solution! 🚀 Paper: arxiv.org/abs/2502.11190 Code: github.com/zjunlp/unlearn ReLearn adopts a forward learning strategy rather than traditional disruptive reverse optimization, effectively forgetting sensitive information while avoiding excessive suppression and eliminating the issue of generating repetitive and meaningless words. ✅ Key Advantages: 🗣️ Improved Language Fluency: Generates fluent and relevant text, avoiding the "aphasia" phenomenon. 🔒 Balance Knowledge Forgetting and Retention: Effectively forgets sensitive information while retaining general knowledge. 📊 New Evaluation Framework: Introduces KFR, KRR, and Linguistic Score to comprehensively measure forgetting effects and linguistic quality. 🤯 In-depth Mechanistic Analysis: Analyzes the cognitive conflicts caused by reverse optimization from the perspective of knowledge distribution and memory, providing a fresh understanding of the unlearning mechanism in large language models. Dive deeper into ReLearn, making your large language models “forget” smarter and “express” better! 👉#LLMs #Unlearning #ResponsibleAI  #PrivacyProtection #NLP #KnowledgeEditing

Over the past year, #KnowledgeEditing has experienced rapid development. As the new year begins, I’ve taken some time to reflect on the progress of this field and share my thoughts on its future directions. I look forward to discussing and collaborating with everyone to further advance this area. 🛠 Progress in Knowledge Editing: 1. Scenarios: In addition to updating the knowledge of LLMs, many works have begun exploring knowledge editing as a means to control model behavior, promoting safer and more controllable generation while enabling capabilities like unlearning. 2. Side Effects: Many works have started to reflect on the fundamental causes of the side effects of knowledge editing and have explored various methods to mitigate them. Editing LLMs (parameter-altering) can lead to overfitting, where models assign disproportionately high importance to edited content and disrupt attention mechanisms, reducing generalization and general abilities. Whether the model has truly updated its relevant knowledge remains questionable. 3. Practicality: While knowledge editing has expanded to fields like software engineering and multimodal tasks, its real-world impact remains limited. 💡 Key Reflections: 1. The field's foundational goal—knowledge updates—has seen limited success outside areas like AI safety. This raises questions about how to better align methods with practical needs. 2. Mechanism research is lagging. Without clear insights into why knowledge editing works (or doesn’t), efforts to improve models risk being akin to “blind men describing an elephant.” 📈 Future Directions: 1. Evaluation: We need a set of metrics/benchmarks to evaluate whether an edited LLM behaves properly, that is, to achieve a balance between generalization and side effects. 2. Steering: Steering vectors (with SAE) are emerging as a promising approach for interventions in model behaviors, particularly in domains like safety and personality alignment. These methods demonstrate the potential to achieve precise control with minimal impact on overall model performance. Furthermore, they may pave the way for bridging the gap between prompts and model parameter updates, enabling prompt-driven, parameterized behavior adjustments within the model. 3. Agent Memory Updates: The debate between symbolic and parametric memory for AI agents is ongoing. Knowledge editing techniques can offer a unified approach to memory updates, bridging the gap between updating both the model's internal memory and external memory. Memory updates may enhance reasoning capabilities over the long term, fostering the gradual evolution of System 2-like slow thinking processes. 4. Mechanism Interpretation: Deepening our understanding of model mechanisms is essential. Currently, research on the mechanisms of LLMs—such as neurons and circuits—lacks systematic exploration. It also fails to explain phenomena like the dynamic acquisition and forgetting of knowledge, as well as higher-order cognitive behaviors such as slow-thinking reasoning. 5. Interdisciplinary: Drawing inspiration from cognitive/brain science, we may: design the next generation of model architectures and model updating paradigms; potentially simulate human brain behavior based on neural networks to construct an electronic digital twin brain, enabling better solutions (e.g., neuromodulation) to problems in neuroscience and cognitive science. If one day machines truly awaken to self-awareness, understanding their mechanisms and having the means to control them will be a critically important technology. 🎉 Exciting News: We’re thrilled to announce that EasyEdit2 is currently in development! This next-generation toolkit will integrate steering capabilities to enable control over model behavior. Stay tuned for updates, and we welcome the community to explore and contribute: github.com/zjunlp/EasyEdit Let’s continue pushing the boundaries of #KnowledgeEditing, tackling its challenges, and exploring its vast potential to redefine AI adaptability and usability. #LLM #AI #NLP #EasyEdit #LLM #ModelEditing #KnowledgeEditing