1 Introduction

General-purpose AI agents are increasingly capable of performing complex tasks across domains (Patwardhan et al., 2026; Jackson et al., 2025; Merrill et al., 2026; Phan et al., 2026). However, reliably accomplishing real-world tasks often requires domain-specific expertise (e.g., procedural knowledge and workflows). Agent skills (Zhang et al., 2025; Li et al., 2026; Chen et al., 2026a; Liu et al., 2026) provide a lightweight, open format for capturing such expertise without updating model parameters. At its core, a skill packages instructions, scripts, and other resources into a reusable filesystem-based module (i.e., an organized directory) (Zhang et al., 2025; Li et al., 2026; Anthropic, 2026; Xia et al., 2026b). This design makes specialized knowledge consistent, auditable, and reusable across skill-compatible agents. It also supports progressive disclosure (Jiang et al., 2026), where agents only load relevant content at any given time, which saves context space. More broadly, skills provide a natural mechanism for accumulating knowledge independently of model parameters.

Developing effective skills, however, remains challenging. Most agent skills are manually authored, which requires anticipating the procedural knowledge and workflows that an agent will need (Li et al., 2026; Liang et al., 2026; Xu and Yan, 2026). This challenge motivates recent work that iteratively develops agent skills by executing agents on training tasks, analyzing successful and failed trajectories, and refining skills based on the resulting experience (Yuksekgonul et al., 2025; Agrawal et al., 2026; Alzubi et al., 2026; Ni et al., 2026; Ouyang et al., 2026; Yang et al., 2026).

A key design question is how to preserve and organize what an agent learns throughout skill evolution. Prior work addresses this question in different ways. EvoSkill (Alzubi et al., 2026) maintains a cumulative history of prior proposals and their evaluation outcomes; Trace2Skill (Ni et al., 2026) extracts and consolidates lessons across execution trajectories into skill updates; and SkillOpt (Yang et al., 2026) uses rejected-edit feedback and epoch-wise meta guidance. However, these methods do not maintain what has been learned as a separate, evolving knowledge representation. Inspired by Karpathy (2026)’s perspective on LLM Wiki, which advocates compiling experience into persistent, compounding knowledge, we ask: Can agent experience be similarly compiled into persistent knowledge to support long-term skill evolution? We introduce WikiSkill, which adds a structured knowledge layer between raw experience and executable procedures (i.e., skills). This layer allows skill development to build on increasingly well-supported and integrated knowledge across iterations, rather than on knowledge scattered across skill-evolution artifacts.

WikiSkill organizes the agent workspace into three layers: a Raw Layer that stores immutable execution traces, a Wiki Layer that maintains structured knowledge, and a Skill Layer that contains evolving procedural knowledge (Figure 2). Each iteration involves four components: an Inference Agent that executes rollouts using the current skills, a Wiki Maintainer that consolidates traces into the wiki, a Skill Proposer that uses the wiki and traces to propose skill updates, and a Gating and Rollback mechanism that retains updates that improve validation performance. While skill updates can be rolled back, the wiki persists so that future updates can build on accumulated knowledge. At a high level, these components form a continual loop in which experience is consolidated into persistent knowledge that supports skill evolution.

We evaluate WikiSkill across five benchmarks spanning mathematical reasoning (LiveMathematicanBench (He et al., 2026)), web search (SealQA (Pham et al., 2026)), spreadsheet manipulation (SpreadSheetBench (Ma et al., 2024)), long-context document question answering (OfficeQA (Singhvi et al., 2025)), and interactive embodied tasks (ALFWorld (Shridhar et al., 2021)), using five models from the Qwen (Qwen Team, 2026a; Qwen Team, 2026b), Gemma (Gemma Team, 2026), and Gemini (Google DeepMind, 2026) families. We find that WikiSkill outperforms existing skill-evolution methods and improves over no skills in most settings. Interestingly, skill evolution complements model scaling. Within the Qwen family, WikiSkill improves average performance by 12.3%, 17.5%, and 23.9% for 4B, 9B, and 27B models, respectively, with gains increasing with model scale. At the same time, evolved skills can compensate for substantial model scale: Qwen-3.5-9B with WikiSkill outperforms Qwen-3.6-27B without skills (47.4% vs. 39.4%). We further find that evolved skills transfer effectively across model families and can outperform self-evolved skills. On ALFWorld, for example, Qwen-3.5-9B reaches 70.2% with a Qwen-3.6-27B-evolved skill, compared with 63.4% using its own skill. These results suggest that skill discovery and skill execution are distinct capabilities. Finally, our analysis shows that the persistent wiki is critical to these gains, supporting our hypothesis that accumulating and refining knowledge across iterations improves skill evolution.

In summary, our main contributions are:

We introduce WikiSkill, a framework that co-evolves agent skills with a persistent knowledge base that continually organizes and refines knowledge from agent experience.

We demonstrate across five benchmarks and five models that WikiSkill consistently outperforms existing skill-evolution methods, with ablations confirming the importance of persistent knowledge accumulation.

We systematically study how evolved skills interact with model capability, showing that skill evolution complements model scaling and that evolved skills can transfer effectively across models, sometimes outperforming self-evolved skills.

Taken together, we hope that our work will spur more fundamental research on how agents can accumulate, organize, and reuse knowledge from experience.

2 Problem Setup

We formalize the task of iterative skill evolution for LLM agents. Let $\mathcal{D}=\{(x_{i},y_{i})\}_{i=1}^{N}$ be a dataset of tasks, where $x_{i}$ denotes a task instance and $y_{i}$ denotes its ground-truth answer. We partition $\mathcal{D}$ into three disjoint splits: training tasks $\mathcal{D}_{\text{train}}$, validation tasks $\mathcal{D}_{\text{val}}$, and testing tasks $\mathcal{D}_{\text{test}}$.

An agent $\pi$ is an LLM-based system equipped with a set of tools $\mathcal{U}$ (e.g., a bash shell, search APIs, or file readers) and an active skill set $S=\{s_{1},s_{2},\dots,s_{M}\}$. A skill is a modular, filesystem-based directory that packages domain-specific procedural knowledge into instructions, scripts, and other resources (Zhang et al., 2025; Li et al., 2026; Chen et al., 2026a; Liu et al., 2026). Specifically, each skill contains a SKILL.md file with frontmatter metadata (a unique name and concise description) alongside full procedural instructions and applicability conditions. The skill set $S$ is initialized to empty ($\emptyset$) and developed for each dataset through the evolution process.

When executing a task $x_{i}$, the agent receives the task context $x_{i}$ and access to the available skills $S$. The agent interacts with the environment over multiple steps using its tools and skills to generate an execution trajectory $\tau_{i}\sim\pi(x_{i};S)$. The trajectory $\tau_{i}=(o_{1},a_{1},o_{2},a_{2},\dots,o_{T},a_{T})$ consists of observations $o_{t}$ and actions $a_{t}$ (which may include calls to tools in $\mathcal{U}$). The final action $a_{T}$ emits a predicted answer $\hat{y}_{i}$. The correctness of the prediction is evaluated by a domain-specific scoring function $f(\hat{y}_{i},y_{i})\in[0,1]$. For any task split $\mathcal{D}_{\text{split}}\subset\mathcal{D}$, rolling out the agent $\pi(\cdot;S)$ across all task instances in $\mathcal{D}_{\text{split}}$ yields a corresponding set of execution trajectories $\mathcal{T}_{\text{split}}=\{\tau_{i}\sim\pi(x_{i};S)\}_{(x_{i},y_{i})\in\mathcal{D}_{\text{split}}}$. The performance on a task split $\mathcal{R}(\mathcal{T}_{\text{split}})$ is the average score across all task instances $(x_{i},y_{i})\in\mathcal{D}_{\text{split}}$.

In WikiSkill, the system state at iteration $k$ is represented by the tuple $(S_{k},W_{k})$, where $S_{k}=\{s_{1},\dots,s_{M}\}$ denotes the active procedural skill set and $W_{k}$ denotes the persistent knowledge base (Wiki). While candidate skill updates are subject to validation gating and rollback upon score degradation, the knowledge base $W_{k}$ persists and compounds across iterations. Starting from $(S_{0},W_{0})=(\emptyset,\emptyset)$, WikiSkill co-evolves the joint state $(S_{k},W_{k})$ across iterations $k\in\{1,\dots,K\}$, leveraging training rollouts $\mathcal{T}_{\text{train},k}$, pattern consolidation, and validation gating based on $\mathcal{T}_{\text{val},k}$ to maximize final test performance $\mathcal{R}(\mathcal{T}_{\text{test}})$ on unseen tasks $\mathcal{D}_{\text{test}}$.

3 Methodology

We present WikiSkill, a framework that co-evolves agent skills and a persistent knowledge base (wiki). Built around a three-layer knowledge architecture (§3.1), WikiSkill executes an orchestrated evolutionary loop in which the agent runs rollouts, a Wiki Maintainer consolidates traces and updates the wiki, a Skill Proposer proposes skill updates, and a gating mechanism filters changes (§3.2).

3.1 Three-Layer Knowledge Architecture

The WikiSkill workspace consists of three distinct layers, as shown in Figure 2 and described below.

Raw Layer (raw/)

This layer stores the raw execution traces $\tau_{i}\in\mathcal{T}_{\text{train},k}$ collected from training examples in each iteration. These traces capture the agent’s complete step-by-step interactions, including reasoning, tool calls, tool-call outputs, and final answers. In our setup, the Wiki Maintainer and Skill Proposer agents can access these raw traces to analyze agent behavior. To preserve the raw history, this layer is immutable.

Wiki Layer (wiki/)

This layer compiles raw traces into structured, compounding knowledge and is maintained throughout skill evolution. It contains a pattern directory (patterns/) populated with individual markdown files that document specific failure modes or successful strategies, along with actionable workarounds. Crucially, this layer provides long-term historical awareness across optimization iterations through an evolution log (logs.md, updated by the Wiki Maintainer) and a skill impact tracker (skill-impact.md, updated programmatically by the outer-loop harness after validation gating). These records allow the Wiki Maintainer and Skill Proposer to (1) observe the complete skill acceptance history so that rejected interventions are not proposed again, (2) track what was proposed in prior iterations and whether those proposals succeeded, and (3) identify which errors recur across iterations. The wiki is not reset between iterations, but rather accumulates and compiles knowledge continuously throughout the evolution process.

Skills Layer (skills/)

This layer contains the active set of evolved skills $S$, which encode the procedural knowledge that the Inference Agent can read. Each skill directory in WikiSkill contains two files: SKILL.md, which contains the full content of the skill; and PURPOSE.md, which maps the skill back to the motivating Wiki patterns that inspired its creation or modification. A detailed example of interactions between the Skill Layer and the Wiki Layer is illustrated in Figure 3 and explained in the case study in Section 5.3.

3.2 Evolutionary Agents and Wiki Orchestration

The WikiSkill loop consists of four components. In each iteration, the Inference Agent (§3.2.1) executes tasks using the active skills in skills/, producing immutable execution traces in raw/. During the training rollouts, the Inference Agent is restricted from accessing the Wiki Layer, as our ablation study (§5.1) shows that allowing wiki access during training negatively affects skill development. Next, the Wiki Maintainer (§3.2.2) analyzes these raw traces alongside the existing wiki/ layer to diagnose failures and extract successful strategies, updating the persistent pattern catalog and evolution logs. The Skill Proposer (§3.2.3) then reviews the updated wiki and reads execution traces from the latest iteration to generate or modify candidate skills in skills/. Finally, a Gating and Rollback mechanism (§3.2.4) evaluates the candidate skills on a validation split, accepting successful modifications or rolling back the skill set if the changes degrade performance. The entire evolution algorithm is described in Algorithm 1 in Appendix A.

3.2.1 Skill Provisioning for the Inference Agent

At iteration $k$, the Inference Agent $\pi$ is conditioned on the active skill set $S_{k-1}$ and executes a multi-turn trajectory using environment tools $\mathcal{U}$:

In WikiSkill, the full content of active skills $S_{k-1}$ is injected directly into the Inference Agent’s system prompt. Following prior work (Yang et al., 2026; Ni et al., 2026), this full-injection setting ensures that procedural instructions are immediately available during task execution, thereby eliminating skill triggering or retrieval failures as confounding variables in our study.

3.2.2 Wiki Maintainer: Pattern Consolidation

At iteration $k$, after obtaining rollout traces $\mathcal{T}_{\text{train},k}$ on the training split $\mathcal{D}_{\text{train}}$, we sample a subset of successful and failing execution traces $\mathcal{T}_{\text{sample},k}\subset\mathcal{T}_{\text{train},k}$ (see Appendix C for sampling budget and stratification criteria) to avoid context window limitations. The Wiki Maintainer agent $\mathcal{M}_{\text{WM}}$ consolidates these observations into the persistent wiki $W_{k-1}$, producing the intermediate wiki state $W^{\prime}_{k}$:

The Wiki Maintainer agent receives the full wiki context $W_{k-1}$ alongside sampled traces $\mathcal{T}_{\text{sample},k}$. It performs root cause analysis on the failing tasks, and extracts successful strategies from the passing tasks. In each iteration, the Wiki Maintainer can create new pattern pages under wiki/patterns/ and update existing pattern pages with new evidence or refined solutions. Updates to pattern pages are applied using incremental, patch-based editing (e.g., appending, replacing, or inserting text spans). Whenever patterns are modified, the Wiki Maintainer revises the index.md catalog to reflect the current state and appends a summary of the iteration’s findings to the evolution log logs.md. There is no hard limit on the number of patterns created or updated per iteration; the Wiki Maintainer decides what updates are warranted based on the traces and the current wiki state.

3.2.3 Wiki-Informed Skill Proposer

The Proposer $\mathcal{M}_{\text{P}}$ is an LLM-based agent responsible for skill discovery and refinement. At iteration $k$, the proposer operates in a multi-turn ReAct style (Yao et al., 2023). To avoid context window exhaustion when analyzing long execution histories, the proposer is not given a fixed set of pre-sampled traces; instead, it is initially provided with the wiki index $I(W^{\prime}_{k})$, the historical skill impact tracker (skill-impact.md), and a concise summary of all training task outcomes (pass/fail status, predictions and ground-truth answers). Operating as an autonomous agent, it actively reasons and uses environment tools (read_file) to select and inspect specific pattern pages and raw execution traces $\tau_{i}\in\mathcal{T}_{\text{train},k}$ on demand to diagnose root causes before synthesizing a proposal $P_{k}$:

In each iteration, the Skill Proposer produces an atomic proposal $P_{k}$ that targets a single skill, either creating a new skill or applying an incremental, patch-based edit to the targeted existing skill.

3.2.4 Gating and Rollback

Once a proposal $P_{k}$ is generated, it is applied to the workspace to yield a candidate skill set $S^{\prime}_{k}=\text{Apply}(S_{k-1},P_{k})$. The system evaluates $S^{\prime}_{k}$ on the validation split $\mathcal{D}_{\text{val}}$, obtaining validation traces $\mathcal{T}_{\text{val},k}$ and score $\mathcal{R}(\mathcal{T}_{\text{val},k})$. The acceptance decision is governed by:

If accepted, the candidate skills are preserved as the new active skill set $S_{k}$, and the benchmark performance threshold $\mathcal{R}_{\text{best}}$ is updated to $\mathcal{R}(\mathcal{T}_{\text{val},k})$. Prior to the evolution loop, $\mathcal{R}_{\text{best}}$ is initialized to the baseline validation score $\mathcal{R}(\mathcal{T}_{\text{val},0})$ obtained by evaluating the empty skill set $S_{0}$ on $\mathcal{D}_{\text{val}}$. If the validation score reaches the maximum ($\mathcal{R}_{\text{best}}=1.0$) at any point during evolution, the evolution loop terminates early. If rejected, the system discards the candidate skill modifications and reverts the skill set to the most recent successful configuration $S_{k-1}$. Notably, the wiki $W_{k}$ is never rolled back regardless of the acceptance decision; accumulated patterns and logs persist across all iterations to ensure long-term knowledge retention. Following each validation evaluation, the outer-loop orchestration harness programmatically appends an entry to wiki/skill-impact.md via $W_{k}\leftarrow\text{Update}(W^{\prime}_{k},P_{k},\mathcal{R}(\mathcal{T}_{\text{val},k}),a_{k})$, recording the proposal metadata, target skill name, unified diff of the modification, validation score $\mathcal{R}(\mathcal{T}_{\text{val},k})$, and final acceptance outcome $a_{k}\in\{\text{Accepted},\text{Rejected}\}$. This completes the wiki state transition $W_{k-1}\to W_{k}$ for iteration $k$, providing an objective, ground-truth audit trail of past interventions that the Skill Proposer can consult in subsequent iterations to avoid repeating failed modifications.

4 Experiments and Results

4.1 Experimental Setup

Datasets

We evaluate across five benchmarks spanning diverse domains: mathematical reasoning (LiveMathematicianBench (LiveMath) (He et al., 2026)), web search (SealQA (Pham et al., 2026)), spreadsheet manipulation (SpreadsheetBench (SpreadSheet) (Ma et al., 2024)), long-context document question answering OfficeQA (Singhvi et al., 2025)), and interactive embodied tasks (ALFWorld (Shridhar et al., 2021)). Dataset details and statistics are provided in Appendix B.

Baselines

We compare WikiSkill against three representative skill-evolution baselines, including Trace2Skill (Ni et al., 2026), EvoSkill (Alzubi et al., 2026), and SkillOpt (Yang et al., 2026), all of which share the same general loop of rolling out an agent, analyzing execution traces, proposing skill modifications, and gating changes via validation. We also evaluate each model without skills as a no-skill baseline. A detailed description and an analysis of the complexity of optimizer API calls across these frameworks are provided in Appendix D. We focus our comparison on dedicated skill-evolution frameworks rather than general automatic prompt optimizers (e.g., GEPA (Agrawal et al., 2026)), following prior work that shows specialized skill-evolution pipelines consistently outperform general prompt optimization methods (Yang et al., 2026).

Models

We experiment with both closed and open-weight models to evaluate WikiSkill and the baselines. For closed models, we use Gemini-3.5-Flash (Google DeepMind, 2026). For open-weight models, we evaluate Qwen-3.5-4B/9B-Instruct (Qwen Team, 2026a), Qwen-3.6-27B (Qwen Team, 2026b), and Gemma-4-31B-It (Gemma Team, 2026), which we deploy using the vLLM framework (Kwon et al., 2023).

4.2 Main Results

We evaluate WikiSkill across models and tasks and study whether evolved skills transfer across models. Table 1 presents the main skill-evolution results across models and tasks, including how the benefits of skill evolution vary with model scale, while Table 2 presents the cross-model skill transfer results. We analyze these results in detail below.

To account for variability, we repeat the full evolution process across three independent runs for each method, and all reported scores represent the average test performance across the three resulting evolved skill sets. Statistical significance of performance differences is evaluated using paired bootstrap testing at $p<0.05$ (Appendix C). Note that for Gemini-3.5-Flash on ALFWorld, all evolution methods yield the same performance (85.9%) as the no-skill baseline because Gemini-3.5-Flash achieves a 100% score on the validation split ($\mathcal{D}_{\text{val}}$) before skill evolution. This also explains why Gemini-3.5-Flash is marked with ‘$-$’ as a skill source on ALFWorld in the cross-model transfer evaluation (Table 2).

4.2.1 Skill Evolution Across Models and Tasks

WikiSkill yields consistent improvements across models and datasets

As shown in Table 1, WikiSkill achieves the highest average performance across all five models. Compared with the strongest competing skill-evolution method for each model, WikiSkill improves average performance by 3.3, 5.1, 10.0, 5.8, and 12.0 points for Qwen-3.5-4B, Qwen-3.5-9B, Qwen-3.6-27B, Gemma-4-31B, and Gemini-3.5-Flash, respectively. These improvements are consistent across settings: WikiSkill improves over the no-skill baseline in most model-dataset pairs and matches or exceeds the strongest competing method across all models in the average performance across 5 datasets. The improvements also span diverse domains and can be substantial. For example, WikiSkill improves Gemini-3.5-Flash from 33.0% to 72.6% on LiveMath and from 50.5% to 76.6% on SpreadSheet, while improving Qwen-3.6-27B from 52.8% to 77.6% on ALFWorld. In contrast, existing skill-evolution methods are less consistent. For example, EvoSkill improves Qwen-9B substantially on LiveMath (28.2% $\rightarrow$ 58.1%) but degrades Gemma-4-31B on the same benchmark (33.9 % $\rightarrow$ 29.8%), while SkillOpt degrades Gemini-3.5-Flash on SealQA (29.4 % $\rightarrow$ 28.2%). These results show that WikiSkill produces both stronger and more reliable improvements across settings.

The benefits of skill evolution increase with model capability and complement model scaling

Within the Qwen family, the average improvement from WikiSkill increases with model scale, from +12.3 points for Qwen-3.5-4B to +17.5 points for Qwen-3.5-9B and +23.9 points for Qwen-3.6-27B. This trend is particularly pronounced on SpreadSheet, where WikiSkill improves the three models by +6.5, +9.3, and +40.9 points, respectively, showing that the benefits of skill evolution can increase substantially with model scale. At the same time, evolved skills can compensate for substantial differences in model scale: Qwen-3.5-9B with WikiSkill reaches 47.4% average accuracy, outperforming Qwen-3.6-27B without skills at 39.4%, while Qwen-3.5-4B with WikiSkill reaches 38.5%. Our results suggest that model capability and evolved procedural knowledge provide complementary sources of performance: stronger models can derive greater value from skill evolution by developing and executing more effective skills, while effective skills can allow smaller models to outperform substantially larger models that do not use skills.

The benefits of skill evolution also vary substantially across datasets

Our results suggest that some datasets are more amenable to skill evolution than others. For Qwen-3.6-27B, WikiSkill improves performance by 11.6 points on OfficeQA and 14.1 points on SealQA, compared with 24.8 points on ALFWorld, 28.0 points on LiveMath, and 40.9 points on SpreadSheet. Similar differences appear across other models. LiveMath consistently benefits from skill evolution, with gains ranging from 20.6 to 39.6 points across all five models, while ALFWorld yields gains of 14.0 to 29.3 points across the four models for which WikiSkill evolves skills (excluding Gemini-3.5-Flash due to early stopping). In contrast, OfficeQA presents unique challenges due to its long-context document-retrieval requirements. Larger models effectively leverage evolved search workflows to navigate lengthy documents (e.g., +11.6 points for Qwen-3.6-27B and +12.1 points for Gemini-3.5-Flash), whereas Qwen-3.5-4B struggles to execute these multi-step search workflows across long contexts and reverts to its default reading behavior, resulting in slight degradation.

4.2.2 Cross-Model Skill Transfer with WikiSkill

Evolved skills transfer effectively across models, and transferred skills can outperform self-evolved skills

Table 2 evaluates how skills evolved by WikiSkill transfer across inference models when developed using different source models. Transferred skills frequently outperform both the no-skill baseline and self-evolved skills. For example, Qwen-3.6-27B skills improve Qwen-3.5-9B to 50.5% on SpreadSheet, compared with 24.3% without skills and 33.6% with self-evolved skills, and improve Gemma-4-31B to 73.7% on LiveMath, compared with 33.9% and 56.7%, respectively. Notably, effective transfer also occurs from smaller to larger models: Qwen-3.5-4B skills improve Gemma-4-31B to 73.1% on LiveMath and 66.9% on ALFWorld. Our results indicate that stronger source models do not necessarily produce better skills and that procedural knowledge developed by one model’s experience can transfer across model scales and families.

The transferability of evolved skills depends on whether they capture general procedures or model-specific workarounds

Our results in Table 2 suggest that WikiSkill can produce both general procedural knowledge that transfers across models and model-specific strategies that can cause negative transfer. LiveMath skills transfer particularly well across models: Qwen-3.5-4B and Qwen-3.6-27B skills improve Gemini-3.5-Flash from 33.0% to 67.5% and 73.9%, respectively. In contrast, SpreadSheet exhibits strong source-target interactions. Qwen-3.5-4B skills reduce Gemini-3.5-Flash performance from 50.5% to 18.1%, while Qwen-3.6-27B skills improve it to 63.4%. Our error analysis identifies two factors behind this negative transfer. First, Qwen-3.5-4B skills encode low-level workarounds, such as single-line Python commands and string-conversion rules, which help the smaller model avoid execution failures but constrain stronger models such as Gemini-3.5-Flash from using comprehensive end-to-end scripts. Second, fragmented diagnostic procedures introduce redundant tool calls that can exhaust Gemini-3.5-Flash’s interaction budget before task completion.

The utility of transferred skills also depends on the inference model’s ability to execute them

We now turn toward how different inference models use skills developed by the same source model. Within the Qwen family, stronger models can derive greater value from the same procedural knowledge. For example, Qwen-3.6-27B SpreadSheet skills improve Qwen-3.5-4B, Qwen-3.5-9B, and Qwen-3.6-27B over their no-skill baselines by 18.4%, 26.2%, and 40.9%, respectively. OfficeQA provides a case where a model develops skills that are more useful to another model than to itself: Qwen-3.5-4B skills decrease its own performance from 30.2% to 28.5%, but improve Qwen-3.6-27B from 42.1% to 52.9%. Our trajectory analysis suggests that in long-context settings, smaller models can become distracted by lengthy document contexts and fail to follow detailed multi-step search instructions, instead reverting to their default document-reading behavior. Stronger models, in contrast, more reliably execute the structured navigation procedures specified by the skill across long contexts. Taken together, these results distinguish two capabilities that self-evolution normally conflates: discovering useful procedural knowledge from experience and effectively executing that knowledge at inference time.

5 Analysis and Discussion

5.1 Role of Persistent Knowledge in Skill Evolution

To understand where persistent knowledge contributes to skill evolution, we ablate wiki access for the two components that can use it during evolution (Table 3). Specifically, using Gemini-3.5-Flash, we independently vary wiki access for the Inference Agent during training rollouts and the Skill Proposer during skill development, which results in four configurations. When the Skill Proposer has no wiki access, we also remove the Wiki Maintainer, eliminating persistent knowledge accumulation across iterations. Our default WikiSkill configuration gives wiki access to the Skill Proposer but not the Inference Agent.

Persistent wiki knowledge dramatically improves skill evolution

As shown in Table 3, when wiki access for the Inference Agent is disabled, providing the Skill Proposer with access to the persistent wiki increases average benchmark performance from 48.7% to 63.7% (+15.0%), with substantial gains on LiveMath (51.3% to 72.6%) and SpreadsheetBench (49.9% to 76.6%). Without persistent knowledge accumulated across iterations, the Skill Proposer struggles to resolve intricate failure modes.

Wiki access for the Inference Agent during evolution degrades final skill quality

When the Skill Proposer has access to the persistent wiki, providing the Inference Agent with wiki access during training rollouts reduces average benchmark performance from 63.7% to 60.9%, with a substantial drop on LiveMath from 72.6% to 64.8%. We hypothesize that when the Inference Agent has access to both skills and the wiki during training rollouts, some task-solving knowledge may be obtained directly from the wiki rather than the skills, which can make the resulting trajectories less informative for skill development.

5.2 Qualitative Analysis: Skill and Wiki Dynamics

To better understand how WikiSkill evolves knowledge and skills across models and datasets, we analyze the wiki patterns accumulated and skills produced during evolution (Table 4) and when successful skill updates are accepted across iterations (Appendix Table 5).

WikiSkill continuously accumulates wiki patterns while producing concise skills

Table 4 summarizes the creation and editing of skills and wiki patterns across models and benchmarks, along with their average lengths. Across models, Qwen models produce longer procedural skills (118.9-128.6 lines), whereas Gemma-4-31B and Gemini-3.5-Flash produce more compact skills (45.1 and 81.2 lines, respectively). Wiki pattern accumulation also varies across models, with 6.3-8.9 patterns created and 7.0–18.4 edits on average. Across benchmarks, SpreadSheet produces the longest skills (142.5 lines) and most wiki patterns (9.8), whereas LiveMath produces the shortest skills (84.6 lines) and fewest wiki patterns (4.4). Overall, these results show that both skill structure and wiki accumulation vary across models and datasets.

Skill refinement continues throughout the evolution process

Appendix Table 5 groups accepted skill updates into early (Iterations 0–1), middle (Iterations 2–4), and late (Iterations 5–7) stages. Across models, the initial stage accounts for 39%-52% of accepted updates, with substantial fractions continuing into the middle and late stages. A similar pattern holds across benchmarks, where 39%-58% of accepted updates occur during the initial stage. Continued refinement is particularly pronounced on SealQA, where 33% of accepted updates occur in the middle stage and 28% in the late stage. Combined with the ablation in Section 5.1, these results suggest that persistent knowledge accumulation supports continued skill refinement across iterations. The Wiki Layer preserves recurring errors, rejected proposals, and evolution history, which provide the Skill Proposer with accumulated context for subsequent updates. Below, we present a case study that illustrates how this accumulated knowledge informs skill evolution.

5.3 Case Study: Anatomy of Wiki-Guided Skill Evolution

To illustrate how the Wiki and Skill Layers interact during evolution, we trace a concrete example from Qwen-3.6-27B on ALFWorld, as shown in Figure 3 (simplified for presentation).

At Iteration 0, the Wiki Maintainer identifies a basic looping behavior (take-examine-move-loop.md), while the Skill Proposer proposes goal-directed-action, which fails to improve performance on the validation set and is rejected. Crucially, skill-impact.md preserves the proposal diff and rejection outcome, allowing subsequent skill updates to account for this failed attempt.

Informed by this audit trail, the Skill Proposer creates break-repetition-loop at Iteration 1 with a concrete action rule (Never Return an Item to Its Origin Location), which is accepted. As new loop variants emerge across rollouts (multi-operation-loop.md), the Wiki Maintainer accumulates new evidence in the persistent wiki. Guided by these accumulated wiki patterns and newly stored trajectories (not shown in the figure), the Skill Proposer further refines the skill at Iteration 4 with a new rule (Each Operation Type ONCE Per Item). This example illustrates how persistent knowledge from prior iterations informs subsequent skill refinement.

6 Related Work

Experience-Driven Agent Skill Evolution

Agent skills encode reusable procedural knowledge that allows LLM agents to leverage past experience for future tasks (Zhang et al., 2025; Li et al., 2026; Anthropic, 2026; Xia et al., 2026b; Zhou et al., 2026; Wang et al., 2026; Xu and Yan, 2026). Recent frameworks enable agents to self-improve by discovering and refining procedural knowledge from past execution traces (Yuksekgonul et al., 2025; Agrawal et al., 2026; Ouyang et al., 2026; Xia et al., 2026a; Lu et al., 2026). Methods like EvoSkill (Alzubi et al., 2026), Trace2Skill (Ni et al., 2026), and SkillOpt (Yang et al., 2026) use specialized agent pipelines to analyze task rollouts and update modular skill documents (Zhang et al., 2026b). However, these methods do not maintain what has been learned as a separate, evolving knowledge representation. WikiSkill introduces a persistent Wiki Layer that consolidates experience into structured knowledge across iterations, allowing subsequent skill updates to build systematically on accumulated knowledge.

Skill-Augmented Agents and Agent Self-Improvement

Beyond constructing high-quality skills, skill-augmented agents must effectively select and utilize relevant skills during execution (Chen et al., 2026a; Liu et al., 2026). As the number of reusable skills grows, recent work has explored skill retrieval to select relevant skills from a library for each task (Zheng et al., 2026; Su et al., 2026; Cho et al., 2026; Shi et al., 2026; Ye et al., 2026). WikiSkill instead focuses on skill quality itself, separately from skill retrieval. Another line of work optimizes the broader agent harness, including prompts, context, tools, memory, and workflows (Lou et al., 2026; Lee et al., 2026; Zhang et al., 2026a; Chen et al., 2026b; Lin et al., 2026). These methods improve the agent system by analyzing execution traces and environment feedback to search for better agent configurations. This direction is complementary to WikiSkill, which focuses specifically on evolving reusable procedural skills while holding the broader agent harness fixed.

7 Conclusion

We presented WikiSkill, a framework that co-evolves agent skills with a persistent, compounding knowledge base (wiki). By structuring the agent workspace into three distinct layers, WikiSkill enables skill development to build on increasingly well-supported and integrated knowledge across iterations. An orchestrated loop consolidates experience into the wiki, proposes skill refinements from accumulated knowledge, and gates changes based on validation performance. Empirically, WikiSkill consistently outperforms existing skill-evolution methods across five benchmarks and five inference models and improves over no-skill baselines in most model-dataset pairs. Beyond these overall gains, skill evolution complements model scaling: larger models generally benefit more from evolved skills, while smaller models with skills can outperform substantially larger models without them. At the same time, evolved skills transfer effectively across models and model families and can outperform self-evolved skills. Finally, our ablations confirm that persistent knowledge accumulation is critical for effective skill evolution.

Limitations

WikiSkill has several limitations that motivate future work. First, to isolate skill quality and avoid confounding effects from skill retrieval, our study follows prior work by directly injecting active skills into the agent prompt. This setup does not evaluate skill retrieval or triggering, which becomes important as the number of available skills grows. Second, our validation gating requires each accepted proposal to improve the validation score, which excludes neutral proposals that preserve immediate performance but could enable gains in subsequent iterations. We adopt this strict criterion following prior skill-evolution frameworks (Yang et al., 2026; Alzubi et al., 2026) to ensure a fair comparison. Exploring more flexible acceptance criteria is an important direction for future work. Third, the Wiki Layer continuously accumulates pattern pages, evolution logs, and proposal diffs across iterations, but WikiSkill currently lacks an automated mechanism to prune the wiki. Such pruning may become necessary as knowledge accumulates over longer evolution runs. Finally, while our benchmark suite includes long-context document reasoning (OfficeQA) and multi-step tool interactions, it does not cover very long-horizon tasks that span hundreds of environment actions or multiple hours. Developing online skill adaptation methods that refine procedural knowledge within a single long execution rollout remains an important direction for future work.

AI Disclosure

Large language models and coding agents are used to aid with and polish writing and generate some tables and plots.

Appendix A Method Details

A.1 Algorithm

The full skill-evolution algorithm for WikiSkill is described in Algorithm 1.

A.2 Distribution of Accepted Skill Updates

We show when the updated skill proposals are accepted in Table 5.

Appendix B Dataset Details and Splits

We describe the five benchmarks used in our evaluation below.

LiveMathematicianBench (LiveMath) (He et al., 2026) consists of multiple-choice mathematics competition problems from recent months. It tests the model’s capacity for complex mathematical reasoning, quantifiers, and extremal conditions. SealQA (Pham et al., 2026) is a factual question-answering benchmark composed of scholarly questions across various topics. It evaluates the agent’s ability to formulate effective search queries and extract answers from web search results using a search tool. SpreadsheetBench (SpreadSheet) (Ma et al., 2024) tests the agent’s ability to write correct code under library constraints (such as formula evaluation limitations) and execute complex table transformations. OfficeQA (Singhvi et al., 2025) evaluates long-context question-answering over a large repository of historical Treasury bulletins. Tasks require synthesizing evidence across long contexts and multi-page financial tables. Following the setup in Yang et al. (2026), the agent is provided with pre-parsed oracle reference pages as initial document evidence in the prompt, while retaining access to local text-processing tools (glob, grep, read) to search, cross-reference, and inspect full Treasury bulletin files on disk. ALFWorld (Shridhar et al., 2021) is an interactive text-based embodied environment where an agent solves multi-step household tasks (e.g., picking and placing objects, heating or cooling items) by outputting text actions to a simulator. Unlike static QA benchmarks, ALFWorld tests sequential decision-making, spatial reasoning, and error recovery from simulator feedback.

Table 6 summarizes the sample counts across training, validation, and test splits, interaction modes, and available tools for each benchmark evaluated in our experiments. All task splits and available toolsets are strictly matched with prior work (Yang et al., 2026; Alzubi et al., 2026). For tool setup, LiveMath operates as a single-step reasoning benchmark without external tools, where the model generates final answers directly; SealQA equips the agent with web search (using Google Search API) and file reading for multi-step factual retrieval (we use the July, 2026 version of SealQA for all experiments); SpreadSheet provides a bash shell tool for Python code execution and table manipulation; OfficeQA provides local text-search utilities for multi-step Treasury bulletin navigation; and ALFWorld provides an interactive simulator action space for multi-step embodied decision making.

Evaluation robustness with small validation sets

Following established setups in prior work (Yang et al., 2026; Alzubi et al., 2026), benchmark validation splits are relatively small, which can introduce evaluation noise into gating decision. To account for this variability, all reported scores represent the average test performance across three independent runs of the entire evolutionary pipeline, with paired bootstrap significance testing (detailed in Appendix C).

Appendix C Implementation Details

To provide diagnostic feedback for the Wiki maintainer, we apply a stratified sampling strategy ($\mathcal{T}_{\text{sample},k}\subset\mathcal{T}_{\text{train},k}$) at each iteration $k$. Specifically, the system samples up to 8 traces per iteration, stratified into a maximum of 5 failing traces (to perform root-cause analysis of errors) and up to 3 passing traces (to identify effective strategies and prevent regressions in working behaviors). Each individual execution log is capped at 15,000 characters prior to injection into the prompt.

Statistical significance testing

We perform paired bootstrap significance tests with $1,000$ iterations for each benchmark. In each bootstrap iteration, task instances are sampled with replacement from the test split $\mathcal{D}_{\text{test}}$ to construct a bootstrap evaluation set of size $|\mathcal{D}_{\text{test}}|$, from which candidate accuracy scores and pairwise performance margins are computed. To evaluate overall cross-benchmark performance, we conduct stratified macro-average bootstrap resampling: in each iteration, task instances are resampled independently with replacement within each benchmark, and we calculate macro-average accuracy by assigning equal weight to all benchmarks.

We determine top-performing methods in our evaluation tables as follows. Methods are initially ranked by their observed performance (or macro-average performance across benchmarks). A top-ranked method $M^{*}$ is the sole top performer if and only if it achieves a statistically significant gain over all competing methods at $p<0.05$. If $M^{*}$ is not statistically distinguishable ($p\geq 0.05$) against one or more lower-ranked methods, no single method is declared the sole top performer. Instead, all methods whose performance is not statistically distinguishable from $M^{*}$ ($p\geq 0.05$) are grouped into a top-tier statistical tie (and bolded accordingly).

Appendix D Baseline Details and Optimizer API Call Analysis

D.1 Baseline Methods

Trace2Skill (Ni et al., 2026)

Trace2Skill employs a three-stage pipeline centered on parallel trace analysis and hierarchical merging. It evaluates the current skill on training tasks and dispatches parallel success analysts and error analysts to extract effective strategies from passing tasks and diagnose root causes of failures. The resulting structured patches are recursively consolidated via a hierarchical merge operator into a single patch set. The consolidated patches are applied to the skill document and accepted based on validation performance.

EvoSkill (Alzubi et al., 2026)

EvoSkill frames skill evolution as a search over a frontier of candidate programs. In each iteration, it samples training tasks via a round-robin category schedule and executes rollouts. EvoSkill feeds only failure traces to the proposer alongside a flat feedback history of past proposal outcomes. The proposer generates candidate skill modifications that are materialized into SKILL.md files, scored on a validation split, and added to a bounded frontier of top-performing programs.

SkillOpt (Yang et al., 2026)

SkillOpt implements a six-stage ReflACT pipeline (Rollout, Reflect, Aggregate, Select, Update, Evaluate) for iterative skill optimization. In each epoch, the system rolls out the agent on training tasks and reflects on full execution traces, including both successes and failures, to generate candidate patches. These patches are hierarchically aggregated and selected to update a single monolithic skill document, which is accepted or rejected based on validation performance.

D.2 Optimizer API Call Complexity

In this section, we analyze optimizer API call complexity, denoted by $\mathcal{C}$, across self-improving agent frameworks. We define one evolution iteration as rolling out the agent on the full training split $\mathcal{D}_{\text{train}}$ once ($N_{\text{train}}=|\mathcal{D}_{\text{train}}|$ training task instances), processed in minibatches of size $B$ ($B\leq N_{\text{train}}$). Table 7 summarizes the per-iteration optimizer API call complexity for WikiSkill and prior methods.

WikiSkill

For each batch of size $B$, the Wiki Maintainer requires one LLM call to analyze sampled traces $\mathcal{T}_{\text{sample},k}$ and consolidate pattern pages into the intermediate wiki $W^{\prime}_{k}$. The Skill Proposer then runs as an autonomous multi-turn ReAct agent, executing interactive tool calls over $T_{\text{ReAct}}$ reasoning turns (roughly $10\leq T_{\text{ReAct}}\leq 20$ across our experiment runs), where each ReAct turn requires $1$ LLM call. When processing training data in batches of size $B$, completing one full iteration over $N_{\text{train}}$ tasks requires $\frac{N_{\text{train}}}{B}$ steps:

In our experiments, we set the batch size to the full training size ($B=N_{\text{train}}$) across all datasets. In this full-batch setting ($\frac{N_{\text{train}}}{B}=1$), $\mathcal{C}_{\text{WikiSkill}}=1+T_{\text{ReAct}}$. Because $T_{\text{ReAct}}$ does not depend on $N_{\text{train}}$, WikiSkill’s optimizer API call complexity is $\mathcal{O}(1)$ with respect to training set size. Specifically, each iteration requires $1+T_{\text{ReAct}}$ optimizer LLM calls, regardless of the number of training instances. While this constant call complexity may incur higher inference cost on some datasets, the additional computation is accompanied by consistent performance gains over prior skill-evolution methods across our evaluation.

EvoSkill

EvoSkill partitions training tasks into minibatches of size $B$. For each minibatch, EvoSkill uses one Proposer LLM call for error diagnosis and one Generator LLM call for skill updates. Processing the full training split of $N_{\text{train}}$ tasks requires $\frac{N_{\text{train}}}{B}$ minibatch steps, resulting in:

Thus, EvoSkill’s optimizer API call complexity scales linearly with training set size $N_{\text{train}}$ ($\mathcal{O}(N_{\text{train}}/B)$).

SkillOpt

SkillOpt evaluates minibatches of size $B$, completing each iteration in $\frac{N_{\text{train}}}{B}$ optimization steps. During each step, SkillOpt executes its ReflACT pipeline (parallel analyst reflections, hierarchical patch synthesis, and candidate selection), requiring $K_{\text{opt}}\approx 6\text{--}8$ optimizer LLM calls per step:

SkillOpt similarly scales linearly with training set size $N_{\text{train}}$, as $\mathcal{O}(N_{\text{train}}/B)$.

Trace2Skill

Trace2Skill processes training trajectories through a three-stage pipeline per iteration:

Trace analysis stage: Every individual execution trajectory is analyzed independently with one LLM call, incurring $N_{\text{train}}$ total calls.

Patch map stage: Analysis records are chunked into batches of size $B$, requiring $\frac{N_{\text{train}}}{B}$ calls to generate local skill patches.

Hierarchical reduce & apply stage: The $\frac{N_{\text{train}}}{B}$ local patches are recursively merged via a $c$-ary reduction tree (where $c$ is the branching factor). Summing across levels yields $\approx\frac{1}{c-1}\frac{N_{\text{train}}}{B}$ merge calls, plus one final call to format the skill document.

Combining all stages:

Because Trace2Skill performs individual LLM analysis on every training trajectory ($N_{\text{train}}$ calls), its complexity is lower-bounded by $\mathcal{O}(N_{\text{train}})$, scaling linearly with training set size.

Full-batch training vs. minibatch optimization

Across all datasets, we set the batch size $B$ to the full training set size ($B=N_{\text{train}}$) for WikiSkill, processing the entire training set at once per iteration. The Skill Proposer dynamically searches, selects, and reads specific execution traces on demand to diagnose root causes before proposing skill updates. In contrast, EvoSkill and SkillOpt achieve their best performance under minibatch settings ($B<N_{\text{train}}$), which causes their optimizer API call complexity to scale linearly with training set size. Finally, Trace2Skill remains strictly $\mathcal{O}(N_{\text{train}})$ regardless of minibatch size because it requires an independent LLM call for every training trajectory.

Appendix E System and Agent Prompts

We provide the exact system prompts used for (1) the Inference Agent for each task across all methods, (2) the Wiki Maintainer, and (3) the Skill Proposer in WikiSkill.

E.1 Task Inference Agent System Prompts

LiveMathematicianBench Inference Agent System Prompt

SealQA Inference Agent System Prompt

SpreadsheetBench Inference Agent System Prompt

OfficeQA Inference Agent System Prompt

ALFWorld Inference Agent System Prompt

E.2 Wiki Maintainer Agent System Prompt

Wiki Maintainer Agent System Prompt

E.3 Skill Proposer Agent System Prompt (ReAct Mode)

Skill Proposer Agent System Prompt

Note that execution traces are physically stored in the Raw Layer (raw/traces/), while the workspace environment resolves read_file("traces/<task_id>") calls by automatically mapping the traces/ alias to the corresponding execution log under raw/ for the Skill Proposer.