Harness Continual Learning: Continual
Adaptation Beyond Model Parameters

Contemporary agent systems place a runtime harness around a foundation model to turn inference into task-directed execution (Li et al., 2026a; Meng et al., 2026; He et al., 2026; Zhou et al., 2026). Across implementations, persistent runtime contents commonly serve four functions. An interface converts raw instructions, observations, documents, or multimodal inputs into a form the agent can use. Memory stores interaction records, summaries, and reusable guidance. A capability registry describes tools, APIs, environment actions, and learned skills together with their invocation conditions. A router or workflow controller selects relevant memories and capabilities, orders their use, and assembles the execution context. Environment adapters execute actions, and task-specific validators check outcomes at the boundary of the pipeline (Gu, 2026; Chen et al., 2026b).

Existing systems develop different parts of this structure. ReAct couples reasoning with environment interaction (Yao et al., 2023). Toolformer, MRKL, and HuggingGPT expose and coordinate external capabilities (Schick et al., 2023; Karpas et al., 2022; Shen et al., 2023). MemGPT, Reflexion, and Voyager retain experience as memory, feedback, or executable skills (Packer et al., 2023; Shinn et al., 2023; Wang et al., 2024a). Together, these components form a coupled execution pipeline. The interface shapes what the router sees. Memory and capability descriptions determine what it can select. The resulting workflow determines how the model acts.

Harness engineering also uses execution feedback to revise prompts, declarative programs, memories, tool-use policies, skills, and workflows (Zhou et al., 2023; Khattab et al., 2024; Abuzakuk et al., 2026; Schick et al., 2023; Shinn et al., 2023; Wang et al., 2024a; Zhong et al., 2026; Zhang et al., 2026d). Recent work broadens this process to configuration search, cross-layer failure diagnosis, and sustained agent improvement (Zhang et al., 2026a; Chen et al., 2026a; Yao et al., 2026; Liu et al., 2026b). These systems show that a harness is editable and can improve with experience. Their main objective, however, is usually the quality of a component or the next configuration on a current task or target distribution. Repeated improvement alone does not provide a general retention criterion for the full harness state (Lin et al., 2026). Our work differs by treating the entire mutable harness as a unified continual learning state and by making retention across committed updates an explicit objective.

Model-centric continual learning adapts a model to a non-stationary stream of tasks or data while seeking to retain capabilities acquired from earlier experience. Its central challenge is catastrophic forgetting, which arises when learning new knowledge disrupts knowledge encoded by the model (Kirkpatrick et al., 2017; Delange et al., 2022; Wang et al., 2024b; Kang et al., 2026). Representation-based approaches learn features or prompts that remain useful across tasks (Wang et al., 2022b; Wang et al., 2022a). Recent analysis also examines how these internal representations shift across a learning sequence (Kim et al., 2025). Architecture-based approaches isolate, expand, or select model components to reduce interference between tasks (Liu et al., 2026a; Lu et al., 2024). Optimization-based approaches alter the update trajectory or constrain gradients using information from earlier tasks (Lopez-Paz and Ranzato, 2017; Abbes et al., 2026; Shang et al., 2025). Regularization-based approaches penalize changes to parameters or functions that support old behavior (Kirkpatrick et al., 2017; Lewandowski et al., 2025). Replay-based approaches retain or reconstruct earlier examples and mix them with new data (Urettini and Carta, 2025; Wang et al., 2025a; Yue et al., 2025; Bellitto et al., 2024). Recent work extends these families to large language models and broader knowledge streams, but the state being learned remains model knowledge, representations, architectures, or parameters (Liang et al., 2025; Zhang et al., 2026b). Our work moves the continual learning object outside the model. The foundation model parameters remain frozen, while the harness state evolves under explicit acquisition and retention constraints.

Consider a fixed foundation model $F_{\theta}$ and a harness $H_{n}$ deployed at interaction step $n$. The model parameters $\theta$ remain unchanged, whereas a committed harness update affects subsequent interactions. We define Harness Continual Learning as the problem of sequentially updating the deployed harness to acquire new behavior while retaining behavior that was reliable before the update. Previously reliable behavior may be a correct response, a valid tool call, or an action trajectory that satisfies an environment goal. Retention requires such behavior to remain successful after later harness updates when evaluated under the same input and execution conditions. This setting differs from conventional harness engineering, which typically optimizes a prompt, tool configuration, or workflow for a current objective. HCL instead studies a sequence of deployed harnesses.

At interaction step $n$, $\mathbf{u}_{n}$ denotes the raw interaction, such as an instruction, an observation, or a multimodal input. The harness transforms $\mathbf{u}_{n}$ into the structured interaction $\mathbf{i}_{n}$. Guided by the frozen foundation model, it then combines $\mathbf{i}_{n}$ with selected memory and capabilities to assemble the execution context $\mathbf{z}_{n}$. The model and external runtime execute $\mathbf{z}_{n}$ to produce the outcome $\mathbf{y}_{n}$. Post-execution feedback is denoted by $\mathbf{f}_{n}$. We collect these interaction-level objects as

The Optimizer provides the foundation model $F_{\theta}$ with an update rule, the deployed harness, and the available interaction evidence as context for generating a candidate harness::

The candidate remains separate from the deployed harness until a commitment decision is made. Let $G_{n}\in\{0,1\}$ denote this decision. The deployed harness evolves as

Therefore, a candidate affects later interactions only when it is committed. Our framework realizes HCL in two parts. First, it defines the deployed harness state $H_{n}$ by specifying its mutable contents and versioning them jointly. Second, it controls the update from $H_{n}$ to ${H}_{n+1}$ by checking current improvement, historical retention, and validity before commitment.

The design of the HCL state builds on established mechanisms from prior harness and agent systems, including prompt-based task interfaces, persistent memory, tool and skill registries, and routing or workflow controllers (Li et al., 2026a; He et al., 2026). Rather than inheriting the architecture of any single system, HCL organizes these recurring execution functions into four jointly versioned components, whose mutable contents evolve from sequential experience under explicit acquisition and retention constraints.

Accordingly, HCL organizes the mutable harness state as

where $I_{n}$, $M_{n}$, $C_{n}$, and $R_{n}$ denote the Task Interface, Experience Memory, Capability Map, and Adaptive Router, respectively. At interaction step $n$, $H_{n}$ represents the complete harness currently deployed. Its prompts and processing rules, stored experience, reusable skills, and routing specifications persist across interactions and jointly determine how the agent handles future tasks.

Although these four execution functions are common in agent harnesses, HCL differs in how their mutable contents are learned and deployed. Because a change to one component may interact with the others and affect both new and previously learned behavior, HCL treats all proposed changes as one complete candidate harness. The candidate replaces $H_{n}$ only after it satisfies current improvement, historical retention, and validity requirements. Otherwise, none of its changes enters the deployed harness. HCL therefore turns harness contents into a coordinated mechanism for continual learning rather than a collection of independently edited artifacts.

Table 1 summarizes the execution function of each component and the contents that can be updated through continual interaction. Figure 2 shows how these components support execution and how post-execution feedback initiates a candidate harness.

A harness update may improve current behavior while degrading previously reliable behavior on earlier tasks. We therefore introduce guarded harness evolution, which separates update generation from deployment through a proposal–evaluation–commitment process. Given feedback, the Continual Optimizer produces an isolated candidate harness. The Continual Evaluator commits it only if it satisfies current-improvement, historical-retention, and validity requirements. Otherwise, $H_{n}$ remains deployed. This process makes retention an explicit condition for harness evolution rather than assuming that a useful update on the current task is safe for earlier tasks.

HCL draws on several complementary principles from model-centric continual learning, but realizes them through harness mechanisms rather than model-parameter updates (Delange et al., 2022; Wang et al., 2024b). Replay-based methods retain earlier examples to preserve acquired knowledge. Experience Memory follows this principle by storing concrete interactions for later reuse. Representation-based methods learn abstractions that support transfer across tasks. The Capability Map similarly transforms accumulated experience into reusable skills and combines them with external capabilities. Architecture-based methods organize reusable modules and routines to reduce interference. HCL represents these routines as invocable capabilities and uses the Adaptive Router to select and compose them for each interaction. Optimization- and regularization-based methods control parameter updates using information from earlier tasks, allowing new knowledge to be acquired while limiting interference with previous knowledge. HCL applies the same principle to harness updates through the Continual Optimizer and Continual Evaluator. The Optimizer proposes candidate changes from current feedback, while the Evaluator tests them on current validation cases and historical anchors. Only candidates that improve current performance while satisfying historical retention and validity requirements are committed. This proposal–evaluation–commitment process integrates adaptation and protection into continual harness evolution.

These relationships are conceptual rather than one-to-one implementations. More importantly, HCL brings the complementary principles of model-centric continual learning into a unified system-level formulation. Traditional approaches (Kang et al., 2025; Liu et al., 2026c) often treat replay, representation, architecture, optimization, and regularization as separate solution families for adapting model parameters. HCL coordinates their functions within a single evolving harness under the same acquisition–retention objective. It therefore extends continual learning from parameter adaptation to the coordinated evolution of agent infrastructure, providing a unified framework for continual learning beyond the model itself.

For each task stream, a single harness evolves sequentially around the same foundation model. Let $H^{(s)}$ denote the deployed harness after learning task $\mathcal{D}_{s}$, where $s$ indexes the evaluation stage. At the end of each stage, we evaluate $H^{(s)}$ on the current task and every previously observed task:

where $R_{s,j}$ is the benchmark score or episode success rate on task $j$. Current-task validation cases and historical anchors are used only by the Continual Evaluator to determine whether a candidate can be committed. The final test sets are disjoint from both and are used only for reporting.

For task streams with metrics on a common scale, we report final average performance and average old-task forgetting:

$\operatorname{Avg}_{T}$ measures final performance across the complete stream, while $\operatorname{Fgt}_{T}$ measures the average decline of earlier tasks from their best observed performance. Forgetting is marked as “–” for Zero-shot and Static Harness because they make no sequential updates.

Stability-HCL and Plasticity-HCL are two configurations of the framework, differing only in the historical-loss tolerance $B_{n}$. Stability-HCL sets $B_{n}=0$ and rejects any candidate that causes a currently solved anchor to fail. Plasticity-HCL sets $B_{n}=\infty$, so historical anchor losses do not block a candidate as long as it satisfies the current-improvement and validity requirements. We evaluate both configurations in ALFWorld and the controlled streams, while Minecraft uses the retention-oriented configuration. Detailed settings are provided in Appendix A.

We study long-horizon harness evolution in ALFWorld and Minecraft. ALFWorld supports stage-wise evaluation across previously observed task categories, while Minecraft provides a longer interaction curriculum for examining capability accumulation, failure recovery, and skill revision.

We next evaluate HCL on task sequences. Within each stream, all HCL profiles share the same foundation model, task order, data allocation, editable artifacts, and candidate generator.

Following Eq. (13), we vary only the historical-loss tolerance $B_{n}$ in $D_{n}\leq B_{n}$, while holding the current-improvement and validity criteria fixed. Specifically, $\delta_{n}$ in Eq. (9) requires an improvement of at least two correct validation cases. Under the validity criterion in Eq. (12), each candidate must achieve at least 90.00% output-format compliance and introduce no syntax, tool-use, or environment violations. These thresholds are chosen heuristically to balance current-task improvement with candidate reliability and remain identical across all settings.

The historical loss $D_{n}$ in Eq. (11) counts anchors that are solved by $H_{n}$ but fail under $\widetilde{H}_{n+1}$. Within each run, we fix $B_{n}\equiv b$ for all candidate decisions and compare $b\in\{0,1,3,\infty\}$. The settings $b=0$ and $b=\infty$ correspond to Stability-HCL and Plasticity-HCL, respectively. The intermediate settings $b=1$ and $b=3$ allow each candidate to introduce at most one and three newly failed anchors across $A_{n}$. Each run uses 300 adaptation, 80 validation, and 600 test examples per task, with 80 anchors for every earlier task. A predefined parameter controls the composition of previously successful and failed examples in each anchor set. If either group contains too few examples to meet its target, the remaining slots are filled from the other group. All other experimental conditions remain fixed.

Table 5 shows that increasing $b$ weakens retention. Average forgetting rises from 0.39 at $b=0$ to 3.45 at $b=\infty$. Final performance does not increase accordingly: the highest final average of 63.46% occurs at $b=1$, while the unrestricted setting reaches 60.13%. One possible explanation is that each committed update changes the subsequent evolution trajectory: without historical constraints, locally beneficial updates may overwrite reusable harness contents, weakening both retention and the experience or capabilities available for later tasks. A moderate value of $b$ therefore provides additional flexibility for adaptation without allowing excessive historical loss. The remaining forgetting at $b=0$ occurs because the constraint covers a finite anchor set, whereas forgetting is evaluated on separate historical test cases. Preserving all anchors currently solved by $H_{n}$ cannot guarantee unchanged behavior on historical cases not represented by $A_{n}$.

Figure 4 complements these final results by showing how forgetting develops across the task sequence. In the textual stream shown in Figure 4(a), smaller values of $b$ generally maintain lower forgetting, with final forgetting increasing consistently from 0.39 at $b=0$ to 3.45 at $b=\infty$. In the multimodal stream shown in Figure 4(b), Stability-HCL remains below Plasticity-HCL at every stage after $T_{1}$ and finishes with forgetting of 0.22 rather than 0.81. This pattern reflects the role of $b$ in the commitment gate: smaller values reject more candidates that improve the current task at the expense of historical behavior, thereby constraining the harness to more retention-preserving update trajectories. Larger values permit greater adaptation flexibility but expose earlier tasks to more regression. Together, the two trajectories illustrate that a stricter historical-loss tolerance suppresses forgetting throughout harness evolution.

We conduct component ablations on the controlled multimodal stream using Qwen3.5-4B with the balanced HCL configuration. The stream follows COCO object detection $\rightarrow$ COCO image captioning $\rightarrow$ RefCOCO visual grounding $\rightarrow$ VQAv2, with 250 adaptation, 50 validation, and 500 test examples for each task. Starting from Full HCL, we disable updates to one harness component at a time while keeping the other three components adaptive. All variants use the same foundation model, task order, evaluation criteria, and update schedule. Table 6 summarizes the resulting component-wise ablation results.

Full HCL achieves the highest final average of 63.41%, showing that the four components contribute complementarily to continual adaptation. Disabling Experience Memory or the Task Interface produces the largest decrease in final performance. In particular, removing Memory updates also increases forgetting to 0.83, indicating that evolving memory supports both the acquisition and retention of behavior. Disabling Capability Map or Adaptive Router updates causes smaller but consistent performance reductions. The small effect of Capability updates may reflect that this multimodal stream relies less on reusable executable procedures than the Minecraft curriculum.

Several ablations show lower forgetting than Full HCL because restricting the editable components also limits the extent of adaptation. Lower forgetting alone therefore does not necessarily indicate a better evolving harness and should be considered together with final performance. Exact interventions and full per-task results are reported in Appendix B.