How do AI agent skills emerge from source code?

A model describing a source-code function does not make that function a reusable agent skill. In Code2Skill, acceptance comes at the end of a causal chain. A procedural source unit must pass selection, survive abstraction into a typed record, support reconstruction without its source body, and clear an LLM-based consistency decision. An accepted object records that this chain succeeded under a particular model, endpoint, repository snapshot, parser, and configuration. It does not prove that the skill is formally equivalent to the implementation or safe in every runtime.

The dependencies between those stages explain the distinction. A skill is accepted because a reconstruction was judged consistent. That reconstruction was possible because the abstract record retained enough information about the interface, workflow, invariants, and failures. The record, in turn, exists because the source unit passed explicit procedural and value gates. Each stage narrows the candidate set and records a reason for acceptance or rejection.

Where does the system boundary for skill extraction lie?

Five areas share responsibility for extraction: the source repository, local runtime, configured model endpoint, run artifacts, and operator. The runtime sends repository-derived content to the configured endpoint through an OpenAI-compatible /chat/completions interface. Named run artifacts retain records, traces, counters, summaries, provenance, and the configuration needed to interpret the result. The operator controls the endpoint and how those artifacts are handled; the model endpoint supplies the judgments used during extraction and verification.

Repository content leaves the local process when the runtime sends it to the configured model endpoint. Code2Skill rejects public model hosts by default. An operator can allow one, but only after deciding that source-code transmission is permitted. Run directories can also contain source units, prompts, responses, and provenance, so they remain sensitive after the model call finishes.

This boundary is not the same as an agent runtime or sandbox. A runtime coordinates agent state and external interaction, while a sandbox constrains a narrower execution surface. For a deeper separation, distinguish an agent runtime from an agent sandbox. Questions about isolation, egress, and credential scope belong to security boundaries for AI agents; questions about lifecycle, persistence, and resources belong to agent workspaces and cloud development environments. Those adjacent systems can consume a skill, but they do not show that the extraction chain produced a correct one.

How does the selection stage decide which code becomes a candidate?

Selection starts with procedural source units, not with an entire repository treated as undifferentiated text. Its rubric checks six signals: reusable intent; ordered operational steps; control-flow, state, or resource transitions; boundary conditions, invariants, and failure handling; enough interface context for reconstruction without the source body; and non-triviality. A unit must clear the configured selection score, and each repository is subject to a candidate cap.

The published example configuration makes these prerequisites concrete. It excludes tests, ignores units shorter than three lines, limits files to 200,000 bytes and units to 12,000 bytes, caps a repository at 400 units, and sets the minimum selection score to 0.5. These values are configuration, not universal properties of skill extraction. Change them, and a different set of source units may enter the next stage.

Stored stage and reason values make selection observable. The implementation distinguishes source-tagging or tagging errors, the source-unit selection gate, and the per-repository cap. A missing candidate can therefore have several causes: the parser or tagger did not expose it, its procedural signals did not clear the threshold, or an otherwise eligible unit fell beyond the repository cap.

What does abstraction preserve from the source code?

A selected unit becomes a typed record rather than prose alone. The record carries level, extraction value, generality and abstraction scores, inputs, outputs, workflow, invariants, error cases, implementation notes, evidence, anti-goals, and confidence. These fields separate reusable behavior from incidental syntax while preserving the conditions needed to reason about that behavior.

Abstraction also has a value gate. A record is removed when worth_extracting is false or its skill_value_score is below 0.45. Passing procedural selection is therefore necessary but insufficient: a unit can look process-like in source and still fail to produce a reusable record.

That record becomes the input to verification and, later, to feature-tagged and purpose-indexed retrieval views. This order keeps retrieval metadata distinct from the core skill evidence. The workflow, boundaries, provenance, and source evidence exist before the record is organized for discovery.

How does reconstruction become an acceptance decision?

Reconstruction tests whether the abstract record contains enough information to recover the relevant behavior without copying the implementation. This step withholds the source body, repository, and file path while leaving the target symbol and interface visible. Formal-validity metadata may be recorded, but it does not independently accept or reject a candidate.

The implementation records equivalence and adjudication as separate decision paths. On the adjudication path, only a result with keep set to true is accepted. Failed generated code is not retained as an accepted artifact.

The resulting causal states appear as equivalence, adjudication, gate, or pipeline-error paths. Request and parse failures fail closed. Feature tagging behaves differently: a tagging failure can be recorded after an accepted decision and does not retroactively reject the grounded skill. A partial run can therefore contain more accepted records than retrieval records.

A small diagnostic checks whether those causes match the observed result. Compare the stored stage and reason for one accepted record and one rejected record with the expected equivalence, adjudication, gate, or error path. Then compare accepted-record and retrieval-record counts when a feature-tagging error is present. A count difference is consistent with a post-acceptance indexing failure; it is not evidence that the accepted skill itself was rejected.

What does the acceptance gate prove—and what does it not prove?

The gate supports a bounded claim: under the recorded run conditions, an LLM judged the reconstruction consistent with implementation evidence. The project methodology calls this an LLM consistency check, not formal program equivalence or a substitute for execution-based verification. Reproducing the result depends on the model, endpoint, serving parameters, repository snapshot, and parser coverage.

Independent work on functional-equivalence judgment reinforces that limit. PolyHuman reports that current LLMs do not reliably capture functional equivalence and that harder tasks increasingly produce false equivalence labels. A positive consistency judgment can therefore preserve residual semantic errors even when the pipeline and record work as designed.

The Code2Skill paper's human checks draw the same line between acceptance and correctness. For the accepted-bank summary, reviewers reported 92% description quality, 84% worth retaining, and 80% correct reconstruction. Those measurements describe reviewed samples in the paper; they do not make every accepted record a proven implementation equivalent.

Assertion-backed execution provides a stronger countercondition. Repo-To-Skill admits skills only after repository-native checks such as examples, tests, CLI checks, small fixtures, or smoke scripts; sources alone are insufficient. This approach tests behavior that the consistency gate only predicts, although its result still depends on the coverage and relevance of the executed assertions.

Which operating boundaries determine how much confidence the chain deserves?

Persistence is snapshot-oriented. Rerunning into the same directory replaces named JSON and JSONL files, but it does not automatically remove older trace folders. To interpret a run, an operator must tie records and traces to the current manifest rather than assume that every remaining file belongs to the latest execution.

Configurable thread-pool workers bound concurrency; the mechanism is not an unlimited platform guarantee. Stored traces, counters, summaries, and the manifest make throughput and failure states inspectable. The supplied evidence, however, does not establish service-level objectives, production capacity, or a managed operating model.

Responsibility follows the concrete data path. The published mechanism lets the operator decide whether a public model URL is permitted and requires sensitive run artifacts to be protected. The supplied evidence describes this local operating mechanism, not a managed extraction service. Under either operating model, a supportable claim must still identify where source content is sent and which evidence supports each accepted skill.

The practical question is not whether the pipeline produces “verified skills” in the abstract. For each claimed quality, identify the causal stage that creates it, the stored evidence that makes the stage observable, and the operator boundary that limits the claim. If the unresolved question concerns execution containment, inspect the runtime or sandbox boundary. For source transmission or credentials, inspect the security boundary. For persistence and resources, inspect the workspace boundary.