Controlled Emergence: AVL Code's Engineering Paradigm (Part 1)
— Emergence Is Not Magic, It Is a System Property

This post is Part 1 of the series "Controlled Emergence: AVL Code's Engineering Paradigm," published over three days: Part 1 (this post, July 16) · Part 2, "Harness Engineering and Loop Engineering" (July 17) · Part 3, "From Theory to Practice, and Seven Engineering Principles" (July 18).

When a large language model enters a code repository and gains Context, Tools, permissions, and factual feedback, it starts to exhibit capabilities that a single pass of text generation does not have: reading the project, planning steps, modifying files, running tests, and then going on fixing based on the errors. Faced with this phenomenon, what is genuinely worth discussing is not "has the model suddenly awakened," nor can it be waved away with "it is just next-token prediction" — it is three more specific questions: who do these new capabilities belong to, how do they arise, and how are they constrained into engineering behavior that is verifiable, stoppable, and deliverable?

The answer depends first on how we draw the system boundary. If the object of study is widened from the isolated LLM to the complete generative AI system, many phenomena become far clearer: the model generates candidate content, Context supplies the current conditions, Tools turn symbolic output into external action, the runtime organizes the loop, permissions and tests constrain the boundaries of action, and a human confirms goals and risks at the critical junctures. Look at any one of these parts on its own and none of them has the capability to "autonomously complete a software engineering task"; connect those parts in a particular structure and the system as a whole can exhibit exactly that capability.

This is precisely the ordinary meaning of the word "emergence" in systems engineering. The core claims of this series can be summarized as follows:

1. Emergence is the system as a whole producing new properties its parts do not have — it is not pseudoscience, nor a mystifying name for an unknown mechanism;
2. What appears once an LLM is combined with Context, Tools, and a runtime is observable, measurable functional behavioral emergence; from it one cannot directly infer that cognition, consciousness, or agency has emerged;
3. The engineering object of generative AI is not the bare model, but the Harness outside the model — the reins, the driving layer — and the complete system it encloses;
4. Loop engineering brings feedback, error, state, control variables, and stopping conditions into a probabilistic AI system; it is the thinking of control engineering put into practice on an object that cannot be fully modeled;
5. The engineering value of agents such as Claude Code, Codex, and AVL Code lies precisely in organizing model capability into a Harness that has phases, tools, feedback, gates, budgets, and recoverability.

Points 1 and 2 are the content of this post (Part 1); points 3 and 4 are developed in Part 2; point 5 lands on AVL Code's concrete mechanisms in Part 3.

In this series, "functional behavioral emergence" and "cognitive intelligence emergence" are a working distinction proposed to clarify engineering boundaries, not an existing terminological classification in SEBoK.

1. Emergence Is Not Magic, It Is a System Property

The SEBoK entry on Emergence sums emergence up like this: certain properties are meaningful only when attributed to the whole, and are not properties the constituent parts hold individually. The Emergent Property glossary entry stresses further that such properties are meaningful only at a higher level of system organization. System-level behavior arises from the joint action of the properties of the elements, the relationships between the elements, the system structure, and stimuli from the environment.

An aircraft wing does not "fly under control" on its own, and neither does the engine or the control system; only when aerodynamic structure, propulsion, control, sensors, and pilot input combine in the right relationships does flight become a property of the aircraft as a whole. Likewise, a North American blackout does not necessarily originate in the single-point failure of one device; it can equally be a cascading failure triggered jointly by how the system is connected and the conditions it is running under. Emergence can be a capability the designer is after, and it can just as well be a risk the designer has to suppress.

Emergence, then, is not "it cannot be explained, so attribute it to a mysterious force." An emergence claim with engineering meaning should answer at least five questions:

If these questions cannot be answered, "emergence" may be nothing more than rhetoric; if they can, it is an ordinary and serious concept from systems science.

1.1 Three Kinds of "Emergence" That Are Often Conflated

Discussion around large models mixes up at least three different questions:

On the first sense, the paper Emergent Abilities of Large Language Models by Wei et al. calls benchmark abilities that small models lack and only larger models exhibit "emergent abilities"; Are Emergent Abilities of Large Language Models a Mirage? by Schaeffer et al. counters that some of these jumps may come from discontinuous evaluation metrics, and that switching to a continuous metric smooths the performance change out. The debate makes one thing clear: "crossing an evaluation threshold" and "the system has produced a new ontological property" are not the same thing.

The focus of this series is the second sense: functional behavioral emergence in an engineering system. It does not depend on "whether the scaling curve jumps suddenly," nor does it require first proving that machines have minds; it asks only that we describe honestly which functions the whole system adds over its parts, and that we check those functions against evidence from tests and from real runs.

1.2 "Controlled" Does Not Mean Every Token Is Predictable

SEBoK distinguishes simple emergence, which is relatively easy to predict; weak emergence, which can be anticipated but whose level is hard to predict precisely; and unexpected emergence, which surfaces only in simulation, in testing, or even at runtime. It stresses at the same time that desired emergence is usually obtained gradually, through modeling, iteration, and build/test cycles.

So "controlled emergence" is not a claim that engineers can settle every output of the model in advance. It means:

Raising the probability that desired behavior appears and passes acceptance — through system structure, context, permissions, feedback, verification, budgets, and human decisions — while limiting the blast radius of undesired behavior.

The object of control is not a single Token, but the task trajectory and the system outcome; the goal of control is not absolute certainty either, but an acceptable success rate, risk, cost, latency, and recoverability.

2. What LLM + Context Makes Emerge Is Functional Behavior, Not a Cognitive Conclusion

A single LLM generation can be abstracted as:

Here the parameters $\theta$ hold the statistical structure formed by training, $C_t$ is the Context visible on this turn, and $y_t$ is the text or structured action proposal the model generates. On this process alone, the model cannot directly read a local repository that was never provided to it, cannot actually run a compiler, cannot know whether a command succeeded, and cannot make an erroneous change roll back on its own.

But when an engineering system continuously writes repository files, user goals, project conventions, Tool Schemas, execution results, and test feedback into Context, and the runtime then executes the actions the model proposes, the system as a whole can form a trajectory like this:

understand the goal → search the repo → make a plan → modify code → run tests
        → read the failure → locate the cause → modify again → accept and deliver

This trajectory is not a capability any single component holds on its own: the LLM does not execute commands, the shell does not understand requirements, the test framework does not devise a repair strategy, the permission module does not write code. "Carrying out multi-step software engineering around a goal and correcting course based on real results" is a system-level function that holds only once these parts are combined in a particular set of relationships.

Nor is it a thought experiment on paper. Every session replay in the public cases on our website is a record of this trajectory running to completion on a real repository with real samples — for example, turning a single requirements mind map end to end into an enterprise-grade monitoring system, or verifying the data boundary of a closed-source client given nothing but an offline installer and no source code. The reasoning, the tool calls, the errors, and the recovery are all inspectable end to end, including the places where it went wrong and backed out.

2.1 Context Is the Model's Operating Condition, and Also the Agent's State Carrier

The effect of Context on the output distribution can be written as:

Adding project conventions, the relevant code, or a single test failure does not modify the model parameters, but it does change the conditions for the next output and the next action. In an engineering system, Context carries at least three roles:

• Task conditions: the goal, the scope, the constraints, the acceptance criteria;
• State estimation: what has already been done, which step it is on now, which assumptions have been falsified;
• External observation: file contents, command output, test results, user approvals.

This is also why "the same model" performs markedly differently once placed in a different Harness. Model capability supplies the space of possibilities, Context determines the local conditions currently visible, and the runtime determines which outputs get to become real actions.

2.2 Several Kinds of "Knowledge" Have to Be Kept Apart

In AI discussions, "the model knows," "Context supplied the knowledge," and "the system learned" are often treated as one and the same thing. Engineering is better served by describing them in layers:

The value of knowledge in a system depends not only on whether it is there, but on whether it is recalled at the right moment, whether it is relevant to the task, whether it is verifiable, and whether it can enter the feedback loop.

AVL Code's "Memory Palace"

If cross-session knowledge is seen as a "memory maze" that needs routes and gates, then the formal name in the AVL Code user manual is the "memory palace." Once a session ends, HookStop extracts memory candidates and puts them first into the _pending/ draft box; only after the user has reviewed and approved them do the candidates enter the corresponding room. The user can also write entries directly with /memorize <text>, and open the management panel with /memory.

What actually enters the next turn is not the whole palace, but the small number of memories relevant to the question at hand: the system adjusts the ranking by combining recall count with how well each memory has worked in use, cutting down irrelevant injection; when it meets a duplicate or a conflict, it puts the entry into the review queue rather than overwriting the old knowledge outright. This "candidate — review — store — recall — effectiveness feedback" loop improves the quality of factual and experiential feedback; it does not make the prompt endlessly longer.

2.3 Behavioral Capability Cannot Directly Prove a Cognitive State

When a system can explain code, call tools, and fix tests, saying it "exhibits programming capability" is a testable functional description; saying it "understood the code," "knows it made a mistake," or "has developed self-awareness" brings in far stronger cognitive and philosophical claims.

Bender and Koller remind researchers to distinguish linguistic form from meaning, and Shanahan cautions against the loose use of words like "know," "believe," and "think," which readily invite anthropomorphism. These questions can go on being researched and argued over, but engineering should hold one line of evidence:

Being able to produce behavior that resembles understanding is not the same as having proved that the system has an understanding mechanism of the same kind as a human's; being able to carry out intelligent tasks is not the same as having proved that agency or consciousness has emerged.

So "intelligence" in this series refers first to measurable task capability, not to any assertion about the system's inner experience.

Summary, and What Part 2 Covers

Two things are settled at this point: emergence is an observable, measurable property that the system as a whole has beyond its parts; and what emerges once an LLM is combined with Context, Tools, and a runtime is functional behavior, not a cognitive conclusion.

But "functional behavior can emerge" only answers what the phenomenon is; it does not answer what the engineering should be. If the model only puts forward proposals, then what actually decides whether those proposals can become constrained real actions is the layer of structure outside the model. Part 2, "Harness Engineering and Loop Engineering" (published July 17), will discuss how a Harness connects model capability to the real world, and how a Loop applies feedback control to an object that cannot be fully modeled, with explicit stopping conditions.

References

Primary Systems Engineering Sources

• SEBoK: Emergence
• SEBoK: Emergent Property (glossary)

Supplementary Papers

• Wei, J. et al. (2022): Emergent Abilities of Large Language Models, arXiv:2206.07682.
• Schaeffer, R., Miranda, B. & Koyejo, S. (2023): Are Emergent Abilities of Large Language Models a Mirage?, arXiv:2304.15004.
• Bender, E. M. & Koller, A. (2020): Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data, ACL 2020, DOI: 10.18653/v1/2020.acl-main.463.
• Shanahan, M. (2022, revised 2023): Talking About Large Language Models, arXiv:2212.03551.

AVL Code — the AVL security engine, with intelligence at your side. From the Antiy Landi team.