Suppose a strong language model solves a problem today, then encounters a thousand more instances governed by the same rules tomorrow. Should it reason through every instance again? Or should the first encounter produce something that can do the work from then on? This is the version of delegation that interests me: turning an expensive act of reasoning into a reusable computational procedure.
A delegate, in this essay, is an executable program or workflow that handles a defined family of inputs. Delegation can mean many things, including handing work to another agent. Here I focus on a narrower research question: when can a strong language model’s online reasoning be compiled into a procedure that is accurate, reusable, and cheaper to run?
The central question is what makes a task delegable, and how much of the work of discovering its algorithm can itself be amortized.
I think this leads to a useful research agenda: a delegability frontier. The frontier would describe which tasks a particular system can reliably delegate under specified resource and accuracy constraints. The framework below is a set of hypotheses and proposed experiments, rather than a report of established results.
ITask difficulty is not delegation difficulty
Consider finding a shortest path in a large graph with nonnegative edge weights. An instance may require substantial computation, but the rule for solving it is explicit and reusable. Once an appropriate implementation is available, new graphs do not require a language model to rediscover the algorithm. The computation remains; repeated algorithm discovery disappears.
Now consider a short instruction asking us to classify a few messages under a newly invented policy. The inputs are small, but the policy may contain ambiguous concepts or exceptions that a few examples do not resolve. Constructing a reliable delegate may be harder than the apparent size of the task suggests. Some of that difficulty comes from discovering a procedure; some comes from an underspecified task that needs clarification.
These examples separate three costs: understanding the task, finding a solution procedure, and executing that procedure. A task can be expensive to execute while being easy to delegate. It can also be cheap to answer once while being difficult to turn into a dependable reusable program.
This distinction also limits the promise. Compilation does not make an intrinsically expensive computation cheap. It can remove repeated language-model reasoning around that computation, provided the resulting program meets the required accuracy and resource budget.
IIA solution can exist without being easy to discover
There are two different capability boundaries here. Existence asks whether a compact, accurate, efficient delegate exists in the allowed program space. Learnability asks whether a smaller model can find or generate such a delegate from the available task description and examples, within a feasible synthesis budget.
A short program may encode an insight that is difficult to discover. Conversely, a compiler may generate a long but adequate program by composing familiar components. Program length alone therefore cannot tell us how hard synthesis will be.
Learnability also depends on the evidence supplied. Several programs may agree on all the examples and disagree on the next input. If the specification does not distinguish them, more search alone cannot establish which one was intended. The quality of the task description belongs in the experimental design.
This gives delegation two potential failure modes. The permitted program space may contain no satisfactory delegate, or the synthesis system may fail to discover one that does exist. Evaluations should try to distinguish these cases. Failure to generate a program is not evidence that no suitable program exists.
IIIFour dimensions of delegability
I would characterize a task along four dimensions. Each is relative to the available tools, representations, and budget.
| Dimension | Question |
|---|---|
| Compressibility | Can the required behavior be expressed as a compact, reusable program? |
| Synthesizability | Can that program be generated from a specification and a small set of examples? |
| Verifiability | Can we establish its correctness, or estimate its error, at an acceptable cost? |
| Distribution stability | Does it remain valid across the intended changes in inputs, and can changes that require revision be identified? |
The first two dimensions capture the distinction between an available solution and a discoverable one. Compressibility is only part of existence: a concise program must also satisfy execution and accuracy constraints. Verifiability determines how confidently we can accept a candidate. Stability determines how long the investment in it remains useful.
Passing a few tests is evidence about those tests, not a general correctness guarantee. Some tasks admit exact checkers or formal proofs; others require statistical evaluation on independent inputs. The strength and scope of the acceptance evidence should travel with the delegate.
Stability also needs a precise interpretation. New inputs under unchanged rules may be covered by the same program. Changed rules may require a new program, unless the existing delegate already accepts those rules as parameters. We should not count unannounced changes in meaning as ordinary input generalization.
IVNovelty may locate the frontier more precisely
Instance novelty changes the input while preserving the solving rule: another graph, another list, another record under the same schema. Task novelty changes the rule, abstraction, or solution method: a different objective, a new constraint, or a previously unavailable operation.
My hypothesis is that task novelty will often explain delegation failures better than raw instance difficulty. A fixed delegate can handle many unfamiliar instances if its assumptions hold. A small change in the task’s meaning can invalidate it immediately.
But novelty is relative to the system’s abstraction library. A task that once required inventing an algorithm may become routine after the system acquires a suitable solver, parser, or reusable operation. With a general solver, a new constraint might be merely another input; without one, it might require a new implementation.
The frontier should therefore move as the library improves. To study that movement, we need to record which primitives the system has and which compositions it has encountered. Otherwise, an apparent advance in reasoning may simply reflect access to a previously missing component. Both are useful, but they answer different research questions.
VLearning to produce delegates
The direction I find most promising is learned delegation. A strong language model generates candidate task–program pairs. Those candidates are checked and filtered, then used to train a smaller neural compiler: a model that maps task specifications and examples to executable delegates.
- Strong LM: propose algorithms and programs for a collection of tasks.
- Validation: check candidates against specifications and independent cases; retain the evidence and limitations.
- Small neural compiler: learn to synthesize programs for new task descriptions.
- Delegate: execute repeated instances after the generated program passes the acceptance procedure.
This creates two levels of amortization. At the instance level, a program replaces repeated reasoning on the same task. At the task level, a learned compiler reuses experience from discovering programs across tasks. Its success would mean that part of algorithm discovery can be learned in a form that is cheaper to invoke.
The challenging test is whether the compiler can generate programs for new task structures. Success on paraphrased specifications or familiar templates would establish a narrower capability. A convincing evaluation needs held-out compositions, rule systems, and task families, with clear controls on what was available during training.
A practical system could use the compiler when it produces an acceptable candidate and ask the strong model for help when synthesis or validation fails. That fallback is part of the system’s cost and capability. It should be measured explicitly rather than hidden inside the delegate’s reported success rate.
VIWhen does delegation pay?
Even a correct delegate can be a poor investment for a task that appears only once. A simple cost model makes the tradeoff visible. Let G be the cost of generating and validating a delegate, E its execution cost per instance, L the cost of direct strong-model reasoning per instance, and N the number of instances before the delegate needs revision.
Direct reasoning: N × L
Delegation: G + N × E
Delegation saves cost when:
N × (L − E) > GThese quantities must use comparable units and refer to systems that meet the same accuracy requirement. If execution is cheaper than direct reasoning, enough reuse can repay the setup cost. Frequent rule changes reduce the useful reuse window. Verification, maintenance, and fallback calls can raise the actual cost beyond this simplified expression.
For a learned compiler, the training and data-generation costs must also be accounted for across the tasks it serves. Lower synthesis cost at deployment does not, by itself, establish lower total cost. Monetary cost, compute, and latency should be reported separately when they lead to different conclusions.
VIIMeasuring the delegability frontier
A concrete starting point would be a controlled suite of task families with known reference programs. We could vary program structure and execution cost, the amount of information in the specification, and novelty relative to the training tasks and abstraction library. These factors interact, so the aim should be to disentangle them as far as the task construction permits.
Program length or nesting depth can serve as structural controls. Synthesis difficulty should also be measured through the search budget and success rate of specified synthesizers; it should not be assumed to follow directly from program length. Reference programs establish existence within the tested setting, allowing synthesis failures to be studied separately.
The central comparison would have three arms:
- Strong LM direct: the strong model answers each instance.
- LM-generated delegate: the strong model synthesizes a program that is reused across instances.
- Learned delegate: the smaller neural compiler synthesizes the reusable program.
Each system should receive the same task information, with tool access and resource budgets recorded. Evaluation should separate new instances of known tasks, new combinations of known components, and tasks requiring held-out rules or abstractions. Inputs used to select or repair programs must remain separate from the final evaluation set.
We should report accuracy alongside generation and validation cost, execution cost, failure and fallback rates, and generalization across tasks. Varying the reuse count would reveal when each approach becomes economical. Comparing several abstraction libraries would show how much of the frontier moves through better components.
The outcome should be a family of accuracy–cost frontiers under explicit conditions. Evidence against the hypothesis matters too: if performance collapses on new compositions, if validation consumes the savings, or if strong-model fallback remains frequent, then algorithm discovery has been amortized only within a limited range.
VIIIWhat remains for the strong model?
The research hypothesis is that a strong language model’s distinctive value may increasingly lie in discovering new algorithms and abstractions. Once a procedure is explicit and dependable, much of its repeated execution can move to cheaper machinery. A learned compiler could move part of the discovery work there as well.
That remains conditional. Some tasks resist a compact executable specification; some change too quickly to repay synthesis; some require a new abstraction that the compiler has not learned. Strong models may still be useful in those cases, though their involvement does not automatically resolve ambiguity or establish correctness.
When to delegate? When a reusable procedure can be found, its reliability can be established for the intended setting, and its expected reuse justifies the full cost. The deeper question is how far we can extend that region by teaching systems to discover procedures themselves. Mapping that boundary would tell us where strong-model reasoning is still needed, and where it can become a reusable computational asset.
About this essayThis essay develops a research perspective from the author’s notes on the delegability frontier. The examples and experimental design are illustrative; no benchmark results are claimed. AI assisted with drafting and preparing the English and Chinese versions.