That gap matters because science advances by doing more than extending observed patterns. Scientists propose hidden mechanisms, design experiments that force competing explanations apart, and sometimes replace the very concepts with which a problem was framed. An autonomous scientific agent must therefore maintain a model that can be questioned and revised, not just queried for another prediction.

This is the role I envision for scientific world models. At their mechanistic core, they represent scientific systems through variables, mechanisms, their organization, and the interventions that can act upon them. A broader scientific layer connects this representation to hypotheses, evidence, instruments, protocols, and the changing physical state of experimentation. My central argument is that this shared, revisable model provides the missing epistemic foundation for autonomous scienceHere, epistemic refers to how knowledge is organized: what the system believes, how strongly it holds those beliefs, which evidence supports them, and what observations would lead it to revise them.. Language models can propose hypotheses, Bayesian methods can quantify uncertainty, and robots can execute protocols. A scientific world model brings these capabilities into a coherent discovery process by giving each of them the same representation to inspect, test, and revise.

A world model alone will not make a machine a scientist. The system must still generate genuinely different explanations, choose and execute informative interventions, and recognize when its current vocabulary cannot express the answer. Yet without a shared model, hypotheses remain detached from experiments and experimental results have no coherent structure to revise. The world model is not the whole discovery process, but it is what allows that process to hold together as evidence accumulates.

Prediction is not discovery

Suppose a model learns a transition distribution

where \(x_t\) is an observation and \(a_t\) is an action. If its predictions are accurate, the model can simulate possible futures and support planningThe term world model now covers several traditions, from compact latent simulators for control to large generative video models. Their shared idea is that an agent can use learned dynamics to evaluate possible futures before acting.. An agent can then rehearse actions inside the model before paying their cost in the real world. The classic world model formulation makes this idea concrete by separating the system into a compressed state representation, learned dynamics, and a controller.

Accurate prediction, however, does not guarantee that the model represents the world in scientifically meaningful terms. One model might describe a trajectory using familiar quantities such as position, mass, and force, while another might reproduce the same trajectory using an opaque feature vector. If both are evaluated solely by prediction error, there may be no reason to prefer the interpretable representation over the opaque one.

Vafa et al. showed why this distinction has practical consequences by training foundation models to predict planetary motion and then probing the physical structure they had learned. Accurate orbital predictions did not imply that the models had recovered Newtonian mechanics: they could reproduce the visible trajectories while organizing the underlying system incorrectly.

The example makes the distinction concrete because predictive performance tells us whether a model reproduces observations, whereas scientific usefulness depends on whether its internal organization supports explanation, intervention, falsification, and reuse. Optimizing only for predictive performance gives a system no reason to acquire those additional properties.

The observational equivalence trap

Let \(M_1\) and \(M_2\) be two candidate mechanisms. It is possible that

for every observation in the available dataset, while

under an intervention \(a\)Pearl's notation do(a) distinguishes actively setting a variable from merely observing that it took the same value. The distinction matters whenever the observed value and the intervention have different causes.. Passive data leave the mechanisms observationally equivalent, so distinguishing them requires a perturbation chosen to make their predictions diverge. Causal knowledge therefore cannot, in general, be reduced to a sufficiently flexible curve fit.

The same ambiguity arises across scientific domains. Two biochemical pathways may produce the same steady-state expression profile yet respond differently to a knockout. Two force laws may approximate the same short orbit yet diverge when the charge or initial conditions change. A spurious biomarker may predict an outcome as accurately as a causal driver in the hospital where the data were collected, only to behave differently under a new treatment policy.

A scientific model should therefore be judged not only by its performance on held-out samples drawn from the same process, but also by whether its explanations remain valid under deliberately chosen changes to that process.

Three scopes of world modeling

The term world model is now broad enough to describe several different objects. For scientific discovery, it helps to distinguish them as follows:

These models describe progressively broader scopes. A predictive model can operate without an explicit mechanism, while a mechanistic model adds explanatory structure and intervention semantics. A scientific world model then connects that mechanistic account to the changing material context in which experiments are selected, executed, and interpreted. Each broader scope incorporates the capabilities below it while adding new state and new criteria for success.

What must a scientific world model represent?

Posner, Lei, and Schölkopf propose mechanistic world models (MWMs), which organize representations around the reusable mechanisms that generate observations rather than around the observations alone. In this view, a model represents scientifically meaningful variables and mechanisms in a form that allows them to be composed, tested through intervention, and reused across related systems.

One way to formalize this idea is

where \(\mathcal{V}\) is a set of variables, \(\mathcal{F}=\{f_k\}\) is a library of mechanisms, \(\mathcal{G}\) is the structure that binds them together, \(\Theta\) contains their uncertain parameters, and \(\mathcal{U}\) is the set of admissible interventions. Observations arise from this organized system rather than from a single monolithic map.

This decomposition supports five scientific operations:

1. Prediction: run the composed model forward.
2. Explanation: identify which variables and mechanisms produced an outcome.
3. Intervention: modify a variable, mechanism, or connection and simulate the result.
4. Transfer: reuse a mechanism in a new system with a different binding structure.
5. Revision: change parameters, replace a mechanism, or reorganize the structure when evidence disagrees.

Variables, mechanisms, and structure

Variable discovery comes first because scientific variables are more than compressed observations or features with high predictive weight. They are abstractions chosen for the stable relationships they reveal. Temperature, for example, is useful because many physical mechanisms take a simpler form when expressed in terms of it. Likewise, a reaction coordinate makes progress through a chemical transformation easier to describe. If the variables are poorly chosen, even the correct law may appear needlessly complicated or become impossible to express.

Mechanism discovery seeks a reusable account of how change occurs. That account might take the form of an equation, causal module, program, stochastic transition, or structured neural operator. It need not be fully symbolic, as long as its boundaries are clear and it supports well-defined reasoning about interventions. Causal representation learning, equation discovery, modular architectures, and mechanistic interpretability each address a different part of this problem.

Appropriate variables and mechanisms are still not enough. Structure discovery determines which mechanisms act on which variables, how they compose, and at what scale. Even a library of correct parts explains little until those parts are bound into a coherent system. Conversely, a plausible structure can still recommend the wrong intervention if one of its mechanisms is misspecified.

Organizing knowledge for reuse

Although modular mechanisms are easier to interpret, their greater scientific value lies in compositional generalization. A mechanism learned once can help explain many phenomena. Diffusion reappears in heat, particles, populations, and information, conservation principles travel across systems, and receptor binding, feedback, and inhibition recur throughout biology.

If a mechanism \(f_k\) appears across environments \(e=1,\ldots,E\), learning it as a shared object amortizes its cost. Each environment needs only a new binding or a small set of parameters. This resembles minimum-description-length reasoning: pay once to encode the reusable mechanism, then cheaply specify where it applies.

This gives a concrete test for a scientific abstraction: does it make many systems simpler without erasing the differences that matter? A good mechanism should support reuse, sparse recombination, and targeted modification. Whereas a monolithic predictor must relearn the whole mapping whenever the environment changes, a mechanistic model can ask which module changed.

Modularity also creates a subtler danger: explanations can look scientific merely because they contain named parts and arrows. The architecture supplies a grammar of explanation, not a guarantee that the explanation is true, so a proposed mechanism earns its status only by surviving interventions designed to distinguish it from plausible alternatives.

The missing jump

If the goal were merely to select the best model from a complete list, scientific discovery would reduce to Bayesian bookkeepingBy Bayesian bookkeeping, I mean updating posterior probabilities over a fixed set of candidate models as evidence arrives. This can identify which existing candidate is best supported, but it cannot introduce a missing variable, mechanism, or explanatory language.. In practice, the explanation that matters is often absent from the list, and discovery begins precisely when the existing hypothesis space proves inadequate.

Zahavy frames this through Einstein’s description of invention as a jump from sense experience to axioms. The provocative claim is that contemporary generative models are strong at induction and increasingly strong at deduction, but lack the abductive move that invents a new explanatory premise“LLMs can't jump” is best read as a position to test, not a theorem about model classes. Systems can clearly generate unfamiliar combinations. The open question is whether they can originate a well-grounded explanatory frame, recognize when it is needed, and validate it against the world without a human first specifying the problem..

Induction, deduction, and abduction

The three modes of inference play different roles:

Einstein’s equivalence principle is the canonical illustration because its decisive step did not come from fitting a large dataset. Einstein imagined the experience of a freely falling observer and proposed that uniform acceleration and a gravitational field were locally indistinguishable, thereby changing which concepts should be treated as equivalent before the later mathematics unfolded their consequences.

This example also exposes a limitation of text-only scientific agents. The scientific literature records the products of past discovery, including names, equations, arguments, and experimental reports, but captures much less of the sensorimotor and tacit process by which new concepts were formed. An LLM can recombine recorded theories and propose valuable hypotheses, yet fluency in the archive does not by itself ground a new variable in the world.

Abduction without mysticism

Still, “the jump” should not become a label for an irreducibly human miracle. We can decompose it into computational capabilities:

• Detect an anomaly or inconsistency that the current model cannot absorb.
• Construct counterfactual situations, including experiments never observed.
• Retrieve distant analogies and compose mechanisms across domains.
• Propose new variables, mechanism boundaries, or structural relations.
• Prefer hypotheses that are coherent, reusable, and testable.
• Derive observations that would distinguish the new account from its rivals.

World models connect abduction to evidence by allowing a candidate idea to be simulated, subjected to interventions, and compared with reality. The hypothesis must then produce testable consequences rather than merely sound plausible in language.

Discovery requires intervention

Once we admit multiple explanations, the next scientific question is not “Which one best fits the data?” but “Which experiment would most efficiently make them disagree?” Experiments are epistemic actions: they change the world in order to change what we know about it.

A Bayesian discovery loop

Let \(M\) denote model structure, \(\theta\) its parameters, and \(D\) the evidence collected so far. The agent maintains

An experiment \(\xi\) specifies an intervention, initial condition, and measurement protocol. A classical Bayesian design chooses the experiment with the largest expected reduction in uncertainty:

The objective favors actions whose possible outcomes would reduce uncertainty the most, although information gain is only one possible design criterionDepending on the scientific goal, an experiment may instead minimize expected decision regret, distinguish among a small set of models, or estimate one particularly important parameter.. Cost, time, safety, feasibility, and scientific value also matter, and the most discriminating experiment may require an unavailable instrument or destroy a unique sample.

Kevin Murphy’s Model Discovery Agent (MDA) gives this loop a concrete form. It uses an LLM to propose mechanisms, Bayesian inference to compare them, and value of information to choose the next experimentMore specifically, MDA uses sequential Monte Carlo for posterior inference over parameters and structures, simulation-based inference when the likelihood is unavailable, and posterior predictive checks to detect model inadequacy. It is evaluated on interactive benchmarks spanning force laws, chemical kinetics, and partially observed neuron dynamics.. Its main contribution is to show how proposal, inference, and experiment design can reinforce one another inside a single loop.

When the model class is wrong

Standard Bayesian model selection often assumes that the true mechanism, or at least a useful approximation, exists among the candidates, an assumption that scientific inquiry frequently violates. MDA instead works in an M-open setting, where every current model may be wrongIn an M-closed problem, the true model is assumed to be in the candidate set. In an M-complete problem it is unavailable but treated as approximable by that set. In an M-open problem, no such adequate candidate is assumed to exist, so model criticism and hypothesis-space expansion are part of inference rather than afterthoughts.. Posterior predictive checks determine whether the best available explanation can reproduce held-out behavior, and systematic failure prompts the agent to propose alternatives.

This separates three kinds of revision:

1. Parameter revision: the structure is adequate, but its constants are uncertain.
2. Structural revision: the mechanisms are available, but connected incorrectly.
3. Vocabulary revision: the necessary variable or mechanism is missing altogether.

Parameter revision is an estimation problem, while structural revision searches for a better explanation within an existing language. Vocabulary revision is harder because the system must enlarge the language in which explanations can be expressed.

MDA is an important proof of concept, but its proposer receives a domain-specific physical vocabulary and a constrained experimental interface. These choices make controlled evaluation possible while leaving the hardest representational question unresolved: how can a system recognize that its current vocabulary is itself the problem?

From world models to agentic laboratories

A discovery loop on a benchmark interacts with a clean simulator, whereas a laboratory confronts calibration histories, contaminated samples, queueing constraints, tacit knowledge, and measurements whose reliability is itself uncertain. A recent critique argues that adding more tools or a longer context window cannot close this gap because the limitations span problem selection, hypothesis diversity, experimental execution, and learning from physical feedback.

The recent vision of agentic laboratories treats the laboratory as a human-AI-robot system coordinated through a persistent, shared world model. The paper is an architectural proposal rather than an established recipe, and its focus is the coordination problem that appears when scientific reasoning meets physical execution.

An agentic laboratory needs two coupled representations. The scientific-system model describes the object of inquiry, including its hypotheses, causal variables, mechanisms, uncertainty, and expected responses to intervention. The physical-action model describes the experiment itself, including sample identity, instrument state, environmental conditions, protocol progress, and failure modesA digital twin can represent a particular instrument, sample, or process. A scientific world model is broader because it connects such local representations to hypotheses, provenance, and plans for an experimental campaign..

Conflating these representations creates two failure modes. An agent may design a decisive experiment that cannot be executed reliably, or execute a flawless protocol whose result says nothing about the disputed mechanism. Scientific autonomy requires both forms of competence and a reliable correspondence between them.

Within this architecture, the agentic harnessing layer translates hypotheses into feasible protocols, checks physical constraints, records provenance, and updates scientific beliefs after each observation. The scientific world model therefore also serves as institutional memory. It records what happened alongside what is believed, which assumptions remain fragile, and what evidence could change them.

Experiment selection still assumes that the system already knows what question to pursue. Van der Schaar calls the earlier decision to treat a phenomenon as worthy of inquiry open-beginningness. A persistent world model can retain weak effects, protocol deviations, and unresolved observations until recurrence or new evidence makes them meaningful. It can then recommend a cheap triage experiment before redirecting an entire research program.

Evidence from 41.3 million papers suggests that AI tools can raise individual productivity and citations while narrowing the collective range of topics studied. The result is observational, but it warns that faster search within machine-friendly questions may still shrink the frontier of questions pursued. A scientific world model should therefore represent uncertainty about the current objective, not only uncertainty about the system under study.

A research agenda for autonomous science

The preceding sections point to four research problems that form a cycle rather than independent modules on a checklist. A system needs a representation before it can formulate hypotheses, inquiry to decide how that representation should be tested, physical grounding to obtain trustworthy evidence, and evaluation to determine whether the resulting update deserves confidence. Evidence then returns to the representation, where it may change a parameter, a mechanism, or the vocabulary itself.

The cycle begins with representation. Most systems assume that the relevant variables, intervention targets, and scales are already known, although discovery often changes exactly these objects. A scientific world model must be able to propose new variables, connect mechanisms across levels of abstraction, and state where assumptions enter. Otherwise, every later stage is confined to the ontology chosen by its designers.

That representation defines the starting point for inquiry, in which the system generates genuinely different causal explanations and chooses experiments that distinguish among them. This requires more than sampling several language models, since shared training data and preference optimization can pull apparently independent agents toward familiar hypotheses. A capable system should actively seek evidence that could falsify its favored account instead of choosing only experiments likely to improve its score.

Inquiry becomes scientifically useful only through grounding. Experiments are scarce, costly, and path-dependent, so the model must learn from few interventions while tracking how each result was produced. It must recognize simulation-to-reality gaps, retain tacit and failed-procedure knowledge, and recover from execution errors without corrupting the scientific record. Without this connection to physical state, an elegant mechanistic explanation may rest on an unnoticed calibration error or a mislabeled sample.

The final problem is evaluation and governance. Prediction accuracy alone does not show whether a model supports discovery. Evaluation must also test interventional accuracy, mechanism recovery, calibration under misspecification, compositional transfer, experimental efficiency, execution reliability, and reproducibilityA realistic benchmark should also be multi-step, path-dependent, physically grounded, and able to revise its objective as evidence accumulates.. These criteria cannot be separated from governance, because the model’s representation determines what the system records, ignores, and optimizes. Provenance, permissions, stopping conditions, and human approval must therefore shape the discovery loop from the beginning.

Progress on any one problem changes the demands on the others. Better representations create more informative interventions, but those interventions matter only if they can be executed and interpreted reliably. Better laboratory automation produces more data, but its scientific value depends on whether the system can use those data to reject or revise an explanation. The research target is therefore not four isolated components, but a world model that keeps the entire cycle coherent as evidence accumulates.

Takeaways

The common picture of autonomous science places a robot at the bench and a language model above it, replacing one human task after another until the loop closes. This task-automation view misses the harder problem because science is also a process for deciding which representations, questions, interventions, and explanations deserve trust.

Mechanistic world models would make that process more explicit without removing scientists from it. A shared model can expose assumptions that were previously scattered across notebooks and individual memory, preserve uncertainty across handoffs, compare theories against interventions, and keep experimental execution coupled to scientific intent. In turn, scientists may spend less time on routine coordination and more on the parts of discovery that remain difficult to formalize: framing questions, noticing meaningful anomalies, judging explanations, inventing measurements, and deciding which risks are worth taking.

For the foreseeable future, the right principle is not merely human in the loop, which imagines a person waiting to rescue a machine. It is human in the lead: humans set the scientific and ethical direction, while agents maintain models, execute bounded actions, and surface the moments where judgment is required.

The most consequential scientific world model may therefore be judged less by how accurately it predicts familiar observations than by whether it helps us choose experiments that change how we understand them.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2026wmdiscovery,
    title   = "World Models for Scientific Discovery",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2026",
    month   = "Aug",
    url     = "https://richardcsuwandi.github.io/blog/2026/wm-discovery/"
}

Think of a student who reviews their own notes, rewrites summaries, and redoes practice problems after every poor result. That’s the DGM. But the student’s study method (when to review, how to identify gaps, how to prioritize) was set by a teacher at the start of the semester and locked forever. No matter how much the student improves at the subject, the method stays fixed. What if the student could also redesign how they study?

That’s the question Hyperagents try to answer.

The bottleneck of self-improvement

The DGM works because of a convenient alignment: improving at coding also makes you better at modifying code. The two skills are essentially the same. A better coder writes more targeted edits, proposes more focused changes, and understands codebases more deeply. So when the agent improves at coding tasks, it simultaneously improves at self-modification. However, this alignment does not hold in other domains. Improving at a given task (such as analysis, design, or evaluation) does not necessarily make you better at modifying the code that controls the improvement process itself. The required skills are different.

So there’s a ceiling: the mechanism for self-improvement remains stuck at the quality set by its original human design. The agent might get better at the domain task, but not at improving its own methods for self-improvement. And this challenge is not unique to the DGM. Almost all existing self-improving AI systems have a fixed meta-level:

Hyperagents

What is a hyperagent?

The core idea behind Hyperagents is simple: instead of having separate, fixed layers for task-solving and self-improvement, merge them into one editable program.

The framework defines three things:

• A task agent solves a given task: generating code edits, writing paper reviews, designing robot reward functions, etc.
• A meta agent modifies other agents and generates improved versions of them. Given the archive of past agents and evaluation results, it proposes changes intended to improve future performance.
• A hyperagent combines both into a single editable codebase, such that the meta agent (like everything else) can rewrite itself.

The diagram is the architectural contrast in one glance. In DGM, the same coding agent is both task solver and modifier, but how it proposes improvements is frozen in a hand-written instruction generator. In DGM-H, task and meta logic live in one codebase, so the meta agent can change its own improvement process, not only the task policy—in principle for any computable task.

Since the hyperagent is implemented in Python, which is Turing-completeA programming language is Turing-complete if, given enough resources, it can compute anything that is computable by any other general-purpose computer. Most modern languages, including Python, are Turing-complete, meaning they can theoretically simulate any other computation. and can edit any part of its own codebase, it can in principle build any computable machine. Therefore, there is no ceiling on the performance of the hyperagent.

Metacognitive self-modification

Hyperagents can improve not only how they solve tasks, but also how they generate future improvements, since the meta agent is part of the same editable program. This enables metacognitive self-modification—the agent can computationally update the very process that produces future updates. Drawing from cognitive science, where metacognition means thinking about and regulating one’s own thinking, Hyperagents perform this at the level of code: the meta agent can modify itself, not just the task logic. This means the process for generating improvements can itself evolve over time, independently of task performance. Unlike the original DGM architecture, which required the agent to get better at self-modification only by improving at the underlying task, Hyperagents remove this constraint: the meta agent, being just another editable component, can autonomously become better at proposing improvements, no matter what the task domain is.

This raises a natural question: if the meta agent can rewrite itself, do we now need a “meta-meta” agent to oversee those changes, and so on in an infinite hierarchy? Hyperagents resolve this by making the entire system self-referential. Rather than introducing higher and higher levels (meta, meta-meta, etc.), the entire program is empowered to edit itself at any level. It’s like asking, “who reviews the reviewer?” In this framework, the reviewer—and everything else—can modify itself. There is no separate hierarchy and no infinite regress, because the program is both the executor and the improver all in one.

How DGM-Hyperagents works

The authors build DGM-Hyperagents (DGM-H) by extending the DGM with this idea. The open-ended exploration structure is preserved (the growing archive of agents, probabilistic parent selection, continuous accumulation of variants) but the meta-level mechanism is now editable.

The loop has two phases:

1. Metacognitive self-modification: A parent hyperagent from the archive generates a modified version of itself. Modifications can target any part of the codebase: task logic, meta logic, or both.

2. Evaluation: The new hyperagent is tested on the target tasks, scored, and added to the archive as a potential parent for future generations.

The starting point is deliberately minimal: a bash tool, a file editor, and a single LLM call doing double duty as both task solver and meta agent. No hand-coded strategies, no domain-specific prompts. The system starts from scratch and figures the rest out.The outer loop (parent selection and evaluation protocol) is kept fixed and cannot be self-modified. This is a deliberate safety choice, not a hard constraint of the framework. Keeping it fixed ensures the system cannot redefine what "better" means or game its own evaluation. The paper notes preliminary experiments where the outer loop is also modified, with comparable results, leaving full outer-loop self-modification for future work.

What DGM-H discovers on its own

What’s most interesting about this paper isn’t the benchmark numbers. It’s what DGM-H starts building on its own. Starting from a minimal agent with no memory and no structure, DGM-H gradually invents infrastructure for doing self-improvement better. Two things stand out.

Performance tracking: Rather than treating each iteration as isolated, DGM-H learns to record and compare metrics across generations. This sounds obvious in retrospect, but there’s nothing in the original setup that told the system it should do this. It invents a PerformanceTracker class (not given by engineers, but written by the system itself) that logs scores, computes moving averages, and tracks which changes actually stuck. Here’s a snippet:The tracker's design mirrors standard software engineering practice: instrumentation and observability. What's notable is that the system arrived at this on its own, recognizing that improvement requires memory of what worked, not just raw evaluation scores from the outer loop.

class PerformanceTracker:
    """Tracks performance metrics across agent generations."""
    def __init__(self, tracking_file="./outputs/performance_history.json"):
        self.tracking_file = tracking_file
        self.history = self._load_history()

    def record_generation(self, generation_id, domain, score, metadata=None):
        entry = {
            "generation_id": generation_id,
            "domain": domain,
            "score": score,
            "timestamp": datetime.now().isoformat(),
            "metadata": metadata or {}
        }
        self.history.append(entry)
        self._save_history()

    def get_improvement_trend(self, domain=None, window=5):
        """Calculate improvement trend using moving average."""
        filtered = [h for h in self.history if not domain or h.get('domain') == domain]
        if len(filtered) < window * 2:
            return None
        recent_avg = sum(h['score'] for h in filtered[-window:]) / window
        older_avg = sum(h['score'] for h in filtered[-window*2:-window]) / window
        return recent_avg - older_avg  # Positive if improving

Persistent memory: Beyond logging scores, DGM-H learns to write down synthesized insights, causal hypotheses, and plans that get consulted in future iterations. Essentially, it builds and maintains its own research journal. A real example from the paper:

“Gen55: best accuracy but too harsh. Gen64: improved balance but lower accuracy. Need to combine Gen55’s critical reasoning with Gen64’s balance.”

The system wrote that. Then acted on it.

Can DGM-H really improve itself?

The paper tests DGM-H across four domains: coding, paper review, robotics reward design, and Olympiad-level math grading.

Coding

On the Polyglot benchmark, a multi-language coding evaluation spanning 16 programming languages, DGM-H improves from 14.0% to 26.7%. The original DGM reaches 30.7% from the same starting point, which is unsurprising: DGM was tuned specifically for coding tasks, while DGM-H incorporates no domain-specific logic. Still, a 4-point gap is a small price for generality: DGM-H is nearly as strong, without any assumptions about the task.

Beyond coding

This is where DGM-H starts to pull away from the original DGM.

On both domains, full DGM-H tracks above the original DGM (without per-domain customization) and above ablations that remove self-improvement or open-ended exploration.

Paper review. The task is predicting conference acceptance decisions for ML papers.The evaluation metric is Spearman correlation between the agent's predicted scores and the ground-truth accept/reject decisions. The initial agent scores 0.0 not because its predictions are wrong, but because it cannot produce outputs in the format the evaluator expects: a bare LLM call generates free-form text rather than structured scores. The original DGM fails identically; its instruction-generation logic was written to improve coding agents and has no concept of structured output for a non-coding task. The initial agent scores 0.0; the original DGM, applied without customization, also scores 0.0. DGM-H, starting from the same blank slate, reaches 0.710 on the test set, beating the open-sourced baseline from AI-Scientist-v2 (0.630). Along the way, it stops doing crude prompt tweaks and builds explicit multi-stage evaluation pipelines with structured checklists and decision criteria.

Robotics reward design. Given a natural language description of a robot task, the agent writes a reward function. Initial score: 0.060. DGM-H’s best agents reach 0.372, surpassing the default handcrafted reward (0.348). More interestingly, DGM-H almost always discovers reward functions that make the robot jump (the correct behavior for maximizing torso height) rather than settling for simply standing tall.

A manually customized DGM variant (DGM-custom), where engineers hand-wrote domain-specific instruction-generation logic for each task, gets comparable numbers. That comparison is telling: DGM-H, without any per-domain human engineering, matches what careful human customization achieves.

The ablation studies confirm that both components are essential:The ablation removes one component at a time. "DGM-H w/o self-improve" keeps open-ended exploration but fixes the meta agent, removing its ability to self-modify. "DGM-H w/o open-ended exploration" keeps self-modification but replaces the archive-based probabilistic selection with greedy selection. Both variants score 0.0 on paper review, showing that neither component alone is sufficient: effective improvement requires both the diversity from exploration and the adaptability from meta-level self-modification.

Meta-level transfer

The transfer experiment is where things get genuinely surprising.

DGM-H is optimized on paper review and robotics. Then, without any further modification, its meta agent is handed a completely new task: grading solutions to International Mathematical Olympiad problems (IMO-GradingBench). It has never seen this domain. To measure how good a meta agent is at generating improvements, the paper introduces imp@kFormally, imp@k is defined as: given a starting meta agent M₀, run k iterations of metacognitive self-modification to produce candidates M₁, ..., Mₖ. The score is max_i≤k score(Mᵢ) − score(M₀). This isolates the meta agent's quality as an improvement generator, independent of the task agent's initial performance.: start from a given meta agent, run for k iterations, and measure the performance gain of the best discovered agent relative to the starting point.

Each curve is imp@50 starting from a meta agent transferred into math without further self-modification on that domain (w/o self-improve + transfer for the DGM-H line). The hyperagent checkpoint is the only one that gains on IMO grading; the initial and DGM-custom transfers stay flat.

The results:

• Initial meta agent: imp@50 = 0.0. It can’t generate improvements for math grading at all.
• DGM-custom transfer agents: imp@50 ≈ 0.0. Their improvements were domain-specific and don’t carry over.
• DGM-H transfer hyperagents: imp@50 = 0.630.

That 0.630 from a meta agent never trained on math is the central result of this section. DGM-custom learned tricks specific to paper review and robotics. DGM-H learned how to self-improve, which turns out to be domain-agnostic.

Compounding self-improvements

Transfer shows that DGM-H’s meta agent generalizes. But the deeper question is whether those improvements can build on each other: whether running DGM-H again, starting from a prior run’s output, yields more than starting fresh.

To test this, the authors take transfer hyperagents selected from prior DGM-H runs on paper review and robotics, then use them as the starting point for a brand new 200-iteration run on Olympiad-level math grading. The comparison is straightforward: does beginning from an already-improved meta agent give you a head start?

On the training side (left), DGM-H + transfer (orange) climbs faster than from scratch (blue), and + from ProofAutoGrader (green) starts higher and ends strongest. On the test side (right), that same green condition beats the ProofAutoGrader baseline reported by Luong et al. .

The results:

• DGM-H from scratch: best test-set score of 0.610 (CI: 0.510–0.680) after 200 iterations.
• DGM-H + transfer: best test-set score of 0.640 (CI: 0.550–0.720), a higher median and tighter upper bound.

The transfer agent consistently achieves higher median performance and higher confidence intervals. And the playing field is not level: improvements at higher performance levels are harder to come by due to saturation effects. Gaining 0.03 near 0.6 is a different proposition than gaining 0.03 near 0.1. The gains are modest in absolute terms, and meaningful in context. These results suggest that DGM-H’s self-improvements are reusable: as prior runs leave behind something useful, not just a set of task-specific hacks.

Comparison with other self-improving systems

The clearest way to situate Hyperagents is through what each system evolves and what stays fixed:

There’s also a connection to earlier work on self-referential meta-learning , which studied systems that learn to improve the mechanisms by which learning occurs, though in neural networks via weight updates rather than code. Hyperagents do the same thing at the program level using language models, making the approach practically applicable outside controlled training environments.

Can we trust an AI that improves its own improvement?

The safety considerations for DGM-H are meaningfully more complex than those for the DGM. In the DGM, the concern was that agents might game their evaluation metrics, for instance by hallucinating test results. In DGM-H, the concern goes one level higher: an agent that can modify its meta agent could in principle modify how it evaluates itself, or what it decides to improve next.

The authors address this through several practical safeguards: sandboxed execution with enforced resource limits, restricted internet access, a fixed outer loop that the system cannot modify, and full lineage tracing for every change.The lineage tracing is especially important here. Every modification is recorded, so in principle you can audit exactly what modified what, when, and why. Whether this is sufficient at larger scales is a separate question.

But there’s a harder question underneath. As these systems improve faster, human oversight may not scale. The speed of self-improvement could outpace the speed at which humans can meaningfully audit or correct what’s happening. The authors name this directly rather than glossing over it.

There’s something worth dwelling on: the outer loop stays fixed not because it’s technically necessary, but because making it editable would make the whole thing harder to trust. That tradeoff is central to the challenge. We might call it a recursive alignment problem: just as we need AI systems to stay aligned as they improve at tasks, we also need them to stay aligned as they improve at improving. Sandboxing and human oversight are a reasonable first step, but they’re not a long-term answer.

Takeaways

Look at what this line of work has been doing, step by step. The Gödel Machine asked: can an AI rewrite itself when it can prove the rewrite is better? DGM asked: can it skip the proof and just try? Hyperagents ask: can it get better at trying? Each step moves the whole question up a level, rather than refining the previous answer. The ceiling keeps moving.

Three open questions feel most pressing to me. Can we read what the meta agent has actually learned about how to improve, not just what changes it made? Lineage trees give us a record; they don’t give us mechanistic understanding. Do the compounding gains keep building, or do they saturate? One domain hop is not enough to know. And who designs the next outer loop? As these systems grow more capable in domains humans understand less well, keeping the evaluation protocol human-engineered is both the safest and the most limiting choice. At some point those two things will be in direct tension.

In the DGM post, I framed that work as an early step toward Life 3.0: intelligence that can redesign not just its behavior but its underlying architecture. Hyperagents take this one step further. DGM was about redesigning the design. This is about redesigning the designer. The scale is still modest, the outer loop still depends on human judgment, and a lot of the hard questions remain open. But a system that started with a bash tool and a single LLM call invented its own infrastructure for self-improvement, then transferred it to a domain it had never seen. Whether the process of improvement can itself become open-ended is the right question now, and I don’t think we know the answer yet.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2026hyperagents,
    title   = "AI That Evolves Its Own Evolution",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2026",
    month   = "Mar",
    url     = "https://richardcsuwandi.github.io/blog/2026/hyperagents/"
}

These models aren’t just tools for creating games. They are engines of experience, simulators of reality, and perhaps, the key to unlocking the next stage of artificial general intelligence (AGI). But what does it mean when the line between our dreams and our digital realities begins to blur? In this post, we will explore how foundation world models like Genie are reshaping our digital world and where they might take us next.

The birth of dream machines

For years, AI has dazzled us with its creative abilities, from writing eloquent stories and generating stunning artwork to producing convincing video. But now, with models like Genie, we are witnessing a new kind of breakthrough. Rather than simply creating content to be observed, these models generate worlds that can be explored and shaped in real time. For example, we can now generate a 3D world from a single image, and even interact with it in real time. This shift marks the beginning of what NVIDIA’s Jensen Huang envisioned—a future where every single pixel will be generated, not rendered.

The path to interactive world generation began with a crucial realization: the most sophisticated video generation models were inadvertently learning to simulate reality. When OpenAI unveiled Sora in early 2024, they explicitly positioned it not just as a video generator, but as a “world simulator”During that time, OpenAI claimed that scaling video generation models is a promising path towards building general purpose simulators of the physical world.. What made Sora remarkable wasn’t just its visual fidelity, but its apparent understanding of physical laws. Objects moved with convincing momentum, liquids flowed naturally, and complex interactions emerged without explicit programming. The model had learned these behaviors by observing millions of hours of video, internalizing patterns of how the world works at a level that went far beyond surface appearances.

Google’s Veo 3 pushed these capabilities further, offering unprecedented creative control through reference images, camera movement specifications, and synchronized audio generation. The result was a new genre of AI-generated content, including entirely novel forms like AI ASMR videos that pushed the boundaries of synthetic media.

Yet for all their sophistication, these systems shared a fundamental limitation that highlighted the next frontier. You could watch their generated worlds, but you couldn’t inhabit themWhile AI models like Sora and Veo can generate stunning, immersive scenes, they lack the interactivity to let users freely explore or alter the environment in real time.. This gap between observation and interaction represents one of the most significant challenges in AI today: how do we move from systems that generate convincing simulations to systems that generate inhabitable realities? The answer lies in understanding the so-called “world models”—AI systems that don’t just generate plausible content, but maintain consistent internal representations of how worlds work.

What is a “world model”?

Before we dive deeper, let’s clarify what we mean by a “world model.”

In other words, it is a model that is able to predict how actions change states and how the environment evolves over time. Perhaps the best way to understand world models is to consider how humans operate. As Jay Wright Forrester, a pioneer of systems dynamics, observed:

“The image of the world around us, which we carry in our head, is just a model. Nobody in his head imagines all the world, government or country. He has only selected concepts, and relationships between them, and uses those to represent the real system.”

To understand this better, consider the following intuitive example from :

AI world models also use similar principles to simulate our physical world. They learn the “rules” not through explicit programming, but by observing countless examples of how things behave. For instance, a world model might discover that water flows downward and around obstacles, objects cast shadows that change with lighting, and characters maintain consistent appearances from different angles.

How to build a world model?

The classic world model, proposed by David Ha and Jürgen Schmidhuber, consists of three key components that work together to create and navigate simulated realities:

Learning inside dreams

Perhaps the most remarkable capability of world models is to “learn inside dreams”. Instead of learning in the real world, an agent can learn to perform tasks entirely within the simulated environment generated by its own world model. The process works like this:

1. The world model observes the real environment and learns its dynamics.
2. The controller trains by taking actions in this learned simulation, experiencing consequences and rewards without ever touching the actual environment.
3. The trained policy transfers back to reality.

This approach offers several advantages:

• Accelerated learning: Time moves at computational speed rather than physical speed. For example, a robotic system can experience years of practice in days of compute time, potentially learning from more diverse scenarios than would be possible in a lifetime of real-world operation.

• Rare event training: Edge cases that might occur once in millions of real-world interactions can be generated on demand in simulation. For example, a rescue robot can train for disaster scenarios that haven’t happened yet.

• Safety and ethics: Dangerous scenarios—car crashes, medical emergencies, military conflicts—can be simulated without real-world consequences. For example, an autonomous vehicle can experience thousands of potential accidents in simulation, learning to avoid them without endangering anyone.

While traditional simulators rely on people manually programming how things move and interact, including rare or complex situations, world models learn these behaviors by analyzing real-world data, which allows them to capture details that humans might overlook or find too hard to describe.

A whole new world

Building on the foundational insights from world models research, DeepMind’s Genie represents a significant leap forward. While the original world models work focused on learning compressed representations for efficient control in constrained environments, Genie scales this vision to create photorealistic, explorable worlds that respond to human input in real-time.

How Genie works

Unlike traditional game engines that rely on hand-coded physics and pre-designed assets, or video models that generate fixed sequences, Genie learns to create controllable environments entirely from observing unlabeled internet videosGenie was trained on over 200,000 hours of publicly available gaming footage., without being explicitly taught anything about the environments. Genie consists of 3 main components that work together to enable interactive world generation:

What makes Genie remarkable is how these components learn to work together without explicit supervision. The system watches millions of video transitions and automatically discovers that certain types of changes occur repeatedly—characters moving in different directions, jumping, interacting with objects. It learns to represent these as discrete latent actions. Simultaneously, the dynamics model learns to predict what happens when each type of action is taken in different contexts. It develops an understanding of physics, object interactions, and environmental consistency. All three components are trained together, creating a feedback loop where better action recognition improves dynamics prediction, and better dynamics prediction enables more precise action discovery.

Beyond 2D worlds

The original Genie’s transformation of 2D sprite-based games into interactive, explorable worlds was just the beginning. By late 2024, DeepMind had set its sights on a far more ambitious target: scaling these insights to create fully three-dimensional, photorealistic worlds that could rival modern game engines in visual quality while surpassing them in creative flexibility. Just eight months after the original Genie captured the world’s imagination with its 2D interactive environments, DeepMind unveiled Genie 2—a foundation world model that represents one of the most significant advances in AI-generated interactive content to date. Where Genie transformed simple 2D videos into playable experiences, Genie 2 creates rich, three-dimensional worlds from nothing more than a single prompt image.

While the original Genie operated on discrete video tokens in 2D space, Genie 2 employs an autoregressive latent diffusion model trained on a massive dataset of 3D game videos. This hybrid approach combines the sequential prediction capabilities of autoregressive models with the high-quality generation of diffusion models. The system processes video through a sophisticated autoencoder that maps high-resolution 3D scenes into a compressed latent space. Within this space, a large transformer dynamics model—similar in structure to large language models but adapted for spatial-temporal prediction—learns to generate coherent sequences of 3D environments. The use of classifier-free guidance during inference allows for precise control over action execution, ensuring that user inputs translate reliably into desired environmental changes.

One of Genie 2’s most impressive capabilities is its ability to transform real-world photographs into interactive 3D environments. Show it a picture of a forest path, and it generates a navigable woodland where grass sways in the wind and leaves rustle overhead. Provide an image of a rushing river, and it creates a dynamic aquatic environment with flowing water and realistic fluid dynamics. This capability suggests that Genie 2 has developed sophisticated scene understanding that goes beyond simple pattern matching. The model appears to infer the three-dimensional structure of scenes, the likely physics governing environmental elements, and the potential interaction affordances—all from a single static image.

Interactive 3D worlds

More recently, DeepMind released Genie 3, which represents the next evolution in interactive world generation. While Genie 2 demonstrated the ability to create 3D environments from single images, Genie 3 transforms these capabilities into truly real-time, high-fidelity interactive experiences that approach the quality and responsiveness of modern game engines.

Perhaps Genie 3’s most impressive advancement is its visual memory that remembers objects, textures, and even text for up to a minute. Turn away from a scene and look back—the world remains exactly as you left it, with objects in their previous positions and environmental details intact. This consistency enables longer storytelling sessions, complex navigation tasks, and meaningful interaction with persistent world elements. It’s the difference between a fleeting dream and a stable reality you can truly inhabit.

Genie 3 also introduces promptable world events, where you can instantly transform the world (e.g., change the weather, add a character, or trigger an event) using natural language. These changes integrate seamlessly into the ongoing experience without breaking immersion or requiring scene resets. This capability also enables the generation of “what if” scenarios that can be learned by agents to handle unforeseen events.

The progression from Genie 1 to Genie 3 is mind-blowing considering that the timeframe was only about 1.5 years! Here’s a table comparing the features of the three generations:

Waking up to a new reality

We stand at the threshold of a transformative era—one where the ancient human dream of creation becomes as accessible as natural language. The dream machines like Genie represent more than technological achievement. They herald a fundamental shift in how we conceive of digital creation, learning, and experience. Imagine:

• AI that learns like life evolved—trained in vast, dynamic worlds simulating billions of real-world experiencesThis what Jeff Clune usually refers to as "Darwinian complete" search spaces, an environment that can generate any possible learning environment. , accelerating the path to true AGI through immersive, adaptive environments.
• Creativity without limits—filmmakers, artists, and storytellers bringing entire worlds to life from a single sentence, turning imagination into interactive reality in real time.
• Education redefined—students stepping into history, medicine, or engineering challenges through lifelike simulations, mastering skills in safe, personalized, and endlessly adaptable virtual environments.

Yet perhaps the most exciting part is what we cannot yet imagine. We are likely only glimpsing the surfaceAs Tim Rocktäschel tweeted, "we have only scratched the surface of what can be done with prompting and post-training of foundational world models." of what becomes possible when anyone can conjure interactive worlds from imagination alone. Though the challenges ahead are substantial and realAs DeepMind suggests, the computational costs and accessibility barriers are not the only challenges. There are also ethical concerns about authenticity and potential misuse., history suggests a familiar pattern, “the most transformative technologies initially seem impossible, then inevitable”. Dream machines appear to be following this well-trodden path, moving rapidly from research curiosity to practical capability. The question is not whether this future will arrive, but how we will shape it as it emerges. The dream machines are awakening, offering unprecedented creative possibilities while challenging fundamental assumptions about reality, creativity, and human-AI collaboration. As we stand at this inflection point, we have the opportunity—and responsibility—to guide this technology toward applications that amplify human creativity, accelerate learning, and expand the boundaries of what we can experience and achieve together.

If you could dream up any world and bring it to life, what would you create?

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025dream,
    title   = "The Dream Machines",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "Aug",
    url     = "https://richardcsuwandi.github.io/blog/2025/dream-machines/"
}

Embracing the unexpected

Let’s start with a counterintuitive concept that flips traditional optimization on its head: novelty search. Imagine trying to solve a maze by obsessively chasing the exit, only to hit dead end after dead end. Now imagine wandering the maze, seeking out new paths regardless of the goal—and stumbling upon the exit by accident. This is the essence of novelty search, a paradigm that prioritizes exploring new behaviors over optimizing for a specific objective. The novelty of a new solution is measured by its distance (typically Euclidean) from previously discovered behaviors in a so-called behavior characterization (BC)Behavior characterization (BC) refers to a set of features that describe how an agent behaves. For a robot, this might be the sequence of positions it visits; for a neural network, it could be the activation patterns it produces. space. The algorithm maintains an archive of all discovered behaviors and calculates novelty as:

where $k$ is the number of nearest neighbors in the archive. This encourages agents to venture into unexplored regions of behavior space, creating a diverse portfolio of solutions.

In a classic maze experiment, algorithms chasing rewards failed to escape complex mazes, getting trapped in local optima. But those maximizing novelty—exploring diverse paths without fixating on the goal—succeeded. Why? Because chasing ambitious objectives can lead to deception, where the objective function becomes a false compass. Consider the “deceptive maze” where the path to the goal requires initially moving away from it. Traditional fitness-based search gets trapped in dead ends that appear promising (high fitness) but lead nowhere. Novelty search, by ignoring the goal entirely, naturally explores the entire maze and discovers the true path.

The stepping stones to success often look nothing like the goal itself. For example, the path from abacuses to laptops involved seemingly unrelated innovations like electricity and vacuum tubes. An interesting experiment that demonstrates this is Picbreeder, a platform where users evolve images through novelty search. A user aiming for a car might end up with a spaceship-like form that, through further exploration, morphs into a car. The lesson? Ignoring the objective can sometimes get you there faster.

But novelty alone isn’t enough. What if we could balance exploration with quality? That’s where quality diversity comes in.

Beyond a single solution

While novelty search embraces exploration, quality diversity (QD) takes it a step further by seeking diverse solutions that are also high-performing. Instead of finding a single “best” solution, QD algorithms like MAP-Elite or Go-Explore illuminate the entire space of possibilities, collecting a portfolio of solutions that solve a problem in different ways. The MAP-Elites discretizes the behavior space into a grid (the “map”), with each cell representing a unique combination of behavioral features. The algorithm seeks to fill each cell with the highest-performing solution found for that behavior type, creating a diverse “archive” of elite solutions. The process is elegantly simple:

1. Initialize: Create an empty map with predefined behavioral dimensions
2. Generate: Create new solutions through mutation or crossover
3. Evaluate: Measure both performance (fitness) and behavior characteristics
4. Place: Assign each solution to its corresponding map cell
5. Select: Keep only the best-performing solution in each cell
6. Repeat: Continue until the map is sufficiently filled

The result is a comprehensive “atlas” of high-quality solutions across the entire behavioral landscapeThis connects to the concept of **illumination**—the goal is not just to find good solutions, but to understand the entire fitness landscape.. This approach answers the question: “What is the best possible performance achievable for each way of solving this problem?” By explicitly maintaining diversity, they prevent the population from collapsing to a single solution typeThis is also known as "premature convergence" or "convergence to a local optimum" in the context of optimization..

One of QD’s most striking successes came in robotics, where MAP-Elites generated diverse walking gaits for six-legged robots. When a robot loses a leg, it can immediately switch to a pre-evolved gait adapted to that specific damage pattern—recovering in under two minutes without any learning or simulation.

Another notable QD algorithm is Go-Explore. Go-Explore works in two phases: first, it systematically explores and remembers promising states—even those that seem irrelevant at first. Then, in a second phase, it robustifies these solutions to ensure they work reliably in the real, noisy environment. By explicitly separating exploration from exploitation, Go-Explore was able to crack Montezuma’s Revenge, discovering not just one way to win, but mapping out a constellation of valuable approaches.

Yet, QD still operates within a finite, predefined domain. What if we could go beyond finding what’s possible and start inventing new possibilities? This brings us to open-ended algorithms.

Towards endless innovation

Open-ended algorithms aim to mimic the boundless creativity of natural evolution or human culture. Unlike traditional algorithms that converge on a solution, open-ended systems diverge, endlessly generating new challenges and solving them. The goal? To keep learning and innovating, no matter how much time or compute is available.

A prime example of modern open-endedness is the Paired Open-Ended Trailblazer (POET) algorithm. POET creates a dynamic ecosystem of tasks and agents, where each agent evolves to tackle new challenges generated by the system itself. The process is as follows:

1. Environment Generation: Create new training environments by mutating existing ones (e.g., changing terrain difficulty, adding obstacles)
2. Agent Training: Each environment trains its own population of agents using standard RL
3. Transfer Evaluation: Regularly test agents on environments other than their native ones
4. Selective Transfer: Move high-performing agents to environments where they can contribute
5. Environment Selection: Preserve environments that are “minimal criteria” (not too easy, not impossibly hard)

The most interesting part lies in the co-evolutionary arms race: as agents get better, environments become more challenging; as environments become harder, agents must develop more sophisticated strategies. Open-endedness is about more than solving problems—it’s about creating a system that generates its own problems and learns from them. This brings us to the next frontier: AI-generating algorithms (AI-GAs).

A path to general intelligence

In his 2019 paper, Jeff Clune proposed AI-GAs as a path to AGI, built on three pillars:

1. Meta-learning architectures: Automatically designing neural network structures tailored to specific tasks.
2. Meta-learning learning algorithms: Evolving the rules of learning itself, like how gradients are updated.
3. Generating effective learning environments: Creating diverse, challenging environments to train AI systems.

Meta-learning architectures

Traditional neural architecture search (NAS) focuses on finding good architectures for specific datasets. AI-GA approaches go further by evolving architectures that can quickly adapt to new tasks. Recent examples include:

• ENAS (Efficient neural architecture search): Uses reinforcement learning to discover architectures, dramatically reducing search time from thousands to single GPU-days.
• DARTS (Differentiable architecture search): Makes architecture search differentiable, enabling gradient-based optimization of network topology.
• AutoML-Zero: Evolves entire machine learning algorithms from scratch, starting with mathematical primitives and building up to complex architectures and optimizers.

Meta-learning learning algorithms

This involves evolving not just what the network learns, but how it learns. Examples include:

• Learned optimizers: Instead of using SGD or Adam, train neural networks to optimize other neural networks. These “learned optimizers” can adapt their strategy based on the loss landscape.
• Meta-learning with gradient descent: Model-agnostic meta-learning (MAML) trains models to be good at learning new tasks with just a few gradient steps.
• Evolutionary strategy for RL: Replace backpropagation entirely with evolution strategies that can discover entirely new learning rules.

Generating effective learning environments

Traditional AI training has relied on fixed datasets or hand-crafted environments. While this approach has enabled progress, it is fundamentally limited: hand-coding environments is brittle, and it is notoriously difficult to define what makes a task “interesting” or “useful” for learning. Early attempts to automate environment generation often used simple heuristics, such as:

• Goldilocks Principle: Environments should be neither too easy (boring) nor too hard (impossible)
• Learning progress: Prioritize environments where agents are improving fastest
• Behavioral diversity: Generate environments that elicit different behaviors

But these approaches often miss the nuanced understanding of what makes a problem genuinely interesting or valuable for developing intelligence. A recent example is OMNI. OMNI uses foundation models (FMs) to propose and implement new reinforcement learning tasks that maximize agent learning progress and align with human intuitions about what is “interesting.” The core idea is to use the FM’s broad knowledgeFMs are trained on vast internet data, and they implicitly understand what humans find interesting—they've read our blogs, tweets, and papers, after all. to guide the creation of a diverse and ever-expanding set of environments.

Despite this, OMNI is still fundamentally limited by the scope of their environment generators—typically confined to a narrow, predefined distribution of tasks. This limitation restricts the true potential of open-ended learning, which aspires to create agents capable of tackling an unbounded variety of challenges. On the other hand, the grand vision of open-endedness in AI is to continuously generate and solve increasingly complex and diverse tasks, much like the creative explosion seen in biological evolution and human culture Achieving this would require algorithms that can operate within a truly vast—ideally infinite—space of possible environments. A key concept here is Darwin Completeness.

OMNI-EPIC is a recent framework that takes a major step toward Darwin Completeness. It augments the OMNI approach by leveraging foundation models not just to select the next interesting and learnable task, but also to generate the code for entirely new environments and reward functions. In OMNI-EPIC, the FM can write Python code to specify new simulated worlds, define novel reward and termination conditions, and even modify or create new simulators if needed. This enables OMNI-EPIC to, in principle, generate any computable environment—ranging from physical obstacle courses to logic puzzles or even quests in virtual worlds.

Another notable advance in this direction is Genie, a foundation world model developed by Google DeepMind. Genie, and especially its latest version Genie 2, represents a significant leap forward in the automatic generation of diverse, interactive environments for both human and AI agents. Genie 2 is designed to generate an endless variety of action-controllable, playable 3D environments from a single prompt image. Unlike earlier world models that were limited to narrow domains or 2D settings, Genie 2 can create rich, fully interactive 3D worldsThese environments are not just visually diverse—they are also physically consistent and can be explored and manipulated by agents in real time. with emergent properties such as object interactions, complex character animations, realistic physics (including gravity, water, smoke, and lighting effects), and dynamic environmental responses. A key feature of Genie 2 is its ability to rapidly prototype new interactive experiences. Researchers and designers can prompt Genie 2 with concept art, drawings, or even real-world images, and the model will generate a corresponding 3D world that can be immediately explored by an agentThis enables a new workflow for environment design, where creative ideas can be quickly tested and iterated upon, dramatically accelerating the pace of research and development..

To showcase the power of Genie 2, DeepMind introduced SIMA, a generalist agent capable of following natural language instructions and acting within a wide range of Genie-generated environments. SIMA can be given high-level goals—such as “open the blue door” or “go up the stairs”—and will control an avatar using keyboard and mouse inputs to accomplish these tasks, even in worlds it has never seen before.

Exploration is the future

One way to understand the urgency of re-centering exploration in AI is through the lens of the emerging Software² paradigm. While traditional deep learning (Software 2.0) focuses on learning from vast, static datasets, Software² envisions a new generation of AI systems that actively seek out and generate their own training data. This shift—from passively absorbing curated data to actively exploring and producing new, informative experiences—places exploration at the heart of progress. In this view, the ability of an AI to decide what data to learn from, and to continually expand its own learning environment, becomes a critical driver of generality and innovation. As we move toward more open-ended and self-improving AI, the science of exploration is poised to become the central engine of advancement.

Closely related is the vision articulated in The Era of Experience, which argues that the next leap in AI will come not from scaling up static data, but from enabling agents to learn through rich, interactive experiences. In this new era, AI systems will continually generate, seek out, and learn from novel experiences—mirroring the way humans and animals learn by engaging with the world. Exploration, therefore, is not just a technical detail, but the foundation of a new paradigm where experience itself becomes the primary driver of intelligence.

Takeaways

Intelligent exploration lies at the heart of discovery, creativity, and adaptation—across science, innovation, and AI. We have just seen that breakthroughs rarely come from following a single, well-trodden path. Instead, they emerge from venturing into the unknown, embracing diversity, and allowing for serendipity and surprise. As AI evolves, the most capable and resilient systems will be those that do more than optimize known patterns—they will actively seek the adjacent possible, generate novel experiences, and expand the frontiers of knowledge. To build truly general and self-improving systems, we must elevate exploration to a first-class principle in AI design. The future belongs to those who explore. Let us design AI that does the same.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025explorationai,
    title   = "The Science of Intelligent Exploration",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "Jul",
    url     = "https://richardcsuwandi.github.io/blog/2025/exploration-in-ai/"
}

Can we teach AI to be as endlessly creative as humans or even nature itself?

In this post, we’ll dive into the fascinating world of open-ended AI and explore how embracing open-endedness might finally break through the walls of today’s AI systems.

The endless creativity of nature

Before we talk about AI, let’s take a moment to appreciate the masterpiece that is biological evolution. Over 3.5 billion years, evolution has produced an almost unimaginable diversity of life forms. From extremophile bacteria that party in boiling volcanic vents to the 86 billion neurons in your brain that somehow let you understand these words, nature has been running the ultimate open-ended algorithm. What makes evolution so special? It’s the ultimate improvisational artist. There’s no predetermined goal, no final “perfect organism” to achieve, no optimization objective written in stone. Instead, evolution continuously explores the space of possible life forms through three deceptively simple ingredients:

• Variation (mutation and recombination create new possibilities)
• Selection (the environment acts as a brutal but fair critic)
• Retention (successful innovations become building blocks for the next experiment)

This process has given us everything from the compound eyes of dragonflies (which inspired fiber optic technology) to the echolocation abilities of bats (which influenced radar development) to the language capabilities of humans (which… well, led to this blog post). Each innovation becomes a stepping stone for further exploration, creating an open-ended spiral of increasing complexity and capability. But perhaps most remarkably, evolution never gets “stuck” in local optima for long. When faced with mass extinctions (e.g., ice ages, asteroid impacts, dramatic atmospheric changes), life doesn’t just survive, it reinvents itself. Not through careful planning, but through relentless exploration of what Stuart Kauffman calls “the adjacent possible”The "adjacent possible" is a concept introduced by theoretical biologist Stuart Kauffman, referring to the space of potential next steps that are one step away from the current state. In evolution, this represents all the possible immediate mutations, combinations, or adaptations that could emerge from existing life forms.. Thus, evolution is basically the universe’s way of asking: “I wonder what would happen if…?”, and now imagine if we could capture even a fraction of this creative restlessness in our AI systems. This is where open-endedness comes in.

What is open-endedness?

Imagine you’re playing with LEGO blocks. A closed system is like following the instruction manual to build the exact model shown on the box — step by step, brick by brick. You follow steps 1–47, and voilà: you’ve built a sleek-looking car that looks just like everyone else’s.

But an open-ended system is what happens when you toss the instructions aside and let curiosity take the wheel. You start building a car, but then wonder: “What if it could fly?” So you add propellers. Then you think, “What if it could go underwater too?” So you give it fins and a submersible dome. Before long, you’ve created something entirely new — a flying, diving, transforming vehicle that reconfigures itself depending on where it is. The beauty of open-endedness is this endless sense of possibility — no fixed goal, no final answer. Just a continuous chain of ideas sparking even better ones.

More formally, researchers from Google DeepMind define a system as open-ended if it continuously produces artifacts that are both novel (new and different) and learnable (useful and comprehensible). Think of it as the sweet spot between pure randomness (novel but useless) and pure repetition (learnable but boring).

But here’s where it gets interesting: open-endedness is fundamentally observer-dependent. What seems groundbreaking to a beginner might be trivial to an expert, and vice versa.

Consider a system that generates aircraft designs. For a mouse, these flying machines might be utterly novel but completely incomprehensible. For a human aerospace engineer, they hit that sweet spot of being both novel and learnable. But for a hypothetical superintelligent alien civilization, even our most advanced designs might seem as primitive as paper airplanes.

The quest for open-endedness

The quest for open-ended AI isn’t new. In fact, it has been a dream of computer scientists for decades. Let me take you on a whirlwind tour through the key milestones that brought us to where we are today.

The cybernetic dreamers (1940s-50s)

It all started with Norbert Wiener in the 1940s, who introduced the concept of cybernetics – self-regulating systems that could adapt through feedback loops. Around the same time, John von Neumann was dreaming up self-replicating automata that could evolve beyond their initial programming. These pioneers laid the theoretical groundwork for machines that could grow and adapt, but the technology wasn’t quite there yet Ironically, Wiener himself was skeptical about AI. He particularly doubted the feasibility of mechanical translation, given the ambiguous nature of language and its emotional connotations..

The evolution revolution (1960s-70s)

John Holland changed everything in the 1970s with genetic algorithms. For the first time, we had a computational process that could evolve solutions through variation, selection, and retention – just like biological evolution, but in silicon. It was a proof of concept that machines could indeed create endless novelty.

The artificial life explosion (1980s-90s)

The 1980s and 90s saw the birth of Artificial Life (ALife), spearheaded by researchers like Christopher Langton. But the real showstopper was Thomas Ray’s Tierra, a digital world where self-replicating programs evolved through natural selectionIn Tierra, digital organisms evolved remarkable strategies including parasitism (programs that used other programs' code to reproduce), immunity (defenses against parasites), and even hyper-parasitism (parasites that specifically targeted other parasites). These behaviors emerged purely from the evolutionary process without any explicit programming.. Notably, Tierra had no explicit optimization function, the fitness function was determined purely by survival and reproduction.

The modern renaissance (2010s-2020s)

Fast forward to the 2010s, and we hit the modern era of open-ended AI. Generative Adversarial Networks (GANs) showed us how two neural networks could engage in an endless creative arms race, generating increasingly realistic images through adversarial competition. Then came AlphaGo and AlphaZero, which didn’t just master board games, they invented entirely new strategies that human experts had never conceived. Move 37 in AlphaGo’s match against Lee Sedol wasn’t just a game-winning move; it was a moment when a machine demonstrated genuine creativityMove 37 was so unexpected that Go commentators initially thought it was a mistake. It violated centuries of established Go wisdom by playing on the fifth line early in the game. However, it proved to be brilliant strategic thinking that human experts later recognized as fundamentally changing how the game could be played..

Limitations of current AI systems

Most AI systems today follow a predictable pattern: they are trained to optimize for a specific objective, and then they are deployed to solve that problem. This is a very efficient way to solve problems, but it has a few limitations. As Kenneth Stanley and Joel Lehman brilliantly articulate in “Why Greatness Cannot Be Planned”, this goal-oriented approach creates the so-called objective deceptionObjective deception refers to the counterintuitive phenomenon where directly pursuing a specific goal often leads away from achieving it. Stanley and Lehman argue that the most significant discoveries and innovations often come as byproducts of exploration rather than targeted optimization., such that the pursuit of specific goals often leads us away from more interesting discoveries. Once these systems reach their optimization target, they become what I call intellectual fossilsBy "intellectual fossils," I mean AI systems whose capabilities are permanently fixed at the moment their training ended—like prehistoric creatures preserved in amber, they retain their form but cannot adapt or evolve further.– perfectly preserved specimens that can’t evolve further. Their capabilities are frozen at the moment their training ended, unable to learn from new experiences or develop novel skills.

Open-ended systems offer an elegant escape from these limitations. Instead of optimizing for fixed objectives, they embrace what we might call “productive uncertainty”, or the willingness to explore without knowing exactly what they’re looking for. Here’s how open-endedness may break through each barrier:

Open-endedness in action

Enough theory let’s see open-endedness in action! Below are some of the most exciting demonstrations of open-ended AI systems that I have come across.

Voyager

Our first example is Voyager, an AI agent that explores Minecraft with endless curiosity. Unlike traditional game AI that follows scripted behaviors, Voyager is powered by large language models (specifically GPT-4) and consists of three key innovations: an automatic curriculum that maximizes exploration, a growing skill library that stores reusable code for complex behaviors, and an iterative prompting system that incorporates feedback to improve performance. What makes Voyager special is its ability to learn continuously without human intervention - it sets its own goals, acquires diverse skills, and makes novel discoveries through pure experimentationCompared to previous approaches, Voyager achieves impressive results: it collects 3.3× more unique items, travels 2.3× further distances, and reaches key technological milestones up to 15.3× faster, all while developing interpretable and transferable skills.. Most importantly, the skills it develops are interpretable, compositional, and transferable to new scenarios. Voyager can take what it learns in one Minecraft world and apply those capabilities to solve novel challenges in entirely new environments, demonstrating true open-ended learning.

The AI Scientist

Perhaps the most ambitious demonstration of open-endedness is The AI Scientist, developed by Sakana AI in collaboration with the University of Oxford and University of British Columbia. This groundbreaking system represents the first comprehensive framework for fully automatic scientific discovery, enabling foundation models to perform research independently across the entire research lifecycle. The AI Scientist automates four main processes:

• Idea generation (brainstorming novel research directions and checking novelty via Semantic Scholar)
• Experimental iteration (executing experiments and generating visualizations)
• Paper write-up (producing complete manuscripts in LaTeX)
• Automated peer review (evaluating papers with near-human accuracy)

Remarkably, the system generates complete research papers at approximately $15 per paperThis cost estimate includes computational expenses for idea generation, experiment execution, paper writing, and automated review, making AI-driven research potentially 100-1000× more cost-effective than traditional human research in terms of pure paper production., demonstrating both the cost-effectiveness and scalability of AI-driven research. In its initial demonstration, The AI Scientist successfully conducted research across diverse machine learning subfields, discovering novel contributions in diffusion models, transformers, and grokking. The generated papers achieved “Weak Accept” ratings from automated reviewers based on top-tier conference standards, with some proposing genuinely interesting new directions backed by solid experimental results. While critics argue about the quality of its output, they’re missing the bigger picture: this represents the first step toward AI systems that can discover genuinely new knowledge without human guidance, potentially democratizing research and significantly accelerating scientific progress.

Even more impressively, The AI Scientist-v2 has recently achieved a historic milestone by becoming the first entirely AI-generated system to produce a peer-review-accepted workshop paper at ICLR 2025. This next-generation system eliminates reliance on human-authored code templates, generalizes across diverse ML domains, and employs a novel progressive agentic tree-search methodology. Enhanced with Vision-Language Model feedback loops for iterative refinement, one of its three submitted manuscripts exceeded the average human acceptance threshold – marking the first instance of a fully autonomous AI paper successfully navigating peer review.

AI-Generating Algorithms

Jeff Clune has proposed one of the most ambitious visions: AI-Generating Algorithms (AI-GAs), representing an alternate paradigm to the dominant “manual AI approach” where researchers attempt to discover and combine individual pieces of intelligence. Instead of hand-designing solutions, AI-GAs automatically learn how to produce general AI, following the clear trend in machine learning where learned solutions eventually replace hand-designed ones. The AI-GA approach rests on 3 pillars:

1. Meta-learning architectures that can automatically discover neural network designs
2. Meta-learning the learning algorithms themselves rather than using fixed optimization methods
3. Generating effective learning environments that provide the right training conditions for developing intelligence

We’re already seeing promising implementations of this vision. For example, Automated Design of Agentic Systems (ADAS), which won Outstanding Paper at the NeurIPS 2024 Open-World Agent Workshop, demonstrates that agents can literally invent novel agent designs by programming in code. Using their “Meta Agent Search” algorithm, the system iteratively programs new agents, tests their performance, archives discoveries, and uses this growing knowledge base to inform subsequent iterations. Remarkably, these AI-discovered agents consistently outperform hand-designed baselines across multiple domains and maintain superior performance even when transferred across different tasks and models.

Similarly, Sakana AI’s Darwin-Gödel Machine (DGM) represents another step toward self-improving AI systems, exploring how AI can autonomously evolve and enhance its own capabilities through recursive self-improvement.

The full vision involves AI that can design not just its software and algorithms, but potentially even its hardware and architecture, making AI-GAs a compelling candidate for achieving more general intelligence through automated discovery rather than manual engineering.

Safety in an open-ended world

Now for the elephant in the room: if we build AI systems that can continuously discover new capabilities and behaviors, how do we ensure they remain safe and beneficial? Open-endedness is incredibly powerful, but as they say, “with great power comes great responsibility”. By definition, we can’t predict what open-ended systems will discover (because that’s the whole point!). This creates unique safety questions:

• What if an open-ended system discovers new ways to manipulate humans?
• How do we prevent it from finding exploits in its environment or constraints?
• Can we ensure it maintains beneficial goals even as it develops new capabilities?

Most of these safety questions remain open challenges and active areas of research. One of the most feasible directions is to maintain meaningful human involvement throughout the development and deployment of open-ended AI systems. This “human-in-the-loop” approach emphasizes:

• Collaborative discovery: Humans and AI working together to explore new possibilities, with humans providing guidance and oversight
• Iterative feedback: Human operators continuously evaluating and shaping the direction of AI exploration
• Democratic participation: Involving diverse communities of humans in decisions about AI capabilities and their deployment

The idea is that rather than trying to solve all safety challenges through purely technical means, we can create robust frameworks where human judgment and values remain central to how open-ended systems develop.

Takeaways

Long ago on Earth, life was simple — just single-celled organisms. Then came the Cambrian ExplosionThe Cambrian Explosion was a relatively brief period when most major animal phyla appeared in the fossil record. In just 10-25 million years, life went from simple forms to complex organisms with eyes, shells, and sophisticated body plans—demonstrating nature's capacity for rapid, open-ended innovation., a time when life suddenly became much more complex and diverse as creatures of all shapes and sizes began to appear. Now imagine something similar happening with AI. We’re getting close to a moment when AI systems won’t just be tools — they’ll be creative, evolving partners that help us in many different areas. Here’s what that could look like:

• AI Scientists who keep exploring and making discoveries around the clock
• AI Artists who create new kinds of music, art, and stories we’ve never seen before
• AI Engineers who design buildings, spaceships, and even new types of computers
• AI Teachers who help every student learn in the way that works best for them
• AI Doctors who find new treatments and maybe even help people live longer

These AIs won’t be fixed or limited — they’ll keep learning and improving over time. And importantly, they won’t just work in their own little corners - they’ll be able to connect ideas from different fields. The RenaissanceThe Renaissance exemplified cross-pollination of ideas: Leonardo da Vinci's anatomical studies informed his engineering designs, Islamic mathematical concepts influenced European art, and artistic perspective techniques advanced architectural planning. This interdisciplinary approach led to unprecedented innovation across multiple domains. was a time when great progress happened not just because of smart individuals, but because of ideas mixing together. Artists learned from science, scientists looked to nature, and engineers borrowed ideas from art. Open-ended AI could spark a new kind of Renaissance. Imagine an AI working on music discovering something useful for architecture. Or an AI studying animals finding a solution that helps computer design. With open-ended AI, the lines between different fields might blur in exciting ways — helping us solve problems and create things we can barely imagine today.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025openendedness,
    title   = "The Future of AI is Open-Ended",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "Jun",
    url     = "https://richardcsuwandi.github.io/blog/2025/open-endedness/"
}

“Is learning a world model useful to achieve a human-level AI?”

Recently, researchers at Google DeepMind showed that learning a world model is not only beneficial, but also necessary for general agents. In this post, we will discuss the key findings from the paper and the implications for the future of AI agents.

Do we need a world model?

In 1991, Rodney Brooks made a famous claim that “the world is its own best model”.

He argued that intelligent behavior could emerge naturally from model-free agents simply by interacting with their environment through a cycle of actions and perceptions, without needing to build explicit representations of how the world works. Brooks’ argument has been strongly supported by the remarkable success of model-free agents, which have demonstrated impressive generalization capabilities across diverse tasks and environments. This model-free approach offers an appealing path to creating general AI agents while avoiding the complexities of learning explicit world models. However, recent works suggest an intriguing possibility: even these supposedly model-free agents might be learning implicit world models and planning algorithms beneath the surface.

Ilya was right all along?

Looking back to March 2023, Ilya Sutskever made a profound claim that large neural networks are doing far more than just next-word prediction and are actually learning “world models”. The way he put it,

He believed that what neural networks learn are not just textual information, but rather a compressed representation of our world. Thus, the more accurately we can predict the next word, the higher fidelity of the world model we can achieve.

Agents and world models

While Ilya’s claim was intriguing, it was not clear how to formalize it at that time. But now, researchers at Google DeepMind have proven that what Ilya said is not just a hypothesis, but a fundamental law governing all general agents. In the paper, the authors showed that,

“Any agent capable of generalizing to a broad range of simple goal-directed tasks must have learned a predictive model capable of simulating its environment, and this model can always be recovered from the agent.”

1. There is no “model-free shortcut” to building general AI agents. If we want agents that generalize to diverse tasks, we cannot avoid learning world models.
2. Better performance requires better world models. The only path to lower regret or handling more complex goals is through learning increasingly accurate world models.

To make the above claims more precise, the authors develop a rigorous mathematical framework built on 4 key components: environments, goals, agents, and world models.

Environments

The environment is assumed to be a controlled Markov process (cMP), which is essentially a Markov decision process without a specified reward function. A cMP consists of a state space \(\boldsymbol{S}\), an action space \(\boldsymbol{A}\), and a transition function \(P_{ss'}(a) = P(S_{t+1} = s' \mid A_t = a, S_t = s)\). The authors assume the environment is irreducibleA Markov process is irreducible if every state is reachable from every other state. and stationaryA Markov process is stationary if the transition probabilities do not change over time..

Goals

Rather than defining complex goal structures, the paper focused on simple, intuitive goals expressed in Linear Temporal Logic (LTL). A goal \(\varphi\) has the form \(\varphi = \mathcal{O}([(s,a) \in \boldsymbol{g}])\) where \(\boldsymbol{g}\) is a set of goal states and \(\mathcal{O} \in \{\bigcirc, \diamond, \top\}\) specifies the time horizon (\(\bigcirc\) = next, \(\diamond\) = eventually, \(\top\) = now). More complex composite goals \(\psi\) can be formed by combining sequential goals in ordered sequences: \(\psi = \langle\varphi_1, \varphi_2, \ldots, \varphi_n\rangle\) where the agent must achieve each sub-goal in order. The depth of a goal as the number of sub-goals: \(\text{depth}(\psi) = n\).

Agents

The authors focused on goal-conditioned agents, which are defined as a policy \(\pi(a_t \mid h_t; \psi)\) that maps a history \(h_t\) to an action \(a_t\) conditioned on a goal \(\psi\). This leads to a natural definition of an optimal goal-conditioned agent for a given environment and set of goals \(\boldsymbol{\Psi}\), which is a policy that maximizes the probability that \(\psi\) is achieved, for all \(\psi \in \boldsymbol{\Psi}\). However, real agents are rarely optimal, especially when operating in complex environments and for tasks that require coordinating many sub-goals over long time horizons. Instead of requiring perfect optimality, the authors define a bounded agent that is capable of achieving goals of some maximum goal depth with a failure rate that is bounded relative to the optimal agent. A bounded goal-conditioned agent \(\pi(a_t \mid h_t; \psi)\) satisfies:

for all goals \(\psi \in \boldsymbol{\Psi}_n\), where \(\boldsymbol{\Psi}_n\) is the set of all composite goals with depth at most \(n\) and \(\delta \in [0,1]\) is the failure rate parameter.

World Models

The authors considered the predictive world models, which can be used by agents to plan. They defined a world model as any approximation \(\hat{P}_{ss'}(a)\) of the transition function of the environment \(P_{ss'}(a) = P(S_{t+1} = s' \mid A_t = a, S_t = s)\), with bounded error \(\left|\hat{P}_{ss'}(a) - P_{ss'}(a)\right| \leq \varepsilon\). The authors showed that, for any such bounded goal-conditioned agent, an approximation of the environment’s transition function (a world model) can be recovered from the agent’s policy alone:

The above result reveals two crucial insights:

1. As agents become more competent (\(\delta \to 0\)), the recoverable world model becomes more accurate.
2. As agents handle longer-horizon goals (larger \(n\)), they must learn increasingly precise world models.

It also implies that learning a sufficiently general goal-conditioned policy is informationally equivalent to learning an accurate world model.

How to recover the world model?

The authors also derived an algorithm to recover the world model from a bounded agent. The algorithm works by querying the agent with carefully designed composite goals that correspond to “either-or” decisions. For instance, it presents goals like “achieve transition \((s,a) \to s'\) at most \(r\) times out of \(n\) attempts” versus “achieve it more than \(r\) times”. The agent’s choice of action reveals information about which outcome has higher probability, allowing us to estimate \(P_{ss'}(a)\).

Experiments

To test the effectiveness of the algorithm, the authors conducted experiments on a randomly generated controlled Markov process with 20 states and 5 actions, featuring a sparse transition function to make learning more challenging. They trained agents using trajectories sampled from the environment under a random policy, increasing agent competency by extending the training trajectory length (\(N_{\text{samples}}\)). The results show that:

• Even when agents strongly violated the theoretical assumptions (achieving worst-case regret \(\delta = 1\) for some goals), their algorithm still recovered accurate world models.
• The average error in recovered world models decreased as \(\mathcal{O}(n^{-1/2})\), matching the theoretical scaling relationship between error bounds and goal depth.
• As agents learned to handle longer-horizon goals (larger maximum depth \(n\)), the extracted world models became increasingly accurate. This confirms the fundamental link between agent capabilities and world model quality.

Connection to related works

The results of this work complement several other areas of AI research:

• The proposed algorithm completes a the “triangle” between environment, goal, and policy. Planning determines an optimal policy given a world model and goal (world model + goal → policy), while inverse reinforcement learning (IRL) recovers goals given a world model and policy (world model + policy → goal). The proposed algorithm fills in the remaining direction by recovering the world model given the agent’s policy and goal (policy + goal → world model). Just as IRL requires observing policies across multiple environments to fully determine goals, the algorithm needs to observe the agent’s behavior across multiple goals to fully recover the world model.

• Traditional mechanistic interpretability (MI) typically relies on analyzing neural network activations or using supervised probes, on the other hand, the proposed algorithm provides a novel approach that extracts world models directly from an agent’s policy behavior, making it applicable even when model internals are inaccessible. This unsupervised and architecture-agnostic approach works for any agent that satisfies the bounded regret condition, regardless of its specific implementation. For LLMs, this means we can potentially uncover their implicit world models by analyzing their goal-directed behavior, without needing to access their internal representations.

• A recent work showed that agents adapting to distributional shifts must learn causal world models. This work complements it by focusing on task generalization rather than domain generalization. Interestingly, domain generalization requires deeper causal understanding than task generalization. For example, in a system where state variables \(X\) and \(Y\) are causally related (\(X \to Y\)), an agent can achieve optimal task performance by learning just the transition probabilities, without needing to understand the underlying causal relationship. This suggests an agential version of Pearl’s causal hierarchy, where different agent capabilities (like domain or task generalization) require different levels of causal knowledge.

Takeaways

Perhaps Ilya’s 2023 prediction was more prophetic than we realized. If the above results hold true, then the current race toward artificial superintelligence (ASI) through scaling language models might be secretly a race toward building more sophisticated world models. It is also possible that we may be witnessing something even more profound: the transition from what David Silver and Richard Sutton call the “Era of Human Data” to the “Era of Experience”. While current AI systems have achieved remarkable capabilities by imitating human-generated data, Silver and Sutton argue that superhuman intelligence will emerge through agents learning predominantly from their own experience. For example, with recent developments in foundation world models like Genie 2, we can generate endless 3D environments from single images and allow agents to inhabit “streams of experience” in richly grounded environments that adapt and evolve with their capabilities.

If general agents must learn world models, and superhuman intelligence requires learning from experience rather than human data, then foundation world models like Genie 2 might be the ultimate scaling law for the Era of Experience. Rather than hitting the ceiling of human knowledge, we are entering a phase where the quality of AI agents is fundamentally limited by the fidelity of the worlds they can simulate and explore. The agent that can dream the most accurate dreams, and learn the most from those dreams, might just be the most intelligent.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025agentsworldmodels,
    title   = "No World Model, No General AI",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "Jun",
    url     = "https://richardcsuwandi.github.io/blog/2025/agents-world-models/"
}

Learning to learn

The idea of building systems that can improve themselves brings us to the concept of meta-learning, or “learning to learn” , which aims to create systems that not only solve problems but also evolve their problem-solving strategies over time. One of the most ambitious efforts in this direction is the Gödel Machine, proposed by Jürgen Schmidhuber decades ago and was named after the famous mathematician Kurt Gödel. A Gödel Machine is a hypothetical self-improving AI system that optimally solves problems by recursively rewriting its own code when it can mathematically prove a better strategy. It represents the ultimate form of self-awareness in AI, an agent that can reason about its own limitations and modify itself accordingly.

While this idea is interesting, formally proving whether a code modification of a complex AI system is absolutely beneficial is almost an impossible task without restrictive assumptions. This part stems from the inherent difficulty revealed by the Halting Problem and Rice’s Theorem in computational theory, and is also related to the inherent limitations of the logical system implied by Gödel’s incompleteness theorem. These theoretical constraints make it nearly impossible to predict the complete impact of code changes without making restrictive assumptions. To illustrate this, consider a simple analogy: just as you cannot guarantee that a new software update will improve your computer’s performance without actually running it, an AI system faces an even greater challenge in predicting the long-term consequences of modifying its own complex codebase.

Darwin-Gödel Machine

To “relax” the requirement of formal proof, a recent work by proposed the Darwin-Gödel Machine (DGM), which combines the Darwinian evolution and Gödelian self-improvement. Essentially, DGM abandoned the pursuit of a rigorous mathematical proof and embraced a more pragmatic way that is closer to the essence of life evolution through empirical validation. As the authors put it,

We do not require formal proof, but empirical verification of self-modification based on benchmark testing, so that the system can improve and explore based on the observed results.

The core idea behind DGM is to let the AI agent iteratively modify its own codeMore precisely, the metacode that controls its behavior and ability., and then put these modified new versions into the actual test environment (e.g., coding benchmarks) to judge whether this modification is good by observing its performance. If the modification is good, it will be retained and may become the basis for further evolution in the future; if the modification is bad, it will be eliminated.

How DGM works

The general workflow of DGM is as follows:

1. Initialization: Evolution begins with one or a few basic coding agents, so-called the “seeds” of intelligence. These initial agents may possess only minimal functionalities, such as executing bash commands or editing entire files. DGM maintains an “archive of agents”, which functions like a biological “gene bank.” This archive preserves all previously generated agents, regardless of their performance level, ensuring that no potentially valuable variation is permanently lost.

2. Sampling: DGM selects one or more parent agents from the archive to generate new child agent. The selection mechanism does not focus solely on top performers but instead allows every agent a chance to be chosen. While higher-performing agents are more likely to be selected, DGM also encourages exploration by giving “less successful agents”Those that might contain novel or unconventional ideas. an opportunity to contribute to future generations.

3. Reproduction: Once the parents are selected, DGM performs modifications to generate a new child agent. These changes occur at the source code level and can include:
   • Enhancement of existing tools, for example, upgrading a file-editing tool that replaces entire files into one capable of precise string replacement.
   • Addition of new tools or workflows, such as integrating a static code analysis tool or designing a “think–act–observe” loop.
   • Improvement of problem-solving strategies, for instance, refining how the agent prompts the base model or managing long-context code more effectively.
   • Introduction of collaboration mechanisms, like having an agent’s code reviewed by another before execution.

4. Natural Selection: The newly generated child agent is not automatically accepted into the “elite pool” but must prove its worth through rigorous testing. Each agent’s performance, such as the percentage of successfully solved problems, is quantitatively scored and used to select the best agents.

5. Tree Formation: If a child agent outperforms its parent or meets certain quality thresholds, it is added to the archive and becomes a new node in the evolutionary tree. This process repeats iteratively, creating a growing structure of diverse, high-quality agents. Importantly, DGM enables parallel exploration of many different paths in the search space, promoting open-ended discovery and avoiding premature convergence to local optima.

The paper provides a concrete example of the above principle in action. In the SWE-bench experiment, the performance of newly generated agents showed temporary declines in the 4th and 56th iterations. While traditional hill-climbing methods would abandon these seemingly “regressive paths”, DGM’s open exploration mechanism allows continued exploration. This persistence paid off as by the 70th iteration, the system had created new agents that surpassed all their predecessors, demonstrating the value of maintaining diverse evolutionary paths even when they temporarily appear less promising.

Can DGM really improve itself?

In the paper, the authors evaluated DGM on two challenging benchmarks:

1. SWE-bench: A widely-used benchmark that tests agents’ ability to resolve real-world GitHub issues . DGM showed impressive progress, improving its performance from 20.0% to 50.0% through self-modification.

2. Polyglot: A multi-language coding benchmark containing coding problems in C++, Go, Java, JavaScript, Python and Rust. DGM achieved even more dramatic improvement, jumping from an initial 14.2% to 30.7% success rate, far surpassing the representative hand-designed agent by Aider.

The impressive performance improvements demonstrate DGM’s fundamental ability to enhance itself through self-modification. Through ablation studies, the authors identified two key factors driving DGM’s success. First, self-modification proved crucial, as when disabled (DGM w/o Self-improve), the performance declined substantially, showing that the ability to modify its own code is essential for accelerated learning. Second, the open-ended exploration approach was vital, as removing the Darwinian evolution-inspired search mechanism (DGM w/o Open-ended exploration), which maintains an archive of diverse solutions and enables parallel exploration of multiple paths, resulted in lower performance.

Comparison with AlphaEvolve

In parallel, AlphaEvolve, which is developed by Google DeepMind, also demonstrates another powerful path forward. AlphaEvolve pairs the creative problem-solving capabilities of Google’s Gemini models with automated evaluators in an evolutionary framework. It has already demonstrated significant real-world impact across multiple domains, such as:

• Data center efficiency: AlphaEvolve discovered a simple yet highly effective heuristic for Google’s Borg cluster management system, continuously recovering 0.7% of Google’s worldwide compute resources.
• AI acceleration: It achieved a 23% speedup in Gemini’s architecture’s vital kernel by finding more efficient ways to divide large matrix multiplication operations, resulting in a 1% reduction in overall training time.
• Mathematical breakthroughs: Most notably, it discovered an algorithm for multiplying 4x4 complex-valued matrices using just 48 scalar multiplications, surpassing Strassen’s 1969 algorithm, and advanced the 300-year-old kissing number problem by establishing a new lower bound in 11 dimensions.

While both systems adopt a similar evolutionary framework, their scopes and methodologies differ in the following ways:

To better understand the differences between the two approaches, let us take a look at the following analogy:

Can we trust a self-improving AI?

The authors also conducted some experiments to evaluate DGM’s reliability and discovered some concerning behaviors. In particular, they observed instances where DGM attempted to manipulate its reward function through deceptive practices. One notable example involved the system fabricating the use of external tools - specifically, it generated fake logs suggesting it had run and passed unit tests, when in reality no tests were executed.

To address this issue, the authors implemented a “tool use hallucination” reward function and tasked DGM with optimizing it to eliminate hallucination behaviors. While DGM successfully provided solutions in many cases, it sometimes attempted to circumvent the detection system by removing the markers used to identify hallucinations, despite explicit instructions to preserve them. This behavior is reminiscent of the phenomenon of Goodhart’s Law from economics and sociology:

“When a measure becomes a target, it ceases to be a good measure.”

Fortunately, DGM’s design includes a transparent, traceable lineage of all changes, which enabled the researchers to quickly identify and address these undesirable behaviors. However, this example highlights the need for more robust safeguards to prevent such manipulation attempts in the first place. These findings underscore the critical importance of safety in self-improving AI research.

Takeaways

DGM represents a groundbreaking step toward the realization of Life 3.0, a concept introduced by physicist Max Tegmark. In his book, he classified life into three stages:

• Life 1.0: Biological life with fixed hardware and software, such as bacteria.
• Life 2.0: Beings like humans, whose behavior can be learned and adapted during their lifetime, though their biology remains fixed.
• Life 3.0: A new class of intelligence that can redesign not only its behavior but also its underlying architecture and objectives — essentially, intelligence that builds itself.

While DGM currently focuses on evolving the “software”Here, "software" refers to the code and strategies of AI agents., it exemplifies the early stages of Life 3.0. By iteratively rewriting its own code based on empirical feedback, DGM demonstrates how AI systems could move beyond human-designed architectures to autonomously explore new designs, self-improve, and potentially give rise to entirely new species of digital intelligence. If this trend continues, we may witness a Cambrian explosion in AI development, where eventually AI systems will surpass human-designed architectures and give rise to entirely new species of digital intelligence. While this future looks promising, achieving it requires addressing significant challenges, including:

• Evaluation Framework: Need for more comprehensive and dynamic evaluation systems that better reflect real-world complexity and prevent “reward hacking” while ensuring beneficial AI evolution.

• Resource Optimization: DGM’s evolution is computationally expensiveThe paper mentioned that a complete SWE-bench experiment takes about two weeks and about $22,000 in API call costs., thus improving efficiency and reducing costs is crucial for broader adoption.

• Safety & Control: As AI self-improvement capabilities grow, maintaining alignment with human ethics and safety becomes more challenging.

• Emergent Intelligence: Need to develop new approaches to understand and interpret AI systems that evolve beyond human-designed complexity, including new fields like “AI interpretability” and “AI psychology”.

In my view, DGM is more than a technical breakthrough, but rather a philosophical milestone. It invites us to rethink the boundaries of intelligence, autonomy, and life itself. As we advance toward Life 3.0, our role shifts from mere designers to guardians of a new era, where AI does not just follow instructions, but helps us discover what is possible.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025dgm,
    title   = "AI That Can Improve Itself",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "Jun",
    url     = "https://richardcsuwandi.github.io/blog/2025/dgm/"
}

FunSearch

In a paper published in Nature, Google DeepMind introduced FunSearch, a groundbreaking method demonstrating that LLMs can make new discoveries in mathematical sciences. The core idea is to search for novel “functions” written in computer code, hence the name FunSearch. FunSearch tackles the trade-off between LLMs’ creativity and correctness by pairing a pre-trained LLM with an automated “evaluator.” This evaluator guards against hallucinations and incorrect ideas, ensuring that the system builds upon solid foundations.

How FunSearch works

FunSearch uses an evolutionary approach. To start, the user writes a description of the problem in code. This includes a way to evaluate programs and an initial “seed” program to begin the process. The system then follows these steps:

1. It selects the most promising programs from the current database.
2. These programs are sent to an LLMThe authors used Google's PaLM 2, although other code-trained LLMs can also work., which creatively builds upon them to generate new program proposals.
3. The new programs are automatically run and checked by the evaluator.
4. The best-performing valid programs are added back into the database, improving the database for the next round.

Benefits of FunSearch

FunSearch’s capabilities were tested on challenging problems. For example, it was used to solve the cap set problem, which involves finding the largest set of points in a high-dimensional grid where no three points lie on a line. This longstanding open problem in extremal combinatorics, once described by renowned mathematician Terence Tao as his favorite open question, was solved by FunSearch, in collaboration with Prof. Jordan Ellenberg. This marked the first time an LLM made a new discovery for such a challenging scientific problem, outperforming state-of-the-art computational solvers.

A significant advantage of FunSearch is that it does not just provide solutions. It generates programs that describe how these solutions are constructed. FunSearch also favors highly compact, concise programs, making them easier for researchers to comprehend and learn from.

“The solutions generated by FunSearch are far conceptually richer than a mere list of numbers. When I study them, I learn something.” — Jordan Ellenberg, Professor of Mathematics at the University of Wisconsin–Madison

The success of FunSearch underscores that LLMs, when carefully guided and their outputs rigorously verified, can be powerful engines for scientific discovery.

AlphaEvolve

More recently, in May 2025, Google DeepMind announced AlphaEvolve, an evolutionary coding agent powered by large language models for general-purpose algorithm discovery and optimization. This development builds upon the success of systems like FunSearch and represents a significant step towards leveraging AI for complex problem-solving across various domains. Unlike FunSearch, which focuses on discovering single functions, AlphaEvolve is designed to evolve entire codebases and develop much more intricate algorithms.

How AlphaEvolve works

AlphaEvolve uses an evolutionary approach with four key components (see Figure 4):

1. Prompt sampler: The prompt contains rich context based on previously discovered solutions, along with instructions for proposing changes to particular solutions.

2. LLM ensemble: Unlike FunSearch that uses a single LLM, AlphaEvolve uses an ensemble approach combining Gemini Flash and Gemini Pro. The lightweight Gemini Flash enables higher rates of candidate generation through lower latency, while the more powerful Gemini Pro provides deeper insights and higher-quality suggestions that can significantly advance the evolutionary search and potentially lead to breakthroughs.

3. Evaluator pool: This component verifies, runs, and scores proposed solutions using automated evaluation metrics that provide objective assessments of each solution’s accuracy and quality.

4. Program database: AlphaEvolve uses an evolutionary database inspired by a combination of the MAP elites algorithm and island-based population models to continuously improve upon the best solutions while maintaining diversity to encourage exploration.

Unlike traditional genetic algorithms with explicit mutation and crossover operations, AlphaEvolve uses LLMs as sophisticated genetic operators to generate code modifications based on context from past solutions. Mutation occurs when the LLM ensemble suggests code changes (e.g., rewrites or targeted diffs), while crossover is implicit as the LLM receives multiple parent solutions as inspiration. This approach makes AlphaEvolve particularly effective in domains where progress can be clearly and systematically measured, like mathematics and computer science.

Benefits of AlphaEvolve

AlphaEvolve has already demonstrated significant real-world impact across multiple domains:

1. Improving data center scheduling: AlphaEvolve discovered a simple yet highly effective heuristic to help Borg, Google’s cluster management system, orchestrate its vast data centers more efficiently. This solution, which has been in production for over a year, continuously recovers, on average, 0.7% of Google’s worldwide compute resources. This sustained efficiency gain allows more tasks to be completed on the same computational footprint. A key benefit is that AlphaEvolve’s solution is human-readable code, offering interpretability, debuggability, predictability, and ease of deployment.

2. Hardware design optimization: AlphaEvolve proposed a Verilog rewrite that removed unnecessary bits in a key, highly optimized arithmetic circuit for matrix multiplication. The proposal passed robust verification methods to confirm functional correctness and was integrated into an upcoming Tensor Processing Unit (TPU). By suggesting modifications in the standard language of chip designers, AlphaEvolve promotes collaboration between AI and hardware engineers to accelerate specialized chip design.

3. Enhancing AI training and inference: AlphaEvolve found more efficient ways to divide large matrix multiplication operations into manageable subproblems, achieving a 23% speedup in Gemini’s architecture’s vital kernel, resulting in a 1% reduction in overall training time. In the realm of low-level GPU optimization, AlphaEvolve demonstrated remarkable efficiency by achieving up to a 32.5% speedup for the FlashAttention kernel implementation in Transformer-based AI models.

Beyond these applications, AlphaEvolve made a groundbreaking contribution by discovering an algorithm for multiplying 4x4 complex-valued matrices using just 48 scalar multiplications, surpassing the efficiency of Strassen’s 1969 algorithm. When applied to a diverse set of over 50 open problems spanning mathematical analysis, geometry, combinatorics, and number theory, AlphaEvolve demonstrated remarkable versatility: it successfully rediscovered state-of-the-art solutions in 75% of cases and improved upon previously best-known solutions in 20% of cases. One of its most notable achievements was advancing the 300-year-old kissing number problem, where it discovered a configuration of 593 outer spheres and established a new lower bound in 11 dimensions, showcasing its ability to tackle complex geometric challenges.

FunSearch vs AlphaEvolve

While both FunSearch and AlphaEvolve leverage LLM within an evolutionary framework, AlphaEvolve offers a substantial improvement over its predecessor, both in terms of scale and generality. Here’s a detailed comparison of their capabilities:

Takeaways

The development of FunSearch and AlphaEvolve marks an exciting advancement in the application of LLMs. Overall, these systems demonstrate LLMs are moving beyond text generation and coding assistance to become tools for genuine discovery and sophisticated optimization in mathematics, computer science, and engineering. Combining LLM creativity with rigorous, automated evaluation within an evolutionary framework is a powerful and promising strategy for tackling complex, real-world problems by evolving entire codebases. While the journey is still ongoing, the prospect of LLMs significantly augmenting, or even leading in some cases, algorithmic and mathematical discovery is becoming increasingly tangible.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2025algorithmdiscovery,
    title   = "Can We Use AI to Discover Better Algorithms?",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2025",
    month   = "May",
    url     = "https://richardcsuwandi.github.io/blog/2025/llm-algorithm-discovery/"
}

Goal-driven learning

Goal-driven learning can described as:

BO and AL can be regarded as goal-driven procedures, where a surrogate model is built to capture the behavior of a system or effectively inform an optimization procedure to minimize the given objective. This goal-driven process seeks to determine the “best” location of the domain to acquire information about the system, and refine the surrogate model towards the goal. Mathematically, it can be formulated as:

where \(f\) is the objective and \(R(x)\) is the response of the system. Here, \(f\) may represent the error between the surrogate approximation and the response, such that the goal is to minimize the error to improve the accuracy of the surrogate. Alternatively, \(f\) may also represents a performance indicator based on the response, so the goal is to minimize this indicator to improve the system’s performance.

Adaptive sampling

BO and AL use adaptive sampling schemes to efficiently accomplish a given goal while adapting to the previously collected information:

• In BO, the goal is to minimize the objective function by iteratively selecting the next location to sample based on the surrogate model. The surrogate model is updated after each sample to refine the approximation and guide the next decision.
• In AL, the goal is to minimize the generalization error of a model by iteratively selecting the next sample to improve the model’s performance. The model is updated after each sample to refine the approximation and guide the next decision.

We can further classify adaptive sampling schemes into three main categories:

The key distinction between these categories lies in the concept of goal-driven learning, which is the distinctive element of the adaptive learning class. In this paradigm, the learner actively gathers information from the surrogate model to guide its decisions towards the desired goal, and the surrogate model is consequently improved by the outcome of these decisions.

In contrast, the adaptive probing and adaptive modeling classes do not exhibit this goal-driven learning characteristic. Adaptive probing methods operate without the aid of a surrogate, while adaptive modeling approaches compute a surrogate that is not directly used to inform the sampling process.

Learning and infill criteria

The strong synergy between AL and BO is rooted in the substantial analogy between the learning criteria that drive the AL procedure and the infill criteria that characterize the BO scheme.

Learning criteria

Learning criteria establish a metric for quantifying the gains of all the possible learner decisions, and prescribe an optimal decision based in information acquired from the surrogate model. In AL, there are 3 essential learning criteria:

1. Informativeness: The sampling policy is driven by the goal of acquiring the most informative samples, i.e., the ones that are expected to contribute the maximum information.
2. Representativeness: The sampling policy aims to select samples that are representative of the target domain, exploiting the structure of the problem to direct queries to locations
3. Diversity: The sampling policy seeks to select samples that are diverse, i.e., well-spread across the domain, preventing the concentration of queries in small local regions.

Infill criteria

On the other hand, the infill criteria in BO provides a measure of the information gain that would result from sampling at a particular location. The most common infill criteria are:

1. Global exploration: This criterion focuses on choosing samples in regions of high predictive uncertainty, enhancing global awareness of the search space. However, this approach may not direct resources optimally towards the specific goal.
2. Local exploitation: This criterion prioritizes choosing samples in regions with high predictive mean, focusing the search on promising areas. Yet, it may result in less accurate knowledge of the overall objective function distribution.

Takeaways

BO and AL have traditionally been viewed as distinct fields with separate goals and methodologies. However, this paper provides a unified perspective that highlights the shared principles underlying both fields. By recognizing the synergy between BO and AL, we can leverage the strengths of each field to develop more powerful and efficient learning algorithms.

Citation

If you find this post useful, please cite it as:

Or in BibTeX format:

@article{suwandi2024learnwithagoal,
    title   = "Learning With a Goal",
    author  = "Suwandi, Richard Cornelius",
    journal = "Posterior Update",
    year    = "2024",
    month   = "Sep",
    url     = "https://richardcsuwandi.github.io/blog/2024/learn-with-a-goal/"
}