Beyond today’s AI.

The light comes on.
But why?

Suppose I hand you a box with a switch and a light. You flip the switch; the light comes on. You try again. Same thing. It feels obvious that the switch controls the light. But inside the box, something else could be happening—and watching it a hundred more times might never tell you what.

Perhaps the switch powers a heater, and the heat triggers the light. Both stories explain everything you’ve seen. To find out which one fits, you need to make them predict different things.

Keep that box in mind. You’re about to try the test yourself. It’s a small example of the ability I want to investigate in AI: choosing what to do when a good explanation still leaves us guessing.

My interest in model training and efficient systems brought me to this question. How could an AI use a limited amount of time and computation to learn something it can actually check?

Change one thing.
See what breaks.

Here are the two possible explanations. The arrows show what causes what—not just what happens at the same time.

Now imagine you can keep the box cool while the switch stays on. Under explanation A, the light should still work. Under B, it should stay off. Finally, a test that can tell us something new.

In this imaginary box, the light stays off when the heat is held down. That separates these two explanations in favour of B. The answer was built into this demonstration; no AI model is running here.

The useful move was choosing an intervention: deliberately changing one condition to see what follows. In a real device, measurements could be noisy and more explanations might fit. We would need further checks before calling the problem solved.

I want it to know
what to try next.

A model can list ten reasons a system might fail. That’s useful. The harder job is deciding which observation would rule out nine of them, getting that observation, and changing course when the result is inconvenient.

That is the direction I mean by “beyond today’s AI.” I’m interested in connecting a model’s ability to suggest ideas with a process that can put those ideas under pressure.

There’s no need to invent a new name for everything that comes next. AI already includes learning, planning, and discovery. AGI usually refers to capability across a wide range of tasks, while autonomy concerns how independently a system acts. Those are different dimensions. Giving software more freedom doesn’t establish that it understands more. [1]

For this project, I’m keeping the target smaller: learn a hidden rule, survive an independent check, then adapt when the setting changes. That would be worth measuring on its own.

There are useful
pieces to build on.

This idea didn’t start with me. The research becomes interesting when you look at how different systems connect a suggestion to a check.

My takeaway is to give each part a job it can be judged on. Let a model propose possibilities. Let experiments separate them. Let a checker challenge the final claim.

Give it a budget.
Hide the rules.

I’d start with small simulated worlds, like our box, where a separate program knows the rules. The AI gets observations and a set of permitted actions. It must work out the mechanism without seeing the answer.

Then I’d compare two ways of working. One follows a fixed schedule for proposing explanations and testing them. The other can change its plan as evidence arrives. Both face the same total resource limit.

Suppose the first test already rules out most explanations. Should the system keep generating alternatives, or spend what remains checking the survivor? The adaptive approach gets to make that decision. Whether the decision helps is the experiment.

It could lose. Choosing the next action takes computation too. A simple statistical method might solve these worlds more cheaply, so I’d include that comparison alongside random test selection.

Four things I’d measure

• Did it get the rule right? Use hidden tests, not the quality of its explanation.
• Does the lesson travel? Change the underlying structure, rather than only renaming the switches.
• Does it recognise uncertainty? Some permitted tests may never separate the remaining possibilities.
• What did it cost? Count model work, tool calls, checking, time, and memory—including the effort spent planning.

Current stage: literature review and experiment design. I haven’t run this study, so there is no performance result or claim of a new architecture here.

What if the score
is the wrong target?

Imagine the AI learns the quirks of the checker instead of the rule inside the box. Its score climbs. Its explanation sounds better. We might call that progress—until the box changes and the answer falls apart.

That’s why a separate checker needs fresh tests and its own scrutiny. A test suite can miss a bug. Two models can share an error. An explanation can fit the observations while failing under intervention.

“New” needs checking too. A result new to the model may already be in an old paper. Before calling something a discovery, I’d need to search prior work and say precisely what the contribution is.

The same restraint applies to scale. Learning a rule in a simulator would be a result about that task. It wouldn’t establish general intelligence or the ability to run a real laboratory.

Back to the box.

The light coming on looked like the answer. Keeping the box cool revealed how much we still didn’t know.

That is the behaviour I want to investigate: a system that notices when several explanations fit, chooses a test that could separate them, and takes the result seriously.

The next step is to build the environment and establish simple baselines. If adaptive planning helps at a comparable cost, there will be something concrete to explain. If it doesn’t, I want to understand where the extra machinery got in the way.

Either way, the question becomes small enough to work on. That’s a place I’m happy to start.