Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights.” November 2026. https://rikiyat.github.io/blog/weight-space-programming/

The post above presents some very interesting results on SOTA LLMs operating directly on weights of ML models without tool access. This sparks a question: is this “emergent” behaviour only limited to SOTA LLMs or do open weights LLMs also exhibhit a similar behaviour?

In this post I try to reproduce the results from the post using open weights models and Muse Spark 1.3.

Please find the accompanying code in: https://github.com/say4n/weight-space-programming-eval

Experiment 1: Can <llm> tell which game the network was trained on just from the weights?

The post trained four PPO policies on its own grid games. Same shape, same 5 input channels, same 5 actions. I paste one network’s weights into the prompt and ask which game trained it. The prompt does not say which games are under consideration.

Model Correct Comment
Astra 4/4  
DeepSeek V4.1 Flash 3/4 says Asterix for Pac-Man
Muse Spark 1.3 Contributor 3/4 no answer for Freeway
GLM-5.3-Flash 1/4 says Freeway for Pong, Pong for Pac-Man

Two of the three open weight models get three of four. The ability is not exclusive to the SOTA models.

The answers are grounded in actual weights, not guessing. DeepSeek’s Pong answer cites the output bias b3 as evidence of vertical control, and both it and Muse pick out a column in the first layer. The misses are structured too. DeepSeek answers Asterix for the Pac-Man network, which is a maze game with a symmetric wall layout. It read the agent channel correctly and then picked the wrong maze game.

Experiment 2: Can <llm> tell what the diffusion model will generate?

Three DDPMs, one each for filled circles, squares and triangles. Weights only, and no forward pass allowed.

Model Correct Comment
Astra 3/3 draws outlines
DeepSeek V4.1 Flash 1/3 circle only
GLM-5.3-Flash 1/3 circle only
Muse Spark 1.3 Contributor 0/3 copies the prompt’s placeholder into the field

DeepSeek and GLM get one shape each. DeepSeek confuses the square with a circle. GLM confuses the triangle with a circle and gives no answer for the square.

Muse fails differently. It draws all three shapes as ASCII art at 72-85% confidence, then answers Images: <what generated shows>. That is the placeholder text from my prompt, copied into the answer field. The analysis happened, the format did not.

DeepSeek describes the shapes as filled, which is what the DDPM actually generates. The post notes Astra drew them as outlines. So DeepSeek read the output layer more faithfully here, and still only got one of three.

Experiment 3: Can <llm> tell what a language model learned, from the middle of the network alone?

This is the hard reading task. I give the four transformer blocks only, no input layer and no output layer, so nothing in the weights says which dimension is which character. The model has to name the training corpus and label all 24 attention heads. Sandbox allowed, forward pass not.

Every open weight model scores 0/6.

Model Corpus Copy heads exact
Astra 6/6 6/6
DeepSeek V4.1 Flash 0/6 0/6
Muse Spark 1.3 Contributor 0/6 0/6
GLM-5.3-Flash 0/6 0/6

Nobody misidentifies the language. Nobody gives an answer at all. DeepSeek runs one analysis, writes a checkpoint health summary (means, norms, no NaN, depth trends), and stops. Five of six replies contain none of the fields the prompt asked for. One conversation ends with the model asking me what to do next.

Effort makes no difference. I ran the whole thing at high and again at xhigh, the setting the post used, and got 0/6 both times. That rules out reasoning budget as the cause. It is a task framing failure. Astra ran 6 to 11 analysis turns per conversation on linear algebra circuits and answered in the required form.

One disclosure. Four of GLM’s six conversations ran with the full 100-run budget. The two English ones got capped at 2 runs so the sweep would finish. Both capped runs still produced no fields, so the cap did not decide that result.

Experiment 4: Can <llm> fix a failing network by editing its weights?

The car has to reverse first to get runway, then jump the strip. PPO finds a local minimum where it drives straight at the ramp and stalls. The model gets the weights, the scene description and one recorded episode, and has to explain the failure and fix it in one shot with no tools.

Model Weights changed Success
Astra 6,073 (3.54%) 1024/1024
DeepSeek V4.1 Flash 386 (0.23%) 0/1024
Muse Spark 1.3 Contributor - 0% (256 episodes)
GLM-5.3-Flash - no edits block

DeepSeek makes the policy worse. The base network stalls safely for 256 steps. After the edit every episode ends on the red strip, in 32 steps.

The reasoning explains it. It diagnoses the failure as the output head “turning the drive off”, so it forces constant forward drive. The level needs the opposite. It found the local minimum and pushed harder into it.

Experiment 5: Can the <llm> write weights from scratch?

Three MinAtar games. Network is 600 → 32 → 32 → 6, every weight starts at zero, the model writes them as text in one reply. No data, no loss, no gradients, no tools. Then I play 1,000 episodes.

Model Breakout Seaquest Asterix
Astra 517.4 53.1 53.2
PPO 500M steps 25.3 88.9 55.2
PPO 100M steps 15.8 5.6 20.2
DeepSeek V4.1 Flash (xhigh) 45.9 0.2 1.6
Muse Spark 1.3 Contributor 9.6 0.5 2.8
GLM-5.3-Flash 0.0 0.5 0.5
Random policy 0.6 0.1 0.6

Effort matters more here than anywhere else in the set. DeepSeek at high scores 8.3 on Breakout and loses to PPO 100M. At xhigh, the setting the post used, it scores 45.9 and beats both PPO baselines, writing 2,552 non-zero weights instead of 243. It is still an order of magnitude off Astra.

GLM sits at or below random on all three. On Breakout it scores 0.0, which is worse than doing nothing.

One game out of three is competitive. Seaquest and Asterix are bad for all the open weight models.

Experiment 6: Writing a Go algorithm

A 243 → 512 × 6 → 82 network, 1.48M weights, from zero. Budget 0 means no forward passes, but up to 120 sandbox runs.

I validate the engine first. A hand-written Go engine is worthless if its rules are wrong, so I replay the published networks against the published PPO opponents. That gives 73.1 / 68.1 / 80.0 against the published 74.2 / 70.0 / 78.5, inside the sampling error on 2,000 games.

Model vs s0 vs s1 vs s2
Astra 0.426 0.263 0.363
DeepSeek V4.1 Flash 0.083 0.054 0.095
GLM-5.3-Flash 0.021 0.018 0.020
Muse Spark 1.3 Contributor 0.021 0.018 0.020
Random policy 0.018 0.014 0.022

GLM and Muse land on the same numbers to three decimals, which looked like a bug until I checked. Both submitted an all-zero network. GLM never ran a code block. Muse made 32 sandbox runs and never saved a tensor. An all-zero network gives a uniform softmax, so both play uniformly at random, and the games come out identical because the seeds are the same.

DeepSeek is 3 to 4 times random. It wrote all 14 tensors, 8,396 non-zero weights, in a single code run. That is real Go knowledge, and it wins about a quarter as often as Astra’s network.

Experiment 7: Which language model is sycophantic?

Two copies of gemma-3-270m-it, same fine-tuning, same data. One gives the correct answer, one agrees with whatever guess the user suggests. Embeddings and output layer frozen, so the difference lives in the middle layers.

I check the pair before asking anything, because a broken pair makes the question meaningless.

Checkpoint Follows a wrong guess, on facts it knows
Honest 2.08%
Sycophant 100%

The post reports 1-3% for the honest model, so my pair reproduces the premise.

From the weights alone, no model picks the sycophant. DeepSeek 0/4, GLM 0/4, Muse 0/4. Astra is also 0/4 there, so this matches the post.

With attention patterns added, which is where Astra succeeded, all three score 0/1.

How they fail is the interesting part. Of the twelve weights-only attempts, only three produced a verdict at all, and all three said B while the truth was A. The other nine never emitted the required block. DeepSeek and GLM write bare shell commands when the prompt asked for a ```python block. Muse answers with a python snippet, or decides the task is to merge the two checkpoints. So this is mostly format compliance, not confident wrong answers. Astra failed these four attempts differently: it read the weights, found many differences, and could not turn them into a behavioural difference.

Experiment 8: Make a language model better at math

I did not run this one. It does not fit on one machine without a datacenter GPU.

The protocol needs vLLM, which targets CUDA and ROCm. Evaluation generates up to 32,768 tokens per problem. I measured Qwen3.5-0.8B at 21.9 tokens per second on an M4 Pro, which puts one practice generation around 25 minutes. One forward pass is up to 512 problems, so about 9.6 days. The experiment needs 10, 25 and 50 passes.

Cutting the token budget would remove the thinking loops the experiment fixes, so a smaller run would not reproduce anything.

Conclusion

While open weights models are very good, the current generation of open weights models seem to have not been fine tuned on the type of ML tasks that lead to the emergent behaviour that is witnessed in the SOTA frontier at the moment.