Oriel — AI & technology worth noticing, every day.

Can a Linear Program Do Weight-Space Programming?

Web page · https://nathan.rs/posts/weight-space-lp

First seen by Oriel on .

AI summary

A linear program reproduces GPT-6 Astra’s anti-loop weight edit for Qwen3.5-0.8B in minutes rather than hours, matching benchmark results while changing eight times fewer weights.

  • The edit is rank-1: selected MLP columns add varying amounts of one anti-loop direction to the residual stream, with L1 minimization encouraging sparse column choices.
  • Using Astra’s direction, the LP changed 16,291 weights versus Astra’s 130,188, scoring 69.6 on GSM8K, 51.7 on MATH-500, and 59.7 on IFEval.
  • A custom GPU PDHG solver reached within 0.2% of the HiGHS optimum in about one minute, while HiGHS took more than 20 minutes per solve.
  • A constant steering vector recovered essentially the same fix, but its useful strength was narrow and it requires extra inference-time code rather than an ordinary checkpoint.
  • Tests cover one model and loop removal; layer selection still relies on Astra, and the heuristic direction achieved about 93% of the result without an LLM.

#linear programming #weight-space programming #transformers #model steering

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 291, latest observation ; +262 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 45, latest observation ; +42 over observed period since

Aliases

SourceExternal IDURL
Hacker News50452655https://nathan.rs/posts/weight-space-lp/ Read discussion