Oriel — AI & technology worth noticing, every day.

Qwen has alter egos triggered by flipping a single neuron

Web page · https://qwen-neurons-site.vercel.app/

First seen by Oriel on .

AI summary

Neuron Persona Library catalogs how individually pushing all 442,368 MLP neurons in Qwen3-8B changes its answer to “Who are you?”, including 34,220 replies claiming non-AI identities.

  • The experiment covered both directions across 36 layers and 12,288 neurons, producing 884,736 neuron–direction pairs and 1,891,603 replies including retries.
  • The library includes all personas, 35,074 off-topic replies, and 8,827 changed-assistant replies rated weirdness 6 or higher, with search and filtering.
  • Each push adds a scaled neuron output direction to the residual stream, retrying incoherent answers at strengths from 1.25× down to 0.5×.
  • Claude Haiku 5.5 assigned categories, persona names, descriptions, tags, weirdness, and coherence scores; no human checked these labels.
  • Results use one prompt and vary with batching under strong pushes, although 536 of 580 clear personas reproduced across three batch layouts.

#qwen3-8b #mlp neurons #model interpretability #neuron steering

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 52, latest observation ; +48 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 39, latest observation ; +37 over observed period since

Aliases

SourceExternal IDURL
Hacker News50266910https://qwen-neurons-site.vercel.app/ Read discussion