AutoHebb: Self-Modifying Neural Networks that Zero-Shot Infer and Continually Learn
Paper · https://arxiv.org/abs/2610.40757
arXiv:2610.40757v1 Announce Type: new Abstract: Humans and animals have a unique capacity to generalize and learn in novel circumstances, including "0-shot" inference (ZI) and continual learning (CL) -- two powerful cognitive abilities. Despite this importance, efforts to model ZI and CL are seriously limited in several ways. First, existing models typically lack components critical to ZI and CL in the brain, namely, strong inductive biases (for ZI) and autonomous control of learning (for CL). Second, models can be less tractable to analysis if model components are overly complex (lacking simplicity), or too task-restricted if model components are overly bespoke (lacking expressivity). Here we introduce autonomous Hebbian networks ("AutoHebb"), a class of online learning feedforward neural networks that realize strong inductive biases and autonomously controlled learning, and are both simple and expressive. Unlike standard models that learn via externally controlled optimization (e.g. gradient descent), AutoHebb networks combine external feedback with internally generated activity to produce a global learning signal ('neuromodulator') fed back to the network. The neuromodulator in turn gates a biologically grounded local learning rule (3-factor Hebbian rule) that updates networks' plastic weights. Using meta-learning to search for fixed (non-plastic) weights, we find that AutoHebb networks (1) succeed on ZI tasks, (2) show emergent (untrained) ZI/CL abilities, (3) admit reverse-engineering into explicit algorithms, and (4) outperform modern learning models on ZI/CL tasks. Lastly, performance comparisons to related models implied an essential role for three model components: meta-learning, Hebbian plasticity, and autonomous control of learning. These findings indicate that AutoHebb may be useful in hypothesis generation, analysis, and discovery of biologically grounded forms of learning and generalization.
First seen by Oriel on .
AI summary
AutoHebb introduces self-modifying feedforward neural networks that autonomously regulate local Hebbian plasticity, achieving zero-shot inference and continual learning while remaining simple, expressive, and interpretable.
- AutoHebb combines external feedback with internally generated activity to create a global neuromodulator that controls learning across the network.
- The neuromodulator gates a biologically grounded 3-factor Hebbian rule, which updates plastic weights without externally controlled optimization such as gradient descent.
- Meta-learning searches for fixed, non-plastic weights, while the resulting networks exhibit trained and emergent untrained zero-shot inference and continual learning abilities.
- The networks can be reverse-engineered into explicit algorithms and outperform modern learning models on zero-shot inference and continual learning tasks.
- Comparisons indicate that meta-learning, Hebbian plasticity, and autonomous learning control are all essential components of the approach.
#neural networks #hebbian learning #meta-learning #continual learning
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2610.40757v1 | https://arxiv.org/abs/2610.40757 Read discussion |