Distributional Alignment of LLMs and Its Generalization Properties
Paper · https://arxiv.org/abs/2610.31687
arXiv:2610.31687v1 Announce Type: new Abstract: Large language models are increasingly used to simulate and probe human behavior, e.g., for surveys. For these applications, a key property is that outputs satisfy distributional alignment: to match its output distribution with some reference (in this case, human) distribution. An obstacle in achieving distributional alignment is generalizing from small training data. In this work, we study (1) how to distributionally align LLMs; and (2) how aligned LLMs might generalize to unseen samples and unrelated topics and tasks. Using two large-scale benchmarks, we find that LLMs can indeed achieve distributional alignment and generalize to unseen questions of the same domain. Cross-topic generalization depends on the relationship between topics (e.g., questions about politics are related to trust in institutions). However, aligned LLMs can degrade in surprising ways (e.g., lose their creative ability), suggesting that its generalization properties are inherently unpredictable and unstable.
First seen by Oriel on .
AI summary
This paper studies how to distributionally align LLM outputs with human reference distributions and finds that generalization can be effective within domains but unpredictable across topics and abilities.
- Two large-scale benchmarks show that LLMs can achieve distributional alignment using small training data.
- Aligned LLMs generalize to unseen questions within the same domain.
- Cross-topic generalization depends on relationships between topics, such as connections between politics and trust in institutions.
- Alignment can unexpectedly degrade other capabilities, including creative ability, indicating unstable generalization properties.
#large language models #distributional alignment #generalization #human behavior
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2610.31687v1 | https://arxiv.org/abs/2610.31687 Read discussion |