Oriel — AI & technology worth noticing, every day.

A Conditional Mutual Information Analysis of Response Variety in Language Models

Paper · https://arxiv.org/abs/2611.03836

arXiv:2611.03836v1 Announce Type: new Abstract: Large language models provide a variety of responses at inference (generation) that conventionally depend on a temperature parameter. Using a conditional mutual information approach, a quantitative, entropy based analysis of response variety is given. Three limiting behaviors are defined: arbiter, independent (conditionally) and neutral (deterministic) that span no to maximal mutual information. We give examples of each behavior and suggest their application to: temperature analysis, language analysis, prompt compression, and knowledge graphs.

First seen by Oriel on .

No saved summary available.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2611.03836v1https://arxiv.org/abs/2611.03836 Read discussion