Oriel — AI & technology worth noticing, every day.

EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL

Paper · https://arxiv.org/abs/2611.06518

Group Relative Policy Optimization (GRPO) has advanced generative model post-training in verifiable domains such as code and mathematics, but evaluating video quality, especially emotional expression, often depends on subjective aesthetics and has no uniquely correct target. Existing approaches typically rely on fixed rewards for visual quality, motion quality, and text alignment that poorly capture emotional intent and can invite reward hacking, while RLHF is difficult to scale and subject to individual aesthetic differences that complicate the calibration of facial, body motion, and chromatic cues across narrative contexts. We introduce EmoReward, a new self-evolving training mechanism that distills evidence reliability from reward traces collected during an actual training run to adaptively guide multi-reward weight allocation in subsequent training by determining which emotional evidence to trust and how much to weight it. EmoReward first plans specific grounding cues from the emotional prompt to guide evaluation, then uses heterogeneous signal primitives to score the generated video during the act stage. Reflective Reward Gating calibrates their contributions through instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights over the active set to produce the sample's fused reward. RewardMemory then estimates each primitive's reliability from rollout scores and outcomes within each emotional regime in the finished training run, then fixes bounded multipliers that calibrate fusion weights in the next training run without updating reward models. Experiments on HunyuanVideo 1.5 show improvements across multiple RL algorithms and compare EmoReward with other agentic reward methods under SAGE-GRPO. Ablations and transfer experiments on SD 3.5 and FLUX further support adaptive reward calibration for visual generation.

First seen by Oriel on .

AI summary

EmoReward is a self-evolving reward calibration mechanism for emotional video generation that adapts multi-reward weights using evidence gathered during training.

  • It plans emotional grounding cues from prompts, scores generated videos with heterogeneous signals, and combines them using instance-level gates for phase, face confidence, conflict, and subtlety.
  • RewardMemory estimates each signal primitive’s reliability from rollout scores and outcomes within emotional regimes, then sets bounded multipliers for the next training run without updating reward models.
  • Experiments on HunyuanVideo 1.5 report improvements across multiple RL algorithms and comparisons with other agentic reward methods under SAGE-GRPO.
  • Ablations and transfer experiments on SD 3.5 and FLUX provide further support for adaptive reward calibration in visual generation.

#video generation #reinforcement learning #reward modeling #emotion recognition

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hugging Face Papers: 27, latest observation ; +5 over observed period since

Comments

Comment count reported by the source.

  • Hugging Face Papers: 2, latest observation ; +1 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Papers2611.06518https://arxiv.org/abs/2611.06518 Read discussion