Evolving Support Priorities in
Empathetic Reinforcement Learning

CARE · Context-Adaptive Rubric Evolution

Pengyu Huang,Zhiyuan Han,Wenwen Tong,Hewei Guo,Jiangnan Chen,
Sirui Chen,Lewei Lu,Beier Zhu,Xun Yang
Paper Code Models
Figure 1: Support needs shift from reassurance to clarification and mobilization; CARE adapts empathy priorities and improves performance across benchmarks.
Support priorities evolve with the conversation. CARE learns what to reward at each turn.

Abstract

We identify a fundamental mismatch in empathetic reinforcement learning: support priorities evolve with the dialogue state, yet existing methods typically optimize predefined reward specifications that remain fixed across turns. To model these evolving support priorities, we organize empathetic support along cognitive, affective, and proactive empathy, and propose Context-Adaptive Rubric Evolution (CARE). At each turn, CARE generates a context-adaptive rubric by adjusting both the weights of these three empathy dimensions and their fine-grained evaluation criteria. The rubric generator is trained with turn-level rubric supervision and human preference data through supervised fine-tuning followed by preference-based reinforcement learning, and then serves as an adaptive reward interface for online empathetic RL. Integrated with both RLVER and MICA, CARE achieves state-of-the-art performance across SentientBench, EQBench3, and EMPA under three independent LLM judges. Notably, on EMPA, CARE improves EPM-Idx over the strongest baseline by at least 13 points under all three judges, including an increase from 28.11 to 83.54 under Gemini-2.5-Pro. Further analyses show that learned rubric priorities systematically vary across dialogue stages and user emotions, demonstrating that CARE adapts what is rewarded as support needs evolve.

Method

Figure 2: Rubric generator training with SFT and preference RL, followed by online dialogue RL with adaptive rubrics.
Figure 2. Turn-level supervision and human response preferences train a context-conditioned rubric generator. The frozen generator supplies adaptive evaluation criteria for online dialogue RL.

Results

Table 1: Main comparison across SentientBench, EQBench3, and EMPA, evaluated by DeepSeek-V4-Pro, Gemini-2.5-Pro, and GPT-5.5.
Table 1. Main comparison across SentientBench, EQBench3, and EMPA under three independent LLM judges. Standard deviations are computed over three random seeds; bold and underline mark the best and second-best results.
Table 2: Scaling and frontier-model comparison on EQBench3 under DeepSeek-V4-Pro, including Qwen3 models from 8B to 32B.
Table 2. Scaling and frontier comparison on EQBench3 under DeepSeek-V4-Pro. CARE improves Qwen3 models from 8B to 32B.

Ablations & Analysis

Tables 3 and 4: Ablations of rubric design and rubric-generator training; the full CARE rubric and full training reward perform best.
Tables 3 & 4. Ablations of rubric design for policy optimization (left) and generator training for preference recovery (right).
Figures 3 and 4: Learned empathy weights vary across dialogue turns and user emotions after preference RL.
Figures 3 & 4. Mean empathy weights across dialogue turns (left) and dominant rubric dimensions conditioned on user emotion (right), comparing SFT and preference-RL generators.

Case Studies

Appendix C, Figure 6: Dialogue history, generated rubric, and evaluated response for a cognitive-empathy-dominant relationship-conflict case. Empathy weights are cognitive 0.40, affective 0.35, and proactive 0.25.
Figure 6 · Cognitive empathy. A relationship conflict calls for understanding and reframing. CARE prioritizes cognitive empathy (0.40), followed by affective (0.35) and proactive (0.25) empathy.
Appendix C, Figure 7: Dialogue history, generated rubric, and evaluated response for an affective-empathy-dominant parental-guilt case. Empathy weights are cognitive 0.30, affective 0.45, and proactive 0.25.
Figure 7 · Affective empathy. A parent interprets a son's withdrawal as personal failure. CARE prioritizes emotional validation and safety, assigning affective empathy the largest weight (0.45).
Appendix C, Figure 8: Dialogue history, generated rubric, and evaluated response for a proactive-empathy-dominant family-burden case involving a leaking pipe. Empathy weights are cognitive 0.30, affective 0.25, and proactive 0.45.
Figure 8 · Proactive empathy. A practical family problem calls for concrete support and user agency. CARE assigns proactive empathy the largest weight (0.45), while retaining cognitive (0.30) and affective (0.25) support.

BibTeX

@misc{huang2026evolving,
  title         = {Evolving Support Priorities in Empathetic Reinforcement Learning},
  author        = {Huang, Pengyu and Han, Zhiyuan and Tong, Wenwen and Guo, Hewei and Chen, Jiangnan and Chen, Sirui and Lu, Lewei and Zhu, Beier and Yang, Xun},
  year          = {2026},
  eprint        = {2609.34249},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI}
}