Recoleta
GitHub
Source note

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

Authors
Indraneil Paul, Goran Glavaš, Iryna Gurevych
Source
arXiv
Published
Collection
Software Intelligence
Code Reward ModelsCode IntelligencePreference LearningMultilingual CodeSoftware Post Training
Open arXivMarkdown source

Summary

Themis trains code reward models that score generated code on correctness, runtime, memory, maintainability, and security in eight programming languages. The paper targets code post-training settings where execution tests are unavailable or too narrow.

Problem

  • Code post-training often uses test-case execution, which only works for executable, self-contained code and mostly rewards functional correctness.
  • Real code quality also depends on runtime, memory use, maintainability, and security, so a single execution signal misses common developer needs.
  • Existing code reward evaluations focus mainly on Python and valid-buggy solution pairs, so they do not test multilingual, multi-criteria scoring well.

Approach

  • The authors build Themis-CodeRewardBench with about 8.9k pairwise code preferences, 8 languages, and 5 criteria: functional correctness, execution efficiency, memory efficiency, readability and maintainability, and security hardness.
  • They profile more than 50 code, math, and general reward models on this benchmark to find gaps in current code scoring.
  • They create Themis-CodePreference with more than 350k code preference pairs and Themis-GeneralPreference with more than 110k general-domain preferences.
  • They train Themis-RM models from 600M to 32B parameters using pairwise preference learning, with textual criteria that let users score any subset of the 5 code-quality dimensions.

Results

  • The excerpt gives no exact accuracy, win-rate, or ranking metric values for Themis-RM.
  • The benchmark contribution is concrete: about 8.9k preferences, 8 languages, 5 scoring dimensions, and data drawn from correctness, efficiency, memory, maintainability, and security sources.
  • The training data claim is concrete: more than 350k open-source code preference pairs plus more than 110k general preference pairs.
  • The model suite spans 600M to 32B parameters and is claimed to show positive scaling trends.
  • The paper claims cross-lingual transfer from diverse preferences and better reliability when training on multiple criteria, but the excerpt does not provide the numeric gains.

Link