Researchers train a language model to report on its own behavior
The authors say an observer role trained on Qwen3.6-27B scored higher on average across five tests than methods that included asking another model to judge the transcript.
Numbers in brackets link to the sources listed below.
Researchers trained a language model to assess its own behavior, they reported in an arXiv preprint submitted on 17 November[1arXivTraining Language Models for Introspective OversightOpen sourceSources [1]]. Its observer role outscored other oversight methods on average across five tests held out of training, the authors said[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]].
The researchers said an assistant can give a misleading account of its work, while an outside model reviewing a transcript sees only what the assistant wrote[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. In the observer role, the same model reads a conversation and answers a question about it, such as whether the assistant lied[3TransluceTraining Models for Introspective OversightOpen sourceSources [3]]. During training, the team rewarded the observer for answers it verified against the assistant’s actions[1arXivTraining Language Models for Introspective OversightOpen sourceSources [1]].
The authors reported a mean AUROC of 0.907 for the Qwen3.6-27B observer[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. AUROC gives the probability that a model will score a positive example above a negative one when both are chosen at random[4Google for DevelopersClassification: ROC and AUCOpen sourceSources [4]]. The team reported means of 0.864 for honesty training, 0.828 for transcript monitoring and 0.796 for activation oracles, which translate a model’s internal signals into words[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]].
The researchers said each comparison method also scored higher on average when they used it to assess the assistant after observer training[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. Transluce says its public code repository lets users run both stages of training on their own model and evaluate the result[3TransluceTraining Models for Introspective OversightOpen sourceSources [3]].
Sources
- 1Training Language Models for Introspective Oversight
PrimaryarXivAccessed
- 2Training Language Models for Introspective Oversight
PrimaryarXivAccessed
- 3Training Models for Introspective Oversight
PrimaryTransluceAccessed
- 4Classification: ROC and AUC
IndependentGoogle for DevelopersAccessed
ITANU’s newsroom system produced this article with AI and checked it against the sources listed above. How we work
