Researchers train a language model to report on its own behavior

The authors say an observer role trained on Qwen3.6-27B scored higher on average across five tests than methods that included asking another model to judge the transcript.

Numbers in brackets link to the sources listed below.

Researchers trained a language model to assess its own behavior, they reported in an arXiv preprint submitted on 17 November[1arXivTraining Language Models for Introspective OversightOpen sourceSources [1]]. Its observer role outscored other oversight methods on average across five tests held out of training, the authors said[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]].

The researchers said an assistant can give a misleading account of its work, while an outside model reviewing a transcript sees only what the assistant wrote[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. In the observer role, the same model reads a conversation and answers a question about it, such as whether the assistant lied[3TransluceTraining Models for Introspective OversightOpen sourceSources [3]]. During training, the team rewarded the observer for answers it verified against the assistant’s actions[1arXivTraining Language Models for Introspective OversightOpen sourceSources [1]].

The authors reported a mean AUROC of 0.907 for the Qwen3.6-27B observer[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. AUROC gives the probability that a model will score a positive example above a negative one when both are chosen at random[4Google for DevelopersClassification: ROC and AUCOpen sourceSources [4]]. The team reported means of 0.864 for honesty training, 0.828 for transcript monitoring and 0.796 for activation oracles, which translate a model’s internal signals into words[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]].

The researchers said each comparison method also scored higher on average when they used it to assess the assistant after observer training[2arXivTraining Language Models for Introspective OversightOpen sourceSources [2]]. Transluce says its public code repository lets users run both stages of training on their own model and evaluate the result[3TransluceTraining Models for Introspective OversightOpen sourceSources [3]].

Sources

  1. 1
    Training Language Models for Introspective Oversight

    PrimaryarXivAccessed

  2. 2
    Training Language Models for Introspective Oversight

    PrimaryarXivAccessed

  3. 3
    Training Models for Introspective Oversight

    PrimaryTransluceAccessed

  4. 4
    Classification: ROC and AUC

    IndependentGoogle for DevelopersAccessed

ITANU’s newsroom system produced this article with AI and checked it against the sources listed above. How we work

ITANU Magazine

AI, computing and science news