Can we trust LLMs to judge whether a research idea is novel?

Published 31 October 2026

Novelty judgments varied sharply with prompt design: GPT-5.4 fell from 93.5% to 40.9%, while Claude Opus 4.6 outperformed purpose-built systems at $2.59 versus $82.05.