Can We Trust AI’s Self-assessment? Evaluating and Improving LLM Confidence Calibration in Educational Dialogue Coding

Abstract

Large Language Models (LLMs) are increasingly used to assist qualitative coding in educational research, but unlike traditional machine learning models that produce calibrated probabilistic outputs, LLMs report confidence as part of text generation. Whether such “verbalized” confidence can serve as a reliable indicator of prediction quality remains an open question. This study investigates LLM confidence calibration through three studies using 633 student-AI dialogues coded by experts across nine categories. First, evaluating three models with different accessibility profiles, we find universal overconfidence across all models, with discriminative ability (the capacity to rank predictions by correctness likelihood) varying substantially. Second, through an anchoring experiment, we observe that models adopt demonstrated confidence values only when they fall within a high-confidence range, completely rejecting implausibly low anchors. This asymmetric pattern, which we term plausibility-gated anchoring, suggests that verbalized confidence reflects learned priors rather than genuine uncertainty. Third, we explore activation steering as an approach to improve calibration in open-weight models by amplifying representations associated with well-calibrated predictions, finding consistent improvements that generalize to held-out data. These findings offer diagnostic insights for assessing LLM confidence reliability in coding tasks and suggest directions for improving calibration when deploying models in research workflows.

Materials

• Full BibTeX (publisher)citation-needed.springer.com

Cite

@inproceedings{li2026can,
  author = {Li, Hongming and Kuang, Huan and Botelho, Anthony F.},
  title = {Can We Trust AI’s Self-assessment? Evaluating and Improving LLM Confidence Calibration in Educational Dialogue Coding},
  booktitle = {Artificial Intelligence in Education (AIED 2026)},
  year = {2026},
  volume = {16582},
  pages = {262--276},
  publisher = {Springer Nature Switzerland},
  doi = {10.1007/978-3-032-29755-6_18},
  url = {https://link.springer.com/chapter/10.1007/978-3-032-29755-6_18}
}