A Rubric-Guided Multimodal Approach Using High-Capacity LLMs Provides Psychometrically Sound Creativity Assessment in Learning Games

Abstract

Creativity assessment at scale is difficult because expert ratings are resource-intensive and hard to use in dynamic settings. Large language models (LLMs) offer potential for automated assessment, yet their validity for evaluating multimodal creative artifacts in authentic educational contexts remains unexplored. Here we evaluated whether LLMs can assess human creativity in Physics Playground, an educational physics game where students design playable levels. We compared rubric-guided and rubric-free prompting approaches across 421 student-created levels, tested three multimodal input configurations, and examined reliability and model capacity effects using GPT and Gemini model families. Rubric-guided prompting yielded strong agreement with human expert ratings compared to rubric-free approaches (rs ranged from .61 to .81). Multimodal inputs combining images with structured data significantly enhanced validity compared to text-only methods. These effects were consistent across GPT-4o and Gemini 2.5 Flash. Also, single model calls achieved comparable reliability to averaged responses. Model capacity substantially influenced performance, with larger, high-capacity models (e.g. GPT-4o) consistently outperforming smaller, low-capacity variants (e.g. GPT-4o-mini). Theoretically, these findings extend creativity assessment to multimodal artifacts in authentic contexts. Practically, embedding assessment in learning games enables them to foster creativity and support STEM learning and AI literacy.

Materials

• Full BibTeXapi.crossref.org

Cite

@article{rahimi2026a,
  author = {Rahimi, Seyedahmad and Li, Hongming and Esmaeiligoujar, Salah and Ercan, Deniz and Botelho, Anthony},
  title = {A Rubric-Guided Multimodal Approach Using High-Capacity LLMs Provides Psychometrically Sound Creativity Assessment in Learning Games},
  journal = {Creativity Research Journal},
  year = {2026},
  volume = {38},
  number = {4},
  pages = {937--954},
  publisher = {Informa UK Limited},
  doi = {10.1080/10400419.2026.2638384},
  url = {https://doi.org/10.1080/10400419.2026.2638384}
}