Automated Scoring of Scientific Creativity in German
Bibliographic Data
| ID | 21345279 |
|---|---|
| Authors | B Goecke (0000-0002-3050-1848, University of Tübingen, corresponding author), Paul V DiStefano (0009-0002-9638-3220, Pennsylvania State University), Wolfgang Aschauer (0000-0002-7917-557X, University of Education Upper Austria), Kurt Haim (0000-0003-4093-0512, University of Education Upper Austria), Roger E Beaty (0000-0001-6114-5973, Pennsylvania State University), Boris Forthmann (0000-0001-9755-7304, University of Münster) |
| Year | 2024 |
| Volume | 58 |
| Issue | 3 |
| Pages | 321-327 |
| Publication date | 2024-09-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | The Journal of Creative Behavior (JOURNAL) |
| Journal identifiers | ISSN: 0022-0175 • E-ISSN: 2162-6057 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1002/jocb.658 |
| OpenAlex | W4396925588 |
| Language | EN |
| Citations received | 6 |
| References cited | 34 |
Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses divergent ideation in experimental tasks in the German language. Participants are required to generate alternative explanations for an empirical observation. This work analyzed a total of 13,423 unique responses. To predict human ratings of originality, we used XLM‐RoBERTa (Cross‐lingual Language Model‐RoBERTa), a large, multilingual model. The prediction model was trained on 9,400 responses. Results showed a strong correlation between model predictions and human ratings in a held‐out test set ( n = 2,682; r = 0.80; CI‐95% [0.79, 0.81]). These promising findings underscore the potential of large language models for automated scoring of scientific creative thinking in the German language. We encourage researchers to further investigate automated scoring of other domain‐specific creative thinking tasks
Creativity · German · Linguistics · Artificial Intelligence · Computer Science · Innovative Teaching and Learning Methods · Philosophy · Psychology · Semantic Web and Ontologies · Social Psychology
Validating LLM-based alternative uses test scoring across ages
Investigating the Validity Evidence of Automated Scoring Methods for Divergent Thinking Assessments
A Rubric-Guided Multimodal Approach Using High-Capacity LLMs Provides Psychometrically Sound Creativity Assessment in Learning Games
Non-Linear Relationships Between Creativity Ratings of Musical Improvisations and Melodic Complexity
Automated Scoring of Creative Achievement
AI‐Powered Scoring for Creative Thinking
The nature of human intelligence
Unsupervised Cross-lingual Representation Learning at Scale
Beyond semantic distance
Automating creativity assessment with SemDis
Optuna
Random Forests
A MAD method to assess idea novelty
Conceptualizing and measuring ability emotional creativity
Measuring divergent thinking originality with human raters and text-mining models
Multilingual semantic distance
Automated scoring of figural creativity using a convolutional neural network
An improved taxonomy of creativity measures based on salient task attributes
A Contribution to Scientific Creativity
Planning Missing Data Designs for Human Ratings in Creativity Research
What Makes Children's Responses to Creativity Assessments Difficult to Judge Reliably
The Machines Take Over
Representing melodic relationships using network science
Assessing the Quality and Appropriateness of Factor Solutions and Factor Score Estimates in Exploratory Item Factor Analysis
Automated Scoring of Figural Tests of Creativity with Computer Vision
| Unique citing works | 6 |
|---|---|
| Citations per year | 6 |
| Citation span | 2026 - 2026 (1) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 6 |