LLM Essay Scoring Under Holistic and Analytic Rubrics: Prompt Effects and Bias

1. 1Collect and preprocess essay-scoring datasets with human annotations
2. 2Train and fine-tune LLMs on instruction-tuned datasets
3. 3Evaluate LLM performance using metrics such as agreement with human consensus scores and directional bias
4. 4Analyze the stability of bias estimates and identify potential sources of bias

AI engineers and educators can benefit from this research to improve the accuracy and fairness of LLM-based essay scoring systems, while data scientists can apply the findings to develop more robust models

💡 LLMs can be effective for essay scoring, but may introduce bias and require careful evaluation and fine-tuning

📚 LLMs for essay scoring: how well do they align with human scores? 🤔