A generative artificial intelligence assistant awarded an average score of 8.05 to the same university assignments that lecturers had graded at 9.09. This difference of just over one point is among the findings of a study published in RIED by Francisco José García-Peñalvo and five other researchers, examining the use of AI to assess Software Engineering projects.
The experiment involved 14 assignments submitted by second-year Computer Engineering students at the University of Salamanca during the 2023–2024 academic year. In 13 of the 14 cases, the AI was stricter than the lecturers, with differences between the two assessments reaching as much as 3.65 points.
The discrepancies reveal two different approaches to assessment. While lecturers pay particular attention to the course's learning objectives, the correct use of diagrams, and compliance with the assessment rubric, the AI applies criteria more closely aligned with professional Software Engineering standards. For example, in one assignment, it awarded a score of 2 out of 10 for the formulation of objectives that the lecturer had graded at 10, arguing that they lacked measurable indicators. It also identified requirements with no clear connection to the stated objectives and shortcomings in traceability matrices that had received high marks.
To carry out these analyses, the system combines several AI models that process PDF documents, interpret technical diagrams, check project consistency, and generate detailed reports. The entire process runs on the university's local servers, without sending student assignments to external services.
The research proposes using AI as a second assessor, rather than as a replacement for lecturers. Its analyses can identify errors that may go unnoticed in conventional marking and provide more detailed feedback, while lecturers retain responsibility for final decisions and contribute the pedagogical context that the technology does not always interpret appropriately.
The prototype still has limitations: it has been tested on a small sample, does not assess criteria accounting for 10% of the rubric, and requires adjustments to its instructions to avoid imposing demands disproportionate to the students' level. The authors advocate moving towards an assessment model that combines academic perspectives with the demands of professional practice, using disagreements between human and machine assessments to review marking criteria and improve learning.
---
How to Cite: García-Peñalvo, F. J., Alier-Forment, M., Vázquez-Ingelmo, A., García-Holgado, A., Casañ-Guerrero, M. J., & Pereira, J. (2026). Generative artificial intelligence-assisted assessment in Software Engineering assignments: A proof of concept. RIED-Revista Iberoamericana de Educación a Distancia, 29(2), 343–368. https://doi.org/10.5944/ried.47173
