Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Athena: Improve AI feedback request validation #10165

Draft
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

maximiliansoelch
Copy link
Member

@maximiliansoelch maximiliansoelch commented Jan 17, 2025

Warning

Only deploy to TS1 for now, as Athena currently does not work on TS2 and TS3

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) and too complex database calls.
  • I strictly followed the principle of data economy for all database calls.
  • I strictly followed the server coding and design guidelines.
  • I added multiple integration tests (Spring) related to the features (with a high test coverage).
  • I documented the Java code using JavaDoc style.

Client

Motivation and Context

Currently, there are some issues with the AI feedback request feature:

  1. It is possible to request feedback despite no submission was yet made by the student. The check for this case is wrong, as it assumes that no submission will be present in the database. However, if a student starts the exercise, the submission is already created in the database.
  2. The server does not check if the submission is empty before sending it to Athena, which results in an unspecific error message. This only affects text exercises, as there is a client-side check so that a student can not submit an empty modeling submission.
  3. There is an issue with the german translation key submissionAlreadyHasAthenaResult: "Bitte reiche eine neue Abgabe ein, bevor du erneut einreichst.", which is incorrect.

In addition, all AI Feedback request services for the different exercise types have their own implementation for checking the request rate limit, which causes code duplication and is hard to maintain.

Description

This PR fixes several issues and reduces code duplication for the request AI feedback feature:

  1. The check was improved and now checks if there exists at least one successful submission in the database.
  2. A check was added that allows AI feedback requests only for nonempty text and modeling submissions. Translation keys for the corresponding error message were added.
  3. The translation was corrected.

Steps for Testing

Prerequisites:

  • 1 Students
  • 1 Text Exercise with Athena Feedback Requests enabled
  • 1 Modeling Exercise with Athena Feedback Requests enabled

1. Testcase: Request Feedback without a successful submission
Do these steps for each exercise type:

  1. Start the exercise
  2. Open your browser's developer tools and remove the disabled keyword from the request feedback button
  3. Click on the now enabled request feedback button on the exercise overview page
  4. There should be an error message that you first need to submit

2. Testcase: Request Feedback on an empty text submission

  1. Click on the text exercise to work on it
  2. Leave the submission text empty and just submit it
  3. Request feedback should now throw an error message

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.







Review Progress

Performance Review

  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Performance Tests

  • Test 1
  • Test 2

Test Coverage

Screenshots

@github-actions github-actions bot added tests server Pull requests that update Java code. (Added Automatically!) client Pull requests that update TypeScript code. (Added Automatically!) labels Jan 17, 2025
@github-actions github-actions bot added athena Pull requests that affect the corresponding module exercise Pull requests that affect the corresponding module modeling Pull requests that affect the corresponding module programming Pull requests that affect the corresponding module text Pull requests that affect the corresponding module labels Jan 17, 2025
Copy link
Contributor

@EneaGore EneaGore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code lgtm and both rate limits and previous feedback checks worked as expected on TS1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
athena Pull requests that affect the corresponding module client Pull requests that update TypeScript code. (Added Automatically!) exercise Pull requests that affect the corresponding module lock:artemis-test1 modeling Pull requests that affect the corresponding module programming Pull requests that affect the corresponding module server Pull requests that update Java code. (Added Automatically!) tests text Pull requests that affect the corresponding module
Projects
Status: Work In Progress
Development

Successfully merging this pull request may close these issues.

2 participants