Add submission invalidation

[reppuli92]

Description

What?

Add a new 'invalidated' status for submissions. Teachers can manually set this status, and invalidated submissions do not count towards point totals.

Why?

This was requested by teachers, as it is useful for e.g. invalidating AI submissions.

How?

Updated the submission database schema to include the new status. This includes a Django migration.

Also created two new API endpoints to invalidate the submission from status Ready and re-validate the submission back to status Ready. These endpoints are only accessible by teachers and only work if the submission status is initially Ready or Invalidated (respectively.

Also created buttons in the Inspect submission view where this endpoint can be accessed. The buttons also only show if the user is a teacher and the submission has correct status.

Fixes #1479

Testing

Remember to add or update unit tests for new features and changes.

• How to test your changes in A-plus
• How to test accessibility

What type of test did you run?

• Accessibility test using the WAVE extension.
• Django unit tests.
• Playwright tests.
• Other test. (Add a description below)
• Manual testing.

Added a couple new unit tests for the invalidations. Also tested in the UI that invalidation and re-validation works and affects the points totals correctly.

Did you test the changes in

• Chrome
• Firefox
• This pull request cannot be tested in the browser.

Think of what is affected by these changes and could become broken

Translation

• Did you modify or add new strings in the user interface? (Read about how to create translation)

Programming style

• Did you follow our style guides?
• Did you use Python type hinting in all functions that you added or edited? (type hints for function parameters and return values)

Have you updated the README or other relevant documentation?

• documents inside the doc directory.
• README.md.
• Aplus Manual.
• Other documentation (mention below which documentation).

Is it Done?

• Reviewer has finished the code review
• After the review, the developer has made changes accordingly
• Customer/Teacher has accepted the implementation of the feature

Clean up your git commit history before submitting the pull request!

[Copilot]

Pull request overview

Adds a new “invalidated” submission status that course teachers can toggle to exclude selected submissions from point totals and related reporting, addressing the “invalidate graded submission” request in #1479.

Changes:

• Extends Submission status with INVALIDATED and updates relevant queryset/export/reporting filters to exclude invalidated submissions.
• Adds teacher-only staff endpoints to invalidate/revalidate a submission and exposes corresponding buttons in the staff submission inspection UI.
• Adds migration and UI/status translations (en/fi) for the new status and buttons.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
locale/fi/LC_MESSAGES/django.po	Adds Finnish translations for new status + UI strings (needs additional staff flash message translations).
locale/en/LC_MESSAGES/django.po	Adds English translations for new status + UI strings (needs additional staff flash message translations).
exercise/urls.py	Registers new invalidate/revalidate staff endpoints for submissions.
exercise/tests.py	Adds unit test coverage for invalidate/revalidate permissions and status transitions.
exercise/templates/exercise/staff/_assessment_panel.html	Adds teacher-only invalidate/revalidate buttons in the staff assessment panel.
exercise/submission_models.py	Introduces INVALIDATED status and setters; excludes invalidated from exclude_errors() filtering.
exercise/staff_views.py	Adds teacher-only POST views to invalidate/revalidate submissions with user feedback messages.
exercise/migrations/0052_alter_submission_status.py	Alters DB field choices to include invalidated.
exercise/management/commands/export_submissions.py	Excludes invalidated submissions from the export query.
exercise/api/csv/views.py	Excludes invalidated submissions from CSV aggregation/counting logic.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

[ihalaij1]

We should figure out if we want to remove the invalidated status from exclude_errors() and make a separate exclude_invalidated() method. The main concern is if invalidating a submission should refund a submission attempt to the student.

The following explains the ramifications of keeping the new invalidated status in exclude_errors() (from Copilot):

INVALIDATED should be included only if an invalidated submission is meant to be treated as a non-counting attempt. It is not technically an error, and adding it to exclude_errors() has wider effects than just hiding it from results.

In the current code, it affects:

• get_submissions_for_student(..., exclude_errors=True): invalidated submissions disappear.
• Submission limits in one_has_submissions() and no_submissions_left(): an invalidated attempt does not consume a submission slot, so the student may submit again.
• ordinal_number(): invalidated submissions are not counted when generating the attempt number sent to the grader.
• Batch approval: invalidated submissions are excluded from the submissions eligible for approval.
• Any future caller of exclude_errors(): it will silently treat invalidation like a grader error or rejected submission.

It does not affect callers that use the default get_submissions_for_student() or query Submission directly. The CSV/API result queries already explicitly exclude INVALIDATED independently.

I would avoid putting it in exclude_errors() because the name is now misleading. Prefer separate semantics:

def exclude_errors(self):
    return self.exclude(status__in=(
        Submission.STATUS.ERROR,
        Submission.STATUS.REJECTED,
    ))

def exclude_invalidated(self):
    return self.exclude(status=Submission.STATUS.INVALIDATED)

Then use both filters where appropriate. If the intended rule is that invalidation voids the attempt and permits a replacement submission, keep INVALIDATED in a separately named “non-counting submissions” filter. If invalidation is disciplinary and must still consume the attempt limit, it should not be part of the filter used by submission-limit calculations. The relevant shared filter is in submission_models.py, and the limit logic is in exercise_models.py.

[reppuli92]

I changed the filtering logic as above - now exclude_errors() doesn't include the invalidated status and there's a separate method exclude_invalidated(). Invalidated submissions also now consume an attempt. I also changed the form loading logic so that erroneous or invalidated submissions aren't loaded on questionnaire forms.

There's now a bunch of commits on this PR, maybe it's best for you to review this first @ihalaij1 and then I'll merge the commits?

[Copilot]

🟡 Changes recommended

Invalidated submissions can still contribute forced points and can be implicitly revalidated through existing grading paths.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 13/13 changed files
• Comments generated: 4
• Review effort level: Balanced

[reppuli92]

Added new commit to also remove explicit final markings upon submission invalidation. This way the invalidated submissions will never count towards point totals. Also fixed the typo suggested by Copilot.

The question whether to count invalidated submissions towards statistics remains. Currently invalidated submissions, as they consume a submission, count towards submission counts in exercise points data, submission limit calculations, exercise satatistics endpoints and numbers of submitters. In my view these probably should count invalidated submissions as well, since they must consume a submission? What do you think @ihalaij1 or should for example the statistics endpoints exclude invalidated submissions?

Invalidated submissons don't currently count towards points, and csv:s/other aggregate submission exports.

[ihalaij1]

Added new commit to also remove explicit final markings upon submission invalidation. This way the invalidated submissions will never count towards point totals. Also fixed the typo suggested by Copilot.

The question whether to count invalidated statistics towards submissions remains. Currently invalidated submissions, as they consume a submission, count towards submission counts in exercise points data, submission limit calculations, exercise satatistics endpoints and numbers of submitters. In my view these probably should count invalidated submissions as well, since they must consume a submission? What do you think @ihalaij1 or should for example the statistics endpoints exclude invalidated submissions?

Invalidated submissons don't currently count towards points, and csv:s/other aggregate submission exports.

That sounds fine. Did you check the regrade_exercises(..., 'all') function that Copilot mentions also regrades invalidated submissions and changes the status back to READY?

[reppuli92]

Whoops, didn't notice that one :) made new commit so that mass re-grading will always exclude invalidated submissions (whether re-grade type is incomplete or not) and also disabled regrading for single invalidated submissions. The button is also disabled in the inspect submission UI when inspecting an invalidated submission. Also added amention of this in the UI tooltip.

[Copilot]

Copilot review overview

🟡 Changes recommended

Revalidation authorization can be bypassed, final-submission state is lost, and invalidated submissions can still confirm exported points.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2

Open (2)

• Prevent assistants from revalidating invalidated submissions · New
• Preserve final-state flag when invalidating submissions · New

Resolved since last review (4)

• Invalidation does not suppress a submission that was previously marked as final.…
• The teacher-only revalidation transition can be bypassed because existing grading paths call…
• Counting INVALIDATED here keeps these submissions in the student/staff submission statistics and…
• The corrected message still misspells “overridden” as “overriden”.

Previously missed (1)

In code that hasn't changed since last review

Filter invalidated submissions from confirmation subquery

exercise/​api/​csv/​views.py:462

This excludes invalidated rows from the result query, but the confirmed subquery above still calls .passes() without any status filter. An invalidated passing submission in a confirm_the_level category can therefore continue to confirm sibling submissions and cause their points to appear in this export. Apply the same eligible-status filtering to that confirmation subquery.

[reppuli92]

Added one more commit to disable also manual regrading for invalidated submissions. This also prevents assistants from regrading invalidated submissions. Also noticed incorrect tooltip behavior, as previously if the best submission was invalidated, the second-best (which then determined the grade) had a tooltip in the inspect view saying incorrectly that another submission determines the grade. Fixed this and also added a tooltip saying this to invalidated submissions.

The second issue raised by Copilot doesn't make sense to me. If a submission is marked as best and manually graded, a teacher probably won't invalidate it in the first place. If for some reason the teacher wants to, though, (if there is suspicion of AI use etc.) the best flagging should also be removed, since invalidated submissions can't count towards the grade. If the submission is then revalidated (which is already 3 steps of a quite weird user flow that probably wouldn't happen under normal circumstances), the teacher should in my opinion manually regrade the submission again instead of the system carrying the flags somehow over. What do you think @ihalaij1

[Copilot]

Copilot review overview

🟡 Changes recommended

Existing approval and REST regrading paths bypass teacher-only revalidation, while cached statistics still count invalidated submissions.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1

Open (1)

• Prevent assistants from revalidating invalidated submissions · New

Resolved since last review (2)

• Preserve final-state flag when invalidating submissions
• Prevent assistants from revalidating invalidated submissions

[ihalaij1]

Looks like there are still some ways to unintentionally revalidate submissions @reppuli92

[reppuli92]

Added new commit to address the above issue. I also went through (with AI assistance) all possible API endpoints that could accidentally re-validate an invalidated submission and prevented that from happening by either filtering invalidated submissions out from batch API access points or causing an error with API access points that target only single submission instances.

Further, I added sanity checks to submission status-changing functions in submission_models.py to throw an error if an invalidated submission is tried to change into something else. This shouldn't happen now given the above changes but it further safeguards that the submissions won't be accidentally re-validated. Do you think this was a good idea @ihalaij1

[Copilot]

Copilot review overview

🟡 Changes recommended

Concurrency handling can report discarded writes as successful, and invalidation does not fully preserve reversible grading state.

Review effort: Balanced
Findings: 2

Open (2)

• Stale callbacks create tags after submission invalidation · New
• Save silently discards writes while callers report success · New

Resolved since last review (1)

• Prevent grading data updates for invalidated submissions

[Copilot]

Copilot review overview

🟡 Changes recommended

Revalidation loses forced-final grading state, and concurrent discarded grades can still trigger external post-grading side effects.

Review effort: Balanced
Findings: 1

Open (1)

• Post-grading hooks run before locked READY save succeeds · New

Resolved since last review (2)

• Save silently discards writes while callers report success
• Stale callbacks create tags after submission invalidation

[Copilot]

Copilot review overview

🟡 Changes recommended

Invalidated attempts remain counted in cached statistics, and prior-feedback loading unintentionally excludes error submissions.

Review effort: Balanced
Findings: 1

Open (1)

• Do not exclude error and rejected submissions · New

Resolved since last review (1)

• Post-grading hooks run before locked READY save succeeds

[Copilot]

Copilot review overview

🔵 Needs a closer look

Revalidation loses final-submission state, and invalidated submissions still affect cached statistics and reveal rules.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Do not exclude error and rejected submissions

[Copilot]

Copilot review overview

🟡 Changes recommended

Invalidated submissions can still be selected as best in pending exports and remain included in statistics contrary to the linked requirement.

Review effort: Balanced
Findings: 2

Open (2)

• Exclude invalidated submissions from best-submission selection · New
• Exclude invalidated submissions from submission statistics · New

[Copilot]

🔵 Needs a closer look

The concurrency-sensitive model save override and numerous grading integrations warrant final human review despite extensive tests.

0 open findings

2 resolved since last review

• Exclude invalidated submissions from submission statistics
• Exclude invalidated submissions from best-submission selection

🧠 Review effort: Balanced

[ihalaij1]

This has been reviewed, fixed and re-reviewed multiple times by Copilot and me. It seems to work fine now. Possible remaining bugs should be minor and can be addressed later if found.