Skip to content

Harden the exercode all-problem check: symlinked problem discovery, close-delay edge, copy-window temp leak, per-answer copy cost, TEST_CASE_RESULT parsing dedup #319

Description

@exKAZUu

Deferred findings from the reviews of #318:

  1. Symlinked problem directories are silently skipped. visitDirectory in src/cli/checkAllProblems.ts uses entry.isDirectory() / entry.isFile(), which are false for symlinks, so a problem reached only through a symlinked directory (or whose problem.md is a symlink) is skipped without failing the check. Resolve symlinks with stat or fail loudly.
  2. close-delay misclassification edge (POSIX). runHarnessProcess settles on the child's close event. If judged code leaves a background process that both escapes the detached process group (e.g. via setsid) and inherits the stdio pipes, a passing run is reported as timed out after 10 minutes even though all TEST_CASE_RESULT lines were printed. Settle on exit plus a short stream-drain grace period, or add a post-kill settle timer.
  3. Interrupt during the problem copy leaks the partial temp directory. Temporary roots are registered for signal cleanup only when the harness process spawns; a Ctrl-C while copyProblemDirToTemporaryRoot is still copying leaves the partial problem-utils-isolation_* tree in the OS temp directory. Register the temp root (and install the signal handlers) before the copy starts.
  4. The whole problem directory is copied once per model answer. A problem with M answers copies all answers and all test cases M times. Copy once per problem and clone per run (ideally with COPYFILE_FICLONE), and stream-filter TEST_CASE_RESULT lines instead of buffering up to 64 MB of output.
  5. TEST_CASE_RESULT line parsing is triplicated. src/cli/checkAllProblems.ts, isAcceptedJudgeOutput in src/helpers/checkProblemDirIsolation.ts, and test/e2e/debugAndJudge.test.ts each parse the lines with slightly different newline handling. Extract a shared parseTestCaseResultLines(stdout) next to src/types/testCaseResult.ts.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions