diff --git a/Burnout3_recomp/.gitignore b/Burnout3_recomp/.gitignore new file mode 100644 index 000000000..bdc94adda --- /dev/null +++ b/Burnout3_recomp/.gitignore @@ -0,0 +1,42 @@ +# Burnout 3 (SLUS_210.50) workspace: keep game files and game-derived data out of git. +# Tracked here: TOML configs (repo-relative paths), tools/, COMPLETED_WORK.md, this file. + +# Game executables and disc images (also renamed copies such as SLUS_210.50.bak) +/SLUS_* +*.elf +*.irx +*.iso +*.bin +*.img +*.cue +*.chd +*.cso + +# Extracted disc contents and assets +/disc*/ +/assets/ +/game_data/ + +# Ghidra project database and lock files +/ghidra_project/ +*.gpr +*.rep/ +*.lock +*.lock~ + +# Logs +/logs/ +*.log + +# Generated C++ from every recompiler run (output/, output_bounded/, ...) +/output*/ + +# Earlier export backups +/backups/ + +# Function maps, instruction ranges and reports derived from the ELF +/*.csv +/reports/ + +# Machine-specific values: ELF SHA-256, original ELF location +/local/ diff --git a/Burnout3_recomp/COMPLETED_WORK.md b/Burnout3_recomp/COMPLETED_WORK.md new file mode 100644 index 000000000..eb5cf7604 --- /dev/null +++ b/Burnout3_recomp/COMPLETED_WORK.md @@ -0,0 +1,142 @@ +# Burnout 3 USA (SLUS_210.50): recompilation workspace + +All paths below are relative to the PS2Recomp repo root. Run every command from there. + +## Git policy + +- Only the TOML configs, `tools/`, this file and `.gitignore` are tracked. +- Never commit game files or anything derived from the game: the ELF, disc images, + IRX modules, extracted assets, the Ghidra project, function maps, reports, logs, + backups, or generated C++ (`output*/`). `Burnout3_recomp/.gitignore` enforces this. +- Machine-specific values (ELF SHA-256, original ELF location) live in the ignored + `Burnout3_recomp/local/burnout3_local.json`; the tools read them from there. +- Generated C++ is reproducible from the ELF, TOML, CSV and the recompiler commit, + so it is never tracked. + +## Workspace layout + +| Path | Tracked | Contents | +|---|---|---| +| `SLUS_210.50` | no | Working copy of the ELF, byte-identical to the original | +| `ghidra_project/` | no | Ghidra 12.1.4 project `Burnout3_USA_Recomp` (r5900:LE:32:default) | +| `burnout3_usa_ghidra_functions.csv` | no | Original exporter map (71,108 entries) | +| `burnout3_usa_ghidra_functions_instruction_bounded.csv` | no | Repaired map (71,112 entries) | +| `burnout3_usa_recomp.toml` | yes | Original exporter config, writes `output/` | +| `burnout3_usa_recomp_instruction_bounded.toml` | yes | Repaired-map config, writes `output/` | +| `burnout3_usa_recomp_instruction_bounded_output_bounded.toml` | yes | Same, writes `output_bounded/` | +| `tools/` | yes | Ghidra scripts and the repair/verify scripts | +| `reports/`, `logs/`, `backups/` | no | Analysis/validation reports, console logs, pre-repair exports | +| `output/`, `output_bounded/` | no | Generated C++ | +| `disc_image/` | no | Read-only mount point for your own Burnout 3 ISO (empty when detached) | +| `disc/` | no | Runtime launch folder: relative symlinks into `disc_image/`, an APFS clone of `SLUS_210.50`, and a writable `mc0/` for saves | + +Mount before running the game, and detach afterwards. The runtime maps `cdrom0:` to the folder containing the launched ELF. + +```bash +hdiutil attach -readonly -nobrowse -mountpoint "$PWD/Burnout3_recomp/disc_image" "" +``` + +```bash +hdiutil detach Burnout3_recomp/disc_image +``` + +## Completed steps + +1. **Ghidra import and analysis.** 7,776 functions, 880,944 instructions. Single + executable section `0x00100000–0x004E2680` (code and data mixed), entry `0x00100008`. + 12 error bookmarks and 1 warning bookmark (see "Known issues"). +2. **Export.** The unchanged repository script `ps2xRecomp/tools/ghidra/ExportPS2Functions.java` + produced `burnout3_usa_recomp.toml` and `burnout3_usa_ghidra_functions.csv` + (7,776 functions + 63,332 code labels, 58 runtime stubs). +3. **Baseline recompilation** with `burnout3_usa_recomp.toml` into `output/`. + It completed, but the original map has two defects: + - **Data treated as code.** 4,270 unique `Unhandled …` instructions (emitted as + `throw std::runtime_error`) in four files: `entry_004b9238`, `entry_004ca498`, + `entry_004dd7e0` and `entry_004e2118`. All lie in `0x4B9440–0x4E2664`, and none + are inside Ghidra's instruction ranges. + - **Noncontiguous function bodies.** 453 rows have `End - Start != Size`, because + the exporter sets End to the body's highest address. That produced 887 MB of + output, including `FUN_0014d410` at 139 MB. +4. **Range repair.** `tools/ExportBurnout3InstructionRanges.java` exported Ghidra's + contiguous instruction runs to `reports/burnout3_usa_ghidra_instruction_ranges.csv`. + `tools/repair_burnout3_export_ranges.py` bounded every range to those runs. It kept + all 71,108 original entries, added 4 fragments, and excluded 193,076 non-instruction + bytes. `tools/verify_burnout3_exports.py` passes all 30 checks. +5. **Repaired-map recompilation** into `output_bounded/`, logged to + `logs/08_ps2_recomp_instruction_bounded.log`. It ran in 4m50s with exit code 0. + + | | Baseline `output/` | Repaired `output_bounded/` | + |---|---|---| + | Functions in map | 71,108 | 71,112 (all recompiled or stubbed, 0 decode failures) | + | Unhandled instructions | 4,270 unique | **0** | + | `// Unimplemented` register markers | 61 | 0 | + | Total size | 887 MB | 517 MB | + | Largest function file | 139 MB (`FUN_0014d410`) | 3.6 MB (`FUN_00186850`, a genuinely 77 KB function) | + | Errors | not captured | 0 | + | Warnings | not captured | 5,636, all `control-flow` | + + Each warning is "unresolved JR/JALR; promoted N fallback entries". These are + indirect jumps, mostly `switch` jump tables (`caseD_*`, `FUN_*`, `entry_*`), that the + recompiler could not resolve statically. It registered 413,683 fallback entry points + instead, so these jumps go through the runtime dispatch table (`register_functions.cpp` + is 21.8 MB). They are not build blockers, but they are slower and are the next thing + to triage. Whether the 4 split fragments are always reached correctly can only be + confirmed at runtime. + + `output_bounded/` is the output to integrate next. Keep `output/` only for comparison. + +## Known issues + +- Ghidra error bookmarks: 11 flows from `0x484414–0x485654` into `0x74000–0x76680`, + which is memory not in the ELF (probably code copied to low RAM at runtime; this + is inferred), and conflicting data at `0x200024`. +- Ghidra decompiler `VarnodeContext` errors for `vf1`, `vf8` and `vf22` at `0x1700B0`, + `0x17C01C` and `0x18EF0C`. These affect the decompiler only, not `ps2_recomp`. + +## Commands + +```bash +python3 Burnout3_recomp/tools/verify_burnout3_exports.py +``` + +```bash +./out/build/ps2xRecomp/ps2_recomp Burnout3_recomp/burnout3_usa_recomp_instruction_bounded_output_bounded.toml > Burnout3_recomp/logs/08_ps2_recomp_instruction_bounded.log 2>&1 +``` + +```bash +sed -n '/PS2Recomp report/,/Events:/p' Burnout3_recomp/logs/08_ps2_recomp_instruction_bounded.log +``` + +## Not done yet + +- Triage the 5,636 control-flow warnings by resolving jump tables in Ghidra. +- Build, link, then handle runtime work (syscalls, GS/VU1, IOP/IRX), using your own disc for assets. + +## Runtime integration (done) + +`output_bounded/` is integrated into `ps2xRuntime`: +- **Code:** all 71,113 `.cpp` files, cloned with `cp -c` (no extra disk space), are in + `ps2xRuntime/src/runner/`. +- **Headers:** `ps2_recompiled_functions.h` and `ps2_recompiled_stubs.h` are in + `ps2xRuntime/include/`. +- **Git:** upstream's `.gitignore` already ignores both locations. The tracked placeholder + `ps2xRuntime/src/runner/register_functions.cpp` is marked `--skip-worktree`, so the + generated version does not show as modified. + +To undo (restores the upstream placeholder): + +```bash +find ps2xRuntime/src/runner -name '*.cpp' ! -name register_functions.cpp -delete +``` + +```bash +rm ps2xRuntime/include/ps2_recompiled_functions.h ps2xRuntime/include/ps2_recompiled_stubs.h +``` + +```bash +git update-index --no-skip-worktree ps2xRuntime/src/runner/register_functions.cpp +``` + +```bash +git checkout -- ps2xRuntime/src/runner/register_functions.cpp +``` diff --git a/Burnout3_recomp/README.md b/Burnout3_recomp/README.md new file mode 100644 index 000000000..fe61054d9 --- /dev/null +++ b/Burnout3_recomp/README.md @@ -0,0 +1,102 @@ +# Burnout 3 USA (SLUS_210.50) recompilation workspace + +Workspace for statically recompiling Burnout 3 (USA) with PS2Recomp. Game files and +game-derived data are never tracked (see `.gitignore`); use your own disc only. + +- `ROADMAP.md`: the overall plan, steps A–D. +- `COMPLETED_WORK.md`: what has been done so far. + +All commands below run from the PS2Recomp repo root. + +## Step C: build `ps2EntryRunner`, in 4 milestones + +Step C turns the integrated generated code (step B) into one executable, +`out/build/ps2xRuntime/ps2EntryRunner`. + +In unity mode, CMake joins the 71,113 generated files in `ps2xRuntime/src/runner/` into +about 2,200 batches of 32. A precompiled header parses the 6 MB +`ps2_recompiled_functions.h` once instead of 71,113 times. The linker then combines the +batches with `libps2_runtime.a` and the dependencies already in `out/build/_deps`. + +**Done when:** the executable links and its log is saved. Booting the game is step D. + +| # | Milestone | Duration | Main risk | +|---|---|---|---| +| 1 | Pre-flight and reconfigure | minutes | Stale or missing runner files; an unexpected download | +| 2 | Runtime library at the new flags | minutes | Toolchain or flag errors | +| 3 | Compile the generated code | hours | Running out of memory on the heaviest batches | +| 4 | Link, triage, record | minutes to hours | Missing symbols | + +### Milestone 1: pre-flight and reconfigure + +Confirm the inputs, then switch `out/build` to the step C settings. + +1. Check the runner files and headers match the integrated output: + - 71,113 `.cpp` files in `ps2xRuntime/src/runner/`, identical to `Burnout3_recomp/output_bounded/` + - both `ps2_recompiled_*.h` headers in `ps2xRuntime/include/` + - `register_functions.cpp` marked `S` (skip-worktree) in `git ls-files -v` +2. Check the dependencies are present in `out/build/_deps` (raylib, SDL2, imgui, rlImGui and + the rest), so no network is needed. +3. Check free disk space: expect several GB of object files, plus up to 10 GB of sccache cache. +4. Reconfigure (aggressive logs off, `-O1`, asserts kept, so no `-DNDEBUG`): + +```bash +cmake -S . -B out/build -DPS2X_ENABLE_AGRESSIVE_LOGS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS_RELEASE="-O1" +``` + +**Done when:** configure finishes without downloading anything, and `out/build/CMakeCache.txt` +shows `PS2X_ENABLE_AGRESSIVE_LOGS=OFF`, `CMAKE_BUILD_TYPE=Release` and `CMAKE_CXX_FLAGS_RELEASE=-O1`. + +### Milestone 2: runtime library at the new flags + +Rebuild the runtime and its dependencies alone, before the hours-long part. This catches +toolchain or flag problems in minutes. + +```bash +cmake --build out/build --target ps2_runtime -j8 2>&1 | tee Burnout3_recomp/logs/09a_build_ps2_runtime.log +``` + +**Done when:** `out/build/ps2xRuntime/libps2_runtime.a` is rebuilt (newer timestamp) with no errors. + +### Milestone 3: compile the generated code + +The long step. Run it in the background and watch memory use. + +```bash +cmake --build out/build --target ps2EntryRunner -j8 2>&1 | tee Burnout3_recomp/logs/09_build_ps2EntryRunner.log +``` + +- **Heaviest batches:** the one containing `register_functions.cpp` (21.8 MB, a lookup table + for about 4 million guest addresses) and those with the largest functions, such as + `FUN_00186850` (3.6 MB of source). +- **If memory runs out or the compiler crashes:** rerun the same command with `-j6`, then + `-j4`. Make resumes where it stopped. +- **If a single file is the problem:** shrinking the unity batch size or excluding that file + from unity mode means editing upstream CMake, so it needs review first. +- **Possible name collisions:** two files in one batch could define file-local helpers with the + same name. This is unlikely, because generated names include addresses. + +**Done when:** every unity batch compiles. The log shows no `error:` lines before the link step. + +### Milestone 4: link, triage, record + +- **Missing symbols:** link errors usually mean declared functions have no implementation, such + as stubs the generated code references that the runtime lacks. Group them by name in the log. +- **Fix them, preferring the runtime side:** + - Burnout 3-specific fixes go in a game override, kept in `Burnout3_recomp/overrides/` and + cloned into `ps2xRuntime/src/runner/`. That rebuilds one batch. + - Generic gaps go in `ps2xRuntime/src/lib`, which is upstream source, after review. + - TOML changes regenerate code and force a full rebuild, so batch them. + - Never hand-edit generated files. +- **Record the result:** note the outcome in `COMPLETED_WORK.md` and update the status table + in `ROADMAP.md`. + +**Done when:** `out/build/ps2xRuntime/ps2EntryRunner` links cleanly and is newer than the old +placeholder build, and the log is saved at `Burnout3_recomp/logs/09_build_ps2EntryRunner.log`. +This proves the code compiles. Missing syscalls, IOP modules and graphics gaps show up in step D. + +### Reverting the build settings + +```bash +cmake -S . -B out/build -DCMAKE_BUILD_TYPE= -DPS2X_ENABLE_AGRESSIVE_LOGS=ON +``` diff --git a/Burnout3_recomp/ROADMAP.md b/Burnout3_recomp/ROADMAP.md new file mode 100644 index 000000000..63110f826 --- /dev/null +++ b/Burnout3_recomp/ROADMAP.md @@ -0,0 +1,163 @@ +# Burnout 3 USA (SLUS_210.50): recompilation roadmap + +Branch: `Burnout3` (local; push only compliant content). All paths are relative to the +PS2Recomp repo root. Completed history is in `COMPLETED_WORK.md`. + +Ground rules: +- Never track game files or game-derived data: ELF, disc contents, IRX modules, Ghidra + project, function maps, reports, logs or generated C++. `Burnout3_recomp/.gitignore` + enforces this. +- Use your own disc only. Never download game files or a BIOS. +- Don't modify the original ELF, the ISO, or existing PS2Recomp source. + +## Status + +| Step | What | Status | +|---|---|---| +| A | Disc files available to the runtime | Done | +| B | Generated code integrated into `ps2xRuntime` | Done | +| C | Build `ps2EntryRunner` | Next, not started | +| D | First boot and bring-up loop | Pending | + +## A. Disc files for the runtime (done) + +**Why:** the generated C++ holds only the game's code. At runtime the code still reads +`cdrom0:\…` paths for assets, audio, video and the IRX modules that `ps2xIOP` executes. +The runtime maps `cdrom0:` to the folder containing the launched ELF, and puts memory-card +saves in `mc0/` next to it. + +**Layout** (all ignored by `/disc*/`; nothing is copied, nothing outside PS2Recomp is written): +- `Burnout3_recomp/disc_image/`: your ISO mounted read-only. +- `Burnout3_recomp/disc/`: the launch folder, containing: + - relative symlinks into `disc_image/` + - `SLUS_210.50` as an APFS clone (a real file, so `mc0/` resolves inside `disc/`, not on the read-only mount) + - a writable `mc0/` + +The mount does not survive a reboot. Attach it before step D: + +```bash +hdiutil attach -readonly -nobrowse -mountpoint "$PWD/Burnout3_recomp/disc_image" "" +``` + +```bash +hdiutil detach Burnout3_recomp/disc_image +``` + +**Open risks:** +- Untested whether the runtime follows the symlinks. The fallback is copying the disc into + `disc/` (2.7 GB, still ignored). +- The disc ships `IOP/IOPRP280.IMG` (IOP reboot image) and `IOP/DNAS280.IMG` (online + authentication). `ps2xIOP` support for either is unverified. + +## B. Runtime integration (done) + +- **Code:** the 71,113 `.cpp` files from `Burnout3_recomp/output_bounded/` are in + `ps2xRuntime/src/runner/`, as APFS clones. +- **Headers:** `ps2_recompiled_functions.h` and `ps2_recompiled_stubs.h` are in + `ps2xRuntime/include/`. +- **Git:** upstream's `.gitignore` already ignores both locations. The tracked placeholder + `ps2xRuntime/src/runner/register_functions.cpp` is marked `--skip-worktree`, so `git status` + stays clean. + +**Re-sync rule** (whenever `ps2_recomp` output is regenerated): first delete the old generated +files, then copy the new ones. CMake compiles **every** `src/runner/*.cpp`, so stale files +from an earlier run would be built too, causing duplicate or outdated functions. + +```bash +find ps2xRuntime/src/runner -name '*.cpp' ! -name register_functions.cpp -delete +``` + +```bash +find Burnout3_recomp/output_bounded -maxdepth 1 -name '*.cpp' -print0 | xargs -0 -J % cp -c % ps2xRuntime/src/runner/ +``` + +```bash +cp -c Burnout3_recomp/output_bounded/ps2_recompiled_*.h ps2xRuntime/include/ +``` + +Undo instructions are in `COMPLETED_WORK.md`. + +## C. Build `ps2EntryRunner` (next) + +Reuse the existing `out/build`. It already holds every dependency (raylib, SDL2, imgui, +rlImGui, fmt, toml11, …), so the build needs no network. raylib is required by the +graphics backend, so it can't be avoided anyway. The FFmpeg download only happens on Windows. + +| Setting | Value | Reason | +|---|---|---| +| Build folder | `out/build` (Unix Makefiles) | Dependencies already present; Ninja would need a new folder and new downloads | +| `PS2X_ENABLE_AGRESSIVE_LOGS` | `OFF` | When on, it injects `PS_LOG_ENTRY` into all 71,113 functions (`PS2_FUNCTION_LOG_TRACKER`), slowing both the build and the game | +| `PS2X_ENABLE_RUNTIME_LOGS` | `ON` (default) | Keeps syscall, IOP and IO diagnostics | +| Optimization | `-O1`, asserts kept | `-O0` (current default) runs too slowly; `-O2`/`-O3` compile far slower on functions up to 3.6 MB of source. Leaving out `-DNDEBUG` keeps runtime `assert()` checks during bring-up | +| Unity build + precompiled header | `ON` (default) | About 2,200 batches of 32 files; the 6 MB generated header is parsed once instead of 71,113 times | +| `PS2X_ENABLE_SCCACHE` | `ON` (default) | Harmless; cache at `~/Library/Caches/Mozilla.sccache`, capped at 10 GB. No help on the first build | +| Parallel jobs | `-j8` | 8 performance cores, about 3 GB RAM per job. Drop to `-j6` if memory runs out | +| Target | `ps2EntryRunner` only | Skips the recompiler and tests | + +```bash +cmake -S . -B out/build -DPS2X_ENABLE_AGRESSIVE_LOGS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS_RELEASE="-O1" +``` + +```bash +cmake --build out/build --target ps2EntryRunner -j8 2>&1 | tee Burnout3_recomp/logs/09_build_ps2EntryRunner.log +``` + +**Expected cost (estimates):** +- **Time:** hours. +- **Disk:** `out/build` is 1.6 GB today and will grow by several GB of object files. 262 GB is free. +- **Shared build folder:** the new flags are stored in `out/build`, so every target there + (including `ps2_recomp`, raylib and SDL2) recompiles once at `-O1` the next time it is built. + To revert: + + ```bash + cmake -S . -B out/build -DCMAKE_BUILD_TYPE= -DPS2X_ENABLE_AGRESSIVE_LOGS=ON + ``` + +**Done when:** `out/build/ps2xRuntime/ps2EntryRunner` links. Fix compile and link errors +through the TOML, or through runtime code (see D). Never hand-edit generated files. + +**Optional:** add `-gline-tables-only` to the flags for crash backtraces with file and line. +Cost: longer compiles and more disk (not measured). + +## D. First boot and bring-up (pending) + +1. Attach the disc (step A), then launch: + + ```bash + ./out/build/ps2xRuntime/ps2EntryRunner Burnout3_recomp/disc/SLUS_210.50 2>&1 | tee Burnout3_recomp/logs/10_first_boot.log + ``` + +2. Expected early blockers, in likely order: + - missing or TODO syscalls + - IOP reboot with `IOPRP280.IMG` + - IRX imports `ps2xIOP` lacks + - "function not found" for addresses reached through the 413,683 indirect-jump fallback entries + - code Ghidra could not follow at `0x74000–0x76680`, probably copied to low RAM at runtime + - GS and VU1 rendering + - DNAS and network (stub them) +3. Iterate with the README loop: classify with temporary `ret0`/`ret1` stubs, then replace + them with real fixes. + +**Prefer runtime-side fixes over regeneration.** This is the main consequence of the step C setup: +- **Regenerating** (any TOML change, including `handler@0xADDR` stubs) changes + `ps2_recompiled_functions.h`. That header is in the precompiled header, so **all 71k files + recompile, taking hours**. Batch TOML changes together. +- **Game overrides** (`PS2_REGISTER_GAME_OVERRIDE`, `ps2_game_overrides::bindAddressHandler`) + bind addresses to handlers at load time and leave generated code untouched. Keep their + sources tracked in `Burnout3_recomp/overrides/` and clone them into `ps2xRuntime/src/runner/`, + which is compiled automatically and ignored by git. An edit then rebuilds one unity batch + and relinks, which takes minutes. +- **Generic syscall and stub gaps** belong in `ps2xRuntime/src/lib`. That is upstream source: + change it only after explicit review, and keep any patch separate from Burnout 3-specific code. + +**Debugging trade-off:** with aggressive logs off there is no per-function entry trace. When +a crash location is unclear, temporarily rebuild with `PS2X_ENABLE_AGRESSIVE_LOGS=ON` (a full +rebuild) or use the debug UI. + +## Later (after D) + +- **Jump tables:** resolve the 5,636 unresolved JR/JALR jumps in Ghidra. This improves + performance and is not needed for correctness. +- **Optimization:** move to `-O2` once boot is stable, if speed requires it. +- **Audio and memory cards:** audio through the disc's sound IRX modules; memory-card saves + go to `Burnout3_recomp/disc/mc0/`. diff --git a/Burnout3_recomp/burnout3_usa_recomp.toml b/Burnout3_recomp/burnout3_usa_recomp.toml new file mode 100644 index 000000000..9992b71d1 --- /dev/null +++ b/Burnout3_recomp/burnout3_usa_recomp.toml @@ -0,0 +1,91 @@ +# Paths are relative to the PS2Recomp repo root; run ps2_recomp from there. +# Auto-generated by ExportPS2Functions.java +# +# Classification policy (aligned with analyzer intent): +# - runtime-known names -> [general].stubs +# - library-like names without runtime handlers -> [general].untracked_stubs +# - [general].skip is retained empty for legacy compatibility +# - no SCE symbol database is used by this Ghidra script + +[general] +input = "Burnout3_recomp/SLUS_210.50" +output = "Burnout3_recomp/output" +ghidra_output = "Burnout3_recomp/burnout3_usa_ghidra_functions.csv" +single_file_output = false +patch_syscalls = false +patch_cop0 = true +patch_cache = true +stubs = [ + "SetGsCrt@0x0010BA20", + "AddIntcHandler@0x0010BB00", + "AddIntcHandler@0x0010BB10", + "RemoveIntcHandler@0x0010BB20", + "AddDmacHandler@0x0010BB30", + "AddDmacHandler@0x0010BB40", + "RemoveDmacHandler@0x0010BB50", + "EnableIntc@0x0010BB60", + "DisableIntc@0x0010BB70", + "EnableDmac@0x0010BB80", + "DisableDmac@0x0010BB90", + "SetAlarm@0x0010BBA0", + "ReleaseAlarm@0x0010BBB0", + "iEnableIntc@0x0010BBC0", + "iDisableIntc@0x0010BBD0", + "iEnableDmac@0x0010BBE0", + "iDisableDmac@0x0010BBF0", + "iSetAlarm@0x0010BC00", + "CreateThread@0x0010BC20", + "DeleteThread@0x0010BC30", + "StartThread@0x0010BC40", + "ExitDeleteThread@0x0010BC60", + "TerminateThread@0x0010BC70", + "ChangeThreadPriority@0x0010BCB0", + "iChangeThreadPriority@0x0010BCC0", + "RotateThreadReadyQueue@0x0010BCD0", + "GetThreadId@0x0010BD10", + "ReferThreadStatus@0x0010BD20", + "SleepThread@0x0010BD40", + "WakeupThread@0x0010BD50", + "iWakeupThread@0x0010BD60", + "SuspendThread@0x0010BD90", + "ResumeThread@0x0010BDB0", + "EndOfHeap@0x0010BE00", + "CreateSema@0x0010BE20", + "DeleteSema@0x0010BE30", + "SignalSema@0x0010BE40", + "iSignalSema@0x0010BE50", + "WaitSema@0x0010BE60", + "PollSema@0x0010BE70", + "SetOsdConfigParam@0x0010BEC0", + "GetOsdConfigParam@0x0010BED0", + "FlushCache@0x0010C0A0", + "sceSifStopDma@0x0010C0F0", + "GsGetIMR@0x0010C140", + "GsPutIMR@0x0010C160", + "SetVSyncFlag@0x0010C190", + "sceSifDmaStat@0x0010C1C0", + "sceSifSetDma@0x0010C1E0", + "isceSifSetDma@0x0010C1F0", + "sceSifSetDChain@0x0010C200", + "isceSifSetDChain@0x0010C210", + "sceSifSetReg@0x0010C220", + "sceSifGetReg@0x0010C230", + "Deci2Call@0x0010C250", + "GetMemorySize@0x0010C280", + "InitTLB@0x0010C290", + "GetThreadId@0x002372F0", +] +untracked_stubs = [ +] +skip = [] + +[ghidra_export] +function_count = 7776 +code_label_count = 63332 +csv_record_count = 71108 +stub_count = 58 +untracked_stub_count = 0 +skip_count = 0 +uncategorized_count = 7718 +runtime_call_name_count = 656 +runtime_call_source = "embedded_ps2_call_list_snapshot" diff --git a/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded.toml b/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded.toml new file mode 100644 index 000000000..571d32190 --- /dev/null +++ b/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded.toml @@ -0,0 +1,92 @@ +# Instruction-bounded copy; original exports preserved in backups/before_range_fix. +# Paths are relative to the PS2Recomp repo root; run ps2_recomp from there. +# Auto-generated by ExportPS2Functions.java +# +# Classification policy (aligned with analyzer intent): +# - runtime-known names -> [general].stubs +# - library-like names without runtime handlers -> [general].untracked_stubs +# - [general].skip is retained empty for legacy compatibility +# - no SCE symbol database is used by this Ghidra script + +[general] +input = "Burnout3_recomp/SLUS_210.50" +output = "Burnout3_recomp/output" +ghidra_output = "Burnout3_recomp/burnout3_usa_ghidra_functions_instruction_bounded.csv" +single_file_output = false +patch_syscalls = false +patch_cop0 = true +patch_cache = true +stubs = [ + "SetGsCrt@0x0010BA20", + "AddIntcHandler@0x0010BB00", + "AddIntcHandler@0x0010BB10", + "RemoveIntcHandler@0x0010BB20", + "AddDmacHandler@0x0010BB30", + "AddDmacHandler@0x0010BB40", + "RemoveDmacHandler@0x0010BB50", + "EnableIntc@0x0010BB60", + "DisableIntc@0x0010BB70", + "EnableDmac@0x0010BB80", + "DisableDmac@0x0010BB90", + "SetAlarm@0x0010BBA0", + "ReleaseAlarm@0x0010BBB0", + "iEnableIntc@0x0010BBC0", + "iDisableIntc@0x0010BBD0", + "iEnableDmac@0x0010BBE0", + "iDisableDmac@0x0010BBF0", + "iSetAlarm@0x0010BC00", + "CreateThread@0x0010BC20", + "DeleteThread@0x0010BC30", + "StartThread@0x0010BC40", + "ExitDeleteThread@0x0010BC60", + "TerminateThread@0x0010BC70", + "ChangeThreadPriority@0x0010BCB0", + "iChangeThreadPriority@0x0010BCC0", + "RotateThreadReadyQueue@0x0010BCD0", + "GetThreadId@0x0010BD10", + "ReferThreadStatus@0x0010BD20", + "SleepThread@0x0010BD40", + "WakeupThread@0x0010BD50", + "iWakeupThread@0x0010BD60", + "SuspendThread@0x0010BD90", + "ResumeThread@0x0010BDB0", + "EndOfHeap@0x0010BE00", + "CreateSema@0x0010BE20", + "DeleteSema@0x0010BE30", + "SignalSema@0x0010BE40", + "iSignalSema@0x0010BE50", + "WaitSema@0x0010BE60", + "PollSema@0x0010BE70", + "SetOsdConfigParam@0x0010BEC0", + "GetOsdConfigParam@0x0010BED0", + "FlushCache@0x0010C0A0", + "sceSifStopDma@0x0010C0F0", + "GsGetIMR@0x0010C140", + "GsPutIMR@0x0010C160", + "SetVSyncFlag@0x0010C190", + "sceSifDmaStat@0x0010C1C0", + "sceSifSetDma@0x0010C1E0", + "isceSifSetDma@0x0010C1F0", + "sceSifSetDChain@0x0010C200", + "isceSifSetDChain@0x0010C210", + "sceSifSetReg@0x0010C220", + "sceSifGetReg@0x0010C230", + "Deci2Call@0x0010C250", + "GetMemorySize@0x0010C280", + "InitTLB@0x0010C290", + "GetThreadId@0x002372F0", +] +untracked_stubs = [ +] +skip = [] + +[ghidra_export] +function_count = 7776 +code_label_count = 63336 +csv_record_count = 71112 +stub_count = 58 +untracked_stub_count = 0 +skip_count = 0 +uncategorized_count = 7718 +runtime_call_name_count = 656 +runtime_call_source = "embedded_ps2_call_list_snapshot" diff --git a/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded_output_bounded.toml b/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded_output_bounded.toml new file mode 100644 index 000000000..798c1376a --- /dev/null +++ b/Burnout3_recomp/burnout3_usa_recomp_instruction_bounded_output_bounded.toml @@ -0,0 +1,92 @@ +# Run copy of burnout3_usa_recomp_instruction_bounded.toml: writes to output_bounded/ so the baseline output/ stays intact. +# Paths are relative to the PS2Recomp repo root; run ps2_recomp from there. +# Auto-generated by ExportPS2Functions.java +# +# Classification policy (aligned with analyzer intent): +# - runtime-known names -> [general].stubs +# - library-like names without runtime handlers -> [general].untracked_stubs +# - [general].skip is retained empty for legacy compatibility +# - no SCE symbol database is used by this Ghidra script + +[general] +input = "Burnout3_recomp/SLUS_210.50" +output = "Burnout3_recomp/output_bounded" +ghidra_output = "Burnout3_recomp/burnout3_usa_ghidra_functions_instruction_bounded.csv" +single_file_output = false +patch_syscalls = false +patch_cop0 = true +patch_cache = true +stubs = [ + "SetGsCrt@0x0010BA20", + "AddIntcHandler@0x0010BB00", + "AddIntcHandler@0x0010BB10", + "RemoveIntcHandler@0x0010BB20", + "AddDmacHandler@0x0010BB30", + "AddDmacHandler@0x0010BB40", + "RemoveDmacHandler@0x0010BB50", + "EnableIntc@0x0010BB60", + "DisableIntc@0x0010BB70", + "EnableDmac@0x0010BB80", + "DisableDmac@0x0010BB90", + "SetAlarm@0x0010BBA0", + "ReleaseAlarm@0x0010BBB0", + "iEnableIntc@0x0010BBC0", + "iDisableIntc@0x0010BBD0", + "iEnableDmac@0x0010BBE0", + "iDisableDmac@0x0010BBF0", + "iSetAlarm@0x0010BC00", + "CreateThread@0x0010BC20", + "DeleteThread@0x0010BC30", + "StartThread@0x0010BC40", + "ExitDeleteThread@0x0010BC60", + "TerminateThread@0x0010BC70", + "ChangeThreadPriority@0x0010BCB0", + "iChangeThreadPriority@0x0010BCC0", + "RotateThreadReadyQueue@0x0010BCD0", + "GetThreadId@0x0010BD10", + "ReferThreadStatus@0x0010BD20", + "SleepThread@0x0010BD40", + "WakeupThread@0x0010BD50", + "iWakeupThread@0x0010BD60", + "SuspendThread@0x0010BD90", + "ResumeThread@0x0010BDB0", + "EndOfHeap@0x0010BE00", + "CreateSema@0x0010BE20", + "DeleteSema@0x0010BE30", + "SignalSema@0x0010BE40", + "iSignalSema@0x0010BE50", + "WaitSema@0x0010BE60", + "PollSema@0x0010BE70", + "SetOsdConfigParam@0x0010BEC0", + "GetOsdConfigParam@0x0010BED0", + "FlushCache@0x0010C0A0", + "sceSifStopDma@0x0010C0F0", + "GsGetIMR@0x0010C140", + "GsPutIMR@0x0010C160", + "SetVSyncFlag@0x0010C190", + "sceSifDmaStat@0x0010C1C0", + "sceSifSetDma@0x0010C1E0", + "isceSifSetDma@0x0010C1F0", + "sceSifSetDChain@0x0010C200", + "isceSifSetDChain@0x0010C210", + "sceSifSetReg@0x0010C220", + "sceSifGetReg@0x0010C230", + "Deci2Call@0x0010C250", + "GetMemorySize@0x0010C280", + "InitTLB@0x0010C290", + "GetThreadId@0x002372F0", +] +untracked_stubs = [ +] +skip = [] + +[ghidra_export] +function_count = 7776 +code_label_count = 63336 +csv_record_count = 71112 +stub_count = 58 +untracked_stub_count = 0 +skip_count = 0 +uncategorized_count = 7718 +runtime_call_name_count = 656 +runtime_call_source = "embedded_ps2_call_list_snapshot" diff --git a/Burnout3_recomp/tools/ExportBurnout3InstructionRanges.java b/Burnout3_recomp/tools/ExportBurnout3InstructionRanges.java new file mode 100644 index 000000000..7b472ad76 --- /dev/null +++ b/Burnout3_recomp/tools/ExportBurnout3InstructionRanges.java @@ -0,0 +1,43 @@ +// Export evidence for range repair without changing the Ghidra program. +// @category PS2Recomp +import ghidra.app.script.GhidraScript; +import ghidra.program.model.listing.Instruction; +import ghidra.program.model.listing.InstructionIterator; +import ghidra.program.model.mem.MemoryBlock; +import java.io.File; +import java.io.PrintWriter; + +public class ExportBurnout3InstructionRanges extends GhidraScript { + @Override + public void run() throws Exception { + File destination = askFile("Instruction ranges CSV", "Save"); + long start = -1, end = -1, count = 0; + try (PrintWriter writer = new PrintWriter(destination, "UTF-8")) { + writer.println("Start,End"); + InstructionIterator instructions = currentProgram.getListing().getInstructions(true); + while (instructions.hasNext()) { + monitor.checkCancelled(); + Instruction instruction = instructions.next(); + MemoryBlock block = currentProgram.getMemory().getBlock(instruction.getAddress()); + if (block == null || !block.isExecute() || !block.isInitialized() || + !instruction.getAddress().getAddressSpace().equals( + currentProgram.getAddressFactory().getDefaultAddressSpace())) { + continue; + } + if (instruction.getLength() != 4) { + throw new IllegalStateException("Non-word instruction: " + instruction.getAddress()); + } + long address = instruction.getAddress().getOffset(); + if (start >= 0 && address != end) { + writer.printf("0x%08X,0x%08X%n", start, end); + start = -1; + } + if (start < 0) start = address; + end = address + 4; + count++; + } + if (start >= 0) writer.printf("0x%08X,0x%08X%n", start, end); + } + println("Exported contiguous ranges covering " + count + " instructions to " + destination); + } +} diff --git a/Burnout3_recomp/tools/WriteBurnout3AnalysisReport.java b/Burnout3_recomp/tools/WriteBurnout3AnalysisReport.java new file mode 100644 index 000000000..47b49618f --- /dev/null +++ b/Burnout3_recomp/tools/WriteBurnout3AnalysisReport.java @@ -0,0 +1,47 @@ +// Write a read-only audit of the imported program after Ghidra auto-analysis. +// @category PS2Recomp +import ghidra.app.script.GhidraScript; +import ghidra.program.model.listing.Bookmark; +import ghidra.program.model.mem.MemoryBlock; +import java.io.File; +import java.io.PrintWriter; +import java.util.Iterator; +import java.util.Map; +import java.util.TreeMap; + +public class WriteBurnout3AnalysisReport extends GhidraScript { + @Override + public void run() throws Exception { + File output = askFile("Analysis report", "Save"); + try (PrintWriter writer = new PrintWriter(output, "UTF-8")) { + writer.println("Burnout 3 USA — Ghidra analysis report"); + writer.println("Program: " + currentProgram.getName()); + writer.println("Executable: " + currentProgram.getExecutablePath()); + writer.println("Format: " + currentProgram.getExecutableFormat()); + writer.println("Language: " + currentProgram.getLanguageID()); + writer.println("Compiler: " + currentProgram.getCompilerSpec().getCompilerSpecID()); + writer.println("Image base: " + currentProgram.getImageBase()); + writer.println("Functions: " + currentProgram.getFunctionManager().getFunctionCount()); + writer.println("Instructions: " + currentProgram.getListing().getNumInstructions()); + writer.println("\nMemory blocks (name, start, end, executable):"); + for (MemoryBlock block : currentProgram.getMemory().getBlocks()) { + writer.printf("%s\t%s\t%s\t%s%n", block.getName(), block.getStart(), + block.getEnd(), block.isExecute()); + } + writer.println("\nError and warning bookmarks (address, type, category, comment):"); + Map counts = new TreeMap<>(); + Iterator bookmarks = currentProgram.getBookmarkManager().getBookmarksIterator(); + while (bookmarks.hasNext()) { + Bookmark bookmark = bookmarks.next(); + String type = bookmark.getTypeString(); + counts.merge(type, 1, Integer::sum); + if ("Error".equals(type) || "Warning".equals(type)) { + writer.printf("%s\t%s\t%s\t%s%n", bookmark.getAddress(), type, + bookmark.getCategory(), bookmark.getComment().replace('\n', ' ')); + } + } + writer.println("\nBookmark counts: " + counts); + } + println("Analysis report saved: " + output.getAbsolutePath()); + } +} diff --git a/Burnout3_recomp/tools/repair_burnout3_export_ranges.py b/Burnout3_recomp/tools/repair_burnout3_export_ranges.py new file mode 100644 index 000000000..ab64b676f --- /dev/null +++ b/Burnout3_recomp/tools/repair_burnout3_export_ranges.py @@ -0,0 +1,170 @@ +"""Bound exported ranges to saved Ghidra instructions, without running recompilation. + +The repository exporter encloses noncontiguous function bodies and extends some +synthetic labels to the next label, even across data. Intersect its ranges with +the instruction runs exported by ExportBurnout3InstructionRanges.java. Preserve +every original entry and every instruction covered by the original map; assign +entry_ names to additional fragments so branches can still reach them. +""" +from bisect import bisect_right +from pathlib import Path +import csv +import hashlib +import json +import re +import struct +import tomllib + +workspace = Path(__file__).resolve().parents[1] +repo = workspace.parent +# Machine-specific values live in the git-ignored local/ directory. +local = json.loads((workspace / 'local/burnout3_local.json').read_text()) + +def rel(path): + """Repo-relative path, so generated TOMLs and reports carry no home directory.""" + return Path(path).resolve().relative_to(repo).as_posix() +source_csv = workspace / 'backups/before_range_fix/burnout3_usa_ghidra_functions.csv' +source_toml = workspace / 'backups/before_range_fix/burnout3_usa_recomp.toml' +evidence = workspace / 'reports/burnout3_usa_ghidra_instruction_ranges.csv' +fixed_csv = workspace / 'burnout3_usa_ghidra_functions_instruction_bounded.csv' +fixed_toml = workspace / 'burnout3_usa_recomp_instruction_bounded.toml' + +def read_ranges(path): + with path.open(newline='') as stream: + return list(csv.DictReader(stream)) + +def merge(intervals): + result = [] + for start, end in sorted(intervals): + if result and start <= result[-1][1]: + result[-1] = (result[-1][0], max(result[-1][1], end)) + else: + result.append((start, end)) + return result + +runs = [(int(r['Start'], 0), int(r['End'], 0)) for r in read_ranges(evidence)] +assert runs and runs == sorted(runs) +assert all(start % 4 == end % 4 == 0 and start < end for start, end in runs) +assert all(runs[i][1] < runs[i + 1][0] for i in range(len(runs) - 1)) +run_starts = [start for start, _ in runs] +rows = read_ranges(source_csv) +originals = {int(r['Start'], 0): r for r in rows} +assert len(originals) == len(rows) +fragments = {} +retained = [] +changes = [] +for row in rows: + start, end = int(row['Start'], 0), int(row['End'], 0) + index = bisect_right(run_starts, start) - 1 + # Fail rather than silently remove an original entry not backed by instructions. + assert index >= 0 and start < runs[index][1], f'Original entry is not code: {row}' + pieces = [] + while index < len(runs) and runs[index][0] < end: + low, high = max(start, runs[index][0]), min(end, runs[index][1]) + if low < high: + pieces.append((low, high)) + retained.append((low, high)) + fragments[low] = max(fragments.get(low, low), high) + index += 1 + if pieces != [(start, end)]: + changes.append({'name': row['Name'], 'start': f'0x{start:08X}', + 'original_end': f'0x{end:08X}', + 'instruction_fragments': [[f'0x{a:08X}', f'0x{b:08X}'] for a, b in pieces]}) + +# Preserve the original extent of every existing entry (bounded to its code run). +# Newly introduced fragments may overlap existing entries; this is supported by +# the recompiler, just like the original function/entry records. +for start, row in originals.items(): + index = bisect_right(run_starts, start) - 1 + fragments[start] = min(int(row['End'], 0), runs[index][1]) +assert set(originals).issubset(fragments) +assert merge(fragments.items()) == merge(retained), 'Repair would lose originally exported instructions' +assert all(start < end <= runs[bisect_right(run_starts, start) - 1][1] + for start, end in fragments.items()) + +# Validate the exact primary-MMI warning condition using this checkout's decoder. +data = (workspace / 'SLUS_210.50').read_bytes() +assert hashlib.sha256(data).hexdigest() == local['elf_sha256'] +header = struct.unpack_from('> 26 == 0x1C and word & 0x3F not in accepted: + result.append(address) + return result + +before_warnings = warning_addresses(old_union) +after_warnings = warning_addresses(new_union) +assert before_warnings, 'Expected diagnostic reproduction for original map' +assert not after_warnings, f'Unsupported primary MMI words remain: {after_warnings[:10]}' +assert fragments[0x4B9238] == 0x4B9248 +assert not any(start <= 0x4BA254 < end for start, end in new_union) +assert any(start <= header[3] < end for start, end in new_union) + +with fixed_csv.open('w', newline='') as stream: + writer = csv.writer(stream, lineterminator='\n') + writer.writerow(['Name', 'Start', 'End', 'Size']) + for start, end in sorted(fragments.items()): + name = originals[start]['Name'] if start in originals else f'entry_{start:08x}' + writer.writerow([name, f'0x{start:08X}', f'0x{end:08X}', end - start]) +config_text = source_toml.read_text() +config = tomllib.loads(config_text) +for key, value in [('input', rel(workspace / 'SLUS_210.50')), + ('output', rel(workspace / 'output')), + ('ghidra_output', rel(fixed_csv))]: + config_text = config_text.replace(f'{key} = ' + json.dumps(config['general'][key]), + f'{key} = ' + json.dumps(value)) +added = len(fragments) - len(rows) +for key, value in [('csv_record_count', len(fragments)), + ('code_label_count', config['ghidra_export']['code_label_count'] + added)]: + config_text = re.sub(rf'(?m)^{key} = \d+$', f'{key} = {value}', config_text) +config_text = ('# Instruction-bounded copy; original exports preserved in backups/before_range_fix.\n' + '# Paths are relative to the PS2Recomp repo root; run ps2_recomp from there.\n' + config_text) +parsed = tomllib.loads(config_text) +assert parsed['general']['ghidra_output'] == rel(fixed_csv) +fixed_toml.write_text(config_text) + +report = { + 'status': 'PASS', 'recompilation_run': False, + 'original_csv': rel(source_csv), 'instruction_evidence': rel(evidence), + 'corrected_csv': rel(fixed_csv), 'corrected_toml': rel(fixed_toml), + 'original_records': len(rows), 'corrected_records': len(fragments), + 'original_entries_preserved': len(originals), 'added_code_fragments': added, + 'changed_original_ranges': len(changes), + 'original_unique_covered_bytes': sum(b - a for a, b in old_union), + 'corrected_unique_instruction_bytes': sum(b - a for a, b in new_union), + 'excluded_non_instruction_bytes': sum(b - a for a, b in old_union) - sum(b - a for a, b in new_union), + 'primary_mmi_warning_addresses_before': len(before_warnings), + 'primary_mmi_warning_addresses_after': len(after_warnings), + 'checks': ['All original entry addresses and names preserved', + 'All instructions covered by the original export are still covered', + 'No corrected range crosses a Ghidra data/undefined gap', + 'Known string at 0x004BA254 is no longer decoded', + 'entry_004b9238 now ends at 0x004B9248', + 'ELF entry point remains covered', + 'No unknown primary MMI encodings in corrected ranges'], + 'limitations': ['Uses the existing Ghidra analysis, including its reported errors.', + 'Does not establish runtime correctness or game playability.', + 'Recompiler may discover additional ranges during its own passes; full-run results are not yet verified.'], + 'sha256': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in [fixed_csv, fixed_toml, evidence]}, + 'range_changes': changes, +} +report_path = workspace / 'reports/06_burnout3_export_range_repair.json' +report_path.write_text(json.dumps(report, indent=2) + '\n') +print(json.dumps({k: v for k, v in report.items() if k != 'range_changes'}, indent=2)) +print('Full repair report:', report_path) diff --git a/Burnout3_recomp/tools/verify_burnout3_exports.py b/Burnout3_recomp/tools/verify_burnout3_exports.py new file mode 100644 index 000000000..aac51138c --- /dev/null +++ b/Burnout3_recomp/tools/verify_burnout3_exports.py @@ -0,0 +1,127 @@ +"""Validate the Ghidra exports locally without invoking the recompiler.""" +from pathlib import Path +from bisect import bisect_right +import csv +import hashlib +import json +import struct +import tomllib + +workspace = Path(__file__).resolve().parents[1] +repo = workspace.parent +# Machine-specific values live in the git-ignored local/ directory. +local = json.loads((workspace / 'local/burnout3_local.json').read_text()) +original = Path(local['original_elf']).expanduser() +working = workspace / 'SLUS_210.50' +config = workspace / 'burnout3_usa_recomp_instruction_bounded.toml' +function_map = workspace / 'burnout3_usa_ghidra_functions_instruction_bounded.csv' +expected_hash = local['elf_sha256'] +checks = [] + +def resolve(path): + """TOML paths are relative to the repo root (absolute paths still resolve).""" + return (repo / path).resolve() + +def rel(path): + return Path(path).resolve().relative_to(repo).as_posix() + +def check(condition, message): + if not condition: + raise ValueError(message) + checks.append(message) + +data = working.read_bytes() +check(hashlib.sha256(data).hexdigest() == expected_hash, 'Working ELF matches the original verified SHA-256') +check(original.read_bytes() == data, 'Original ELF still exists and matches the working copy') +check(data[:6] == b'\x7fELF\x01\x01', 'ELF is 32-bit little-endian') +header = struct.unpack_from(' 0, 'Ghidra exported discovered functions') +check(len(general['stubs']) == metadata['stub_count'], 'Stub count matches export metadata') +check(len(general['untracked_stubs']) == metadata['untracked_stub_count'], 'Untracked stub count matches export metadata') +starts = [] +ranges = [] +outside = [] +noncontiguous = 0 +for row_number, row in enumerate(rows, 2): + if None in row or any(value is None for value in row.values()): + raise ValueError(f'Malformed CSV row {row_number}') + start, end, size = int(row['Start'], 0), int(row['End'], 0), int(row['Size']) + if not row['Name'] or not (0 <= start < end <= 0x100000000) or size <= 0: + raise ValueError(f'Invalid CSV range at row {row_number}') + if start % 4 or end % 4: + raise ValueError(f'Unaligned MIPS range at row {row_number}') + starts.append(start) + ranges.append((start, end)) + if size != end - start: + noncontiguous += 1 + if not any(low <= start < high for low, high in sections): + outside.append(row) +check(starts == sorted(starts), 'CSV records are sorted by address') +check(len(set(starts)) == len(starts), 'CSV start addresses are unique') +check(any(low <= entry < high for low, high in ranges), 'ELF entry point is covered by the CSV') +check(not outside, 'All exported entry addresses are within ELF executable sections') +with (workspace / 'reports/burnout3_usa_ghidra_instruction_ranges.csv').open(newline='') as stream: + code_runs = [(int(row['Start'], 0), int(row['End'], 0)) for row in csv.DictReader(stream)] +code_starts = [start for start, _ in code_runs] +for start, end in ranges: + index = bisect_right(code_starts, start) - 1 + if index < 0 or not (code_runs[index][0] <= start < end <= code_runs[index][1]): + raise ValueError(f'Export crosses a non-instruction gap: {start:08X}-{end:08X}') +check(True, 'Every exported range consists entirely of contiguous Ghidra instructions') +repair_report = json.loads((workspace / 'reports/06_burnout3_export_range_repair.json').read_text()) +check(repair_report['primary_mmi_warning_addresses_after'] == 0, 'Range repair reports no unknown primary MMI encodings') +for path in [config, function_map, workspace / 'reports/burnout3_usa_ghidra_instruction_ranges.csv']: + check(hashlib.sha256(path.read_bytes()).hexdigest() == repair_report['sha256'][path.name], + f'{path.name} matches the range repair report SHA-256') +check((workspace / 'ghidra_project/Burnout3_USA_Recomp.gpr').exists(), 'Saved Ghidra project exists') +analysis_log = (workspace / 'logs/02_ghidra_analysis_console.log').read_text() +check('Analysis succeeded for file: /SLUS_210.50' in analysis_log, 'Ghidra reported successful auto-analysis') +check('Save succeeded' in analysis_log, 'Ghidra reported successful project save') +export_log = (workspace / 'logs/03_04_ghidra_export_console.log').read_text() +check('Exported TOML config to' in export_log, 'Repository exporter completed') + +report = { + 'status': 'PASS', + 'scope': 'Export validation only; does not cover recompiler output.', + 'original_elf': local['original_elf'], 'working_elf': rel(working), + 'elf_size_bytes': len(data), 'elf_sha256': expected_hash, + 'elf_entry_point': f'0x{entry:08X}', + 'config': rel(config), 'function_map': rel(function_map), + 'ghidra_export': metadata, + 'noncontiguous_body_record_count': noncontiguous, + 'checks': checks, + 'limitations': [ + 'Validation checks export integrity and consistency, not complete or perfect function recovery.', + 'See the saved analysis report and logs for Ghidra warnings and error bookmarks.', + 'Repository exports were repaired using saved Ghidra instruction ranges; originals were preserved.' + ], + 'export_sha256': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in [config, function_map]}, +} +destination = workspace / 'reports/07_burnout3_corrected_export_validation.json' +destination.write_text(json.dumps(report, indent=2) + '\n') +print(json.dumps(report, indent=2))