Fix incremental compilation for fat LTO by bjorn3 · Pull Request #163317 · rust-lang/rust

bjorn3 · 2026-09-25T10:18:19Z

Previously we would copy post-LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre-LTO bitcode didn't get tracked and thus determine_cgu_reuse believed it had to regenerate bitcode for all CGUs even when all CGUs would be green.

Also move incr comp session dirs out of CodegenContext. CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Part of rust-lang/compiler-team#908

rustbot · 2026-09-25T10:18:22Z

rustc_codegen_gcc is developed in its own repository. If possible, consider making this change to rust-lang/rustc_codegen_gcc instead.

cc @antoyo, @GuillaumeGomez

rustbot · 2026-09-25T10:18:25Z

r? @mejrs

rustbot has assigned @mejrs.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: codegen, compiler
codegen, compiler expanded to 77 candidates
Random selection from 21 candidates

bjorn3 · 2026-09-25T12:12:09Z

            if !self.check_config(cfg) {
                debug!("check_attr: config does not match, ignoring attr");
-                return;
+                continue;


This caused us to skip all assertions after the first one with a cfg that doesn't match. Surprisingly nothing regressed by the lack of test coverage since the introduction of this bug.

View changes since the review

mejrs

Not sure about this otherwise.

r? @nnethercote perhaps?

View changes since this review

nnethercote

r=me with the nits addressed.

View changes since this review

nnethercote · 2026-10-02T03:03:32Z

+                };
+
+                // FIXME include pre-LTO bitcode in workproduct tracking
+                // FIXME add separate incr comp session for post-LTO outputs to use during link step


How important are these FIXMEs? Are you planning to fix them?

I'm working on a follow up PR to do this.

bjorn3 · 2026-10-02T09:01:09Z

@bors r=nnethercote

rust-bors · 2026-10-02T09:01:12Z

📌 Commit ebdd423 has been tentatively approved by nnethercote

It will be put into the queue for this repository once PR CI succeeds.

Fix incremental compilation for fat LTO Previously we would copy post-LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre-LTO bitcode didn't get tracked and thus `determine_cgu_reuse` believed it had to regenerate bitcode for all CGUs even when all CGUs would be green. Also move incr comp session dirs out of `CodegenContext`. `CodegenContext` will in the future be serialized and deserialized into a different rustc instance when `-Zno-link`/`-Zlink-only` is used. A separate incr comp session has to be used for both rustc invocations. Part of rust-lang/compiler-team#908

…uwer Rollup of 13 pull requests Successful merges: - #163317 (Fix incremental compilation for fat LTO) - #163582 (Reapply "bootstrap: Enable rustdoc mergeable CCI for std and internal docs") - #151793 (Add mul_add_relaxed methods for floating-point types) - #162782 (Fix rustdoc ICE caused by mishandling of ambiguity errors) - #163010 (Miri can do dirfd now) - #163535 (Improve `DocStrings` perf) - #163576 (Fix `TypeOutlives` fast-path) - #163587 (Several small span improvements) - #163612 (fix `ValidateBoundVars`) - #163632 (bump rustc-build-sysroot) - #163635 (Revert note about signum of NaN) - #163644 (Add mailmap entry) - #163651 (Remove variants from `feature-gate-autodiff-use` test)

JonathanBrouwer · 2026-10-02T14:07:42Z

💔 I suspect this PR failed tests as part of a rollup
@bors r-

After fixing the problem, consider running a try job for the failed job before re-approving.

Link to failure: #163650 (comment)

@bors try jobs=test-x86_64-mingw-1

rust-bors · 2026-10-02T14:07:51Z

This pull request was unapproved.

This PR was contained in a rollup (#163656), which was unapproved.

View changes since this unapproval

rustbot · 2026-10-04T14:36:33Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

bjorn3 · 2026-10-04T14:37:12Z

Doesn't reproduce on Linux, so let's try bisecting it using CI.

@bors try jobs=test-x86_64-mingw-1

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

bjorn3 · 2026-10-04T15:42:23Z

@bors try jobs=test-x86_64-mingw-1

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

rust-bors · 2026-10-04T17:02:15Z

💔 Test for 22eb763 failed: CI. Failed job:

try - test-x86_64-mingw-1 (web logs, enhanced plaintext logs)

bjorn3 · 2026-10-04T17:25:00Z

@bors try jobs=test-x86_64-mingw-1

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

rust-bors · 2026-10-04T20:23:38Z

☀️ Try build successful (CI)
Build commit: a640230 (a6402304796db36e1ddf97b58ecdaa1676982c31)
Base parent: a639ea0 (a639ea0890c0977b1c45596fd6b5f9a70a1d67da)

Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>

CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Previously we would copy post LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre LTO bitcode didn't get tracked and thus determine_cgu_reuse believed it had to regenerate bitcode for all CGUs even when all CGUs would be green.

bjorn3 · 2026-10-08T10:14:31Z

Reverted some changes that turned out needed to be reverted anyway in the follow up PR.

@bors try jobs=test-x86_64-mingw-1

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

rust-bors · 2026-10-08T11:59:38Z

💔 Test for d316683 failed: CI. Failed job:

try - test-x86_64-mingw-1 (web logs, enhanced plaintext logs)

rust-log-analyzer · 2026-10-08T11:59:39Z

The job test-x86_64-mingw-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- [run-make] tests\run-make\lto-empty stdout ----

error: rmake recipe failed to complete
status: exit code: 1
command: "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake.exe"
stdout: none
--- stderr -------------------------------
"D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\stage2\\bin\\rustc.exe" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "lib.rs" "-Clto=fat" "-Copt-level=3" "-Cincremental=inc-fat" "--target=x86_64-pc-windows-gnu"
output status: `exit code: 0`
=== STDOUT ===



=== STDERR ===



command failed at line 16
"D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\stage2\\bin\\rustc.exe" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "lib.rs" "-Clto=fat" "-Copt-level=3" "-Cincremental=inc-fat" "--target=x86_64-pc-windows-gnu"
output status: `exit code: 1`
=== STDOUT ===



=== STDERR ===
error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32-gcc" "-Wl,D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out\\rustcsXwnaP\\list.def" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "<21 object files omitted>" "-Wl,-Bstatic" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out\\rustcsXwnaP/libstd-f19ec5cf82cf2e36.rlib" "<sysroot>\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib/libcompiler_builtins-*.rlib" "-Wl,-Bdynamic" "-lkernel32" "-lkernel32" "-lntdll" "-luserenv" "-lws2_32" "-ldbghelp" "-lsynchronization" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-lmingwex" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "-o" "lib.dll" "-Wl,--gc-sections" "-shared" "-Wl,--out-implib=liblib.dll.a" "-Wl,-O1" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.object-748e56779e68aa6c.object.6c40391e7dfd59d8-cgu.0.rcgu.o.16j2f4c.rcgu.o:object.6c40391e7dfd59d8-cgu.0:(.text+0x45b): undefined reference to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.object-748e56779e68aa6c.object.6c40391e7dfd59d8-cgu.0.rcgu.o.16j2f4c.rcgu.o:object.6c40391e7dfd59d8-cgu.0:(.text+0x536): undefined reference to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x21509): undefined reference to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x215b0): undefined reference to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x21729): undefined reference to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x2179e): more undefined references to `__imp__RNvNvNtNtNtCsc1WNlW8Q6B2_6memchr4arch6x86_646memchr10memchr_raw2FN' follow
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x267e6): undefined reference to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x27ae7): undefined reference to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x27edf): undefined reference to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x27fe8): undefined reference to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x2adf6): undefined reference to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.746a59d7eb87f6c-cgu.0.rcgu.o.16j2f4c.rcgu.o:std.746a59d7eb87f6c-cgu.0:(.text+0x2b88b): more undefined references to `__imp__RNvNtNtNtCshvGoyLvJpmK_4core2io5error12os_functions12OS_FUNCTIONS' follow
          collect2.exe: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link

error: aborting due to 1 previous error
---
Currently active steps:
test::RunMake { test_compiler: Compiler { stage: 2, host: x86_64-pc-windows-gnu, forced_compiler: false }, target: x86_64-pc-windows-gnu } at src\bootstrap\src\core\build_steps\test.rs:2102
test::Compiletest { test_compiler: Compiler { stage: 2, host: x86_64-pc-windows-gnu, forced_compiler: false }, target: x86_64-pc-windows-gnu, mode: run-make, suite: "run-make", path: "tests/run-make", compare_mode: None } at src\bootstrap\src\core\build_steps\test.rs:2102
Build completed unsuccessfully in 1:38:02
make: *** [Makefile:126: ci-mingw-x] Error 1
  local time: Thu Oct  8 11:59:24 CUT 2026
  network time: Thu, 08 Oct 2026 11:59:24 GMT
##[error]Process completed with exit code 2.
##[group]Run echo "disk usage:"
echo "disk usage:"

rustbot assigned mejrs Sep 25, 2026

bjorn3 mentioned this pull request Sep 25, 2026

Move LTO from the codegen coordinator thread to link_binary rust-lang/compiler-team#908

Closed

14 of 22 tasks

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the lto_refactors22 branch from bc690ff to 980f03b Compare September 25, 2026 12:10

bjorn3 commented Sep 25, 2026

View reviewed changes

bjorn3 changed the title ~~Fix incr comp for fat LTO~~ Fix incremental compilation for fat LTO Sep 27, 2026

mejrs reviewed Sep 30, 2026

View reviewed changes

Comment thread compiler/rustc_codegen_llvm/src/back/llvm_backend.rs

rustbot assigned nnethercote and unassigned mejrs Sep 30, 2026

nnethercote approved these changes Oct 2, 2026

View reviewed changes

nnethercote added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 2, 2026

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 2, 2026

JonathanBrouwer mentioned this pull request Oct 2, 2026

Rollup of 8 pull requests #163650

Closed

JonathanBrouwer mentioned this pull request Oct 2, 2026

Rollup of 13 pull requests #163656

Closed

rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 2, 2026

This comment has been minimized.

Sign in to view

bjorn3 added 2 commits October 4, 2026 14:36

Remove unused method

39a66e7

Move some LLVM specific options from Linker::link to cg_llvm

dc513a5

bjorn3 force-pushed the lto_refactors22 branch from ebdd423 to e672a1c Compare October 4, 2026 14:36

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 4, 2026

Auto merge of #163317 - bjorn3:lto_refactors22, r=<try>

d20724a

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 4, 2026

Auto merge of #163317 - bjorn3:lto_refactors22, r=<try>

22eb763

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

This comment has been minimized.

Sign in to view

bjorn3 force-pushed the lto_refactors22 branch from 6dce75b to 993bdd7 Compare October 4, 2026 17:24

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 4, 2026

Auto merge of #163317 - bjorn3:lto_refactors22, r=<try>

a640230

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

bjorn3 and others added 4 commits October 8, 2026 10:03

Apply suggestion

2b7a3ab

Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>

Move incr comp session dirs out of CodegenContext

e243b6d

CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Add test for incr comp fat LTO and fix assert_module_sources testing

5951d04

bjorn3 force-pushed the lto_refactors22 branch from 993bdd7 to 5951d04 Compare October 8, 2026 10:13

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 8, 2026

Auto merge of #163317 - bjorn3:lto_refactors22, r=<try>

d316683

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

Conversation

bjorn3 commented Sep 25, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Sep 25, 2026

Uh oh!

rustbot commented Sep 25, 2026

Uh oh!

This comment has been minimized.

bjorn3 Sep 25, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mejrs left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nnethercote left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nnethercote Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bjorn3 Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bjorn3 commented Oct 2, 2026

Uh oh!

rust-bors Bot commented Oct 2, 2026

Uh oh!

JonathanBrouwer commented Oct 2, 2026

Uh oh!

rust-bors Bot commented Oct 2, 2026

Uh oh!

This comment has been minimized.

rustbot commented Oct 4, 2026

Uh oh!

bjorn3 commented Oct 4, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

bjorn3 commented Oct 4, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Oct 4, 2026

Uh oh!

This comment has been minimized.

bjorn3 commented Oct 4, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Oct 4, 2026

Uh oh!

bjorn3 commented Oct 8, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Oct 8, 2026

Uh oh!

rust-log-analyzer commented Oct 8, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bjorn3 commented Sep 25, 2026 •

edited by rustbot

Loading

bjorn3 Sep 25, 2026 •

edited by rustbot

Loading

mejrs left a comment •

edited by rustbot

Loading

nnethercote left a comment •

edited by rustbot

Loading