Conversation
Since Binutils 2.41 (PR30343) the PE linker creates an early undefined reference to _pei386_runtime_relocator whenever the LTO plugin is loaded, which an LTO-enabled GCC does on every link. With -r that reference leaks into the output as a stray undefined symbol, breaking later -nostdlib links of the result. Skip it for relocatable output, where the final link makes its own reference. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Build GCC with LTO support and compile the three runtime libraries that benefit most as fat LTO objects. They link as before without -flto, while an -flto link can inline into them and, more importantly, drop their unused functions, which --gc-sections cannot do on x86_64 PE because .pdata roots every function. libstdc++'s largest members (locale, format, and stream instantiations) are only partly used by any one program, so this is where the size win concentrates. libstdc++ is the only C++ target library, so CXXFLAGS_FOR_TARGET reaches it alone. libgfortran and libquadmath share CFLAGS_FOR_TARGET with libgcc, libgomp, and libatomic, which stay plain (libgcc feeds the -r tricks in this repository; the others synchronize threads and must not have their translation-unit boundaries dissolved), so those two are rebuilt after the main build with their own flags. The LTO plugin is also installed in lib/bfd-plugins so that plain ar, nm, and ranlib handle slim LTO objects without gcc-ar. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
When a function's mangled name changed in some ABI version, GCC also emits its old name as a weak alias so that objects from older compilers still link. By default it does so for changes since ABI 18 (GCC 13), and -Wabi=N quietly implies -fabi-compat-version=N, which libstdc++ passes (-Wabi=19) and so ends up with a different set of aliases than user code. That asymmetry breaks LTO on PE: the aliases share a COMDAT group with their target, ld cannot reconcile a group with two members in one IR object and one in another, and any C++20 constrained function explicitly instantiated in libstdc++ and implicitly in an -flto user object fails with a multiple definition error. This toolchain is static and self-contained, so there are no older objects to stay compatible with. Emit no compatibility aliases unless -fabi-compat-version is given explicitly, and leave -Wabi= as a pure diagnostic. This goes in the bootstrap queue because the cross compiler is what compiles libstdc++. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Disable the dual ABI, and patch libstdc++ to keep the new SSO string rather than the old gcc4-compatible COW string that configure imposes on single-ABI builds. The dual ABI exists for link compatibility with binaries built against GCC 4-era libstdc++, which a from-scratch static-only toolchain does not need, and it costs every program dead weight at archive-member granularity: both ABIs' locale facets and string instantiations, plus the facet shims to bridge them. The library nearly supports this configuration already. Beyond the configure default, the patch defines the new-ABI ios_base::failure in a C++11 translation unit, and extends the instantiate-once guards on ABI-agnostic locale instantiations to cover single-ABI builds. The Transactional Memory TS clones in cow-stdexcept.cc are written against COW string internals (_Rep, _M_dataplus) and so can only be compiled when std::basic_string is the COW string. This is invisible on x86_64, where _GLIBCXX_USE_WEAK_REF is false and the section is skipped entirely, but breaks the i686 build, where it is true. Compile the clones only when the string ABI matches. Their absence is legal by design: they are weakly referenced, and -fgnu-tm code falls back to serial-irrevocable execution. (-fgnu-tm with libstdc++ headers is broken in GCC 16.2 regardless, in both dual-ABI and single-ABI builds: new_allocator.h trips over __builtin_operator_new selection.) Now that libstdc++ is built as fat LTO objects, this also removes most of the library's deliberate ODR violations from what an -flto link can see: the punned __cow_string and __sso_string redefinitions and the old-ABI compile of the standard exception classes no longer exist, so -Wodr stops reporting them on every link. A trivial C++ program shrinks from 104K to 73K, iostream hello world from 1010K to 656K, and libstdc++.a from 8.1M to 6.0M. Exceptions, iostreams with error codes, locales, and std::filesystem verified working. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The driver copies argv[0] verbatim into COLLECT_GCC, which lto-wrapper uses as the compiler for the LTRANS stage. For parallel LTO (-flto=N, -flto=auto, or a detected jobserver) lto-wrapper writes that path unquoted into a generated Makefile and runs make, whose shell then consumes the backslashes of a Windows path, leaving "C:Users...g++.exe" and a failed link. Since the driver now always replaces argv[0] with the GetModuleFileNameA result, every -flto=N link was affected. Convert the path to forward slashes, which GCC accepts everywhere. The i686-dumpmachine patch is refreshed for the shifted hunk. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With full plugin support in the linker, the driver passes the LTO plugin on every link, and the plugin claims any input containing LTO bytecode. Since libstdc++, libgfortran, and libquadmath are now fat LTO objects, that recompiled the library members of every C++ program whether or not the user asked for LTO: links took 4x longer for tiny programs and 30-45x for typical ones, lto-wrapper nagged about serial compilation, and every LTO edge case reached every user. Only pass the plugin when -flto, -flto=N, or -fuse-linker-plugin is on the link line. Ordinary links use the libraries' regular code exactly as before, and -flto links get the full benefit, including dead-code elimination inside the libraries. The one behavior change is that a slim LTO object linked without -flto now fails with "plugin needed to handle lto object", which is what the GCC manual documents anyway. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two parts of libstdc++ cannot safely be link-time optimized, even though the rest of the library is built as fat LTO objects. libsupc++ is the C++ runtime proper. Compiler-generated code refers to some of its entry points in ways the LTO symbol resolution never sees: a must-not-throw landing pad calls __cxa_call_terminate, but that call is only materialized late, in pass_lower_resx, after whole-program analysis has already dropped the apparently unreferenced definition from eh_call.o. An -flto link of a program using <thread> or <format> then fails with "defined in discarded section". Compile all of libsupc++ without LTO, as libgcc is. ios_init.cc defines cout, cin, and the other standard streams with deliberately unrelated types (character arrays sized and aligned like the stream classes) to control their construction. That is a textbook ODR violation that the compiler can only see across translation units with LTO, and -Wodr then reports every one of them on every -flto link that uses iostreams. Keep that one file plain as well. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On Windows, make finds its shell by searching PATH for sh.exe and falls back to cmd.exe, whose quoting rules break ordinary recipes. When make is invoked by path without w64devkit's bin/ on PATH, that search fails even though sh.exe sits right next to make.exe, and it does not help to set SHELL in the environment, which make ignores. Check make's own directory first, so that make works on its own like the rest of the toolchain. This is what GCC's lto-wrapper needs for parallel LTO in a PATH-free invocation. Verified with a native build: invoked with an empty PATH, the patched make runs a recipe through the adjacent sh.exe, and falls back to cmd.exe as before when sh.exe is removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Parallel LTRANS (-flto=N, -flto=auto, or a detected jobserver) hands the partitions to make through a generated Makefile, and lto-wrapper finds make only through $MAKE or PATH. When the toolchain is invoked by path without its directory on PATH, which the self-locating driver otherwise supports, that search fails and lto-wrapper silently falls back to serial compilation, with a warning suggesting -flto=auto that the user may already be passing. After $MAKE and PATH, also try make in the driver's own directory, known from COLLECT_GCC, and run that one. Together with make locating its shell beside itself, an -flto=auto link now works in parallel with nothing on PATH; verified by pointing $MAKE at the kit's make from an empty PATH, which exercises the same invocation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The linker deduplicates COMDAT sections by key. For a real COFF section the key is its COMDAT symbol name, which on i386 PE carries the user label prefix (__ZTISt9bad_alloc). For a symbol from an LTO IR object, ld creates a placeholder section named after the COMDAT group the compiler reported, which is the group name without any prefix (_ZTISt9bad_alloc). The two never match, so whenever a symbol is defined both by a regular object and by LTO bytecode, ld reports a multiple definition instead of keeping one copy. That is every -flto link on i686 once libstdc++ carries bytecode, since libsupc++ and the CRT are regular objects that define the same typeinfo and template instantiations, and it also breaks linking a plain object into an -flto program. x86_64 PE has no prefix and is unaffected. Strip the target's symbol leading char from keys derived from real objects so that they match the compiler's group names. Verified with native x86_64 and i686 builds of ld against both kits: -flto links, mixed plain and -flto links, and ordinary links all behave, and x86_64 is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
LTO bytecode packs a REAL_VALUE_TYPE's significand as SIGSZ words of HOST_BITS_PER_LONG bits, which is the host's own layout: three 64-bit words on the Linux build host, five 32-bit words on Windows. The runtime libraries' bytecode is produced by the Linux-hosted cross compiler and consumed by the Windows-hosted compiler, so every floating-point constant in it arrived garbled: FLT_MAX read as zero, libgfortran printed every real as Infinity, and std::from_chars returned 0 for any floating input. Integers are unaffected because they stream as HOST_WIDE_INT, which is 64 bits on both hosts. Stream the significand's top 128 bits as four 32-bit words instead, most significant first, which covers every target format (binary128 needs 113), and the 16-byte decimal128 encoding in memory order for decimal values. Both hosts have at least 128 significand bits. The chunk arithmetic was checked to round-trip between the two layouts. This changes the bytecode format relative to upstream, so objects compiled by another GCC 16.2 build would misread constants here; the toolchain is self-contained and the version check never covered host differences anyway. Applied before both compilers are built. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
By default a DLL exports every global symbol, minus those from the standard libraries, startup objects, and (here) any archive under the sysroot, which ld filters by the symbol's archive of origin, and minus weak symbols, which is what COMDAT template instantiations are. LTO defeated all of that: a DLL using std::string exported 704 symbols instead of 2, with every one of them pinned against dead-code elimination. The exports came from the placeholder bfds that stand in for claimed IR objects. They stay in the input list after LTO, still listing every definition the plugin reported as a strong global, and they have no archive to be excluded by. Skip them in the export loop; the definitions that survive LTO are in the output objects the plugin added, and only those count. That alone would still export what the compiler keeps global, since ld tells the plugin that in a DLL link every symbol may be visible from outside. Add an emulation hook consulted during symbol resolution, which the PE emulations answer with the same auto-export decision, using the IR object's original archive member and treating weak or COMDAT definitions as unexportable. A library symbol is then reported as not visible, so the compiler localizes and prunes it as it would in an executable, and its export name is remembered so that auto-export also rejects the ones that survive as globals because regular objects reference them. Symbols listed in a .def file remain exported, and dllexport attributes are unaffected since the compiler honors them regardless of resolution. Verified with a native x64 build of ld on both the x64 and -m32 multilib: export-all LTO DLLs export exactly what non-LTO ones do and come out half the size, dllexport and .def exports are kept, the DLLs work when called, and executables are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
|
Last I checked, gcc couldn't build itself at -O2 (for windows) with lto without giving an ICE. Hopefully that changed. |
Owner
Author
|
I didn't think building GCC itself with LTO would be worth much except as a way to detect LTO bugs earlier. Worth at least testing, though. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch is a serious exploration of re-enabling LTO and shipping fat LTO libstdc++, libgfortran, and libquadmath. My personal motivation is to produce better C++ binaries: smaller and faster. In this branch LTO is opt-in and so costs nothing (other than installation size) unless enabled with
-fltoat both compile and link time. C++ and Fortran binaries compiled with LTO enabled are significantly smaller, probably faster (TODO on benchmarks), and users aren't locked into-O2, though the link-time cost is substantial.This branch includes patches that improve and fix a number of issues with LTO in GCC and Binutils. Cross-compiled LTO doesn't work out-of-the-box with GCC due to bytecode generation bugs in GCC (w64dk may be the first project to ever attempt it), patched in this branch by necessity. In general, most of the effort of this branch has been playing whack-a-mole with GNU toolchain LTO bugs. That's exactly why I disabled LTO in the first place: It's difficult to trust that any given binary linked using GNU LTO is built correctly!
Concurrency runtimes are deliberately excluded from LTO: winpthreads, libgomp, and libatomic. I know for a fact that winpthreads is broken with LTO, and I will not trust the others with LTO without and audit (TODO).
The business with
makeis because-flto=autointegrates with it (jobserver).