Bug: Compiling in release mode with cross-module-optimization causes compiler crash

[true-false-maybe]

When compiling a project which depends on OpenAPIKit i get this compiler crash:

[20/27] Compiling OpenAPIKitCore AnyCodable+Validatable.swift
error: compile command failed due to signal 6 (use -v to see invocation)
Abort: function forAbstract at ASTContext.cpp:5924
Abstract conformance with bad subject type:
(element_type level=1
  (pack=pack_archetype_type address=0x55c429ef0428 conforms_to="OpenAPIKitCore.(file).Validatable@/builds/project/.build/checkouts/OpenAPIKit/Sources/OpenAPIKitCore/Validatable.swift:37:17" conforms_to="Swift.(file).Encodable" name="each T"
    (interface_type=generic_type_param_type depth=0 index=0 name="T" param_kind=pack)))
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace.
Stack dump:

Stack dump

1.	Swift version 6.3.3 (swift-6.3.3-RELEASE)
2.	Compiling with the current language version
3.	While evaluating request ExecuteSILPipelineRequest(Run pipelines { PrepareOptimizationPasses, EarlyModulePasses, HighLevel,Function+EarlyLoopOpt, HighLevel,Module+StackPromote, MidLevel,Function, ClosureSpecialize, LowLevel,Function, LateLoopOpt, SIL Debug Info Generator } on SIL for OpenAPIKit)
4.	While running pass #863268 SILModuleTransform "CrossModuleOptimization".
5.	Abort: function forAbstract at ASTContext.cpp:5924
| 	Abstract conformance with bad subject type:
| 	(element_type level=1
| 	  (pack=pack_archetype_type address=0x55c429ef0428 conforms_to="OpenAPIKitCore.(file).Validatable@/builds/project/.build/checkouts/OpenAPIKit/Sources/OpenAPIKitCore/Validatable.swift:37:17" conforms_to="Swift.(file).Encodable" name="each T"
| 	    (interface_type=generic_type_param_type depth=0 index=0 name="T" param_kind=pack)))
 #0 0x000055c4117d7f38 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/bin/swift-frontend+0x72e4f38)
 #1 0x000055c4117d5a15 llvm::sys::RunSignalHandlers() (/usr/bin/swift-frontend+0x72e2a15)
 #2 0x000055c4117d8cc1 SignalHandler(int, siginfo_t*, void*) Signals.cpp:0:0
 #3 0x00007f6ce5a93050 (/lib/x86_64-linux-gnu/libc.so.6+0x3c050)
 #4 0x00007f6ce5ae1eec (/lib/x86_64-linux-gnu/libc.so.6+0x8aeec)
 #5 0x00007f6ce5a92fb2 raise (/lib/x86_64-linux-gnu/libc.so.6+0x3bfb2)
 #6 0x00007f6ce5a7d472 abort (/lib/x86_64-linux-gnu/libc.so.6+0x26472)
 #7 0x000055c40ca94842 (/usr/bin/swift-frontend+0x25a1842)
 #8 0x000055c40ca9492c (/usr/bin/swift-frontend+0x25a192c)
 #9 0x000055c40c5c4076 (/usr/bin/swift-frontend+0x20d1076)
#10 0x000055c40b3139ac swift::SubstitutionMapWithLocalArchetypes::operator()(swift::InFlightSubstitution&, swift::Type, swift::ProtocolDecl*) crtstuff.c:0:0
#11 0x000055c40c98521f swift::InFlightSubstitution::lookupConformance(swift::Type, swift::ProtocolDecl*, unsigned int) (/usr/bin/swift-frontend+0x249221f)
#12 0x000055c40c8f7f9c swift::ProtocolConformanceRef::subst(swift::InFlightSubstitution&) const (/usr/bin/swift-frontend+0x2404f9c)
#13 0x000055c40c8daa73 void llvm::function_ref<void (swift::Type)>::callback_fn<swift::PackConformance::subst(swift::InFlightSubstitution&) const::$_3>(long, swift::Type) PackConformance.cpp:0:0
#14 0x000055c40c8daa21 void llvm::function_ref<void (swift::Type)>::callback_fn<swift::InFlightSubstitution::expandPackExpansionType(swift::PackExpansionType*, llvm::function_ref<void (swift::Type)>)::'lambda'(swift::Type)>(long, swift::Type) crtstuff.c:0:0
#15 0x000055c40c984c0f swift::InFlightSubstitution::expandPackExpansionShape(swift::Type, llvm::function_ref<void (swift::Type)>) (/usr/bin/swift-frontend+0x2491c0f)
#16 0x000055c40c8da686 swift::PackConformance::subst(swift::InFlightSubstitution&) const (/usr/bin/swift-frontend+0x23e7686)
#17 0x000055c40c94c6df swift::SubstitutionMap::subst(swift::InFlightSubstitution&) const (/usr/bin/swift-frontend+0x24596df)
#18 0x000055c40c94c257 swift::SubstitutionMap::subst(llvm::function_ref<swift::Type (swift::SubstitutableType*)>, llvm::function_ref<swift::ProtocolConformanceRef (swift::InFlightSubstitution&, swift::Type, swift::ProtocolDecl*)>, swift::SubstOptions) const (/usr/bin/swift-frontend+0x2459257)
#19 0x000055c40bd72192 swift::SILCloner<(anonymous namespace)::InstructionVisitor>::getOpSubstitutionMap(swift::SubstitutionMap) CrossModuleOptimization.cpp:0:0
#20 0x000055c40bd685ed swift::SILCloner<(anonymous namespace)::InstructionVisitor>::visitPartialApplyInst(swift::PartialApplyInst*) CrossModuleOptimization.cpp:0:0
#21 0x000055c40bd61910 (anonymous namespace)::CrossModuleOptimization::canSerializeFunction(swift::SILFunction*, llvm::DenseMap<swift::SILFunction*, bool, llvm::DenseMapInfo<swift::SILFunction*, void>, llvm::detail::DenseMapPair<swift::SILFunction*, bool>>&, int) CrossModuleOptimization.cpp:0:0
#22 0x000055c40bd60ce3 (anonymous namespace)::CrossModuleOptimizationPass::run() CrossModuleOptimization.cpp:0:0
#23 0x000055c40baabb8e swift::SILPassManager::runModulePass(unsigned int) (/usr/bin/swift-frontend+0x15b8b8e)
#24 0x000055c40baad2cb swift::SILPassManager::execute() (/usr/bin/swift-frontend+0x15ba2cb)
#25 0x000055c40baa8528 swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) (/usr/bin/swift-frontend+0x15b5528)
#26 0x000055c40baa84db swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const (/usr/bin/swift-frontend+0x15b54db)
#27 0x000055c40bac4d3a swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) crtstuff.c:0:0
#28 0x000055c40bab291d swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) crtstuff.c:0:0
#29 0x000055c40baa863f swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) (/usr/bin/swift-frontend+0x15b563f)
#30 0x000055c40bab382b swift::runSILOptimizationPasses(swift::SILModule&) (/usr/bin/swift-frontend+0x15c082b)
#31 0x000055c40b1324ee swift::CompilerInstance::performSILProcessing(swift::SILModule*) (/usr/bin/swift-frontend+0xc3f4ee)
#32 0x000055c40ae442a1 performCompileStepsPostSILGen(swift::CompilerInstance&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*, llvm::ArrayRef<char const*>) FrontendTool.cpp:0:0
#33 0x000055c40ae4316f swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*, llvm::ArrayRef<char const*>) (/usr/bin/swift-frontend+0x95016f)
#34 0x000055c40ae47d17 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*, llvm::ArrayRef<char const*>) FrontendTool.cpp:0:0
#35 0x000055c40ae4535f swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) (/usr/bin/swift-frontend+0x95235f)
#36 0x000055c40abca355 swift::mainEntry(int, char const**) (/usr/bin/swift-frontend+0x6d7355)
#37 0x00007f6ce5a7e24a (/lib/x86_64-linux-gnu/libc.so.6+0x2724a)
#38 0x00007f6ce5a7e305 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x27305)
#39 0x000055c40abc9511 _start (/usr/bin/swift-frontend+0x6d6511)
*** Signal 6: Backtracing from 0x7f6ce5b58829... done ***
*** Program crashed: Aborted at 0x000000000000122b ***
Platform: x86_64 Linux (Debian GNU/Linux 12 (bookworm))
Thread 0 "swift-frontend" crashed:
  0  0x00007f6ce5b58829 <unknown> in libc.so.6
...
Registers:
rax 0x0000000000000000  0
rdx 0x0000000000000006  6
rcx 0x00007f6ce5b58829  48 3d 01 f0 ff ff 73 01 c3 48 8b 0d a7 15 0d 00  H=·ðÿÿs·ÃH··§···
rbx 0x0000000000000006  6
rsi 0x000000000000122b  4651
rdi 0x000000000000122b  4651
rbp 0x000000000000122b  4651
rsp 0x000055c4267c59a8  eb 8c 7d 11 c4 55 00 00 30 5c 7c 26 c4 55 00 00  ë·}·ÄU··0\|&ÄU··
 r8 0x000055c4267c5c30  06 00 00 00 00 00 00 00 fa ff ff ff 00 00 00 00  ········úÿÿÿ····
 r9 0x000055c4267c5c30  06 00 00 00 00 00 00 00 fa ff ff ff 00 00 00 00  ········úÿÿÿ····
r10 0x000055c4267c5c30  06 00 00 00 00 00 00 00 fa ff ff ff 00 00 00 00  ········úÿÿÿ····
r11 0x0000000000000246  582
r12 0x0000000000000006  6
r13 0x000055c412b1460a  66 6f 72 41 62 73 74 72 61 63 74 00 21 70 61 72  forAbstract·!par
r14 0x0000000000000000  0
r15 0x000055c4267c5a48  ff ff ff 7f fe ff ff ff 00 00 00 00 00 00 00 00  ÿÿÿ·þÿÿÿ········
rip 0x00007f6ce5b58829  48 3d 01 f0 ff ff 73 01 c3 48 8b 0d a7 15 0d 00  H=·ðÿÿs·ÃH··§···
rflags 0x0000000000000246  ZF PF
cs 0x0033  fs 0x0000  gs 0x0000
Images (36 omitted):
0x00007f6ce5a57000–0x00007f6ce5bd20fc 93ac61ec5a8eb1396f9fbd350e3169a558528a40 libc.so.6 /usr/lib/x86_64-linux-gnu/libc.so.6

While this appears to be a bug in the compiler, there still seems to be a way to fix it in OpenAPIKit, by doing the same work-around like here: swiftlang/swift#91771

That is adding:

@_semantics("optimize.no.crossmodule")

to the functions in Sources/OpenAPIKit/Validator/Validator.swift which use each T.

[true-false-maybe]

looking into existing issues swiftlang/swift#86348 might already exist, but has not been addressed since it's creation (Jan 7th)

[mattpolzin]

I think this seems like an acceptable workaround. Could this crash be reproduced by a new CI check that itself was a (hopefully trivial) library that built against OpenAPIKit? If so, that'd be a nice addition to our CI suite, though I'll probably merge the workaround regardless.

[true-false-maybe]

i don't quite follow. The bug cannot be triggered in the CI of OpenAPIKit as-is, it can only be triggered in a project which depends on OpenAPIKit. As an example here:

https://github.com/true-false-maybe/OpenAPIKit-cross-module-bug

this workflow fails with the compiler crash:

https://github.com/true-false-maybe/OpenAPIKit-cross-module-bug/actions/runs/34815198697/job/103884319730

i'm not sure how this could be integrated into this repository

[mattpolzin]

I think I got that part figured out in #521. The new CI workflow catches both the compiler issue you are working around and a different compiler bug with nightly that is known and reported upstream as well.