Tracking Issue for asm_unwind

[Amanieu]

The feature gate for the issue is #![feature(asm_unwind)].

Summary

This feature adds a may_unwind option to asm! which allows an asm block to unwind stack and be part of the stack unwinding process. This option is only supported by the LLVM backend right now.

Status

Blocked on the stabilization of extern "C-unwind" (#74990).

Also it is uncertain whether this can be supported directly by non-LLVM backends. It is possible to support by outlining the assembly code to a separate function and calling it.

[bstrie]

Note that extern "C-unwind" has been stabilized as of #116088 . Are there any remaining blockers for this feature?

[Amanieu]

I'm having crashes in corosensei when using this feature which I've tracked down to LLVM's register allocator not handling unwinding from inline assembly correctly. However I have not been able to extract a minimal reproducer, and even extracting the LLVM IR and compiling it manually with llc fails to reproduce the problem.

[hsanzg]

I've been trying out this feature and have run into an issue regarding DWARF CFI info I don't know how to solve:

I have a function in charge of context switching, i.e. replacing the current program stack by another one, and jumping to another program location. (The stack setup is not directly relevant to the problem.) More precisely, the function consists of an asm! statement that jmps into another function (following the sysv64-unwind ABI); the callee may return control by jmp'ing back into the asm! statement (well, the location of a local label within it).

Since the callee may overwrite the contents of all registers, we need to restore their original values prior to exiting the asm! block. To do so, I add the clobber_abi("sysv64") rule and declare some additional clobbers not covered by it (namely r12--15). Since the rbp and rbx registers cannot be declared as clobbered, the first thing the asm! block does is to push their contents into the stack. I would like any stack unwind operation to go across the context switch boundary (that is, to treat it as a regular function call). To that end, I also add the option(may_unwind) rule. But this is not enough; we need to make the unwinder aware of the old contents of rbp and rbx. We have at least three options:

1. Declare the relevant DWARF info for these two registers. That is, add the .cfi_rel_offset directives for the rbp and rbx registers after the corresponding push instructions in the asm! block.

   From what I see, it's a terrible idea to add your own CFI info to an asm! statement (but it's ok in naked_asm!), since it may invalidate/contradict any other info emitted by the compiler. For one, we cannot assume that the CFA register is rsp, so there's no correct way to specify that the old contents of a register are stored at some [rsp - offset], I think. More important, however, is the fact that the compiler may insert a .cfi_offset directive for the relevant register in between the function prelude and the asm! statement. Declaring our own CFI expression for the same register would make the unwinder restore the wrong contents of the register (namely the ones in between the prelude and the asm! block, rather than the ones it had in the parent stack).

   All of these cases do not seem to occur when compiling in debug mode. But in release mode the compiler inlines the context-switching function, so I begin to emit incorrect DWARF info. (In particular, the .cfi_offset directive in the asm! statement starts to overwrite a previous one in the inlining target.) This leads to segfaults immediately after a manual or panic-induced unwind finishes (i.e. after a catch_unwind function returns).

2. Do not declare any CFI directive in the asm! block. Now we run into just the opposite problem: If the function is not inlined, then the unwinder never restores the contents of the rbp and rbx registers. An we enter segfault territory again :(

3. Wrap the asm!-containing function in a naked one; keep the clobber_abi and r12--r15 clobbers in the former, and do the manual clobbers with CFI info in the latter. This seems to inhibit LLVM from inlining the assembly, but context-switch times suffer significantly (I see a slowdown of 6x or so). Also, is there something preventing LLVM from inlining the naked function in the future? If not, this approach would also begin to suffer from the problems in (1).

What's the intended approach in these cases? This reference page does not explicitly ban asm! blocks from containing .cfi directives, but I don't think there's currently a way to make the compiler take them into account when applying optimizations. In this sense, my question is really similar to the one raised in #144610. Here the problem occurs during stack unwinding rather than during backtrace construction. Fortunately I get clean backtraces containing all the frames before and after a context switch in lldb; it's just that the CFI info (or its lack thereof) gets the unwinder confused.

I've been trying to get a reduced reproducer for a few days now, but I haven't been able to trigger all the required conditions without having the whole context-switching-related code available :/ I hope this comment provides enough context to help identify a valid way to proceed---or at least to point out if I'm misunderstanding something. I'm a bit of a newbie regarding DWARF and unwinding, so the latter is likely the case; if so, I'm sorry if my diagnosis here is incorrect.

Thank you in advance!

[Amanieu]

@hsanzg I would recommend having a look at my implementation in corosensei which has working support for asm_unwind. Notice how although the rbx push is in switch_and_link, the unwind info for restoring that is actually in stack_init_trampoline, which is a naked function.

[hsanzg]

I like that approach; since it keeps the naked assembly restricted to a single function that's only executed once, it shouldn't affect the inlining of the other context-switch-related machinery. Will give it a go, thanks!

To make this comment a bit more on-topic: If this feature gets accepted, it would be great if the inline assembly docs had a sentence saying something along the lines of "For unwinding to proceed correctly, all stack frames between the unwinding start point and the catch point (or the bottom of the stack) must contain valid unwinding information, including those for functions containing asm! or inline_asm! macros or written in foreign languages". This is imo stricter than the current wording in [asm.naked-rules.unwind], which only requires that to be the case for inline_asm! macros. It maybe sounds a bit obvious, but until #144610 is solved, it is not clear what's necessary for a function with asm! to have valid unwinding info.