macOS: replace deprecated Carbon foregrounding with post-wx AppKit activation

[damianrickard]

Summary

VeraCrypt brought argument-bearing macOS GUI launches to the foreground with the deprecated Carbon Process Manager APIs GetCurrentProcess, TransformProcessType, and SetFrontProcess.

Removing that block alone is insufficient for the bundled inner-executable case. The executable resolves to its enclosing .app and already has regular activation policy, so wxWidgets does not run its unbundled/accessory activation path. The process can therefore be classified as Foreground without actually becoming frontmost, leaving command-line modal dialogs inactive.

This change removes VeraCrypt's Carbon foregrounding and activates the existing wx-created NSApp after Cocoa initialization instead.

Implementation

• Remove the Carbon foregrounding block and ApplicationServices include from Main.cpp.
• Add a private Objective-C++ helper that only uses the existing NSApp; it never calls sharedApplication and never changes activation policy.
• Call the helper at the start of GraphicUserInterface::OnInit(), after wxWidgets has created wxNSApplication and before Init() or command-line processing can display modal UI.
• Preserve the previous argument condition with argc > 1. wxWidgets has already removed a leading -psn_, so Finder, LaunchServices, and no-argument launches retain their existing behavior.
• Use [NSApp activate] on macOS 14 and later, with compile-time and runtime availability guards and activateIgnoringOtherApps:YES as the macOS 12-13 fallback.
• Leave text mode outside the AppKit activation path.

No public API or command-line interface changes are introduced.

Verification

Built a signed thin arm64 application with wxWidgets 3.2.10, SDK 26.5, and deployment target macOS 12.

• git diff --check passes.
• The helper compiles with deprecated-declaration and unguarded-availability diagnostics treated as errors.
• --text --test passes; debugger breakpoints on both AppKit activation selectors receive zero hits in text mode.
• Finder and LaunchServices bundle launches, including an explicit -psn_ launch, become frontmost normally.
• Direct inner-executable launch with no arguments preserves the previous no-activation behavior.
• Direct inner-executable launch with a disposable valid volume path makes VeraCrypt the result of lsappinfo front, does not leave LSWantsAttention set, focuses the password field, and accepts keyboard input immediately.
• With another application confirmed frontmost immediately before the process started, the volume-path password dialog still became frontmost; three typed characters appeared immediately and Cmd+A replaced them with one character.
• The activation receiver remains wxNSApplication with regular activation policy under LLDB. Cmd-key/menu behavior, Cmd+A in the password field, and Cmd+Q remain functional.
• A representative change-password dialog was opened and cancelled before any operation. The disposable test volume was removed.
• --background-task returns success and exits immediately in both builds. The AppKit build did not briefly take focus in 10/10 trials, while the released Carbon build did in 10/10. The functional background-task behavior is unchanged; the short-lived, UI-less process exits before cooperative asynchronous activation becomes visible.

The macOS 12-13 activateIgnoringOtherApps: branch is compile-verified but was not runtime-tested on those OS versions.

[idrassi]

Thanks for the PR. I agree that removing VeraCrypt’s direct use of deprecated Process Manager APIs is desirable, but I don’t think this implementation is safe as-is.

The new helper calls [NSApplication sharedApplication] from main before wxWidgets initializes the GUI app. On wxOSX, wxWidgets creates its own wxNSApplication subclass during DoInitGui for Cocoa event handling, including command-key key-up behavior. If a plain NSApplication singleton already exists, wxWidgets can't replace it with its subclass, which can cause subtle keyboard/menu event regressions.

Also, wxWidgets 3.2.10 already handles the non-bundled app activation case internally after creating wxNSApplication, by setting NSApplicationActivationPolicyRegular and activating the app. So I think the better fix is either to remove VeraCrypt’s pre-wx Carbon foregrounding block and rely on wxWidgets, or otherwise rework this so wx’s NSApplication subclass is created first.

One factual note: with the current Xcode SDK, these Process Manager APIs are still declared, so the issue is deprecation warnings, not immediate SDK removal.

Could you please revise the approach and include macOS runtime testing for:

• normal launch of /Applications/VeraCrypt.app from Finder,
• LaunchServices launch of the app bundle, including the -psn_ case,
• direct Terminal invocation of the executable inside the app bundle:
  /Applications/VeraCrypt.app/Contents/MacOS/VeraCrypt
• direct Terminal invocation with GUI command-line operations,
• background-task startup:
  /Applications/VeraCrypt.app/Contents/MacOS/VeraCrypt --background-task
• text-mode self-test:
  /Applications/VeraCrypt.app/Contents/MacOS/VeraCrypt --text --test
• basic Cmd-key/menu behavior in password fields.

[idrassi]

This creates/uses NSApplication before wxWidgets initializes the macOS GUI application. That is risky for VeraCrypt because wxOSX creates its own wxNSApplication subclass during DoInitGui to handle Cocoa events correctly, including command-key key-up handling. If a plain NSApplication singleton is created here first, wxWidgets cannot later replace it with wxNSApplication.

Please rework this so VeraCrypt does not create NSApplication before wxWidgets initialization. A safer direction is to remove the pre-wx Carbon foregrounding block and rely on wxWidgets existing macOS activation path, or otherwise ensure wxNSApplication is created first.

[idrassi]

This rationale should be adjusted. In the current Xcode/macOS SDK I tested, GetCurrentProcess, SetFrontProcess, and TransformProcessType are still declared; GetCurrentProcess and SetFrontProcess produce deprecation warnings, but this is not currently an SDK-removal build break. The change should be framed as deprecation cleanup, not as replacing APIs that are already removed.

[damianrickard]

Thanks, and good catches on both points. You're right that these APIs are only deprecated, not removed, on the current SDK — I've corrected the rationale to deprecation cleanup.

I went with your first suggestion and removed the pre-wx Carbon foregrounding block entirely rather than adding a native helper, so nothing touches NSApplication before wxWidgets installs wxNSApplication in DoInitGui. wxWidgets 3.2.10 already covers this in src/osx/cocoa/utils.mm: after creating wxNSApplication, its applicationDidFinishLaunching sets NSApplicationActivationPolicyRegular and activates the app when the executable isn't inside an .app bundle (and activates bundled accessory apps too). The PR is now just the deletion of the block and the ApplicationServices include.

Rebased onto current master. Testing on arm64 (macOS 26.5): --text --test passes, and launching the executable directly from Terminal foregrounds correctly (regular activation policy, Dock icon, working menus) with no Carbon code. I have not yet run the bundled-.app Finder / LaunchServices -psn_ launch, --background-task, or the Cmd-key-in-password-field checks against a packaged build — I'll do that against a signed .app and follow up here.

[damianrickard]

Built a signed .app and ran through the launch matrix on Apple Silicon (macOS 26.5.1). Using lsappinfo to read the activation policy: launching the bundle via LaunchServices comes up as a Foreground app and in front; launching the inner executable directly from Terminal — both with no args and with a volume-path argument (the case the removed Carbon block used to handle) — also comes up as a Foreground app; the -psn_ argument case launches the GUI normally; and --text --test passes. --background-task with nothing mounted exits immediately, but the current released build (with the Carbon block) behaves identically, so that's pre-existing and not affected by this change. I confirmed no code creates an NSApplication before wxWidgets — the only reference is the standard NSPrincipalClass in Info.plist, which wx subclasses as wxNSApplication, so command-key handling stays wx-native. Removing the pre-wx block rather than adding one makes that strictly safer than before.

[idrassi]

Thanks for revising this. Removing the helper resolves the concern about creating NSApplication before wxWidgets, but I reproduced a foregrounding regression with the current head.

On macOS 26.5.1 with wxWidgets 3.2.10 and a signed arm64 build, I invoked the inner executable from Terminal with a volume path. lsappinfo reported VeraCrypt as Foreground, but lsappinfo front remained Terminal and LSWantsAttention became true. The password dialog existed, but VeraCrypt didn't become active. The current Carbon code brings it forward immediately.

wxWidgets only activates an unbundled app or a bundled app with accessory policy. The inner executable still resolves to the enclosing .app and already has regular policy, so wxWidgets doesn't activate it. Foreground application type isn't equivalent to being frontmost.

Please activate the existing NSApp after wxWidgets has created wxNSApplication, before command line processing can display modal UI. Don't create or access the shared application before wx initialization. Please retest lsappinfo front, not only application type, using a volume path and other modal GUI commands.

The PR title should also be updated since the current change removes the code rather than replacing it with NSApplication.

[damianrickard]

Thanks for catching this. My earlier check treated ApplicationType=Foreground as equivalent to being frontmost, so it missed the regular-policy bundled inner-executable case.

I have amended the PR into one commit. Activation now occurs in GraphicUserInterface::OnInit(), after wxWidgets creates wxNSApplication and before command-line modal processing. The helper only uses the existing NSApp; it does not call sharedApplication or change activation policy. It uses activate on macOS 14+ and the deprecated activateIgnoringOtherApps: only for the macOS 12-13 fallback.

Signed arm64 results on macOS 26.5.1 with wxWidgets 3.2.10 and deployment target 12:

• A direct inner-executable launch with a valid volume path made VeraCrypt the result of lsappinfo front; LSWantsAttention was absent and the password field accepted keyboard input immediately.
• I repeated that launch with another application confirmed frontmost immediately before VeraCrypt started. The password dialog still became frontmost, its password field was focused, and three typed characters appeared immediately. Cmd+A replaced them with one character.
• LLDB stopped in -[NSApplication activate] with a wxNSApplication receiver and regular activation policy.
• Finder and LaunchServices launches, including explicit -psn_, became frontmost normally. Direct no-argument execution retained the previous no-activation behavior.
• Cmd-key/menu handling, Cmd+A in the password field, and Cmd+Q worked. A change-password dialog was opened and cancelled before any operation.
• --text --test passed, with zero hits on both AppKit activation breakpoints.
• --background-task returned success and exited immediately in both builds. The AppKit build did not briefly take focus in 10/10 trials; the released Carbon build did in 10/10. No UI or background-task functionality depends on that transient focus change.

The macOS 12-13 fallback is compile-verified but not runtime-tested on those releases. The PR title and description have also been corrected to describe deprecation cleanup and the post-wx lifecycle accurately.