[HIPIFY][feature] Local header hipification via Clang PPCallbacks (experimental, opt-in) - Part 1 by ranapratap55 · Pull Request #2402 · ROCm/HIPIFY

ranapratap55 · 2026-02-23T09:33:45Z

Problem Statement

hipify-clang currently only transforms the main source file passed as input. Local (quoted) headers included in the main file remain same as CUDA code. These headers often cannot be hipified individually because they are not self-contained headers which requires types, operators, and definitions from preceding includes in the parent source.

Example: dxtc.cu includes CudaMath.h, which uses float3 operators (+=, *, -) defined in helper_math.h. Hipifying CudaMath.h alone fails because these operators are unavailable. Similar cases:

Solution

Two new opt-in flags enable local header hipification:

hipify-clang --local-headers main.cu -I ../Common/ 
hipify-clang --local-headers-recursive main.cu -I ../Common/

The feature is experimental and disabled by default. It will be promoted to default behavior in later release cycles with --skip-local-headers as the opt-out fallback.

Supersedes #2275

Summary of changes in this PR

Clang PPCallbacks instead of regex

Include discovery is using Clang's PPCallbacks::InclusionDirective via preprocessing pre-pass, not regex. This correctly handles conditional compilation (#ifdef), macro-computed includes (#include MACRO), compiler search paths (-I, -isystem), and comments.

Fixed Compilation Database fallback

Header files which have no entry in compile_commands.json falls back to a FixedCompilationDatabase rooted at the main source file's directory so that relative paths resolve correctly.

Cycle and duplicate detection

A processed set prevents re-hipification, and a queued set prevents duplicate work-queue entries during recursive traversal.

ToDo work (follow-up PRs):

Warning for non-self-contained headers: Warn (besides compiler’s error messaging) the user about imposibility to hipify a header file, which is not self-contained.
Folder-level hipification: Hipify everything in the provided folder.
Promote to default (add --skip-local-headers)
Angle-bracket local header hipification (#include <mylib/utils.h> via -I) — [HIPIFY][#2402] hipify project-local headers included with angle brackets (<>) #2505
Transitive preceding includes for deeply nested recursive headers — [HIPIFY][#2402] Transitive preceding includes for deeply nested recursive headers #2507

emankov · 2026-02-24T14:42:39Z

Can we say that the main purpose of this PR is to allow hipifying header files individually as main translation units?

ranapratap55 · 2026-02-25T06:08:55Z

Can we say that the main purpose of this PR is to allow hipifying header files individually as main translation units?

Yes, correct. The main purpose is to allow hipifying local header files as main translation units. We are doing it by injecting their preceding includes from the parent source file using Clang's -include ArgumentsAdjusters which is to provide the necessary compilation context.

This feature is enabled via --local-headers / --local-headers-recursive and only applies to local (quoted) headers referenced in the main source file.

emankov · 2026-02-25T18:18:07Z

the parent source

How do we guarantee finding the parent source(s) for a particular header file so we can hipify it as the main translation unit?

ranapratap55 · 2026-02-26T06:01:58Z

the parent source

How do we guarantee finding the parent source(s) for a particular header file so we can hipify it as the main translation unit?

We don't need to discover/guess the parent source file. The user provides it explicitly while hipifying. The main source file passed to hipify-clang is always the parent. --local-headers simply extends hipification to that source file's local headers, using the source file itself as the context for preceding includes.

For example: hipify-clang --local-headers sample.cu: here sample.cu is known parent for its local headers (header1.h, header2.h, etc.,).

emankov · 2026-02-26T13:20:37Z

hipify-clang --local-headers sample.cu

That is what worries me most.

The user should know or learn about the main source file, or, in general, the source files that include that particular header file.
sample.cu is not a header file. Here we have misleading information for the end users.
We need to explain somewhere, somehow, this tricky feature (actually a workaround).
Finally, it makes hipify-clang even more complicated in use, which we definitely do not want, as users already consider hipify-clang complicated enough to prefer hipify-perl usage instead. And we want the opposite. We want users to prefer hipify-clang. It means that we need to simplify it AMAP and do more work under the hood implicitly.

Technically, the main problem is that we are trying to treat a file, a header file, which is not self-contained, as a source file for hipification (actually, compilation).

I want us to think about other approaches here.
For instance:

Warn (besides compiler’s error messaging) the user about imposibility to hipify a header file, which is not self-contained.
Implement the ability to implicitly hipify all header files that are included in the main file.
Hipify everything in the provided folder.

ranapratap55 · 2026-03-04T06:47:11Z

hipify-clang --local-headers sample.cu

That is what worries me most.

The user should know or learn about the main source file, or, in general, the source files that include that particular header file.

sample.cu is not a header file. Here we have misleading information for the end users.

We need to explain somewhere, somehow, this tricky feature (actually a workaround).

Finally, it makes hipify-clang even more complicated in use, which we definitely do not want, as users already consider hipify-clang complicated enough to prefer hipify-perl usage instead. And we want the opposite. We want users to prefer hipify-clang. It means that we need to simplify it AMAP and do more work under the hood implicitly.

Technically, the main problem is that we are trying to treat a file, a header file, which is not self-contained, as a source file for hipification (actually, compilation).

I want us to think about other approaches here. For instance:

Warn (besides compiler’s error messaging) the user about imposibility to hipify a header file, which is not self-contained.

Implement the ability to implicitly hipify all header files that are included in the main file.

Hipify everything in the provided folder.

As per our discussions/suggestions, I've reworked this PR to implement suggestion 2.
Local header hipification is now implicit by default. hipify-clang sample.cu automatically detects and hipifies all local headers recursively, no flags needed. Added --skip-local-headers as the single opt-out mechanism.

I will raise 2 separate PRs as a follow up for:

Warn the user for non-self-contained headers
Folder-level hipification

Updated PR title and description to reflect the changes.

emankov · 2026-03-31T17:35:35Z

-cl::opt<bool> OptLocalHeadersRecursive("local-headers-recursive",
-  cl::desc("Enable hipification of quoted local headers recursively"),
+cl::opt<bool> SkipLocalHeaders("skip-local-headers",
+  cl::desc("Skip implicit hipification of local (quoted) headers included in the main source file"),


What if the "local" headers are included with <>?

I believe this is out of scope. Angled-bracket includes files are resolved using -I search paths. They represent system headers or external library dependencies.

But I do think there is edge case like, if a project uses #include <mylib/utils.h> via -I project/src flag, that would require resolving each angled-bracket against every -I path. That is a meaningful work and requires a separate PR of its own.

#2505: Angle-bracket local header hipification (#include <>)

emankov · 2026-03-31T18:18:42Z

+  bool collectPrecedingIncludes(const std::string &mainSourceAbspath,
+                                const std::string &targetHeaderAbspath,
+                                std::vector<std::string> &outIncludes) {
+    std::string mainSourceContent;
+    if (!readFile(mainSourceAbspath, mainSourceContent)) {
+      errs() << sHipify << sError << "Cannot read source files: "
+            << mainSourceAbspath << "\n";
+      return false;
+    }
+
+    std::string targetFileName = std::string(sys::path::filename(targetHeaderAbspath));
+    std::istringstream iss(mainSourceContent);
+    std::string line;
+    std::smatch m;
+
+    while (std::getline(iss, line)) {
+      if (std::regex_match(line, m, LocalIncludeRe)) {
+        std::string quotedName = m[1].str();
+        std::string quotedFileName = std::string(sys::path::filename(quotedName));
+        if (quotedFileName == targetFileName)
+          break;
+        std::string absPath;
+        if (resolveLocalIncludeInternal(mainSourceAbspath, quotedName, absPath))
+          outIncludes.push_back(absPath);
+      }
+
+      if (std::regex_search(line, m, SystemIncludeRe)) {
+        outIncludes.push_back(m[1].str());
+      }
+    }
+
+    return true;
+  }
+}


Looks like the collectPrecedingIncludes only scans the immediate parentPath.
What about deeply nested headers?

yes, correct.
collectPrecedingIncludes scans only one level (the immediate parent). This is intentional for this PR, it was designed for (main.cu → A.h → B.h) where B.h depends on explicitly included before it in A.h and collectPrecedingIncludes(A.h, B.h) finds exactly what B.h needs.

I considered collecting full ancestor chain recursively (for nested headers) but that results in over-injection. When two of those injected headers define same symbol, we get redefinition errors.

I would like to take this (deeply nested headers) as a follow-up PR. We track the full ancestors list and modify from {hdr, parentPath} to {hdr, vector<parentChain>}. And only injects that appear in ancestors(parents) up to the nearest file that explicitly includes the current target. This avoids over-injection issue.

#2507: Transitive preceding includes for deeply nested recursive headers

ranapratap55 · 2026-04-07T11:21:02Z

Follow up work

[HIPIFY][#2402] Transitive preceding includes for deeply nested recursive headers #2507: Transitive preceding includes for deeply nested recursive headers
[HIPIFY][#2402] hipify project-local headers included with angle brackets (<>) #2505: Angle-bracket local header hipification (#include <>)

emankov · 2026-04-16T20:50:19Z

The added tests cover basic recursion and depth-1 injection of “includes”. However, the test that verifies the transitive context is preserved during recursive traversal is still needed.

Please add a test that includes a non-self-contained header with a recursion depth of 2 or more. For example, the root .cu file includes <cmath> and parent.h; parent.h includes child.h; and child.h uses sqrtf but does not include <cmath> itself. This is essential to ensure that the inherited compilation context is correctly propagated down the inclusion chain and is not dropped when processing nested files.

emankov · 2026-04-16T20:54:28Z

I honestly believe we need to preserve -local-headers and --local-headers-recursive, since the feature you're introducing is experimental and shouldn't be used by default. What do you think, @ranapratap55?

ranapratap55 · 2026-04-20T10:18:20Z

The added tests cover basic recursion and depth-1 injection of “includes”. However, the test that verifies the transitive context is preserved during recursive traversal is still needed.

Please add a test that includes a non-self-contained header with a recursion depth of 2 or more. For example, the root .cu file includes <cmath> and parent.h; parent.h includes child.h; and child.h uses sqrtf but does not include <cmath> itself. This is essential to ensure that the inherited compilation context is correctly propagated down the inclusion chain and is not dropped when processing nested files.

I've adapted to use <algorithm> + std::sort

transitive_system_include.cu: includes <algorithm> and transitive_parent.h
transitive_parent.h: includes transitive_child.h and <cuda_runtime.h>, but no <algorithm>
transitive_child.h: uses std::sort and cudaMalloc, includes neither <algorithm> nor <cuda_runtime.h>

collectAncestorSystemIncludes walks the ancestor chain and collects <algorithm> from the main src file, then injects it before compiling transitive_child.h. If this propagation breaks, the test fails directly.

ranapratap55 · 2026-04-20T10:31:20Z

I honestly believe we need to preserve -local-headers and --local-headers-recursive, since the feature you're introducing is experimental and shouldn't be used by default. What do you think, @ranapratap55?

Thanks for raising this and it is a fair concern.

Based on our earlier discussions, we collectively moved toward implicit-by-default to lower the barrier for users. The current revision already has multiple layers in place for end users like

Opt-out flag: --skip-local-headers for users who don't want automatic hipification
Non-self-contained header warning: [HIPIFY] warning: non-self-contained header is emitted when a header cannot be hipified in isolation and with a note warning the user to run the parent source file instead

And I am thinking to add below

Auto-hipification notice: an info-level message [HIPIFY] note: local headers detected: hipifying automatically. Use --skip-local-headers to disable.

With these measures together I believe the implicit behavior is transparent. That said, I am happy to address any specific scenario where this causes harm directly rather than revert the design.

What do you think? @emankov

emankov · 2026-05-05T10:53:34Z

I honestly believe we need to preserve -local-headers and --local-headers-recursive, since the feature you're introducing is experimental and shouldn't be used by default. What do you think, @ranapratap55?

Thanks for raising this and it is a fair concern.

Based on our earlier discussions, we collectively moved toward implicit-by-default to lower the barrier for users. The current revision already has multiple layers in place for end users like

Opt-out flag: --skip-local-headers for users who don't want automatic hipification

Non-self-contained header warning: [HIPIFY] warning: non-self-contained header is emitted when a header cannot be hipified in isolation and with a note warning the user to run the parent source file instead

And I am thinking to add below

Auto-hipification notice: an info-level message [HIPIFY] note: local headers detected: hipifying automatically. Use --skip-local-headers to disable.

With these measures together I believe the implicit behavior is transparent. That said, I am happy to address any specific scenario where this causes harm directly rather than revert the design.

What do you think? @emankov

I still believe that experimental features should stay experimental for at least one release. Meanwhile, it might be thoroughly tested. Next step: inform end users in the documentation that, with the next release, this particular feature will be enabled by default, and for fallback functionality, here are the options to set explicitly.

emankov · 2026-05-05T20:07:31Z

[IMPORTANT]

After reviewing this regex-based solution for header tree calculation, I decided it is not a good solution. Using regex in hipify-perl is a forced decision, but using a regex-based approach with a C++ compiler under the hood can't be treated as a good solution.

[Reasons]

Parsing C/C++ includes with regular expressions is fundamentally brittle and serves as a temporary workaround.
Regex has major blind spots that cannot be solved without rewriting a full C++ preprocessor:
- Conditional Compilation: Regex blindly captures #include directives inside #if 0 or unmet #ifdef blocks, injecting phantom dependencies and false context.
- Macro Expansion: Regex completely fails on computed inclusions like #include MACRO_NAME.
- Path Resolution: Custom regex/path logic lacks knowledge of the compiler search flags (-I, -isystem) from compile_commands.json. Whereas Clang’s internal HeaderSearch engine perfectly applies these flags and resolves the exact absolute paths.
- Comments and Formatting: Regex breaks easily on multi-line comments /* ... */ or line continuations \ within the directive.
Tapping into Clang's built-in Preprocessor delegates all of this to the compiler, completely solving these issues out-of-the-box.

[Recommendations]

instead of regexs, use Clang's functionality, based on PPCallbacks and InclusionDirective.
It might need additional pre-pass.

ranapratap55 · 2026-05-13T04:46:20Z

I still believe that experimental features should stay experimental for at least one release. Meanwhile, it might be thoroughly tested. Next step: inform end users in the documentation that, with the next release, this particular feature will be enabled by default, and for fallback functionality, here are the options to set explicitly.

Agreed, both --local-headers and --local-headers-recursive are preserved as explicit opt-in flags in latest revision.
For the graduation plan,

This release: Feature ships as opt-in behind --local-headers/--local-headers-recursive and document that it will become default. Test it thoroughly.
Next release: Enable by default with --skip-local-headers as fallback for opt-out

ranapratap55 · 2026-05-13T04:54:09Z

[IMPORTANT]

After reviewing this regex-based solution for header tree calculation, I decided it is not a good solution. Using regex in hipify-perl is a forced decision, but using a regex-based approach with a C++ compiler under the hood can't be treated as a good solution.

[Reasons]

Parsing C/C++ includes with regular expressions is fundamentally brittle and serves as a temporary workaround.

Regex has major blind spots that cannot be solved without rewriting a full C++ preprocessor:

Conditional Compilation: Regex blindly captures #include directives inside #if 0 or unmet #ifdef blocks, injecting phantom dependencies and false context.

Macro Expansion: Regex completely fails on computed inclusions like #include MACRO_NAME.

Path Resolution: Custom regex/path logic lacks knowledge of the compiler search flags (-I, -isystem) from compile_commands.json. Whereas Clang’s internal HeaderSearch engine perfectly applies these flags and resolves the exact absolute paths.

Comments and Formatting: Regex breaks easily on multi-line comments /* ... */ or line continuations \ within the directive.

Tapping into Clang's built-in Preprocessor delegates all of this to the compiler, completely solving these issues out-of-the-box.

[Recommendations]

instead of regexs, use Clang's functionality, based on PPCallbacks and InclusionDirective.

It might need additional pre-pass.

Thanks for this feedback. I've removed the entire regex-based include scanning system and replaced with Clang's PPCallbacks::InclusionDirective in the latest revision.

The new PPCallbacks approach uses IncludeCollectorCallbacks (a PPCallbacks subclass), IncludeCollectorAction (a PreprocessorFrontendAction), and collectIncludeTree() as an entry point

We are directly addressing below concerns:

Conditional compilation
Macro expansion
Path resolution
Comments/formatting

emankov · 2026-05-19T17:52:31Z

+#if LLVM_VERSION_MAJOR < 15
+                          const clang::FileEntry *file,
+#elif LLVM_VERSION_MAJOR == 15
+                          Optional<clang::FileEntryRef> file,
+#else
+                          clang::OptionalFileEntryRef file,
+#endif
+                          StringRef search_path, StringRef relative_path,
+#if LLVM_VERSION_MAJOR < 19
+                          const clang::Module *SuggestedModule
+#else
+                          const clang::Module *SuggestedModule,
+                          bool ModuleImported
+#endif
+#if LLVM_VERSION_MAJOR > 6
+                          ,
+                          clang::SrcMgr::CharacteristicKind FileType
+#endif


Might it be simplified?
Did you test it against different LLVM versions?

Tested by building HIPIFY against LLVM 14, 15, 16, 18 and all compile without any errors. This covers (FileEntry*, Optional<FileEntryRef>, OptionalFileEntryRef, and with/without ModuleImported).

emankov · 2026-05-20T19:30:11Z

+  // Strip the implicit CUDA header that appendArgumentsAdjusters adds —
+  // not needed for include scanning and would pollute the results.
+  Tool.appendArgumentsAdjuster(
+      [](const ct::CommandLineArguments &Args, StringRef) {
+        ct::CommandLineArguments filtered;
+        for (size_t i = 0; i < Args.size(); ++i) {
+          if (Args[i] == "-include" && i + 1 < Args.size() &&
+              Args[i + 1] == "cuda_runtime.h") {
+            ++i;
+            continue;
+          }
+          filtered.push_back(Args[i]);
+        }
+        return filtered;
+      });


Without cuda_runtime.h, Clang lacks base CUDA macros like __CUDACC__ and __device__, causing it to ignore user code hidden behind #ifdef blocks.

cuda_runtime.h is filtered from the results via the isAngled + isWrittenInMainFile checks.

emankov · 2026-05-20T22:19:05Z

+extern bool appendArgumentsAdjusters(ct::RefactoringTool &Tool,
+                                     const std::string &sSourceAbsPath,
+                                     const char *hipify_exe);


Wrong place to extern. Actually, extern is not needed in LocalHeader.h; it is also misleading.
Instead, better to add a forward function declaration in LocalHeader.cpp.

Removed the extern and added a local forward declaration of appendArgumentsAdjusters in LocalHeader.cpp instead.

emankov · 2026-05-20T22:41:32Z

+// CHECK: #include "transitive_parent.h"
+#include <cuda_runtime.h>
+#include <algorithm>
+#include "transitive_parent.h"


The transitive test does not exercise depth 2, in fact. The option --local-headers-recursive should be used instead of --local-headers.

Add the check for cudaMalloc in the transitive_child.h.

I anticipate this particular test will fail. In that case, the whole algorithm should be revised and fixed (next iteration).

I've switched to --local-headers-recursive and added the cudaMalloc -> hipMalloc check in transitive_child.h. The test now exercises depth-2, discovery finds transitive_parent.h, then recursed into transitive_child.h. The only fix needed was making transitive_child.h self-contained (#include <algorithm> for std::sort), reason being a recursively-hipified header is compiled standalone.

emankov · 2026-07-23T13:17:03Z

    bool ok = hipifySingleSource(hdr, hipOut, compDB, OptionsParserPtr,
-                                  hipify_exe, mainSourceAbsPath, false);
+                                 hipify_exe, mainSourceAbsPath, false);


It looks like a regression. The context vector is never passed to hipifySingleSource, and the local headers are being hipified in a vacuum, which should lead to a crash.

Additionally, the current tests yield false positives as they are self-contained. A negative test could help here to prove the injection is actually working.

buildIncludeContext() now derives it from the entries of that same preprocessor run. The ancestor chain's directives that precede the header, oldest first. Ancestors themselves are excluded, since reincluding a parent pulls the header back through its own guard and empties the copy being hipified. The injection also moved from BEGIN to a single END insertion, so it follows -include cuda_runtime.h.

Added a negative test for non self-contained headers. nsc_context.h declares NscVec and is self-contained,nsc_dependent.h uses NscVec and cudaMemset while including nothing. non_self_contained.cu includes them in that order under --local-headers. Because -include cuda_runtime.h is always injected, cudaError_t and cudaMemset resolve on their own. So NscVec is the only symbol that can come from nowhere but the injected context. With an empty vector the header fails with unknown type name NscVec, hipify returns non-zero and the test fails. nsc_dependent.h sits in config.excludes because it cannot carry a RUN.

emankov · 2026-07-23T13:47:35Z

+  if (cmds.empty()) {
+    std::string dir = sys::path::parent_path(mainContextPath).str();
+    fallbackDB = std::make_unique<ct::FixedCompilationDatabase>(
+        dir, std::vector<std::string>());
+  }


Passing an empty vector to FixedCompilationDatabase wipes out all project compiler flags and breaks header resolution for any real-world project with nested include directories.

Removed the branch entirely and it is unreachable now. Without -p we hold a FixedCompilationDatabase, which synthesizes a command for any path. With -p, loadFromDirectory wraps the JSON database in inferMissingCompileCommands, which borrows the command of the closest matching file, swaps in the filename and drops output args. hipifySingleSource already relies on this, passing a /tmp/*.hip path that's in no database.

emankov · 2026-07-23T13:53:26Z

+    if (!SM.isWrittenInMainFile(hash_loc))
+      return;


Using !SM.isWrittenInMainFile(hash_loc) inside the preprocessor callback forcibly prevents Clang from natively traversing nested includes and forces the tool to spawn a brand-new compiler instance for every child header, which destroys macro inheritance. The possible solution here is to remove the check.

Thanks for this. I've removed the check now and includerPath, isFromMainFile, isSystem are modified as data, so one preprocessor run gives the whole tree. The recursive/non-recursive split is a single filter. Removal alone wasn't ideal though, since CUDA's headers use quoted includes. So paired with SM.isInSystemHeader(HashLoc) and an empty includer path check.

emankov · 2026-07-23T14:14:22Z

+  void ExecuteAction() override {
+    clang::CompilerInstance &CI = getCompilerInstance();
+    clang::Preprocessor &PP = CI.getPreprocessor();
+    PP.addPPCallbacks(std::make_unique<IncludeCollectorCallbacks>(
+        CI.getSourceManager(), Entries));

-  SmallString<256> norm = in;
-  llvm::sys::path::remove_dots(norm, true);
-  return llvm::sys::fs::exists(norm);
-}
+    PP.EnterMainSourceFile();
+    clang::Token Tok;
+    do {
+      PP.Lex(Tok);
+    } while (Tok.isNot(clang::tok::eof));
+  }


Overriding ExecuteAction() to manually call EnterMainSourceFile() and Lex bypasses standard Clang initialization, such as IgnorePragmas(). If a header contains unknown pragmas, your pre-pass might crash.

Fixed this. IncludeCollectorAction derives from PreprocessOnlyAction and registers the callbacks in BeginSourceFileAction(), so the lexing loop and IgnorePragmas() come from the framework. The old loop was the same code without IgnorePragmas(), so unknown pragmas warned. Collection now propagates Tool.run() failure, a -Werror build would abort on something like #pragma unroll. Known pragmas such as #pragma once are untouched.

emankov · 2026-07-23T14:18:52Z

+  void InclusionDirective(clang::SourceLocation hash_loc,
+                          const clang::Token &include_token,
+                          StringRef file_name, bool is_angled,
+                          clang::CharSourceRange filename_range,
+#if LLVM_VERSION_MAJOR < 15
+                          const clang::FileEntry *file,
+#elif LLVM_VERSION_MAJOR == 15
+                          Optional<clang::FileEntryRef> file,
+#else
+                          clang::OptionalFileEntryRef file,
+#endif
+                          StringRef search_path, StringRef relative_path,
+#if LLVM_VERSION_MAJOR < 19
+                          const clang::Module *SuggestedModule
+#else
+                          const clang::Module *SuggestedModule,
+                          bool ModuleImported
+#endif
+#if LLVM_VERSION_MAJOR > 6
+                          ,
+                          clang::SrcMgr::CharacteristicKind FileType
+#endif


Several parameters in InclusionDirective signature are named but never actually used in the function body.

Fixed in latest code.

emankov

Thank you. I do not see any other issues

emankov · 2026-08-07T12:20:33Z

+  ct::RefactoringTool Tool(
+      compDB ? *compDB : OptionsParserPtr->getCompilations(), {srcPath});


We need to revise this explicit JSON compilation database detection here and elsewhere, since the getCompilations() database performs it implicitly.

[Explanation]
Situation 1: The workspace has a valid compile_commands.json. When a user runs HIPIFY in a project where the build system (like CMake) has generated a compile_commands.json file, the compDB variable becomes valid (not null). The code initializes the tool using a ternary operator: compDB ? *compDB : OptionsParserPtr->getCompilations(). Because of this operator, the tool strictly prioritizes the raw database (*compDB) and completely bypasses OptionsParserPtr->getCompilations().

Situation 2: The tool attempts to process a local header file (.h, or .hpp, or cuh). By their very nature, compilation databases (compile_commands.json) only record compile commands for translation units (like .cu, .cpp, or .c files). Header files are never recorded in this database because they are not compiled independently.

The Mechanism of Failure:

HIPIFY discovers a local header and attempts to hipify it.

The tool queries the raw *compDB database, asking for the specific compile flags for that .h file.

Because the header file is not listed in the database, the raw database returns nothing.

Normally, the smart wrapper (CommonOptionsParser) would act as a fallback safety net: it automatically overlays global command-line arguments (like manually passed -I flags) to generate a compile command on the fly for missing files.

However, because the code forces the use of the raw *compDB, it bypasses this built-in safety net.

Consequently, ClangTool::run() finds no command, outputs "Skipping... Compile command not found", and silently aborts.

I debugged more into this, and I do not think failure mode is reachable. compDB is not a raw database. main.cpp:393 calls CompilationDatabase::loadFromDirectory then reaches JSONCompilationDatabasePlugin::loadFromDirectory and returns inferTargetAndDriverMode(inferMissingCompileCommands((...))). The comment at line 159 says it also infers compile commands for files not present in the database. So the header inference you're describing is already inside *compDB.

getCompilations() is that same object plus ArgumentsAdjustingCompilations, which rewrites commands the inner database already returned and so cannot synthesize one for a missing file. Our /unit_tests/compilation_database/ tests show it is working. They run with -p= and hipify a /tmp/*.hip path that is in no JSON. That is the interpolating wrapper answering through *compDB.

You are right that the ternary redundant. The compDB branch drops --extra-arg, and the JSON is parsed twice per run. So I'd rather fix it as a separate NFC PR, with a -p plus --local-headers test as the first commit, since no combination covers it as of today.

…jection via ArgumentsAdjusters

…ntext test

…and restore opt-in flags

…s, depth-2 test

… injection

ranapratap55 requested review from emankov and searlmc1 as code owners February 23, 2026 09:33

ranapratap55 requested review from SyamaAmd and kzhuravl February 23, 2026 09:34

ranapratap55 self-assigned this Feb 23, 2026

ranapratap55 added fix It fixes bug feature Feature request or implementation labels Feb 23, 2026

ranapratap55 mentioned this pull request Feb 23, 2026

[HIPIFY][feature] Add header injection fallback for local headers hipification #2275

Draft

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 5e6249f to 353ebef Compare March 4, 2026 06:20

ranapratap55 changed the title ~~[HIPIFY][feature] Local header hipification with preceding include injection via ArgumentsAdjusters~~ [HIPIFY][feature] Implicit hipification of local header files Mar 4, 2026

ranapratap55 added the partial Partial fix or implementation label Mar 20, 2026

ranapratap55 changed the title ~~[HIPIFY][feature] Implicit hipification of local header files~~ [HIPIFY][feature] Implicit hipification of local header files - Part 1 Mar 20, 2026

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 353ebef to d10566f Compare March 24, 2026 07:43

ranapratap55 mentioned this pull request Mar 25, 2026

[HIPIFY][#2402] Warn users for non-self-contained header hipification in isolation - Part 2 #2447

Draft

emankov requested changes Mar 31, 2026

View reviewed changes

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from d10566f to 7d568b6 Compare April 6, 2026 17:10

ranapratap55 requested a review from emankov April 7, 2026 10:55

This was referenced Apr 7, 2026

[HIPIFY][#2402] hipify project-local headers included with angle brackets (<>) #2505

Open

[HIPIFY][#2402] Transitive preceding includes for deeply nested recursive headers #2507

Open

emankov requested changes Apr 9, 2026

View reviewed changes

Comment thread src/LocalHeader.cpp Outdated

Comment thread src/LocalHeader.cpp Outdated

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 8d1113a to 8a1adf5 Compare April 13, 2026 05:54

ranapratap55 requested a review from emankov April 13, 2026 06:11

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 8a1adf5 to e198b45 Compare April 20, 2026 10:12

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from e198b45 to 7d31bff Compare May 12, 2026 08:37

ranapratap55 changed the title ~~[HIPIFY][feature] Implicit hipification of local header files - Part 1~~ [HIPIFY][feature] Local header hipification via Clang PPCallbacks (experimental, opt-in) - Part 1 May 13, 2026

emankov requested changes May 20, 2026

View reviewed changes

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 7d31bff to 3e3db8f Compare June 16, 2026 08:52

emankov requested changes Jul 23, 2026

View reviewed changes

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch 2 times, most recently from 7321229 to 7bc6edf Compare August 3, 2026 08:09

emankov requested changes Aug 7, 2026

View reviewed changes

ranapratap55 added 10 commits September 24, 2026 20:53

[HIPIFY][feature] Local header hipification with preceding include in…

8177f16

…jection via ArgumentsAdjusters

[HIPIFY][doc][NFC] Updated the licence

4a372ea

[HIPIFY][feature] Implicit hipification of local header files

c5f4220

[HIPIFY][fix] Fix regex, premature loop break, and duplicate warnings

3444199

[HIPIFY][fix] Fix compile_commands json issue

6ceba8e

[HIPIFY][fix] Fix regex and structured bindings

1b6be0a

[HIPIFY][fix] Refactor include scanning and add depth-2 transitive co…

f88daef

…ntext test

[HIPIFY] Replace regex-based include scanning with Clang PPCallbacks …

5785cba

…and restore opt-in flags

[HIPIFY][fix] Apply review fixes: forward decl, <memory>, lit RUN arg…

083c09f

…s, depth-2 test

[HIPIFY][fix] Single-pass include collection and local header context…

a82777b

… injection

ranapratap55 force-pushed the users/ranapratap55/implict-local-headers branch from 7bc6edf to a82777b Compare September 24, 2026 15:24

kzhuravl requested a review from emankov September 24, 2026 15:29

kzhuravl approved these changes Sep 24, 2026

View reviewed changes

		ct::RefactoringTool Tool(
		compDB ? *compDB : OptionsParserPtr->getCompilations(), {srcPath});

Conversation

ranapratap55 commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem Statement

Solution

Summary of changes in this PR

Clang PPCallbacks instead of regex

Fixed Compilation Database fallback

Cycle and duplicate detection

Uh oh!

emankov commented Feb 24, 2026

Uh oh!

ranapratap55 commented Feb 25, 2026

Uh oh!

emankov commented Feb 25, 2026

Uh oh!

ranapratap55 commented Feb 26, 2026

Uh oh!

emankov commented Feb 26, 2026

Uh oh!

ranapratap55 commented Mar 4, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ranapratap55 commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

emankov commented Apr 16, 2026

Uh oh!

emankov commented Apr 16, 2026

Uh oh!

ranapratap55 commented Apr 20, 2026

Uh oh!

ranapratap55 commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emankov commented May 5, 2026

Uh oh!

emankov commented May 5, 2026

Uh oh!

ranapratap55 commented May 13, 2026

Uh oh!

ranapratap55 commented May 13, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ranapratap55 Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ranapratap55 commented Feb 23, 2026 •

edited

Loading

ranapratap55 commented Apr 7, 2026 •

edited

Loading

ranapratap55 commented Apr 20, 2026 •

edited

Loading

ranapratap55 Jun 16, 2026 •

edited

Loading