[5.5/10] arch/arm, cmake: Build FDPIC modules in the normal ELF build. by casaroli · Pull Request #19940 · apache/nuttx

casaroli · 2026-08-23T21:07:52Z

depends-on: [/pull/19942]

Summary

This replaces what this PR was. It added tools/fdpic, a module build of its own; review asked instead that FDPIC go into the normal in-tree ELF build. So it does, and tools/fdpic is not added at all.

With CONFIG_FDPIC, a module built by apps/Application.mk is an FDPIC shared object. Same MODULE = m, same crt0.c, same linker script.

Two things differ from the position independent build beside it: the compiler is told -mfdpic -fPIC, and the link is done by an arm-uclinuxfdpiceabi linker. Only the link needs it; the stock compiler emits correct FDPIC objects for C and C++, its assembler included. That linker is in the CI image since #19992, and the build says so if it is missing.

The stock linker must not be fallen back to, because it does not refuse FDPIC objects. It marks the output UNIX - System V and turns every import into an R_ARM_JUMP_SLOT, one word, where the ABI wants an R_ARM_FUNCDESC_VALUE, which is two: a code address and the data base that goes with it. The module links cleanly and then calls out of itself with the caller's data base still in r9.

gnu-elf.ld.in gains the two segments an FDPIC module needs, under CONFIG_FDPIC, and names .dynamic. The sections and the symbols crt0.c walks are untouched, so one script serves both cases and both build systems.

One commit is not FDPIC's. A C++ module with a static object does not link under CONFIG_PIC today, on __dso_handle, which comes from crtbegin and a module does not link crtbegin. -fno-use-cxa-atexit fixes it and puts the destructors in .fini_array, where libelf_uninit() looks.

This comes after [5/10] #19942 rather than before it, because CONFIG_FDPIC is defined there.

Impact

Nothing changes with CONFIG_FDPIC off, which is every configuration in the tree.

With it on, a module needs the arm-uclinuxfdpiceabi linker. Anyone selecting the option needs it anyway.

The __dso_handle commit changes one thing for everyone building C++ ELF modules: static destructors go in .fini_array instead of __cxa_atexit. That is the only place the loader looks.

Testing

mps3-an547:picostest with apps/examples/elf, CONFIG_FDPIC both ways, through make and cmake.

$ readelf -h apps/bin/hello++3
  OS/ABI:      ARM FDPIC
  Type:        DYN (Shared object file)
  Entry point: 0x81
$ readelf -d apps/bin/hello++3
  0x19 (INIT_ARRAY)  0x1018
  0x1a (FINI_ARRAY)  0x101c

Every module in apps/bin is an FDPIC shared object with two PT_LOAD segments entering at _start. hello++3 has a static C++ object, so it carries both arrays; it does not link at all before the __dso_handle commit.

With CONFIG_FDPIC off the tree builds as before and the generated script has no PHDRS. With the linker off PATH a module link stops with a message naming it, and make distclean still works.

tools/checkpatch.sh -c -u -m -g and cmake-format --check pass.

The fixtures in apps/examples/fdpicxip keep a build of their own, because they are loader edge cases Application.mk cannot express. apache/nuttx-apps#3762 moves it there so NuttX carries none of it.

github-actions · 2026-08-23T21:14:48Z

MemBrowse Memory Report

s698pm-dkit

Code: .text +2,384 B (+0.7%, 367,552 B)
No memory changes detected for:
arduino-mega2560
esp32-devkitc
hifive1-revb
mirtoo
qemu-armv8a
qemu-intel64
rx65n-rsk2mb
stm32-nucleo-f103rb

The commits that follow teach the ELF loader to load an FDPIC object. This puts the option they hang off and the definitions they share in one place first, so each of them builds on its own. CONFIG_FDPIC depends on ARCH_HAVE_ELF_FDPIC, which an architecture selects when it has a PIC base register and the FDPIC relocations. Only armv7-m and armv8-m select it today, and it defaults off, so nothing changes for anyone who does not ask for it. include/nuttx/fdpic.h holds what both sides of the loader need: the two word function descriptor an FDPIC module passes instead of a code address, the test for whether the caller is such a module, and the call sequence that enters one with its own data base. All of it is behind CONFIG_FDPIC, thus the header is empty without it and a file may include it unconditionally. The call sequence itself is architecture specific, so arch/arm/include/arch.h supplies it as up_fdpic_invoke(), beside the other PIC base register macros. up_setpicbase() cannot serve here: the register has to hold the module's base for exactly one call and then go back, and nothing in C tells the compiler the register is live across that call, so the save, the install, the branch and the restore have to be one sequence. Built for mps3-an547:bl and mps3-an547:picostest, with CONFIG_FDPIC off, which is every configuration in the tree. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

An ET_DYN object is loaded into one allocation with its data behind its text, because its data references sit at a fixed distance from the code that makes them. An FDPIC object does not work that way: it reaches its data through a base register, so the two segments can be placed wherever suits, and the point of the format is that the read-only one is left on the media and executed there while only the writable one is copied. One copy of the text then serves every instance. So libelf_load() grows a second case. The object announces itself in the OS/ABI byte, which is noted once in libelf_loadhdrs() rather than re-derived; e_flags cannot be used for this, as an FDPIC object's are an unremarkable EABI version and testing them would reject every valid module. Text is taken from the media address plus the segment's own file offset -- the same arithmetic the ET_REL path already does with sh_offset -- and libelf_loadfile() does not read it. If the filesystem cannot show its media, the loader copies the text to RAM instead. The module then loses the shared text and the flash saving, but it runs. Obtaining that address needs two mechanisms, and they are not interchangeable. A compacting filesystem can move a file's blocks, so it hands out an address only with a pin that holds them still and expects the pin back; xipfs is the one in tree. A filesystem whose layout never changes has nothing to hold and answers FIOC_XIPBASE with a bare address; romfs and tmpfs are those. libelf_xipacquire() asks for the pin first, because a filesystem that needs one is not safe without it, and libelf_unload() gives it back. The loader asks for a pin only if it can hold one, or the pin would stay for ever. The pin is thus not specific to FDPIC. Any module that executes in place from a compacting filesystem takes one, and gives it back at unload. mmap() is not used, though both filesystems implement it. The mapping would be recorded against whichever task called the loader, while the release happens when the module's own task exits, which is a different group -- so the pin would outlive the module and the extent would never become movable again. Unloading has to change with placement: the existing path frees only textalloc because ET_DYN had a single allocation, which would leak an FDPIC object's data and free media the filesystem only lent us. Nothing here runs for a non-FDPIC object; every branch is behind the flag and the single-allocation path is untouched. Built and booted mps3-an547:picostest, which is CONFIG_ELF with CONFIG_PIC, with no change in behaviour. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

libelf_relocatedyn() reads the handful of DT_* tags it needs to walk the relocation tables and ignores the rest. Three more matter now. DT_PLTGOT is where the object's data base lives. An FDPIC module runs with that in the PIC base register, and every function descriptor built for it names the same base as the one its callee should run with, so without it there is nothing to put in a descriptor's second word. The DT_*_ARRAY tags are the constructor and destructor tables. These are already found through the section headers a few lines further down, and that path is kept, but the dynamic tags are the authoritative copy and an object is not obliged to carry section headers at all. Both paths now translate through libelf_addr(), so they agree on the answer rather than depending on which ran last. The tag values themselves were missing from include/elf.h and are added. Sizing the descriptor pool has to happen here rather than later. R_ARM_FUNCDESC asks the loader to manufacture a descriptor and hand back its address, which means the space must exist by the time the relocation is applied, and by then the segment has been placed. So libelf_elfsize() reserves it behind the writable data, bounded by the relocation count -- one relocation cannot ask for more than one descriptor. That bound has slack in it, but a descriptor is two words and modules are small, which is cheaper than walking every relocation twice to get an exact count. Nothing here runs for a non-FDPIC object. Built and booted mps3-an547:picostest with no change in behaviour. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

Running one for the first time turned up two holes in the ET_DYN path. Neither shows up in a build. An undefined symbol is resolved with libelf_findglobal(), which searches only the table of globally registered symbols. The export table that exec() hands its caller went no further than the ET_REL path, so an ET_DYN module could not import anything the caller supplied. Invisible while such modules resolved everything internally; an FDPIC module imports its libc, and every import failed with "Unable to resolve addr of ext ref printf" although the caller had passed a table containing printf. The export table is now threaded into libelf_relocatedyn() and consulted when the global table has no answer, leaving the existing lookup order intact. A relocation naming a symbol defined inside the object was dropped silently. The code handles a relocation with no symbol, and one against an undefined symbol, but a defined symbol fell through both. That was harmless while every dynamic relocation arriving here had symbol index zero, which is the case for R_ARM_RELATIVE. FDPIC brings the first ones that do not: a pointer to a static function is emitted against the *section* symbol, so the value is the section base and the offset within it -- including the Thumb bit -- is carried as the addend. Deriving a value from the word being patched, as the no-symbol case does, would translate that addend as though it were an address. Confirmed against a real module: .text at 0x23c plus an addend of 0x95 gives 0x2d1, which is the function with its Thumb bit. Also stop libelf_symname() reporting a nameless symbol as an error. A section symbol has no name, and libelf_findsymbol() walks the whole table looking for optional entries such as nx_stacksize, so it meets these routinely and checks for -ESRCH itself. At error level it printed ten or more lines per module load and buried the diagnostics that matter. Built and run on lm3s6965-ek with the examples/elf ROMFS. The ET_REL test modules load as before, and an FDPIC module now loads, relocates, resolves printf and puts from the table exec() supplied, and calls through a function descriptor of its own. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

A module that dlopen()s a library gets back function addresses from dlsym() and calls them. Under FDPIC a bare code address is not enough: the callee needs its own data base as well, so what dlsym() returns has to be a function descriptor. The exported symbol table carries no type information -- symtab_s is a name and a value, and its own comment says typing would have to be added to support anything but function pointers -- so by the time dlsym() is asked there is no way to tell a function from an object. libelf_insertsymtab() is the last point that can: st_info is still in hand there. So an FDPIC object's exported functions are published as the address of a descriptor carved from the module's pool, and dlopen(), dlsym() and the module registry need no knowledge of FDPIC at all. The pool is sized for the dynamic symbol table as well as the relocations, since both can draw from it. That leaves the symbol values themselves, which were wrong for any ET_DYN object. libelf_loadsymtab() adds the symbol's section address to its value, which is right for ET_REL, where the section address is where the section was actually placed and the value is relative to it. In a shared object both are already full link-time addresses, so adding them counts the section twice. It needs translating onto wherever the object was placed instead. Library data is shared between everything that dlopen()s it, because the registry holds one instance per name. Giving each user its own copy would mean teaching the registry about instances, which is a much larger change to shared code; an executable loaded through exec() already gets its own data, since that path loads a fresh copy each time. Built and run on lm3s6965-ek with the examples/elf ROMFS; the FDPIC module continues to load, relocate and call through its own descriptors. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

The FDPIC work touches these files, and nxstyle reports errors on the lines around every hunk, which fails the check job. The errors are older than this series: a switch body indented two columns too deep in elf_symbols.c, and declarations with no blank line after them. Whitespace and one reworded comment, no change in behaviour. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

The modules this example and testing/fs/xipfs carry are built by an explicit 'make regen', which reached into nuttx/tools/fdpic for a makefile that builds a module, a script that turns one into a header, and two more that checked its imports. Review of apache/nuttx#19940 asked that NuttX not carry a module build of its own, and it no longer does: with CONFIG_FDPIC an ordinary FDPIC module is built by apps/Application.mk like any other. These are not ordinary modules, which is why they keep a build of their own. They are fixtures for loader edge cases: a library with a SONAME, a module with more DT_NEEDED entries than the loader will follow, one whose imports stay in the lazy binding table, and one naming a symbol the firmware does not export, which exists to be refused. Application.mk cannot say any of that. So the build stays, and it is here beside them rather than in NuttX. It is also much smaller. The generic module makefile is gone: it existed to be included by anything, and only this one directory ever did, so its dozen useful lines are rules here. fdpic-embed.py is gone: xxd does that, as examples/elf already does it, and the license header it also wrote is a template beside it. fdpic-verify.sh and nuttx-exports.sh are gone with no replacement; they checked at build time what the xipfs suite already asserts at run time, for two hundred lines. What the fixtures no longer carry is a crt0 and a linker script. Both come from the tree named by NUTTX_DIR, which is where the in-tree module build takes them, so a fixture is built the way a module is. The crt0 source is compiled here rather than the built object taken, because these are deliberately built for cortex-m3 while the firmware is not: a v7-M module runs on both the v7-M and v8-M targets, so one set of headers serves the RP2350 and mps2-an500 alike. Regenerated qsorter, libshape and cxxuser against a tree configured with CONFIG_FDPIC. qsorter is ARM FDPIC, v7-M, two PT_LOAD segments, entering at _start; libshape carries its SONAME and its DT_INIT_ARRAY. The committed headers are left as they are. They will change when they are next regenerated, because a fixture now carries the tree's crt0 rather than one of its own, and that is a change the xipfs suite should be run against rather than made blind. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

github-actions · 2026-08-28T22:10:54Z

🔗 Cross-repo PR dependencies

The read-only Build run reported the following dependent PR(s) and fetched head SHA(s):

[5/10] libs/libc/elf: Load an FDPIC object. #19942 @ cc2b937

CI run: https://github.com/apache/nuttx/actions/runs/33191894877

Five points from the review, none of which changes what the loader does. The pin is no longer tied to a flat build. It is held with file_get() and file_dup2(), which are kernel side, so the condition that matters is which copy of libc this is, not which build. Guarding on __KERNEL__ the way libs/libc already does elsewhere gives protected and kernel builds the pin as well. FS_PIN says a filesystem can pin a file's blocks and give out their media address. FS_XIPFS selects it, so the loader no longer names a filesystem. textpin is gone. It was set exactly where libelf_pinhold() had succeeded, and that is what sets pinfile, so pinfile != NULL already said it. libelf_pinrelease() moves below the Public Functions banner, where a function that is not static belongs. fdpic_invoke() and up_fdpic_invoke() take arg before entry. Built mps3-an547:picostest three ways: CONFIG_FDPIC off, on, and on with CONFIG_FS_XIPFS so that FS_PIN is selected and the pin path is compiled. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

A C++ module with a static object does not link. GCC registers each such object's destructor with __cxa_atexit(dtor, obj, &__dso_handle), and __dso_handle comes from crtbegin, which a module does not link: hello++3.cxx:119: undefined reference to `__dso_handle' It is reachable today with CONFIG_PIC, where a module is linked as an executable and the symbol has to resolve. Without it the link is relocatable, the symbol stays undefined and nothing complains until something makes it resolve. -fno-use-cxa-atexit puts the destructors in .fini_array instead, which is where libelf_uninit() looks for them when the module is unloaded, so the flag that makes the link work is also the flag that makes the destructors run. Reproduced with apps/examples/elf on mps3-an547:picostest with CONFIG_PIC enabled: hello++3 fails to link before and links after. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

With CONFIG_FDPIC selected, a module built by apps/Application.mk is now an FDPIC shared object. Nothing about how a module is written or built changes: the same MODULE = m in the same Makefile, the same crt0 and the same linker script. Two things differ from the position independent build beside it. The compiler is told -mfdpic -fPIC, and the link is done by an arm-uclinuxfdpiceabi linker. The stock arm-none-eabi compiler emits correct FDPIC objects for both C and C++, so only the link needs it: the stock linker carries the armelf emulation alone and would turn every import into an R_ARM_JUMP_SLOT, one word, where the ABI wants an R_ARM_FUNCDESC_VALUE, which is two, a code address and the data base that goes with it. Such a module links cleanly and then calls out of itself with the caller's data base still in r9. That linker is in the CI image. gnu-elf.ld.in gains the two segments an FDPIC module needs, under CONFIG_FDPIC, because the loader places its read-only and writable segments independently, and names .dynamic, because a shared object is bound through it. The sections themselves are untouched and so are the symbols crt0.c walks, so one script serves both and both build systems get it. Built for mps3-an547:picostest with apps/examples/elf, CONFIG_FDPIC both ways. With it on, every module in apps/bin is ARM FDPIC with two PT_LOAD segments and enters at _start; hello++3, which has a static C++ object, carries DT_INIT_ARRAY and DT_FINI_ARRAY. With it off the generated script has no PHDRS and the modules are what they were. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

The same two differences as in common/Toolchain.defs: the compiler is told -mfdpic -fPIC, and the module link is done by an arm-uclinuxfdpiceabi linker. That linker is not the one that links the firmware, so the module link needs a variable of its own. CMAKE_ELF_LD is the ordinary linker unless the architecture sets it, which arm does under CONFIG_FDPIC. The linker script needs nothing here: it is generated from libs/libc/elf/gnu-elf.ld.in, which both build systems preprocess, and the FDPIC segments are already in it. -r is now conditional on CONFIG_PIC being off, which is what common/Toolchain.defs has always done and the cmake build did not: a position independent module is linked as an executable, and an FDPIC one as a shared object, so neither wants it. -fno-use-cxa-atexit mirrors CXXELFFLAGS for the same reason it was added there. Configured and built mps3-an547:picostest with CONFIG_FDPIC through cmake and ninja: the modules in bin/ are ARM FDPIC with two PT_LOAD segments. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

Without this the build says "arm-uclinuxfdpiceabi-ld: Command not found", which does not say what that is, where to get it, or that the prefix can be changed. The make build reports at the link rather than while parsing, so that a tree configured for FDPIC on a host without the linker can still be cleaned and reconfigured: an error at parse time takes make distclean with it. The cmake build reports while configuring, where nothing is built yet. Both name FDPIC_CROSSDEV, so a linker under another prefix can be used. Checked on mps3-an547:picostest with CONFIG_FDPIC and the linker off PATH: make distclean succeeds, and a module link stops with the message. With the linker present the modules build as before. Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>

github-actions · 2026-08-31T23:21:39Z

🔗 Cross-repo PR dependencies

The read-only Build run reported the following dependent PR(s) and fetched head SHA(s):

[5/10] libs/libc/elf: Load an FDPIC object. #19942 @ 3f1c66a

CI run: https://github.com/apache/nuttx/actions/runs/33433035705

github-actions Bot added Area: Documentation Improvements or additions to documentation Size: L The size of the change in this PR is large labels Aug 23, 2026

casaroli changed the title ~~[3/4] tools/fdpic: Add the module build helpers the demo apps use.~~ [3/10] tools/fdpic: Add the module build helpers the demo apps use. Aug 23, 2026

casaroli mentioned this pull request Aug 23, 2026

[5/10] libs/libc/elf: Load an FDPIC object. #19942

Open

acassis requested review from cederom, jerpelea, linguini1, raiden00pl and xiaoxiang781216 August 25, 2026 14:04

casaroli force-pushed the tools-fdpic branch 3 times, most recently from f874bb2 to 78e4fec Compare August 27, 2026 12:33

casaroli marked this pull request as ready for review August 27, 2026 21:48

xiaoxiang781216 reviewed Aug 28, 2026

View reviewed changes

casaroli mentioned this pull request Aug 28, 2026

tools/ci: Add the FDPIC binutils to the Linux CI image. #19992

Merged

casaroli force-pushed the tools-fdpic branch from 78e4fec to 0c30ddd Compare August 28, 2026 16:47

casaroli requested review from anchao, masayuki2009, pussuw, simbit18 and yamt as code owners August 28, 2026 16:47

casaroli changed the title ~~[3/10] tools/fdpic: Add the module build helpers the demo apps use.~~ arch/arm, cmake: Build FDPIC modules in the normal ELF build. Aug 28, 2026

casaroli mentioned this pull request Aug 28, 2026

examples/fdpicxip: Build the module fixtures without nuttx/tools/fdpic. apache/nuttx-apps#3762

Open

casaroli added 2 commits August 28, 2026 18:49

casaroli added 4 commits August 28, 2026 18:49

casaroli force-pushed the tools-fdpic branch from 0c30ddd to c08569c Compare August 28, 2026 16:50

casaroli changed the title ~~arch/arm, cmake: Build FDPIC modules in the normal ELF build.~~ [5.5/10] arch/arm, cmake: Build FDPIC modules in the normal ELF build. Aug 28, 2026

casaroli requested a review from xiaoxiang781216 August 28, 2026 17:34

casaroli added 5 commits August 31, 2026 21:45

casaroli force-pushed the tools-fdpic branch from c08569c to 7bc1b05 Compare August 31, 2026 19:53

Navigation Menu

Uh oh!

[5.5/10] arch/arm, cmake: Build FDPIC modules in the normal ELF build. - #19940

[5.5/10] arch/arm, cmake: Build FDPIC modules in the normal ELF build.#19940
casaroli wants to merge 11 commits into
apache:masterfrom
casaroli:tools-fdpic

casaroli commented Aug 23, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

casaroli commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Impact

Testing

Uh oh!

github-actions Bot commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

s698pm-dkit

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 28, 2026

🔗 Cross-repo PR dependencies

Uh oh!

github-actions Bot commented Aug 31, 2026

🔗 Cross-repo PR dependencies

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

casaroli commented Aug 23, 2026 •

edited

Loading

github-actions Bot commented Aug 23, 2026 •

edited

Loading