Skip to content

Load plugin packages from their folder without installing them - #412

Merged
PierreRaybaut merged 3 commits into
developfrom
fix/411-plugin-folders
Oct 21, 2026
Merged

PierreRaybaut merged 3 commits into
developfrom
fix/411-plugin-folders

Conversation

@PierreRaybaut

Copy link
Copy Markdown
Contributor

Fixes #411.

Problem

Plugins organized as a package (a datalab_<name>/ folder whose plugin class lives in adapters/desktop.py, as created by datalab-plugin create and used by the Camera, Pulse, Synchrotron and TANGO plugins) were silently ignored when their folder was added to the plugin directories. Folder discovery only registered classes defined while importing the package root, which layered plugins keep free of Qt; the Desktop adapter was only reachable through the entry point of an installed distribution. A checkout holding .egg-info from an earlier installation was loaded, a fresh clone was not. Users also had to declare the project's src subfolder rather than the project folder.

Fix

The existing naming convention is extended, without any new declaration:

  • After importing a datalab_* package, discovery also imports its adapters.desktop module when it exists. Its import errors are reported with the module name and file, without blocking the other plugins.
  • A configured plugin directory whose src subfolder holds datalab_* modules or packages is searched together with that subfolder. The search paths are computed once, for discover_plugins() and discover_v020_plugins(), and added to sys.path as before, so compute and preview processes inherit them in the same way.
  • A module is reloaded at most once per discovery pass, whether it is found by an entry point, by naming convention, or both. The existing merging of identical targets and the rejection of ID collisions are unchanged.
  • datalab-plugin create generates README, contribution and architecture texts that first explain how to try the plugin by adding the project folder, installation being kept for tests and publishing.

Behaviour change

  • Every datalab_* package on the search path now has its adapters/desktop.py module imported at discovery, if it has one. Packages without it (e.g. datalab_kernel, datalab_capsule) are unaffected.
  • A failing Desktop module is listed in the plugin configuration dialog under its module name (e.g. datalab_x.adapters.desktop) and file, instead of the package name.
  • A convention module first imported during the current pass, for instance through an entry point, is no longer reloaded a second time in that pass.
  • Precedence is unchanged: when the same plugin is both in a plugin directory and installed from a file, the folder copy is imported, as the plugin directories come first on sys.path.

Tests

  • New tests in plugins_registry_unit_test.py, on temporary projects without any distribution metadata: discovery from the project folder or from src (stable ID, source, file path); single-file plugins and packages without adapters/desktop.py unchanged; failing Desktop module reported without blocking another plugin; hot reload executing the edited module once; a package also declared by an entry point registered once with both sources and a single reload per module; search paths including only src folders that hold datalab_* modules, including DATALAB_PLUGINS directories.
  • New test in plugin_generator_unit_test.py: a project freshly created by datalab-plugin create is loaded from its folder without being installed.
  • Reproduction with fresh copies of the four plugin projects (without build artifacts), configured as project folder or as src: none is discovered on develop; all four are discovered once with this branch, also after a reload.
  • Catalog: the published Pulse 0.2.0 wheel, installed in a temporary plugin store, still loads from the wheel alone (entry point and convention merged), and loads once when its project folder is also a plugin directory.
  • Regression: datalab/tests/backbone/plugins and datalab/tests/features/plugins (226 passed), run with a PYTHONPATH without plugin folders.
  • Plugin test suites run against develop and against this branch, with identical results: Camera 158 passed; Pulse 113 passed; Synchrotron 94 passed and 1 failure on both (test_import_spec_scans); TANGO 33 passed and 2 failures on both, which rely on instrument window features not on develop.
  • Ruff format and check; Pylint reports no new message on the changed modules (two existing messages in datalab/plugins/base.py).

Documentation and translations

  • doc/features/advanced/plugins.rst: the two forms a plugin may take (single file or folder), the project folder accepted as plugin directory, and how to try a generated project without installing it.
  • Release notes 1.4 (Bug Fixes, "Plugins").
  • French translation of the changed entries only; the French documentation builds with -W.

Out of scope

  • A single-file template in datalab-plugin create, better suited to non-developers.
  • Hot reload of modules other than the package root and adapters/desktop.py, which still requires a restart, as with entry points.
  • Publishing the Synchrotron and TANGO plugins in the plugin catalog.

Follow-up: the development instructions of the four plugin repositories will mention loading the plugin from its folder.

…ng them (#411)

A plugin package found in a plugin directory was only registered if its
package root defined the plugin class. Layered packages, as created by
`datalab-plugin create`, keep their root free of Qt and define the class
in `adapters/desktop.py`, which only the entry point of an installed
distribution named. A fresh clone added as plugin directory was silently
ignored, while a checkout holding `.egg-info` from an earlier install was
loaded.

- After importing a `datalab_*` package, discovery also imports its
  `adapters.desktop` module when it exists. Errors in that module are
  reported with its name and file, without blocking other plugins.
- A module is reloaded at most once per discovery pass, whether it is
  found by an entry point, by naming convention or both.
- A configured plugin directory whose `src` subfolder holds `datalab_*`
  modules or packages is searched together with that subfolder, so that
  users may add the project folder itself.
- Generated projects explain how to try the plugin in DataLab without
  installing it.

Assisted-by: Claude Opus 5.5
Plugin projects without any distribution metadata, configured either as
project folder or as `src` folder, are discovered with their stable ID.
Single-file plugins and packages without `adapters/desktop.py` behave as
before; a failing Desktop module is reported without blocking other
plugins; hot reload executes the edited module once; a package also
declared by an entry point is registered once with both sources; and only
`src` folders holding `datalab_*` modules are searched.

A project freshly created by `datalab-plugin create` is loaded from its
folder without being installed.

Assisted-by: Claude Opus 5.5
The plugin documentation now presents the two forms DataLab loads from a
plugin directory, a single `datalab_*.py` file or a `datalab_*` folder,
and explains that a plugin project folder may be added directly. Release
notes 1.4 (Bug Fixes, "Plugins") with their French translation.

Assisted-by: Claude Opus 5.5
@PierreRaybaut PierreRaybaut added the bug Something isn't working label Oct 21, 2026
@PierreRaybaut
PierreRaybaut merged commit ee4c6a0 into develop Oct 21, 2026
6 checks passed
PierreRaybaut added a commit to DataLab-Platform/datalab-camera-characterization that referenced this pull request Oct 21, 2026
DataLab 1.4 loads a plugin package from its folder, without installing
it (DataLab-Platform/DataLab#411, DataLab-Platform/DataLab#412): the
contribution guide now explains how to try the plugin by adding the
project folder to DataLab's plugin directories.

Assisted-by: Claude Opus 5.5
PierreRaybaut added a commit to DataLab-Platform/datalab-pulse-characterization that referenced this pull request Oct 21, 2026
DataLab 1.4 loads a plugin package from its folder, without installing
it (DataLab-Platform/DataLab#411, DataLab-Platform/DataLab#412): the
contribution guide now explains how to try the plugin by adding the
project folder to DataLab's plugin directories.

Assisted-by: Claude Opus 5.5
@PierreRaybaut
PierreRaybaut deleted the fix/411-plugin-folders branch October 23, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant