Skip to content

plugins.md

Latest commit

 

History

History
203 lines (160 loc) · 13.5 KB

plugins.md

File metadata and controls

203 lines (160 loc) · 13.5 KB
Raw

Authoring DataLab-Web plugins

Shared Desktop plugins target Python 3.10 or later. Web plugins still target the Python 3.12 interpreter provided by Pyodide; DataLab-Web's CPython test minimum remains Python 3.11.

DataLab-Web supports portable DataLab plugin classes inside Pyodide. A plugin may be loaded as a consented single Python source file or installed as a managed pure-Python wheel. Plugin code shares the main Python interpreter and is not sandboxed. It must not block the JavaScript event loop.

For action-oriented source plugins, any path that opens a parameter dialog must be async and use await param.edit_async(self.main) instead of synchronous param.edit(self.main). Application wheels should declare headless recipes and examples; the generic Applications host owns their dialogs and commits.

The good news: the same plugin source can be used in both DataLab Qt and DataLab-Web. DataSet.edit_async() is implemented in upstream guidata; on Qt it simply delegates to edit().

Anatomy of a plugin

# my_plugin.py
from __future__ import annotations

from datalab.config import _
from datalab.plugins import PluginBase, PluginInfo


class MyPlugin(PluginBase):
    """Minimal DataLab-Web plugin example."""

    PLUGIN_INFO = PluginInfo(
      id="org.example.my-plugin",
        name=_("My plugin"),
        version="0.1.0",
        description=_("Adds a few useful actions"),
    )

    async def hello_world(self) -> None:
        # Use the proxy to manipulate the workspace.
        # `add_object`, `calc`, `get_object*`, `call_method` are all
        # available — see ``datalab.control.proxy.LocalProxy``.
        sig = ...  # build a SignalObj or ImageObj
        self.proxy.add_object(sig)

    def create_actions(self) -> None:
        sah = self.signalpanel.acthandler
        with sah.new_menu(_("My plugin")):
            sah.new_action(
                _("Hello world"),
                triggered=self.hello_world,
                select_condition="always",
            )

Available APIs

PluginBase exposes the same surface as the desktop counterpart:

Attribute / method Purpose
self.main Bridge to the React main window (parent for dialogs)
self.signalpanel, self.imagepanel Panels with processor, acthandler, get_newparam_from_current
self.proxy LocalProxy-compatible API: add_object, add_group, calc, get_object*, call_method
await self.show_info_async(msg) Async info popup (show_warning_async / show_error_async for the other kinds)
await self.ask_yesno_async(msg, cancelable=…) Async yes/no(/cancel) prompt
await self.edit_new_signal_parameters_async(…) Async variant of the desktop helper
await self.edit_new_image_parameters_async(…) Async variant of the desktop helper

The synchronous helpers (self.show_info / show_warning / show_error / ask_yesno) exist for desktop source compatibility but raise BrowserNotSupportedError in Pyodide — use the _async variants above.

Browser-only differences

  • Synchronous dialogs raise BrowserNotSupportedError. param.edit(self.main), BaseProxy.calc(...) blocking calls and any similar synchronous prompt are unavailable. Always use the _async counterparts.
  • No filesystem access beyond Pyodide's in-memory FS. Plugins that read local files on the desktop should use the file picker exposed by the host UI instead.
  • No worker / process isolation. All plugin code runs in the same Python interpreter as Sigima. Long-running computations will freeze the UI; consider chunking via asyncio.sleep(0) to yield.

Loading a plugin

  • Python source: open Plugins → Manage plugins…, click Load Python file… and select a .py file. The first time a plugin is loaded you will be prompted to confirm execution. The decision is remembered (per source SHA-256) in localStorage.
  • Local wheel: click Install wheel… and select a .whl file. DataLab-Web inspects the archive before consent and before importing plugin code. The review shows its SHA-256, distribution, version, Python requirement, tags, Web entry points, and dependency compatibility. Accepted artifacts and activation preferences are persisted under the dedicated dlw-plugin-wheels OPFS subtree. This command is disabled when persistent OPFS storage is unavailable.
  • Bundled examples: the Plugins → Manage plugins… dialog lists the example plugins shipped under the repo-root plugins/examples/ folder (mirroring DataLab desktop). Click Load next to one to try it; the same consent gate applies. These are not auto-loaded.
  • Bundled built-ins: drop a file into src/runtime/builtin_plugins/; it will be discovered automatically at startup.

The hash identifies exact bytes; it does not prove that code is safe. A local wheel remains visibly unverified after consent. It can access APIs exposed by Pyodide and the browser origin. Disable or remove it from the manager to unload its contributions and managed import modules; removing it also deletes its OPFS artifact and revokes its stored trust decision.

Managed wheel contract

A compatible wheel exposes at least one entry point in the dedicated Web group. A distribution shared with Desktop may publish both groups; the browser imports only datalab.web_plugins and never imports the Qt target.

[project.entry-points."datalab.web_plugins"]
my-plugin = "my_package.adapters.web:MyWebPlugin"

[project.entry-points."datalab.plugins"]
my-plugin = "my_package.adapters.desktop:MyDesktopPlugin"

The Web target must be a subclass of the portable PluginBase and provide a stable namespaced PluginInfo.id. Application plugins include PluginCapability.APPLICATION, declare RECIPES and optionally EXAMPLES, and may implement materialize_example(). Recipe slots carry signal/image type, ONE/MANY cardinality, required state, a title and description, a minimum count, and metadata requirements; recipes may add suggest_bindings and check_inputs hooks. An example lists the recipes it is designed for in recipe_ids, and generated examples key their parameter values by recipe ID. Parameters are guidata DataSet classes. Outputs are returned as a RecipeOutcome; DataLab-Web commits objects, anchored scalar results, diagnostics, and provenance transactionally.

The Applications dialog presents each recipe as a method card: expected inputs, a readiness status re-assessed when the selection changes, a Run on selection… button, and Try with this example for each example designed for it. Examples without a recipe are listed as datasets. Readiness and binding checks use the same datalab.plugins.recipe_binding module as DataLab Desktop, copied into the host shim, so both hosts accept the same selections.

Tools and instruments

Application plugins may also declare TOOLS, a tuple of datalab.plugins.tools.PluginTool values, as on DataLab Desktop. The Applications dialog lists them in a Tools section, and the Plugins menu lists them in the plugin's submenu, after its methods and examples. object_type places a tool in the signal or image menu (both when None), and selection (none, exactly_one, at_least_one, at_least_two) enables it only when enough objects of that type are selected; otherwise the dialog tells which objects to select.

A tool is opened in one of two ways:

  • Launcher (launcher="method_name"): DataLab calls the plugin method, and awaits it when it is a coroutine, so it can open dialogs with edit_async(). self.get_selected_objects() returns the objects selected when the tool was launched; objects the tool adds are refreshed and selected afterwards.
  • Instrument (instrument="method_name"): the method returns a datalab.plugins.instruments.PluginInstrument, which holds its settings in a guidata DataSet and implements preview() (an InstrumentFrame: signals drawn together, or one image, with a summary and an optional fixed range) and acquire() (an InstrumentAcquisition: a group title and the acquired objects). DataLab draws the window itself: a live view on the left, the settings form on the right, a Live toggle refreshing the view every live_interval_ms, a New group option and an Acquire button. Each acquisition goes to a new group or, when the user turns New group off, to the current group of the panel (a new group is still created when the panel has no current group). The new_group class attribute of the instrument sets the default state of the option (True by default); the choice of the user is remembered for each instrument and prevails. The instrument instance lives as long as the plugin is registered, so settings are kept between openings. The plugin writes no user interface code, so the same instrument runs on DataLab Desktop and DataLab-Web.

datalab.plugins.tools, datalab.plugins.instruments and datalab.plugins.resources are copied from DataLab Desktop into the host shim, like datalab.plugins.recipe_binding.

The wheel rules below come from datalab.plugins.wheels, also copied verbatim from DataLab Desktop and used by the plugin catalog, so a wheel is judged the same way everywhere; dlw_wheels.py only applies them to the datalab.web_plugins group. A Python test compares the copy with a sibling DataLab checkout when one is available.

Version 1 of the installer deliberately accepts only local *-none-any pure-Python wheels compatible with Pyodide's Python version. Native payloads, archive traversal, reserved host namespaces, and distribution/package/plugin collisions are rejected. Every Requires-Dist dependency must already be provided by the host at a compatible version. There is no package-index lookup, dependency resolution, network download, or implicit installation of a second wheel.

Bundled multi-module distributions

Single-file built-ins are not sufficient for a plugin that has a reusable core -> workflow -> adapters package. DataLab-Web therefore ships qualified multi-module plugins as explicit wheel assets under src/runtime/builtin_wheels/. Each artifact has a checked version, size, and SHA-256 entry in bundledPlugins.ts. The runtime fetches only the Vite-bundled URL and passes the wheel through the same inspector and managed registry as a local wheel before importing its Web entry point.

The Camera and Pulse characterization wheels are the first such bundles. They are deliberately not installed from a public package index and their Desktop entry points are not used in the browser. DataLab-Web supplies the portable datalab.* host shim. Camera reuses the existing HDF5 byte loader for its packaged quickstart, while Pulse creates its documented deterministic campaign through the shared headless simulator. See src/runtime/builtin_wheels/README.md for the update procedure.

Bundling proves distribution, not compatibility. A bundled artifact receives the registry trust state verified only after its visible outputs and Pyodide resource budget have passed the project-specific qualification gate.

The Camera gate in tests/e2e/camera_bundle.spec.ts runs the packaged campaign through the real worker-hosted Pyodide runtime. It requires a visible Plotly response trace, decoded non-blank PRNU-map pixels, and the anchored metrics table in the Results panel. It also limits incremental WASM-heap growth to 64 MiB and retained output arrays to three times the input arrays. The status qualifies DataLab-Web 0.8.0, Pyodide 0.26.4, Camera 0.2.0, and relative-DN recipe 1.1.0.

The Pulse gate in tests/e2e/pulse_bundle.spec.ts executes the deterministic 500-shot campaign through Pyodide and transactionally commits three signal outputs plus the anchored 500-row metrics table. It requires visible Plotly traces for amplitude, raw mean, and aligned mean; all six quality statuses; 489 valid/aligned shots; no more than 64 MiB incremental WASM heap; and exactly 24,032 bytes of retained output arrays. This qualifies DataLab-Web 0.8.0, Pyodide 0.26.4, Pulse 0.2.0, and pulse campaign recipe 1.1.0.

Since version 0.2.0, Camera also offers photon transfer and dark-current recipes, and Pulse offers shot-to-shot stability, step-response, two-channel delay and pulse-height spectrum recipes, each with a generated example. tests/e2e/application_methods.spec.ts opens every new example through its deep link, checks that its method is ready, runs it from the Applications dialog and checks the created outputs in the visible object tree; it also runs the relative-DN method through Try with this example on the photon transfer ladder. tests/python/test_camera_application.py and tests/python/test_pulse_application.py run the same pairs, plus the photon transfer → relative-DN and laser warm-up → campaign pairs, through the generic host and check that every example binds its recipe slots without ambiguity.

Hot reload

Plugins → Reload all plugins re-imports every loaded plugin module from disk. The plugin registry strips all features and menu entries contributed by previous versions before re-applying them, so live edits are safe.