From 9a637d613081583220ea480a5a24ae70bdf1cb9f Mon Sep 17 00:00:00 2001 From: Pierre Raybaut <1311787+PierreRaybaut@users.noreply.github.com> Date: Thu, 22 Oct 2026 21:51:50 +0200 Subject: [PATCH 1/3] Clarify plugin source forms and distribution formats Distinguish single-file development, packages, wheels and functional capabilities before extending catalog support. Correct the module consent documentation and provide the French translation. Refs #419. Assisted-by: GPT-6.1 Sol --- doc/features/advanced/plugins.rst | 19 ++++++++++-- .../LC_MESSAGES/features/advanced/plugins.po | 29 ++++++++++++++++--- 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/doc/features/advanced/plugins.rst b/doc/features/advanced/plugins.rst index 6aa7799b2..5e0a2746d 100644 --- a/doc/features/advanced/plugins.rst +++ b/doc/features/advanced/plugins.rst @@ -65,6 +65,21 @@ DataLab supports three categories of plugins, each with its own purpose and regi - **HDF5 plugins** Special plugins that support HDF5 files with domain-specific tree structures. These allow DataLab to interpret signals or images organized in non-standard ways. +.. _plugin_forms: + +Source organization and distribution +----------------------------------- + +**Source organization**, **distribution format** and **functional capabilities** are independent concepts. A single-file plugin and a plugin organized as a Python package use the same :class:`datalab.plugins.PluginBase` API; neither form determines whether it provides processing, input/output, visualization or application features. + +- **Single-file plugin:** one ``datalab_.py`` module, easy to edit and share without a packaging project. This historical form is also the default output of ``datalab-plugin create``; it is not deprecated. +- **Package or project:** Python modules and, optionally, resources, tests and packaging metadata. ``datalab-plugin create --project`` offers one layered organization, not a mandatory architecture for all plugins. +- **Wheel distribution:** an installable ``.whl`` archive with standardized metadata and plugin entry points. A wheel can contain a single module or a larger package; it is a distribution format, not a functional plugin category. + +At present, a single-file plugin can be loaded from a plugin directory or installed with **Install from file...**, but online and local catalogs accept only wheels. Publishing an existing single-module implementation as a wheel does not require reorganizing it into the generated layered project, although packaging metadata must be supplied. + +A directly loaded source file remains in its development directory. A file installed through DataLab is copied to its managed user storage. A plugin is not a macro: a plugin registers persistent application features, whereas a macro is a script run to automate a task. Supporting the historical single-file form does not mean that obsolete APIs from DataLab v0.20 are automatically supported. + Live preview compatibility -------------------------- @@ -379,7 +394,7 @@ This tab installs a plugin shared as a file, without any Python tool: this is th - a wheel (``.whl``) built from a plugin project, for example with ``python -m build``: it must be pure Python (``py3-none-any``), declare its plugin class in the ``datalab.plugins`` entry-point group, and have its requirements provided by the host or the adjacent dependency wheelhouse (see :ref:`plugin_dependencies`); - a single Python module named ``datalab_.py``. -Before installing, DataLab checks the file without running it and shows its name, version, plugin classes, dependencies and SHA-256 digest. A plugin runs with the same rights as DataLab: install only plugins from authors you trust. +Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows the module name, file, size and SHA-256 digest, without claiming to know its version, classes or dependencies. A plugin runs with the same rights as DataLab: install only plugins from authors you trust. Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Installations not adding dependencies retain the plugin reload workflow; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded plugins are only activated at the next start of DataLab. @@ -781,7 +796,7 @@ Options adapt the generated file: An existing file is never overwritten. To share the plugin, send its file: other users install it with **Plugins > Configure plugins... > Install plugins** (see :ref:`plugin_install_from_file`). -To publish a plugin in the plugin catalog, or to develop a larger plugin with tests, create a layered project with ``--project`` instead (see :ref:`plugin_project`). +The catalogs currently require a wheel, not a particular source architecture (see :ref:`plugin_forms`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`). .. _plugin_project: diff --git a/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po b/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po index 637333166..c8d9d75cd 100644 --- a/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po +++ b/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po @@ -75,6 +75,27 @@ msgstr "**Plugins HDF5** Plugins spéciaux qui prennent en charge les fichiers H msgid "Live preview compatibility" msgstr "Compatibilité avec l'aperçu interactif" +msgid "Source organization and distribution" +msgstr "Organisation du code source et distribution" + +msgid "**Source organization**, **distribution format** and **functional capabilities** are independent concepts. A single-file plugin and a plugin organized as a Python package use the same :class:`datalab.plugins.PluginBase` API; neither form determines whether it provides processing, input/output, visualization or application features." +msgstr "**L'organisation du code source**, le **format de distribution** et les **capacités fonctionnelles** sont des notions indépendantes. Un plugin en fichier unique et un plugin organisé en paquet Python utilisent la même API :class:`datalab.plugins.PluginBase` ; aucune de ces formes ne détermine s'il fournit des fonctions de traitement, d'entrée/sortie, de visualisation ou d'application." + +msgid "**Single-file plugin:** one ``datalab_.py`` module, easy to edit and share without a packaging project. This historical form is also the default output of ``datalab-plugin create``; it is not deprecated." +msgstr "**Plugin en fichier unique :** un module ``datalab_.py``, facile à modifier et à partager sans projet de packaging. Cette forme historique est aussi celle produite par défaut par ``datalab-plugin create`` ; elle n'est pas obsolète." + +msgid "**Package or project:** Python modules and, optionally, resources, tests and packaging metadata. ``datalab-plugin create --project`` offers one layered organization, not a mandatory architecture for all plugins." +msgstr "**Paquet ou projet :** des modules Python et, éventuellement, des ressources, des tests et des métadonnées de packaging. ``datalab-plugin create --project`` propose une organisation en couches, pas une architecture obligatoire pour tous les plugins." + +msgid "**Wheel distribution:** an installable ``.whl`` archive with standardized metadata and plugin entry points. A wheel can contain a single module or a larger package; it is a distribution format, not a functional plugin category." +msgstr "**Distribution wheel :** une archive ``.whl`` installable, contenant des métadonnées normalisées et des points d'entrée de plugins. Un wheel peut contenir un seul module ou un paquet plus important ; c'est un format de distribution, pas une catégorie fonctionnelle de plugin." + +msgid "At present, a single-file plugin can be loaded from a plugin directory or installed with **Install from file...**, but online and local catalogs accept only wheels. Publishing an existing single-module implementation as a wheel does not require reorganizing it into the generated layered project, although packaging metadata must be supplied." +msgstr "Actuellement, un plugin en fichier unique peut être chargé depuis un répertoire de plugins ou installé avec **Installer depuis un fichier...**, mais les catalogues en ligne et locaux n'acceptent que les wheels. Distribuer une implémentation existante en un seul module sous forme de wheel n'impose pas de la réorganiser selon le projet en couches généré, mais nécessite des métadonnées de packaging." + +msgid "A directly loaded source file remains in its development directory. A file installed through DataLab is copied to its managed user storage. A plugin is not a macro: a plugin registers persistent application features, whereas a macro is a script run to automate a task. Supporting the historical single-file form does not mean that obsolete APIs from DataLab v0.20 are automatically supported." +msgstr "Un fichier source chargé directement reste dans son répertoire de développement. Un fichier installé par DataLab est copié dans son stockage utilisateur géré. Un plugin n'est pas une macro : un plugin enregistre des fonctions persistantes dans l'application, tandis qu'une macro est un script exécuté pour automatiser une tâche. La prise en charge de la forme historique en fichier unique n'implique pas celle des API obsolètes de DataLab v0.20." + msgid "Standard parameterized functions registered with ``register_1_to_1`` can participate in live preview without a custom renderer. They must return a ``SignalObj`` or ``ImageObj``, be importable in a spawned process, and accept serializable source/parameter copies. Preview uses a private process and does not publish results to the workspace. It is not a sandbox for filesystem, network or other external side effects. Register functions that must not run speculatively with ``preview_enabled=False``. Serialization or computation failures are reported in the preview; the normal OK path remains available. Unknown custom ``DataSet.edit`` implementations and alternate guidata backends are not replaced." msgstr "Les fonctions paramétrées standard enregistrées avec ``register_1_to_1`` peuvent utiliser l'aperçu interactif sans moteur de rendu personnalisé. Elles doivent retourner un ``SignalObj`` ou un ``ImageObj``, être importables dans un processus démarré par spawn et accepter des copies sérialisables des sources et paramètres. L'aperçu utilise un processus privé et ne publie aucun résultat dans l'espace de travail. Il ne protège pas des effets externes sur les fichiers, le réseau ou d'autres ressources. Enregistrez les fonctions qui ne doivent pas être exécutées provisoirement avec ``preview_enabled=False``. Les erreurs de sérialisation ou de calcul sont signalées dans l'aperçu ; le chemin normal via OK reste disponible. Les implémentations personnalisées de ``DataSet.edit`` non prises en charge et les backends alternatifs de guidata ne sont pas remplacés." @@ -360,8 +381,8 @@ msgstr "un paquet wheel (``.whl``) construit à partir d'un projet de plugin, pa msgid "a single Python module named ``datalab_.py``." msgstr "un module Python unique nommé ``datalab_.py``." -msgid "Before installing, DataLab checks the file without running it and shows its name, version, plugin classes, dependencies and SHA-256 digest. A plugin runs with the same rights as DataLab: install only plugins from authors you trust." -msgstr "Avant l'installation, DataLab vérifie le fichier sans l'exécuter et affiche son nom, sa version, ses classes de plugin, ses dépendances et son empreinte SHA-256. Un plugin s'exécute avec les mêmes droits que DataLab : n'installez que des plugins provenant d'auteurs de confiance." +msgid "Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows the module name, file, size and SHA-256 digest, without claiming to know its version, classes or dependencies. A plugin runs with the same rights as DataLab: install only plugins from authors you trust." +msgstr "Avant l'installation, DataLab vérifie le fichier sans l'exécuter. Pour un wheel, la confirmation affiche la distribution, la version, les points d'entrée et les exigences issus de ses métadonnées. Pour un module en fichier unique, elle affiche le nom du module, le fichier, la taille et l'empreinte SHA-256, sans prétendre connaître sa version, ses classes ou ses dépendances. Un plugin s'exécute avec les mêmes droits que DataLab : n'installez que des plugins provenant d'auteurs de confiance." msgid "Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Installations not adding dependencies retain the plugin reload workflow; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded plugins are only activated at the next start of DataLab." msgstr "Les plugins installés sont conservés dans le dossier ``installed_plugins`` du répertoire de configuration DataLab (par exemple ``~/.DataLab_v1/installed_plugins``) et listés dans l'onglet, où **Désinstaller** les supprime. Les installations sans ajout de dépendances conservent le mécanisme de rechargement des plugins ; un plugin nouvellement installé est activé. Les dépendances supplémentaires et les nouvelles versions de plugins déjà chargés ne sont activées qu'au prochain démarrage de DataLab." @@ -600,8 +621,8 @@ msgstr "``--package`` et ``--plugin-id`` : à défaut, le nom du module et l'ID msgid "An existing file is never overwritten. To share the plugin, send its file: other users install it with **Plugins > Configure plugins... > Install plugins** (see :ref:`plugin_install_from_file`)." msgstr "Un fichier existant n'est jamais écrasé. Pour partager le plugin, envoyez son fichier : les autres utilisateurs l'installent avec **Plugins > Configurer les plugins... > Installer des plugins** (voir :ref:`plugin_install_from_file`)." -msgid "To publish a plugin in the plugin catalog, or to develop a larger plugin with tests, create a layered project with ``--project`` instead (see :ref:`plugin_project`)." -msgstr "Pour publier un plugin dans le catalogue de plugins, ou pour développer un plugin plus important avec des tests, créez plutôt un projet structuré en couches avec ``--project`` (voir :ref:`plugin_project`)." +msgid "The catalogs currently require a wheel, not a particular source architecture (see :ref:`plugin_forms`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`)." +msgstr "Les catalogues exigent actuellement un wheel, pas une architecture particulière du code source (voir :ref:`plugin_forms`). Pour développer un plugin plus important avec le packaging et les tests déjà configurés, créez plutôt un projet structuré en couches avec ``--project`` (voir :ref:`plugin_project`)." msgid "Creating a layered plugin project" msgstr "Créer un projet de plugin structuré en couches" From 4263258f6e3b0ad41f9a417473237a39da633b15 Mon Sep 17 00:00:00 2001 From: Pierre Raybaut <1311787+PierreRaybaut@users.noreply.github.com> Date: Thu, 22 Oct 2026 22:55:10 +0200 Subject: [PATCH 2/3] Support single-file plugins in online and offline catalogs Inspect modules without execution, preserve historical sources, track unknown versions by digest, and reuse managed installation with consent, identity checks and failed-write restoration. Keep wheel behavior and dependency policies unchanged. Refs #419. Assisted-by: GPT-6.1 Sol --- datalab/gui/plugins/available.py | 82 +++- datalab/gui/plugins/install.py | 69 +++- datalab/locale/fr/LC_MESSAGES/datalab.po | 33 ++ datalab/plugins/catalog.py | 156 +++++++- datalab/plugins/catalogtool.py | 80 +++- datalab/plugins/generator.py | 4 +- datalab/plugins/modules.py | 249 +++++++++++++ datalab/plugins/store.py | 115 ++++-- .../plugin_module_catalog_unit_test.py | 351 ++++++++++++++++++ .../plugins/plugin_modules_unit_test.py | 168 +++++++++ scripts/frozen_plugin_probe.py | 91 +++++ 11 files changed, 1322 insertions(+), 76 deletions(-) create mode 100644 datalab/plugins/modules.py create mode 100644 datalab/tests/backbone/plugins/plugin_module_catalog_unit_test.py create mode 100644 datalab/tests/backbone/plugins/plugin_modules_unit_test.py diff --git a/datalab/gui/plugins/available.py b/datalab/gui/plugins/available.py index 492863205..7226030f6 100644 --- a/datalab/gui/plugins/available.py +++ b/datalab/gui/plugins/available.py @@ -16,6 +16,7 @@ from collections.abc import Callable, Sequence from html import escape from importlib import metadata as importlib_metadata +from importlib.machinery import PathFinder from guidata.configtools import get_icon from packaging.utils import canonicalize_name @@ -42,7 +43,7 @@ load_catalogs, ) from datalab.plugins.dependencies import DEPENDENCIES_DIRECTORY -from datalab.plugins.store import WHEEL_KIND +from datalab.plugins.store import MODULE_KIND, WHEEL_KIND from datalab.widgets.expandabletext import apply_palette_color, apply_subdued_color CATALOG_PAGE_URL = "https://datalab-platform.com/plugins/" @@ -88,7 +89,7 @@ class AvailablePluginItemWidget(QW.QWidget): def __init__( self, plugin: CatalogPlugin, - installed: tuple[str, str], + installed: tuple[str, str, str], parent: QW.QWidget | None = None, ) -> None: super().__init__(parent) @@ -135,7 +136,7 @@ def __init__( layout.addWidget(self.action_button) def _describe( - self, release: CatalogRelease | None, installed: tuple[str, str] + self, release: CatalogRelease | None, installed: tuple[str, str, str] ) -> tuple[str, str | None]: """Return the status text and the action button text, if any.""" plugin = self.plugin @@ -146,9 +147,34 @@ def _describe( python = f"{sys.version_info.major}.{sys.version_info.minor}" return _("Not available for Python %s") % python, None return _("Not available for DataLab desktop"), None - how, version = installed + how, version, digest = installed + if how == "unmanaged": + return _("Provided outside the plugin store"), None + if how == "conflict": + return _("Conflicting installed plugin: %s") % version, None + if how == "error": + return version, None if how == "environment": return _("Installed in the Python environment (%s)") % version, None + if how == "file" and plugin.kind == MODULE_KIND: + notice = ( + _("Deprecated: %s") % plugin.status_reason + if plugin.status == "deprecated" + else "" + ) + if digest == release.sha256: + return notice or _("Installed"), None + if version and release.version: + try: + if Version(release.version) < Version(version): + return _("Installed: %s") % version, None + if Version(release.version) > Version(version): + return notice or _("Installed: %s") % version, _( + "Update to %s" + ) % release.version + except InvalidVersion: + pass + return notice or _("Different installed revision"), _("Replace...") if how == "file" and not _is_newer(release.version, version): return _("Installed"), None status = "" @@ -363,7 +389,11 @@ def finished() -> None: self._populate() self._installer.install_path( - path, origin=origin, wheelhouse=wheelhouse, on_finished=finished + path, + origin=origin, + wheelhouse=wheelhouse, + on_finished=finished, + catalog_plugin=plugin, ) def _on_download_failed(self, message: str) -> None: @@ -374,25 +404,49 @@ def _on_download_failed(self, message: str) -> None: self.status_label.setText("") QW.QMessageBox.warning(self, _("Install plugin"), message) - def _get_installed(self, plugin: CatalogPlugin) -> tuple[str, str]: + def _get_installed(self, plugin: CatalogPlugin) -> tuple[str, str, str]: """Return how a catalog plugin is installed, and its version.""" - if not plugin.distribution: - return "", "" - key = canonicalize_name(plugin.distribution) try: installed_plugins = self._installer.store.get_plugins() - except ValueError: - installed_plugins = [] + except (OSError, ValueError) as exc: + return "error", str(exc), "" + if plugin.kind == MODULE_KIND: + for installed in installed_plugins: + same_module = installed.kind == MODULE_KIND and osp.normcase( + installed.name + ) == osp.normcase(plugin.module) + if same_module: + if installed.name != plugin.module: + return "conflict", installed.name, "" + if installed.catalog_id and installed.catalog_id != plugin.id: + return "conflict", installed.catalog_id, "" + return "file", installed.version, installed.sha256 + if installed.catalog_id == plugin.id: + return "conflict", installed.name, "" + managed = osp.normcase(osp.realpath(self._installer.store.modules_dir)) + paths = [ + entry + for entry in sys.path + if osp.normcase(osp.realpath(entry)) != managed + ] + if PathFinder.find_spec(plugin.module, paths) is not None: + return "unmanaged", "", "" + return "", "", "" + if not plugin.distribution: + return "", "", "" + key = canonicalize_name(plugin.distribution) for installed in installed_plugins: if ( installed.kind == WHEEL_KIND and canonicalize_name(installed.name) == key ): - return "file", installed.version + return "file", installed.version, installed.sha256 + if installed.catalog_id == plugin.id and installed.kind != WHEEL_KIND: + return "conflict", installed.name, "" try: - return "environment", importlib_metadata.version(plugin.distribution) + return "environment", importlib_metadata.version(plugin.distribution), "" except importlib_metadata.PackageNotFoundError: - return "", "" + return "", "", "" def _populate(self) -> None: """Rebuild the plugin rows matching the search text.""" diff --git a/datalab/gui/plugins/install.py b/datalab/gui/plugins/install.py index 5099ccdaf..9cfa87109 100644 --- a/datalab/gui/plugins/install.py +++ b/datalab/gui/plugins/install.py @@ -27,7 +27,9 @@ enable_installed_plugin_classes, get_installed_plugin_store, ) +from datalab.plugins.catalog import CatalogPlugin from datalab.plugins.dependencies import DependencyPlan +from datalab.plugins.modules import apply_catalog_metadata from datalab.plugins.store import ( WHEEL_KIND, InstalledPlugin, @@ -93,7 +95,9 @@ def _format_date(timestamp: str) -> str: def _plugin_title(plugin: InstalledPlugin) -> str: if plugin.kind == WHEEL_KIND: return f"{plugin.name} {plugin.version}" - return plugin.filename + if plugin.version: + return f"{plugin.filename} {plugin.version}" + return f"{plugin.filename} ({_('revision %s') % plugin.sha256[:12]})" class PluginConsentDialog(QW.QDialog): @@ -164,6 +168,26 @@ def __init__( form.addRow(_("Dependency source:"), source) else: form.addRow(_("Plugin module:"), QW.QLabel(manifest["name"])) + if manifest["display_name"]: + form.addRow(_("Plugin:"), QW.QLabel(escape(manifest["display_name"]))) + if manifest["version"]: + form.addRow(_("Version:"), QW.QLabel(escape(manifest["version"]))) + else: + form.addRow(_("Revision:"), QW.QLabel(manifest["sha256"][:12])) + if manifest["summary"]: + description = QW.QLabel(escape(manifest["summary"])) + description.setWordWrap(True) + form.addRow(_("Description:"), description) + if manifest["warnings"]: + warning = QW.QLabel( + _( + "Some metadata cannot be read without executing the file. " + "Catalog metadata is used when provided; dependencies are " + "not installed for single-file plugins." + ) + ) + warning.setWordWrap(True) + form.addRow(warning) form.addRow(_("File:"), QW.QLabel(manifest["filename"])) form.addRow(_("Size:"), QW.QLabel(_format_size(manifest["size_bytes"]))) digest = QW.QLabel(manifest["sha256"]) @@ -245,7 +269,7 @@ def __init__( ) -> None: super().__init__(parent) self._reload_plugins = reload_plugins - self._pending_install: tuple[str, str | None] | None = None + self._pending_install: tuple[str, str | None, str] | None = None self._pending_finish: Callable[[], None] | None = None self._dependency_plan: DependencyPlan | None = None self._manifest: dict | None = None @@ -354,6 +378,7 @@ def install_path( *, wheelhouse: str | None = None, on_finished: Callable[[], None] | None = None, + catalog_plugin: CatalogPlugin | None = None, ) -> InstallResult | None: """Install a plugin file after the user's consent. @@ -377,7 +402,11 @@ def install_path( item["applies"] and not item["compatible"] for item in manifest["dependencies"] ): - self._pending_install = path, origin + self._pending_install = ( + path, + origin, + catalog_plugin.id if catalog_plugin else "", + ) self._pending_finish = on_finished self._manifest = manifest self._set_busy(True, _("Checking the local dependency wheels...")) @@ -390,12 +419,35 @@ def install_path( return None else: manifest = self.store.inspect_module_file(path) + if catalog_plugin is not None: + release = catalog_plugin.get_installable_release() + if release is None or catalog_plugin.module != manifest["name"]: + raise PluginInstallError( + "The module does not match the catalog" + ) + manifest = apply_catalog_metadata( + manifest, + plugin_id=catalog_plugin.id, + name=catalog_plugin.name, + version=release.version, + summary=catalog_plugin.summary, + ) if not self.confirm_installation(manifest, origin=origin): return None - install = ( - self.store.install_wheel if is_wheel else self.store.install_module - ) - result = install(path, expected_sha256=manifest["sha256"]) + if is_wheel: + result = self.store.install_wheel( + path, + expected_sha256=manifest["sha256"], + catalog_id=catalog_plugin.id if catalog_plugin else "", + ) + else: + result = self.store.install_module( + path, + expected_sha256=manifest["sha256"], + catalog_id=catalog_plugin.id if catalog_plugin else "", + catalog_name=catalog_plugin.name if catalog_plugin else "", + catalog_version=manifest["version"], + ) except (OSError, ValueError) as exc: QW.QMessageBox.warning( self, @@ -433,7 +485,7 @@ def _set_busy(self, busy: bool, text: str = "") -> None: def _on_prepared(self, prepared: tuple[dict, DependencyPlan | None]) -> None: manifest, self._dependency_plan = prepared assert self._pending_install is not None - path, origin = self._pending_install + path, origin, catalog_id = self._pending_install if self._manifest is None or manifest["sha256"] != self._manifest["sha256"]: self._on_install_failed( _("The plugin changed while checking dependencies.") @@ -448,6 +500,7 @@ def _on_prepared(self, prepared: tuple[dict, DependencyPlan | None]) -> None: path, expected_sha256=manifest["sha256"], dependency_plan=self._dependency_plan, + catalog_id=catalog_id, ), self._on_installed, self._on_install_failed, diff --git a/datalab/locale/fr/LC_MESSAGES/datalab.po b/datalab/locale/fr/LC_MESSAGES/datalab.po index 28de12a0a..411bfacdf 100644 --- a/datalab/locale/fr/LC_MESSAGES/datalab.po +++ b/datalab/locale/fr/LC_MESSAGES/datalab.po @@ -10,6 +10,39 @@ msgstr "" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" +msgid "Source code" +msgstr "Code source" + +msgid "not applicable" +msgstr "sans objet" + +msgid "Provided outside the plugin store" +msgstr "Fourni hors du stockage géré des plugins" + +msgid "Conflicting installed plugin: %s" +msgstr "Plugin installé en conflit : %s" + +msgid "Different installed revision" +msgstr "Révision installée différente" + +msgid "Replace..." +msgstr "Remplacer..." + +msgid "Revision:" +msgstr "Révision :" + +msgid "revision %s" +msgstr "révision %s" + +msgid "Some metadata cannot be read without executing the file. Catalog metadata is used when provided; dependencies are not installed for single-file plugins." +msgstr "Certaines métadonnées ne peuvent pas être lues sans exécuter le fichier. Les métadonnées du catalogue sont utilisées si elles sont fournies ; les dépendances ne sont pas installées pour les plugins en fichier unique." + +msgid "Write the catalog.json file listing the plugin files." +msgstr "Écrire le fichier catalog.json qui liste les fichiers des plugins." + +msgid "Check the catalog.json file against the plugin files." +msgstr "Vérifier le fichier catalog.json par rapport aux fichiers des plugins." + msgid "Additional packages from the local wheelhouse:" msgstr "Paquets supplémentaires du dépôt local :" diff --git a/datalab/plugins/catalog.py b/datalab/plugins/catalog.py index b2064899a..a6cc15041 100644 --- a/datalab/plugins/catalog.py +++ b/datalab/plugins/catalog.py @@ -4,13 +4,13 @@ Plugin catalog client --------------------- -Read the DataLab plugin catalog (``catalog.json``) and download release wheels +Read the DataLab plugin catalog (``catalog.json``) and download plugin artifacts whose size and SHA-256 digest match the catalog. This module is Qt-free: the plugin dialog calls it from a worker thread. Besides the online catalog, local catalogs are folders, typically shared on the network, read as files: their ``catalog.json`` if any, otherwise the plugin -wheels they hold. +wheels and single-file modules they hold. """ from __future__ import annotations @@ -23,6 +23,7 @@ import os import os.path as osp import re +import sys import urllib.error import urllib.parse import urllib.request @@ -32,11 +33,13 @@ from email.parser import BytesParser from email.policy import default as email_policy +from packaging.specifiers import InvalidSpecifier, SpecifierSet from packaging.tags import sys_tags from packaging.utils import canonicalize_name, parse_wheel_filename from packaging.version import InvalidVersion, Version from datalab import __version__ +from datalab.plugins.modules import MAX_MODULE_BYTES, inspect_module, module_name from datalab.plugins.wheels import ( DESKTOP_ENTRY_POINT_GROUP, MAX_WHEEL_BYTES, @@ -97,6 +100,7 @@ class CatalogRelease: yanked: str = "" requires_python: str | None = None dependency_wheels: tuple[CatalogWheel, ...] = () + kind: str = "wheel" @dataclasses.dataclass(frozen=True) @@ -119,6 +123,8 @@ class CatalogPlugin: license: str = "" keywords: tuple[str, ...] = () source: str = "" + kind: str = "wheel" + module: str = "" def get_installable_release( self, target: str = "desktop", python_version: str | None = None @@ -141,8 +147,14 @@ def get_installable_release( and target in release.targets and ( python_version == "" - or is_compatible_wheel( - release.filename, release.requires_python, python_version + or ( + _module_python_compatible( + release.requires_python, python_version + ) + if release.kind == "module" + else is_compatible_wheel( + release.filename, release.requires_python, python_version + ) ) ) ), @@ -212,7 +224,21 @@ def resolve(url: str) -> str: return resolve -def _parse_release(item: dict, resolve: Callable[[str], str]) -> CatalogRelease: +def _module_python_compatible( + requirement: str | None, python_version: str | None +) -> bool: + if not requirement: + return True + try: + version = python_version or f"{sys.version_info.major}.{sys.version_info.minor}" + return Version(version) in SpecifierSet(requirement) + except (InvalidSpecifier, InvalidVersion): + return False + + +def _parse_release( + item: dict, resolve: Callable[[str], str], kind: str = "wheel" +) -> CatalogRelease: if not _SHA256_PATTERN.fullmatch(item["sha256"]): raise ValueError(f"Invalid SHA-256 digest: {item['sha256']!r}") url = item.get("url", "") @@ -228,6 +254,17 @@ def _parse_release(item: dict, resolve: Callable[[str], str]) -> CatalogRelease: size=int(artifact["size"]), ) ) + if kind == "module": + if dependencies: + raise ValueError("Module plugins cannot install additional dependencies") + if url: + module_name(item["filename"]) + if tuple(item.get("targets", ())) != ("desktop",): + raise ValueError("Module plugins are Desktop-only") + if item.get("version"): + Version(str(item["version"])) + if item.get("requires_python"): + SpecifierSet(item["requires_python"]) return CatalogRelease( version=str(item["version"]), sha256=item["sha256"], @@ -238,27 +275,44 @@ def _parse_release(item: dict, resolve: Callable[[str], str]) -> CatalogRelease: yanked=item.get("yanked", ""), requires_python=item.get("requires_python"), dependency_wheels=tuple(dependencies), + kind=kind, ) def _parse_plugin( item: dict, resolve: Callable[[str], str], source: str ) -> CatalogPlugin: + kind = item.get("kind", "wheel") + if kind not in ("wheel", "module"): + raise ValueError(f"Unknown plugin artifact kind: {kind!r}") + module = item.get("module", "") + releases = tuple( + _parse_release(release, resolve, kind) for release in item["releases"] + ) + if kind == "module": + module_name(f"{module}.py") + if item.get("distribution"): + raise ValueError("Module plugins are not Python distributions") + if any( + release.url and module_name(release.filename) != module + for release in releases + ): + raise ValueError("Module file names must stay stable between releases") return CatalogPlugin( id=item["id"], name=item["name"], tier=item["tier"], status=item["status"], repository=item["repository"], - releases=tuple( - _parse_release(release, resolve) for release in item["releases"] - ), + releases=releases, status_reason=item.get("status_reason", ""), distribution=item.get("distribution", ""), summary=item.get("summary", ""), license=item.get("license", ""), keywords=tuple(item.get("keywords", ())), source=source, + kind=kind, + module=module, ) @@ -448,8 +502,51 @@ def list_folder_wheels(directory: str) -> list[str]: return [osp.join(directory, name) for name in names] +def list_folder_modules(directory: str) -> list[str]: + """List potential single-file plugins at the catalog root only.""" + try: + names = sorted( + name + for name in os.listdir(directory) + if name.startswith("datalab_") + and name.endswith(".py") + and osp.isfile(osp.join(directory, name)) + ) + except OSError as exc: + raise CatalogError(f"Cannot read the plugin catalog folder: {exc}") from exc + if len(names) > MAX_FOLDER_WHEELS: + raise CatalogError(f"{directory} holds too many plugin modules") + return [osp.join(directory, name) for name in names] + + +def read_module_entry(path: str) -> tuple[dict, dict, list[str]]: + """Describe a source module without importing it or inventing a version.""" + manifest = inspect_module(path, filename=osp.basename(path)) + name = manifest["name"] + plugin = { + "id": manifest["plugin_id"] or f"local.module.{name.replace('_', '-')}", + "name": manifest["display_name"] or name, + "kind": "module", + "module": name, + "tier": LOCAL_TIER, + "status": "active", + "repository": "", + "summary": manifest["summary"], + "license": "", + } + release = { + "version": manifest["version"], + "filename": manifest["filename"], + "url": urllib.parse.quote(manifest["filename"]), + "size": manifest["size_bytes"], + "sha256": manifest["sha256"], + "targets": ["desktop"], + } + return plugin, release, manifest["warnings"] + + def scan_wheel_folder(directory: str) -> tuple[dict, list[str]]: - """Return the catalog describing the plugin wheels of a folder. + """Return the catalog describing the wheels and modules of a folder. Releases of a plugin are grouped by distribution, from the newest. @@ -485,6 +582,24 @@ def scan_wheel_folder(directory: str) -> tuple[dict, list[str]]: versions = releases[key] plugin["releases"] = [versions[version] for version in sorted(versions)[::-1]] records.append(plugin) + for path in list_folder_modules(directory): + try: + plugin, release, _remarks = read_module_entry(path) + except (OSError, ValueError) as exc: + warnings.append(f"{path} ignored: {exc}") + continue + if any( + record["id"] == plugin["id"] + or record.get("distribution", "").replace("-", "_") == plugin["module"] + for record in records + ): + warnings.append( + f"{path} ignored: conflicting plugin identity or module name" + ) + continue + plugin["releases"] = [release] + records.append(plugin) + records.sort(key=lambda record: record["name"].casefold()) return {"schema_version": CATALOG_SCHEMA_VERSION, "plugins": records}, warnings @@ -492,7 +607,7 @@ def read_folder_catalog(directory: str) -> tuple[list[CatalogPlugin], list[str]] """Read a local catalog folder. The folder ``catalog.json`` lists the plugins if it exists, with release URLs - relative to the folder; otherwise, the plugin wheels of the folder are listed. + relative to the folder; otherwise, its plugin wheels and modules are listed. Args: directory: Catalog folder, possibly shared on the network @@ -547,6 +662,7 @@ def load_catalogs(url: str, directories: Sequence[str] = ()) -> CatalogsContent: errors: list[str] = [] ids: set[str] = set() distributions: set[str] = set() + modules: set[str] = set() readers: list[Callable[[], tuple[list[CatalogPlugin], list[str]]]] = [ lambda directory=directory: read_folder_catalog(directory) for directory in directories @@ -562,7 +678,11 @@ def load_catalogs(url: str, directories: Sequence[str] = ()) -> CatalogsContent: errors.extend(warnings) for plugin in items: distribution = canonicalize_name(plugin.distribution) - if plugin.id in ids or (distribution and distribution in distributions): + if ( + plugin.id in ids + or (distribution and distribution in distributions) + or (plugin.module and osp.normcase(plugin.module) in modules) + ): if plugin.status == "revoked": errors.append( f"{plugin.name} ({plugin.id}) is withdrawn from " @@ -572,12 +692,14 @@ def load_catalogs(url: str, directories: Sequence[str] = ()) -> CatalogsContent: ids.add(plugin.id) if distribution: distributions.add(distribution) + if plugin.module: + modules.add(osp.normcase(plugin.module)) plugins.append(plugin) return CatalogsContent(tuple(plugins), tuple(errors)) def download_release(release: CatalogRelease | CatalogWheel, directory: str) -> str: - """Download a release wheel into a directory after checking it. + """Download a wheel or source module after checking size and digest. Returns: Path of the downloaded wheel, named as in the catalog @@ -585,9 +707,13 @@ def download_release(release: CatalogRelease | CatalogWheel, directory: str) -> Raises: CatalogError: The download failed or does not match the catalog """ - if not release.url or not release.filename.endswith(".whl"): + is_module = isinstance(release, CatalogRelease) and release.kind == "module" + if is_module: + module_name(release.filename) + if not release.url or (not is_module and not release.filename.endswith(".whl")): raise CatalogError("This release cannot be downloaded") - if not 0 < release.size <= MAX_WHEEL_BYTES: + limit = MAX_MODULE_BYTES if is_module else MAX_WHEEL_BYTES + if not 0 < release.size <= limit: raise CatalogError(f"Invalid wheel size in the catalog: {release.size}") data = _read(release.url, release.size) if len(data) != release.size or hashlib.sha256(data).hexdigest() != release.sha256: @@ -627,10 +753,12 @@ def download_dependency_wheels(release: CatalogRelease, directory: str) -> None: "download_dependency_wheels", "download_release", "fetch_catalog", + "list_folder_modules", "list_folder_wheels", "load_catalog_json", "load_catalogs", "read_folder_catalog", + "read_module_entry", "read_wheel_entry", "scan_wheel_folder", ] diff --git a/datalab/plugins/catalogtool.py b/datalab/plugins/catalogtool.py index 26fad7f23..6e1ceef2a 100644 --- a/datalab/plugins/catalogtool.py +++ b/datalab/plugins/catalogtool.py @@ -5,7 +5,7 @@ ------------------------- Build and check the ``catalog.json`` file of a local plugin catalog: a folder -holding plugin wheels, typically shared on the network. This module is Qt-free +holding plugin artifacts, typically shared on the network. This module is Qt-free and backs the ``datalab-plugin catalog`` command. """ @@ -26,12 +26,14 @@ CATALOG_FILENAME, MAX_CATALOG_BYTES, CatalogError, + list_folder_modules, list_folder_wheels, load_catalog_json, read_folder_catalog, scan_wheel_folder, ) from datalab.plugins.dependencies import DEPENDENCIES_DIRECTORY, download_dependencies +from datalab.plugins.modules import apply_catalog_metadata, inspect_module from datalab.plugins.store import get_available_distributions from datalab.plugins.wheels import ( DESKTOP_ENTRY_POINT_GROUP, @@ -238,7 +240,7 @@ def _write_atomically(path: str, catalog: dict) -> None: def build_local_catalog(directory: str, *, write: bool = True) -> CatalogReport: - """Build the ``catalog.json`` file listing the plugin wheels of a folder. + """Build the ``catalog.json`` file listing plugin wheels and modules. Every wheel is checked as DataLab does before installing it. Fields edited in an existing ``catalog.json`` are kept: plugin ID, name, tier, status and @@ -271,11 +273,43 @@ def build_local_catalog(directory: str, *, write: bool = True) -> CatalogReport: plugins: list[dict] = [] for plugin in payload["plugins"]: releases = [] + previous = next( + ( + item + for item in previous_plugins + if plugin.get("kind") == "module" + and item.get("kind") == "module" + and item.get("module") == plugin["module"] + ), + None, + ) for release in plugin["releases"]: path = osp.join(root, release["filename"]) try: + if plugin.get("kind") == "module": + manifest = inspect_module(path, filename=release["filename"]) + if previous is not None: + old = previous.get("releases", []) + explicit_version = release["version"] or ( + str(old[0].get("version", "")) if old else "" + ) + manifest = apply_catalog_metadata( + manifest, + plugin_id=previous["id"], + name=previous["name"], + version=explicit_version, + summary=previous.get("summary", ""), + ) + release["version"] = manifest["version"] + if old and old[0].get("requires_python"): + release["requires_python"] = old[0]["requires_python"] + report.warnings.extend( + f"{path}: {message}" for message in manifest["warnings"] + ) + releases.append(release) + continue manifest = inspect_plugin_wheel(path, available, dependency_versions) - except (OSError, WheelInspectionError) as exc: + except (OSError, ValueError) as exc: report.errors.append(f"{path} refused: {exc}") continue release["requires_dist"] = [ @@ -286,9 +320,27 @@ def build_local_catalog(directory: str, *, write: bool = True) -> CatalogReport: releases.append(release) if releases: plugin["releases"] = releases + if previous is not None: + _keep_curated_fields(plugin, previous) + for field in ("summary", "license"): + if field in previous: + plugin[field] = previous[field] plugins.append(plugin) for previous in previous_plugins: + if previous.get("kind") == "module": + matched = any( + item.get("kind") == "module" + and item.get("module") == previous.get("module") + for item in plugins + ) + if not matched and previous.get("status") == "revoked": + plugins.append(previous) + elif not matched: + report.warnings.append( + f"{previous.get('name')}: no module left, removed from the catalog" + ) + continue distribution = canonicalize_name(str(previous.get("distribution", ""))) plugin = next( (item for item in plugins if distribution and item["id"] == distribution), @@ -352,6 +404,24 @@ def check_local_catalog(directory: str) -> CatalogReport: label = f"{plugin.name} {release.version}" listed.add(osp.normcase(release.url)) try: + if plugin.kind == "module": + manifest = inspect_module(release.url, filename=release.filename) + apply_catalog_metadata( + manifest, + plugin_id=plugin.id, + name=plugin.name, + version=release.version, + summary=plugin.summary, + ) + if ( + manifest["size_bytes"] != release.size + or manifest["sha256"] != release.sha256 + ): + raise ValueError( + f"{release.filename} does not match the size and digest " + "of the catalog" + ) + continue release_available = dict(available) additional: dict[str, list[str]] = {} if release.dependency_wheels: @@ -382,7 +452,7 @@ def check_local_catalog(directory: str) -> CatalogReport: except OSError as exc: report.errors.append(f"{label}: cannot read {release.url}: {exc}") continue - except WheelInspectionError as exc: + except ValueError as exc: report.errors.append(f"{label}: {release.url} refused: {exc}") continue if ( @@ -395,7 +465,7 @@ def check_local_catalog(directory: str) -> CatalogReport: ) report.warnings.extend( f"{wheel} is not listed in {CATALOG_FILENAME}" - for wheel in list_folder_wheels(root) + for wheel in [*list_folder_wheels(root), *list_folder_modules(root)] if osp.normcase(wheel) not in listed ) return report diff --git a/datalab/plugins/generator.py b/datalab/plugins/generator.py index 6a96db37c..01300502d 100644 --- a/datalab/plugins/generator.py +++ b/datalab/plugins/generator.py @@ -1005,8 +1005,8 @@ def build_parser() -> argparse.ArgumentParser: dest="catalog_command", required=True ) for name, text in ( - ("build", _("Write the catalog.json file listing the plugin wheels.")), - ("check", _("Check the catalog.json file against the plugin wheels.")), + ("build", _("Write the catalog.json file listing the plugin files.")), + ("check", _("Check the catalog.json file against the plugin files.")), ( "prepare", _("Download dependency wheels for this host and build the local catalog."), diff --git a/datalab/plugins/modules.py b/datalab/plugins/modules.py new file mode 100644 index 000000000..dbd88b8af --- /dev/null +++ b/datalab/plugins/modules.py @@ -0,0 +1,249 @@ +# Copyright (c) DataLab Platform Developers, BSD 3-Clause license, see LICENSE file. + +"""Inspect single-file plugins without importing or executing their code. + +The public catalog keeps a verbatim copy of this Qt-free module. Only literal +strings and references to previously declared string constants are read. +""" + +from __future__ import annotations + +import ast +import hashlib +import os.path as osp +from typing import Any + +from packaging.version import InvalidVersion, Version + +MAX_MODULE_BYTES = 4 * 1024 * 1024 +_FIELDS = { + "id": "plugin_id", + "name": "display_name", + "version": "version", + "description": "summary", +} + + +class ModuleInspectionError(ValueError): + """A source file is not an acceptable single-file plugin artifact.""" + + +def module_name(filename: str) -> str: + """Return the importable name of a ``datalab_*.py`` file.""" + name, extension = osp.splitext(filename) + if ( + "/" in filename + or "\\" in filename + or extension != ".py" + or not name.startswith("datalab_") + or len(name) <= len("datalab_") + or not name.isidentifier() + ): + raise ModuleInspectionError( + "A plugin module file name must look like datalab_.py" + ) + return name + + +def _string(node: ast.expr, constants: dict[str, str]) -> str | None: + if isinstance(node, ast.Constant) and isinstance(node.value, str): + return node.value + if isinstance(node, ast.Name): + return constants.get(node.id) + return None + + +def _assignment(node: ast.stmt) -> tuple[str, ast.expr] | None: + if ( + isinstance(node, ast.Assign) + and len(node.targets) == 1 + and isinstance(node.targets[0], ast.Name) + ): + return node.targets[0].id, node.value + if ( + isinstance(node, ast.AnnAssign) + and isinstance(node.target, ast.Name) + and node.value is not None + ): + return node.target.id, node.value + return None + + +def _metadata(tree: ast.Module) -> tuple[dict[str, str], list[str]]: + constants: dict[str, str] = {} + constructors: set[str] = set() + namespaces: set[str] = set() + records: list[dict[str, str]] = [] + for statement in tree.body: + if isinstance(statement, (ast.Import, ast.ImportFrom)): + for alias in statement.names: + bound = alias.asname or alias.name.split(".", maxsplit=1)[0] + constants.pop(bound, None) + constructors.discard(bound) + namespaces.discard(bound) + if isinstance(statement, ast.ImportFrom) and statement.module in ( + "datalab.plugins", + "datalab.plugins.base", + ): + constructors.update( + alias.asname or alias.name + for alias in statement.names + if alias.name == "PluginInfo" + ) + elif isinstance(statement, ast.Import): + namespaces.update( + alias.asname or alias.name + for alias in statement.names + if alias.name == "datalab.plugins" and alias.asname + ) + assignment = _assignment(statement) + if assignment is not None: + name, value = assignment + text = _string(value, constants) + constants.pop(name, None) + constructors.discard(name) + namespaces.discard(name) + if text is not None: + constants[name] = text + if not isinstance(statement, ast.ClassDef): + if assignment is None and not isinstance( + statement, (ast.Import, ast.ImportFrom) + ): + for node in ast.walk(statement): + if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Store): + constants.pop(node.id, None) + constructors.discard(node.id) + namespaces.discard(node.id) + continue + local = dict(constants) + local_constructors = set(constructors) + local_namespaces = set(namespaces) + for member in statement.body: + assignment = _assignment(member) + if assignment is None: + for node in ast.walk(member): + if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Store): + local.pop(node.id, None) + local_constructors.discard(node.id) + local_namespaces.discard(node.id) + continue + name, value = assignment + if name != "PLUGIN_INFO": + text = _string(value, local) + local.pop(name, None) + local_constructors.discard(name) + local_namespaces.discard(name) + if text is not None: + local[name] = text + continue + record: dict[str, str] = {} + if isinstance(value, ast.Call): + function = value.func + recognized = ( + isinstance(function, ast.Name) and function.id in local_constructors + ) or ( + isinstance(function, ast.Attribute) + and function.attr == "PluginInfo" + and isinstance(function.value, ast.Name) + and function.value.id in local_namespaces + ) + if recognized and not any( + keyword.arg is None for keyword in value.keywords + ): + for keyword in value.keywords: + if keyword.arg in _FIELDS: + text = _string(keyword.value, local) + if text is not None: + record[_FIELDS[keyword.arg]] = text + records.append(record) + warnings: list[str] = [] + if len(records) != 1: + warnings.append( + "Plugin metadata is not unique or not statically readable; " + "the catalog describes the module as a whole" + ) + return {}, warnings + record = records[0] + if "version" in record and record["version"]: + try: + Version(record["version"]) + except InvalidVersion: + warnings.append("The declared version is not comparable; using the digest") + record.pop("version") + if not all(record.get(field) for field in ("plugin_id", "display_name", "version")): + warnings.append( + "Some plugin metadata is unavailable without executing the file" + ) + return record, warnings + + +def inspect_module(path_or_bytes: str | bytes, *, filename: str) -> dict[str, Any]: + """Return file facts and safely readable plugin metadata. + + Valid Python is not necessarily a working or trustworthy plugin. Missing + metadata stays unknown; callers may supply it explicitly in a catalog. + """ + name = module_name(filename) + if isinstance(path_or_bytes, str): + with open(path_or_bytes, "rb") as stream: + data = stream.read(MAX_MODULE_BYTES + 1) + else: + data = path_or_bytes + if not data: + raise ModuleInspectionError("Plugin module is empty") + if len(data) > MAX_MODULE_BYTES: + raise ModuleInspectionError( + f"Plugin module exceeds the {MAX_MODULE_BYTES} byte size limit" + ) + try: + tree = ast.parse(data, filename=filename) + # Parsing alone accepts e.g. a return outside a function. + compile(tree, filename, "exec") + except (SyntaxError, ValueError, UnicodeError, RecursionError) as exc: + raise ModuleInspectionError(f"Invalid Python plugin {filename}: {exc}") from exc + metadata, warnings = _metadata(tree) + return { + "filename": filename, + "name": name, + "sha256": hashlib.sha256(data).hexdigest(), + "size_bytes": len(data), + "version": metadata.get("version", ""), + "plugin_id": metadata.get("plugin_id", ""), + "display_name": metadata.get("display_name", ""), + "summary": metadata.get("summary", ""), + "warnings": warnings, + } + + +def apply_catalog_metadata( + manifest: dict[str, Any], + *, + plugin_id: str, + name: str, + version: str, + summary: str = "", +) -> dict[str, Any]: + """Supplement unknown metadata, refusing a known identity/version conflict.""" + if version: + Version(version) + for field, expected in (("plugin_id", plugin_id), ("version", version)): + actual = manifest[field] + if ( + actual + and expected + and ( + Version(actual) != Version(expected) + if field == "version" + else actual != expected + ) + ): + raise ModuleInspectionError( + f"Catalog {field} {expected!r} differs from the file: {actual!r}" + ) + return { + **manifest, + "plugin_id": plugin_id, + "display_name": name, + "version": version or manifest["version"], + "summary": summary or manifest["summary"], + } diff --git a/datalab/plugins/store.py b/datalab/plugins/store.py index f5645f33c..9b8de9962 100644 --- a/datalab/plugins/store.py +++ b/datalab/plugins/store.py @@ -20,7 +20,6 @@ import dataclasses import datetime import gc -import hashlib import importlib import importlib.util import json @@ -48,6 +47,13 @@ plan_dependencies, validate_installed, ) +from datalab.plugins.modules import ( + MAX_MODULE_BYTES, + ModuleInspectionError, + apply_catalog_metadata, + inspect_module, + module_name, +) from datalab.plugins.wheels import ( DESKTOP_ENTRY_POINT_GROUP, MAX_WHEEL_BYTES, @@ -55,7 +61,6 @@ ) INDEX_SCHEMA_VERSION = 1 -MAX_MODULE_BYTES = 4 * 1024 * 1024 WHEEL_KIND = "wheel" MODULE_KIND = "module" @@ -78,12 +83,13 @@ class InstalledPlugin: Args: kind: ``"wheel"`` or ``"module"`` name: Distribution name (wheel) or module name (module) - version: Distribution version, empty for a module + version: Distribution version, or known module version (otherwise empty) filename: Name of the file the plugin was installed from sha256: SHA-256 digest of the installed file size_bytes: Size of the installed file packages: Top-level packages or modules provided by the plugin installed_at: Installation date (ISO 8601, UTC) + catalog_id: Catalog identity when installed from a catalog """ kind: str @@ -94,6 +100,7 @@ class InstalledPlugin: size_bytes: int packages: tuple[str, ...] installed_at: str + catalog_id: str = "" def __post_init__(self) -> None: object.__setattr__(self, "packages", tuple(self.packages)) @@ -101,8 +108,8 @@ def __post_init__(self) -> None: raise ValueError(f"Unknown installed plugin kind: {self.kind!r}") if not _SHA256_PATTERN.fullmatch(self.sha256): raise ValueError(f"Invalid SHA-256 digest: {self.sha256!r}") - if self.kind == MODULE_KIND and not _is_module_name(self.name): - raise ValueError(f"Invalid plugin module name: {self.name!r}") + if self.kind == MODULE_KIND: + module_name(f"{self.name}.py") if not all( isinstance(name, str) and name.isidentifier() for name in self.packages ): @@ -129,14 +136,6 @@ class InstallResult: restart_required: bool -def _is_module_name(name: str) -> bool: - return ( - name.startswith(_MODULE_PREFIX) - and len(name) > len(_MODULE_PREFIX) - and name.isidentifier() - ) - - def _now() -> str: return datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds") @@ -365,6 +364,7 @@ def install_wheel( *, expected_sha256: str | None = None, dependency_plan: DependencyPlan | None = None, + catalog_id: str = "", ) -> InstallResult: """Install a plugin wheel file. @@ -412,7 +412,16 @@ def install_wheel( size_bytes=manifest["size_bytes"], packages=manifest["top_level_packages"], installed_at=_now(), + catalog_id=catalog_id, ) + for other in self.get_plugins(): + if other.kind == MODULE_KIND and ( + set(plugin.packages).intersection(other.packages) + or (catalog_id and other.catalog_id == catalog_id) + ): + raise PluginInstallError( + "The plugin conflicts with an installed module" + ) installed_directory: str | None = None success = False try: @@ -495,15 +504,16 @@ def inspect_module_file(self, path: str) -> dict: PluginInstallError: Invalid file name or conflicting module name """ name, data = self._read_module(path) - return { - "filename": osp.basename(path), - "name": name, - "sha256": hashlib.sha256(data).hexdigest(), - "size_bytes": len(data), - } + return inspect_module(data, filename=f"{name}.py") def install_module( - self, path: str, *, expected_sha256: str | None = None + self, + path: str, + *, + expected_sha256: str | None = None, + catalog_id: str = "", + catalog_name: str = "", + catalog_version: str = "", ) -> InstallResult: """Install a single-file ``datalab_*.py`` plugin module. @@ -515,18 +525,45 @@ def install_module( PluginInstallError: Invalid file name or conflicting module name """ name, data = self._read_module(path) - sha256 = hashlib.sha256(data).hexdigest() + manifest = inspect_module(data, filename=f"{name}.py") + if catalog_id: + manifest = apply_catalog_metadata( + manifest, + plugin_id=catalog_id, + name=catalog_name, + version=catalog_version, + ) + sha256 = manifest["sha256"] self._check_expected_digest(sha256, expected_sha256) plugin = InstalledPlugin( kind=MODULE_KIND, name=name, - version="", + version=manifest["version"], filename=osp.basename(path), sha256=sha256, size_bytes=len(data), packages=(name,), installed_at=_now(), + catalog_id=catalog_id, ) + for other in self.get_plugins(): + if other.kind == MODULE_KIND and _normalized( + self.get_path(other) + ) == _normalized(self.get_path(plugin)): + if other.name != name: + raise PluginInstallError( + "Module file name conflicts with an installed module" + ) + if other.catalog_id and catalog_id and other.catalog_id != catalog_id: + raise PluginInstallError( + f"Module {name} belongs to another catalog plugin" + ) + elif other.kind == WHEEL_KIND and name in other.packages: + raise PluginInstallError("Module conflicts with an installed wheel") + elif catalog_id and other.catalog_id == catalog_id: + raise PluginInstallError( + "Catalog plugin uses another installed artifact" + ) replaced = self._add( plugin, data, @@ -616,11 +653,10 @@ def _check_expected_digest(sha256: str, expected_sha256: str | None) -> None: def _read_module(self, path: str) -> tuple[str, bytes]: """Return the name and content of a valid plugin module file.""" - name, extension = osp.splitext(osp.basename(path)) - if extension != ".py" or not _is_module_name(name): - raise PluginInstallError( - "A plugin module file name must look like datalab_.py" - ) + try: + name = module_name(osp.basename(path)) + except ModuleInspectionError as exc: + raise PluginInstallError(str(exc)) from exc with open(path, "rb") as file: data = file.read(MAX_MODULE_BYTES + 1) if len(data) > MAX_MODULE_BYTES: @@ -652,12 +688,25 @@ def _add( replaced = next((other for other in plugins if same_plugin(other)), None) if replaced is not None and replaced.sha256 == plugin.sha256 and not repair: raise PluginInstallError(f"{plugin.name} is already installed") - self._write_file(self.get_path(plugin), data) - self._write_index( - [other for other in plugins if other != replaced] + [plugin], - dependency_roots=dependency_roots, - pending_enabled=pending_enabled, - ) + path = self.get_path(plugin) + previous_data = None + if plugin.kind == MODULE_KIND and osp.isfile(path): + with open(path, "rb") as stream: + previous_data = stream.read(MAX_MODULE_BYTES + 1) + self._write_file(path, data) + try: + self._write_index( + [other for other in plugins if other != replaced] + [plugin], + dependency_roots=dependency_roots, + pending_enabled=pending_enabled, + ) + except (OSError, ValueError): + if plugin.kind == MODULE_KIND: + if previous_data is None: + os.remove(path) + else: + self._write_file(path, previous_data) + raise return replaced def _write_index( diff --git a/datalab/tests/backbone/plugins/plugin_module_catalog_unit_test.py b/datalab/tests/backbone/plugins/plugin_module_catalog_unit_test.py new file mode 100644 index 000000000..173b43337 --- /dev/null +++ b/datalab/tests/backbone/plugins/plugin_module_catalog_unit_test.py @@ -0,0 +1,351 @@ +# Copyright (c) DataLab Platform Developers, BSD 3-Clause license, see LICENSE file. + +"""Single-file plugins in mixed catalogs, with old installed indexes too.""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path + +import pytest +from qtpy import QtWidgets as QW + +from datalab.gui.plugins.available import ( + AvailablePluginItemWidget, + AvailablePluginsWidget, +) +from datalab.gui.plugins.install import InstalledPluginsWidget +from datalab.plugins import base as plugin_base +from datalab.plugins import catalog as catalog_client +from datalab.plugins.catalog import ( + CatalogError, + download_release, + read_folder_catalog, +) +from datalab.plugins.catalogtool import ( + build_local_catalog, + check_local_catalog, + prepare_local_catalog, +) +from datalab.plugins.generator import PluginProject, render_single_file_plugin +from datalab.plugins.store import InstalledPluginStore, PluginInstallError +from datalab.tests.backbone.plugins.plugin_local_catalog_unit_test import add_wheel +from datalab.tests.backbone.plugins.plugin_modules_unit_test import FILENAME, SOURCE + + +def write_module(folder: Path, source: bytes = SOURCE) -> Path: + path = folder / FILENAME + path.write_bytes(source) + return path + + +@pytest.fixture(name="store") +def fixture_store( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> InstalledPluginStore: + monkeypatch.setattr(sys, "path", list(sys.path)) + store = InstalledPluginStore(str(tmp_path / "installed")) + monkeypatch.setattr(plugin_base, "_INSTALLED_PLUGIN_STORE", store) + return store + + +def test_mixed_folder_and_declared_catalog_are_read_without_execution( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + marker = tmp_path / "side-effect" + path = write_module( + tmp_path, SOURCE + f"\nopen({str(marker)!r}, 'w').write('executed')\n".encode() + ) + add_wheel(tmp_path, "1.0") + plugins, warnings = read_folder_catalog(str(tmp_path)) + assert not warnings + module = next(plugin for plugin in plugins if plugin.kind == "module") + assert module.id == "org.example.module" + assert module.module == path.stem + release = module.get_installable_release() + assert release is not None and release.version == "1.2" + assert release.kind == "module" + assert not marker.exists() + report = build_local_catalog(str(tmp_path)) + assert report.written and not report.errors + assert not check_local_catalog(str(tmp_path)).errors + with (tmp_path / "catalog.json").open(encoding="utf-8") as stream: + payload = json.load(stream) + record = next( + plugin for plugin in payload["plugins"] if plugin.get("kind") == "module" + ) + record["name"] = "An administered name" + record["license"] = "BSD-3-Clause" + record["releases"][0]["yanked"] = "Needs review" + (tmp_path / "catalog.json").write_text(json.dumps(payload), encoding="utf-8") + rebuilt = build_local_catalog(str(tmp_path)) + assert not rebuilt.errors + record = next( + plugin + for plugin in rebuilt.catalog["plugins"] + if plugin.get("kind") == "module" + ) + assert record["name"] == "An administered name" + assert record["license"] == "BSD-3-Clause" + assert record["releases"][0]["yanked"] == "Needs review" + assert not marker.exists() + + # A mirror may be consumed with no URL opener available at all. + monkeypatch.setattr( + "urllib.request.urlopen", + lambda *_args, **_kwargs: pytest.fail("Offline catalog contacted the network"), + ) + offline, _ = read_folder_catalog(str(tmp_path)) + release = next(plugin for plugin in offline if plugin.kind == "module").releases[0] + target = tmp_path / "copy" + target.mkdir() + assert ( + Path(download_release(release, str(target))).read_bytes() == path.read_bytes() + ) + + +def test_generated_and_unversioned_modules_need_no_packaging( + tmp_path: Path, store: InstalledPluginStore, monkeypatch: pytest.MonkeyPatch +) -> None: + project = PluginProject( + destination=tmp_path, + name="Example", + package=Path(FILENAME).stem, + plugin_id="org.example.generated", + description="Generated example", + ) + path = write_module(tmp_path, render_single_file_plugin(project).encode("utf-8")) + report = prepare_local_catalog(str(tmp_path)) + assert not report.errors and report.written + (plugin,) = read_folder_catalog(str(tmp_path))[0] + assert plugin.releases[0].version == "0.1.0" + (tmp_path / "catalog.json").unlink() + path.write_text("VALUE = 1\n", encoding="utf-8") + report = build_local_catalog(str(tmp_path)) + assert not report.errors and report.written + (plugin,) = read_folder_catalog(str(tmp_path))[0] + assert plugin.releases[0].version == "" + first = store.install_module(str(path)).plugin + assert first.version == "" and first.catalog_id == "" + index_path = Path(store.index_path) + index = json.loads(index_path.read_text("utf-8")) + index["plugins"][0].pop("catalog_id") + index_path.write_text(json.dumps(index), encoding="utf-8") + assert store.get_plugins()[0] == first + _app = QW.QApplication.instance() or QW.QApplication([]) + installer = InstalledPluginsWidget(lambda: None) + browser = AvailablePluginsWidget(installer) + installed = browser._get_installed(plugin) + item = AvailablePluginItemWidget(plugin, installed) + assert item.action_button is None + path.write_text("VALUE = 2\n", encoding="utf-8") + (tmp_path / "catalog.json").unlink() + (changed,) = read_folder_catalog(str(tmp_path))[0] + item = AvailablePluginItemWidget(changed, browser._get_installed(changed)) + assert item.action_button is not None + assert item.action_button.text() == "Replace..." + consent: list[dict] = [] + monkeypatch.setattr( + installer, + "confirm_installation", + lambda manifest, origin=None: consent.append(manifest) or False, + ) + assert installer.install_path(str(path), catalog_plugin=changed) is None + assert store.get_plugins() == [first] + assert consent[0]["version"] == "" + monkeypatch.setattr(installer, "confirm_installation", lambda *a, **k: True) + monkeypatch.setattr(installer, "_ask_reload", lambda *_args: False) + result = installer.install_path(str(path), catalog_plugin=changed) + assert result is not None and not result.restart_required + assert result.plugin.catalog_id == changed.id + assert result.plugin.sha256 != first.sha256 + assert not result.plugin.version + installer.deleteLater() + browser.deleteLater() + item.deleteLater() + + +def test_versioned_module_updates_and_conflicting_id( + tmp_path: Path, store: InstalledPluginStore +) -> None: + path = write_module(tmp_path) + first = store.install_module(str(path)) + path.write_bytes(SOURCE.replace(b'version="1.2"', b'version="1.3"')) + (plugin,) = read_folder_catalog(str(tmp_path))[0] + _app = QW.QApplication.instance() or QW.QApplication([]) + item = AvailablePluginItemWidget( + plugin, ("file", first.plugin.version, first.plugin.sha256) + ) + assert item.action_button is not None + assert item.action_button.text() == "Update to 1.3" + result = store.install_module( + str(path), catalog_id=plugin.id, catalog_name=plugin.name, catalog_version="1.3" + ) + assert result.replaced == first.plugin + assert result.plugin.version == "1.3" + with pytest.raises(ValueError, match="plugin_id"): + store.install_module(str(path), catalog_id="org.example.other") + store.uninstall(result.plugin) + assert not store.get_plugins() + item.deleteLater() + + +def test_failed_index_write_restores_the_previous_module( + tmp_path: Path, store: InstalledPluginStore, monkeypatch: pytest.MonkeyPatch +) -> None: + path = write_module(tmp_path) + first = store.install_module(str(path)).plugin + original = Path(store.get_path(first)).read_bytes() + path.write_bytes(SOURCE + b"\nVALUE = 2\n") + + def fail(*_args, **_kwargs): + raise OSError("Index write failed") + + monkeypatch.setattr(store, "_write_index", fail) + with pytest.raises(OSError, match="Index write"): + store.install_module(str(path)) + assert Path(store.get_path(first)).read_bytes() == original + assert store.get_plugins() == [first] + + +def test_module_changed_after_consent_does_not_replace_the_installed_file( + tmp_path: Path, store: InstalledPluginStore +) -> None: + path = write_module(tmp_path) + first = store.install_module(str(path)).plugin + manifest = store.inspect_module_file(str(path)) + path.write_bytes(SOURCE + b"\nVALUE = 2\n") + with pytest.raises(PluginInstallError, match="changed since it was inspected"): + store.install_module(str(path), expected_sha256=manifest["sha256"]) + assert store.get_plugins() == [first] + assert Path(store.get_path(first)).read_bytes() == SOURCE + + +def test_no_script_execution_and_explicit_metadata_for_dynamic_scripts( + tmp_path: Path, +) -> None: + path = write_module(tmp_path, b"raise RuntimeError('not executed')\n") + report = build_local_catalog(str(tmp_path)) + assert report.written and report.warnings + record = report.catalog["plugins"][0] + record.update(id="org.example.dynamic", name="Dynamic plugin", license="MIT") + record["releases"][0]["version"] = "2.0" + (tmp_path / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + rebuilt = build_local_catalog(str(tmp_path)) + assert rebuilt.written and not rebuilt.errors + assert rebuilt.catalog["plugins"][0]["id"] == "org.example.dynamic" + assert rebuilt.catalog["plugins"][0]["releases"][0]["version"] == "2.0" + assert not check_local_catalog(str(tmp_path)).errors + path.write_text("invalid = (", encoding="utf-8") + failed = build_local_catalog(str(tmp_path)) + assert failed.errors and not failed.written + + +def test_modules_keep_withdrawal_and_source_priority_rules(tmp_path: Path) -> None: + write_module(tmp_path) + report = build_local_catalog(str(tmp_path)) + record = report.catalog["plugins"][0] + record.update(status="revoked", status_reason="Withdrawn") + (tmp_path / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + report = build_local_catalog(str(tmp_path)) + (plugin,) = read_folder_catalog(str(tmp_path))[0] + assert plugin.status == "revoked" + assert plugin.get_installable_release() is None + assert set(report.catalog["plugins"][0]["releases"][0]) == {"version", "sha256"} + + +def test_module_release_checks_compatibility_size_and_identity(tmp_path: Path) -> None: + path = write_module(tmp_path) + report = build_local_catalog(str(tmp_path)) + record = report.catalog["plugins"][0] + release = record["releases"][0] + release["requires_python"] = ">=99" + (tmp_path / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + (plugin,) = read_folder_catalog(str(tmp_path))[0] + assert plugin.get_installable_release() is None + assert plugin.get_installable_release(python_version="") is not None + release["dependency_wheels"] = [ + { + "filename": "extra-1.0-py3-none-any.whl", + "url": "extra.whl", + "size": 1, + "sha256": "a" * 64, + } + ] + (tmp_path / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + with pytest.raises(CatalogError, match="Invalid plugin catalog"): + read_folder_catalog(str(tmp_path)) + release.pop("dependency_wheels") + release["requires_python"] = ">=3.9" + record["id"] = "org.example.wrong" + (tmp_path / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + assert check_local_catalog(str(tmp_path)).errors + assert path.exists() + + +@pytest.mark.skipif(sys.platform != "win32", reason="Windows file names ignore case") +def test_case_alias_cannot_overwrite_another_module( + tmp_path: Path, store: InstalledPluginStore +) -> None: + path = tmp_path / "datalab_Case.py" + path.write_text("VALUE = 1\n", encoding="utf-8") + first = store.install_module(str(path)).plugin + replacement = tmp_path / "datalab_case.py" + replacement.write_text("VALUE = 2\n", encoding="utf-8") + with pytest.raises(PluginInstallError, match="file name conflicts"): + store.install_module(str(replacement)) + assert Path(store.get_path(first)).read_text("utf-8") == "VALUE = 1\n" + assert store.get_plugins() == [first] + + +def test_module_prepare_never_invokes_pip( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + write_module(tmp_path) + monkeypatch.setattr( + "datalab.plugins.dependencies._pip", + lambda *_args, **_kwargs: pytest.fail("Modules must not invoke pip"), + ) + report = prepare_local_catalog(str(tmp_path)) + assert report.written and not report.errors + + +def test_module_download_refuses_changed_bytes_and_excessive_size( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + path = write_module(tmp_path) + (plugin,) = read_folder_catalog(str(tmp_path))[0] + release = plugin.get_installable_release() + assert release is not None + target = tmp_path / "download" + target.mkdir() + path.write_bytes(SOURCE.replace(b"An example", b"An altered")) + with pytest.raises(CatalogError): + download_release(release, str(target)) + assert not list(target.iterdir()) + path.write_bytes(SOURCE) + monkeypatch.setattr(catalog_client, "MAX_MODULE_BYTES", len(SOURCE) - 1) + with pytest.raises(CatalogError, match="size"): + download_release(release, str(target)) + assert not list(target.iterdir()) + + +def test_module_sources_keep_first_source_and_report_masked_revocation( + tmp_path: Path, +) -> None: + first, second = tmp_path / "first", tmp_path / "second" + first.mkdir() + second.mkdir() + write_module(first) + write_module(second, SOURCE.replace(b'version="1.2"', b'version="2.0"')) + content = catalog_client.load_catalogs("", [str(first), str(second)]) + assert not content.errors + assert len(content.plugins) == 1 + assert content.plugins[0].releases[0].version == "1.2" + report = build_local_catalog(str(second)) + report.catalog["plugins"][0].update(status="revoked", status_reason="Withdrawn") + (second / "catalog.json").write_text(json.dumps(report.catalog), encoding="utf-8") + content = catalog_client.load_catalogs("", [str(first), str(second)]) + assert len(content.plugins) == 1 + assert len(content.errors) == 1 and "Withdrawn" in content.errors[0] diff --git a/datalab/tests/backbone/plugins/plugin_modules_unit_test.py b/datalab/tests/backbone/plugins/plugin_modules_unit_test.py new file mode 100644 index 000000000..cda6236d5 --- /dev/null +++ b/datalab/tests/backbone/plugins/plugin_modules_unit_test.py @@ -0,0 +1,168 @@ +# Copyright (c) DataLab Platform Developers, BSD 3-Clause license, see LICENSE file. + +"""Static metadata inspection never imports plugin code.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from datalab.plugins import modules +from datalab.plugins.modules import ( + ModuleInspectionError, + apply_catalog_metadata, + inspect_module, +) + +SOURCE = b"""from datalab.plugins import PluginBase, PluginInfo +PLUGIN_ID = "org.example.module" +PLUGIN_NAME = "An example" +class Example(PluginBase): + PLUGIN_INFO = PluginInfo(id=PLUGIN_ID, name=PLUGIN_NAME, version="1.2") + def create_actions(self): + pass +""" +FILENAME = "datalab_module_probe.py" + + +def test_literal_metadata_and_no_execution(tmp_path: Path) -> None: + marker = tmp_path / "must-not-exist" + source = SOURCE + f"\nopen({str(marker)!r}, 'w').write('executed')\n".encode() + manifest = inspect_module(source, filename=FILENAME) + assert manifest["plugin_id"] == "org.example.module" + assert manifest["display_name"] == "An example" + assert manifest["version"] == "1.2" + assert not manifest["warnings"] + assert not marker.exists() + + +@pytest.mark.parametrize( + "source", + [ + b"raise RuntimeError('must not run')", + SOURCE.replace(b'version="1.2"', b"version=dangerous()"), + SOURCE.replace(b'PLUGIN_NAME = "An example"', b"PLUGIN_NAME = dangerous()"), + SOURCE + SOURCE.replace(b"class Example", b"class Other"), + SOURCE.replace(b"PluginInfo(id=PLUGIN_ID", b"factory(id=PLUGIN_ID"), + ], +) +def test_unknown_metadata_is_not_evaluated(source: bytes) -> None: + manifest = inspect_module(source, filename=FILENAME) + assert manifest["warnings"] + explicit = apply_catalog_metadata( + manifest, plugin_id="org.example.module", name="Example", version="1.2" + ) + assert explicit["version"] == "1.2" + + +def test_generated_constants_and_class_constants() -> None: + manifest = inspect_module( + SOURCE.replace( + b"class Example(PluginBase):", + b'class Example(PluginBase):\n VERSION = "2.0"', + ).replace(b'version="1.2"', b"version=VERSION"), + filename=FILENAME, + ) + assert manifest["version"] == "2.0" + + +def test_encoding_and_invalid_source(monkeypatch: pytest.MonkeyPatch) -> None: + source = b"# coding: latin-1\n" + SOURCE.replace(b"An example", b"Caf\xe9") + assert inspect_module(source, filename=FILENAME)["display_name"] == "Caf\u00e9" + for data in ( + b"class Broken(", + b"\x00", + b"# coding: does-not-exist\n", + b"return 1", + b"from __future__ import made_up_feature", + ): + with pytest.raises(ModuleInspectionError, match="Invalid Python"): + inspect_module(data, filename=FILENAME) + monkeypatch.setattr(modules, "MAX_MODULE_BYTES", 1) + with pytest.raises(ModuleInspectionError, match="size limit"): + inspect_module(SOURCE, filename=FILENAME) + + +def test_empty_module_cannot_produce_an_undownloadable_release() -> None: + with pytest.raises(ModuleInspectionError, match="empty"): + inspect_module(b"", filename=FILENAME) + + +def test_explicit_version_must_be_valid_even_when_source_metadata_is_unknown() -> None: + manifest = inspect_module(b"VALUE = 1\n", filename=FILENAME) + with pytest.raises(ValueError, match="Invalid version"): + apply_catalog_metadata( + manifest, + plugin_id="org.example.module", + name="Example", + version="not a version", + ) + + +@pytest.mark.parametrize( + "name", + [ + "other.py", + "datalab_.py", + "datalab_bad-name.py", + "../datalab_x.py", + r"..\datalab_x.py", + "datalab_x.py.exe", + ], +) +def test_invalid_names(name: str) -> None: + with pytest.raises(ModuleInspectionError, match="datalab_"): + inspect_module(SOURCE, filename=name) + + +def test_explicit_metadata_cannot_hide_known_conflicts() -> None: + manifest = inspect_module(SOURCE, filename=FILENAME) + with pytest.raises(ModuleInspectionError, match="plugin_id"): + apply_catalog_metadata( + manifest, plugin_id="org.example.other", name="Example", version="1.2" + ) + with pytest.raises(ModuleInspectionError, match="version"): + apply_catalog_metadata( + manifest, plugin_id="org.example.module", name="Example", version="2.0" + ) + + +@pytest.mark.parametrize( + "before_class", + [ + b"PLUGIN_NAME = dangerous()\n", + b"if condition:\n PLUGIN_NAME = dangerous()\n", + b"from other import PLUGIN_NAME\n", + ], +) +def test_reassigned_constants_are_not_stale_metadata(before_class: bytes) -> None: + source = SOURCE.replace(b"class Example", before_class + b"class Example") + manifest = inspect_module(source, filename=FILENAME) + assert not manifest["display_name"] + assert manifest["warnings"] + + +def test_shadowed_constructor_and_expansion_are_not_metadata() -> None: + for source in ( + SOURCE.replace(b"class Example", b"PluginInfo = dangerous\nclass Example"), + SOURCE.replace( + b"class Example(PluginBase):", + b"class Example(PluginBase):\n PluginInfo = dangerous", + ), + SOURCE.replace(b"PluginInfo(id=", b"PluginInfo(**dynamic, id="), + ): + manifest = inspect_module(source, filename=FILENAME) + assert not manifest["plugin_id"] + assert manifest["warnings"] + + +def test_conditional_class_metadata_is_not_guessed() -> None: + source = SOURCE.replace( + b"class Example(PluginBase):", + b"class Example(PluginBase):\n" + b' VERSION = "1.0"\n' + b" if condition:\n" + b" VERSION = dangerous()", + ).replace(b'version="1.2"', b"version=VERSION") + assert not inspect_module(source, filename=FILENAME)["version"] diff --git a/scripts/frozen_plugin_probe.py b/scripts/frozen_plugin_probe.py index f2e675224..ab5e8e271 100644 --- a/scripts/frozen_plugin_probe.py +++ b/scripts/frozen_plugin_probe.py @@ -28,9 +28,40 @@ import sys import tempfile import zipfile +from pathlib import Path +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from datalab.plugins.store import InstalledPluginStore MARKER_ENV = "DATALAB_PLUGIN_PROBE_MARKER" WHEELHOUSE_ENV = "DATALAB_PLUGIN_PROBE_WHEELHOUSE" +MODULE_FILENAME = "datalab_frozen_module_probe.py" +GENERATED_MODULE = "datalab_frozen_generated_probe" +MODULE_SOURCE = f'''\ +import json +import os +import sys + +import numpy as np +from datalab.plugins import PluginBase, PluginInfo + + +class SingleFileProbe(PluginBase): + # Intentionally no ID or version: this is a historical, unchanged module. + PLUGIN_INFO = PluginInfo(name="Frozen single-file probe") + + def create_actions(self): + generated = sys.modules["{GENERATED_MODULE}"] + result = {{ + "module": __file__, + "generated": generated.__file__, + "sum": int(np.arange(4).sum()), + }} + marker = os.environ["{MARKER_ENV}"] + ".module" + with open(marker, "w", encoding="utf-8") as stream: + json.dump(result, stream) +''' #: Modules commonly imported by plugins MODULES = ( @@ -186,6 +217,9 @@ def install() -> int: from datalab.plugins.base import get_installed_plugin_store filename, data = make_probe_wheel() + module_marker = os.environ[MARKER_ENV] + ".module" + if os.path.isfile(module_marker): + os.remove(module_marker) with tempfile.TemporaryDirectory() as directory: path = os.path.join(directory, filename) with open(path, "wb") as file: @@ -201,10 +235,56 @@ def install() -> int: finally: if plan is not None: plan.close() + install_modules(directory, store) print(f"Probe plugin installed in {store.root}") return 0 +def install_modules(directory: str, store: InstalledPluginStore) -> None: + """Install historical and generated scripts from a local catalog.""" + from datalab.plugins.catalog import download_release, read_folder_catalog + from datalab.plugins.catalogtool import build_local_catalog + from datalab.plugins.generator import PluginProject, render_single_file_plugin + + folder = Path(directory) / "module-catalog" + folder.mkdir() + (folder / MODULE_FILENAME).write_text(MODULE_SOURCE, encoding="utf-8") + project = PluginProject( + destination=folder, + name="Frozen generated probe", + package=GENERATED_MODULE, + plugin_id="org.datalab.frozen-generated-probe", + description="Generated single-file build qualification", + ) + (folder / f"{GENERATED_MODULE}.py").write_text( + render_single_file_plugin(project), encoding="utf-8" + ) + report = build_local_catalog(str(folder)) + if report.errors: + raise ValueError("\n".join(report.errors)) + plugins, warnings = read_folder_catalog(str(folder)) + if warnings or len(plugins) != 2: + raise ValueError(f"Invalid single-file probe catalog: {warnings}") + target = Path(directory) / "download" + target.mkdir() + for plugin in plugins: + release = plugin.get_installable_release() + if release is None: + raise ValueError(f"No probe release for {plugin.name}") + path = download_release(release, str(target)) + result = store.install_module( + path, + expected_sha256=release.sha256, + catalog_id=plugin.id, + catalog_name=plugin.name, + catalog_version=release.version, + ) + if result.restart_required or result.plugin.version != release.version: + raise ValueError("Incorrect module installation state") + if os.path.exists(os.environ[MARKER_ENV] + ".module"): + raise ValueError("The module ran during catalog inspection or installation") + + def prepare() -> int: """Prepare the optional native dependency wheelhouse before the offline check.""" from datalab.plugins.catalogtool import ( @@ -242,6 +322,17 @@ def verify() -> int: ) if result.get("capsule") != "ok": failures.append(f"capsule round trip: {result.get('capsule')}") + module_marker = marker + ".module" + if not os.path.isfile(module_marker): + failures.append("The single-file plugin probes were not loaded") + else: + with open(module_marker, encoding="utf-8") as stream: + module = json.load(stream) + if module["sum"] != 6 or not module["module"].endswith(MODULE_FILENAME): + failures.append(f"Invalid single-file module result: {module}") + if not module["generated"].endswith(f"{GENERATED_MODULE}.py"): + failures.append(f"Generated module did not load: {module}") + result["single_file"] = module print(json.dumps(result, indent=2)) for failure in failures: print(failure, file=sys.stderr) From 067e0e931f0e02a3d7a75ed53db8052f0760e9de Mon Sep 17 00:00:00 2001 From: Pierre Raybaut <1311787+PierreRaybaut@users.noreply.github.com> Date: Thu, 22 Oct 2026 22:55:14 +0200 Subject: [PATCH 3/3] Document mixed plugin catalogs and module revisions in English and French Complete the source-form matrix and self-contained offline workflows after implementation, including catalog metadata, source priorities, updates, explicit replacement and module dependency limits. Refs #419. Assisted-by: GPT-6.1 Sol --- doc/features/advanced/plugins.rst | 91 ++++++++++--- .../LC_MESSAGES/features/advanced/plugins.po | 121 +++++++++++++----- .../LC_MESSAGES/release_notes/release_1.04.po | 15 ++- doc/release_notes/release_1.04.md | 5 +- 4 files changed, 180 insertions(+), 52 deletions(-) diff --git a/doc/features/advanced/plugins.rst b/doc/features/advanced/plugins.rst index 5e0a2746d..349d745ce 100644 --- a/doc/features/advanced/plugins.rst +++ b/doc/features/advanced/plugins.rst @@ -76,7 +76,23 @@ Source organization and distribution - **Package or project:** Python modules and, optionally, resources, tests and packaging metadata. ``datalab-plugin create --project`` offers one layered organization, not a mandatory architecture for all plugins. - **Wheel distribution:** an installable ``.whl`` archive with standardized metadata and plugin entry points. A wheel can contain a single module or a larger package; it is a distribution format, not a functional plugin category. -At present, a single-file plugin can be loaded from a plugin directory or installed with **Install from file...**, but online and local catalogs accept only wheels. Publishing an existing single-module implementation as a wheel does not require reorganizing it into the generated layered project, although packaging metadata must be supplied. +Both single-file modules and wheels can be installed with **Install from file...** or distributed through online and local catalogs. The catalogs do not require the layered project generated by ``--project``. Packaging a single module as a wheel remains possible, but is optional for Desktop. + +.. list-table:: Development forms and distribution + :header-rows: 1 + :widths: 28 36 36 + + * - Development form + - Direct source loading + - Managed distribution + * - Single Python file + - ``datalab_.py`` in a plugin directory + - Raw ``.py`` file, or an optional wheel + * - Python package or project + - Package folder, including a project's ``src`` layout + - Wheel containing the package and its resources + +Both managed formats work in local and online Desktop catalogs. The historical single-file form is not a separate functional plugin category, and its source does not need to be changed merely to list it in a catalog. DataLab-Web support is not implied by a Desktop catalog entry. A directly loaded source file remains in its development directory. A file installed through DataLab is copied to its managed user storage. A plugin is not a macro: a plugin registers persistent application features, whereas a macro is a script run to automate a task. Supporting the historical single-file form does not mean that obsolete APIs from DataLab v0.20 are automatically supported. @@ -215,7 +231,7 @@ Available plugins This tab lists the plugins of the `DataLab plugin catalog `_ and of the local plugin catalogs (see :ref:`plugin_local_catalogs`), with their description, license and source code link. Plugins maintained by the DataLab team are marked **Official**; the others are **Community** plugins, maintained by their authors, or **Local catalog** plugins, provided by your organization (the tooltip of such a plugin shows the folder of its catalog). A search field filters the list. -**Install** downloads the newest release available for DataLab desktop and for the running Python version, checks that its size and SHA-256 digest match the catalog, then installs it as described in :ref:`plugin_install_from_file`, after the same confirmation. When the catalog offers a newer version of a plugin installed this way, the button becomes **Update to...**. Plugins installed with ``pip`` in the Python environment are shown as such and are not modified. +**Install** copies or downloads the wheel or single-file module available for Desktop and the running Python version, checks its size and SHA-256 digest, then installs it as described in :ref:`plugin_install_from_file`, after confirmation. Versioned releases may offer **Update to...**. For a module without a comparable version, changed content offers **Replace...** instead; an identical digest is already installed. Plugins installed with ``pip`` or supplied by unmanaged source directories are identified but are not modified. The catalogs are read only when the tab is shown or refreshed, and again when the local catalog folders change. A catalog that cannot be read, for example a network share that is not connected, is reported below the plugin count without hiding the plugins of the other catalogs. The address of the DataLab plugin catalog is the ``plugins_catalog_url`` option of the DataLab configuration file: it may point to a private copy of the catalog, or be emptied to disable it, for example on computers without Internet access. For a catalog stored in a folder, including a folder shared on the network, use a local plugin catalog instead. @@ -238,22 +254,65 @@ Local catalogs are declared: A local catalog folder holds either: -- **plugin wheels only**: DataLab lists every pure-Python wheel of the folder (subfolders are not scanned) declaring a ``datalab.plugins`` entry point, and groups the versions of each plugin. Other files are ignored and reported. The plugin name is its distribution name; its description, license, keywords and source code link come from the wheel metadata. Copying a new version to the folder is enough to offer it as an update; +- **plugin files without an index**: DataLab scans pure-Python wheels declaring a ``datalab.plugins`` entry point and source files named ``datalab_.py`` at the folder root, without scanning subfolders. Wheels are grouped by distribution and version; each module is one installable file, even if it declares several plugin classes. Module metadata is read statically where possible; unknown names use the module name, and unknown versions remain empty and are tracked by digest. Invalid artifacts are reported. Copying a changed module or a new wheel version is enough to offer it; - **or a** ``catalog.json`` **file**, in the format of the DataLab plugin catalog, whose release URLs are relative to the folder: only the plugins it lists are shown. It may be a copy of the DataLab plugin catalog with its wheels, or a catalog built from the wheels of the folder with ``datalab-plugin catalog build``, then edited to give a display name to the plugins, deprecate or revoke them (``status`` and ``status_reason`` fields), or withdraw a release (``yanked`` field). -Whatever the catalog, DataLab checks the size and SHA-256 digest of the wheel before installing it, then inspects it without running it and asks for confirmation; the confirmation dialog shows the catalog folder as the source of the plugin. Being listed in a local catalog does not mean that a plugin was audited: the organization managing the folder is responsible for its content, and write access to the folder should be restricted accordingly. +Whatever the catalog, DataLab checks the size and SHA-256 digest of the artifact before installing it, inspects it without running it and asks for confirmation; the dialog shows the catalog folder as the source. A module is checked for a valid importable file name, Python encoding and syntax, with a 4 MiB size limit. Inspection does not import its code or certify its behavior. The organization managing a shared catalog is responsible for its content and should restrict write access. The ``datalab-plugin`` command maintains the ``catalog.json`` file of a folder: .. code-block:: bat - :: Write catalog.json, listing the plugin wheels of the folder + :: Write catalog.json, listing the plugin files of the folder datalab-plugin catalog build \\fileserver\datalab\plugins - :: Check catalog.json against the plugin wheels of the folder + :: Check catalog.json against the plugin files of the folder datalab-plugin catalog check \\fileserver\datalab\plugins -``build`` checks the plugin wheels (pure Python, plugin entry point, requirements available from the host or the catalog wheelhouse, supported Python version) and does not write the file if a wheel is refused. It keeps the fields edited in an existing ``catalog.json`` file (plugin ID, name, tier, status and its reason, repository, documentation, capabilities and keywords, and the reasons why releases are withdrawn), and revoked plugins remain listed. The file is replaced atomically, so that users may read the catalog meanwhile. ``check`` reports wheels that are missing, changed, refused or not listed, including the dependency artifacts. Run ``build`` again after adding wheels. Both commands exit with a non-zero status when they find a problem. +``build`` checks wheels and source modules without executing them and does not write the index if an artifact is refused. Wheels retain their pure-Python, entry-point and dependency checks; modules follow the rules below. Curated metadata, withdrawal reasons and revoked entries are preserved. The JSON file is replaced atomically. ``check`` reports missing, changed, refused or unlisted plugin files and dependency artifacts. Run ``build`` again after changing the root files. Both commands exit with a non-zero status when they find a problem. + +.. _plugin_module_catalog: + +Single-file modules in a catalog +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For a simple offline catalog, put the unchanged ``datalab_.py`` file at its root and add that folder under **Local plugin catalogs**. No ``pyproject.toml``, wheel or Internet access is needed. To generate an index that an administrator can curate, run ``datalab-plugin catalog build ``, then ``check``. Mixed folders may contain both modules and wheels: + +.. code-block:: text + + approved_plugins\ + datalab_spectroscopy.py + datalab_other_plugin-1.0-py3-none-any.whl + catalog.json + dependencies\ + additional_wheel_dependency-1.0-py3-none-any.whl + +The generated JSON uses ``kind: "module"`` and the real import name in ``module``; it does not give the script a fictitious Python distribution. Its release contains ``filename``, a relative ``url``, ``size``, ``sha256``, ``targets: ["desktop"]`` and ``version`` (possibly an empty string). ``kind`` is omitted or ``"wheel"`` for existing wheel entries. Add an optional ``requires_python`` constraint to a module release when its compatibility is known. + +An administrator may edit the display name, summary, license and other descriptive fields in the JSON, and supply a catalog identity and version when they are not statically available in the source. ``build`` preserves these fields. A known source identity or version must agree with the catalog. For dynamic metadata or several plugin classes, the catalog describes the complete module without executing it. Its catalog identity does not rewrite the runtime ``PluginInfo`` or require editing a historical file. + +If a module has no usable version, leave ``version`` empty: DataLab compares SHA-256 fingerprints instead. The confirmation and installed-files list distinguish a known version from a revision. A changed unversioned file, or different contents with the same declared version, requires explicit **Replace...** consent; it is not labelled as a newer version. Existing modules installed manually through DataLab are recognized too. Files in development directories are not overwritten; resolve a name conflict or remove the unmanaged copy before using the managed installation. + +Single-file modules use packages already available to DataLab. Their ``import`` statements are not interpreted as package requirements, no dependencies are downloaded, and a ``dependencies`` subfolder is not installed for them. ``prepare`` includes modules in the catalog but performs dependency downloads only for wheels. Imports that cannot be checked statically may still fail when the plugin is loaded, and are then shown in **Enable/disable plugins**. + +For the public catalog, publish the unchanged file as a named GitHub release asset and provide its version, license, description and SHA-256 in the YAML entry. The source is not required to expose those values statically. An entry has this structure (replace the example repository, file and fingerprint with the actual release information): + +.. code-block:: yaml + + id: io.github.alice.spectroscopy + name: Spectroscopy tools + kind: module + summary: A single-file Desktop plugin + license: BSD-3-Clause + repository: https://github.com/alice/spectroscopy + capabilities: [processing] + releases: + - version: "1.0.0" + filename: datalab_spectroscopy.py + sha256: + requires_python: ">=3.9" + +The public catalog verifies and mirrors the script under ``modules//...``. To use that published catalog offline, copy its JSON and complete relative tree, including both ``modules`` and ``wheels`` when present. Do not rebuild an unchanged mirror whose artifacts are in these subdirectories: the local authoring commands scan the root. Source URLs, revision checks, withdrawals and local-catalog priority follow the same rules for both formats. Module entries target Desktop only, not DataLab-Web or PyPI. .. _plugin_dependencies: @@ -296,7 +355,7 @@ On Windows, native library paths must also fit the DLL loader's path limit. Data Offline deployment, step by step ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -This procedure is self-contained: the disconnected computer does not need a package index, a Python development environment, or access to the public plugin website. A local network share may be used without Internet access; a USB drive or an ordinary local folder works as well. Preparation takes place on a connected computer, installation on the disconnected computer. +This procedure is self-contained: the disconnected computer does not need a package index, a Python development environment, or access to the public plugin website. A local network share, USB drive or ordinary local folder works. Preparing missing wheel dependencies requires a connected reference computer; a catalog containing only source modules can be assembled entirely offline (see :ref:`plugin_module_catalog`). .. important:: @@ -307,14 +366,14 @@ This procedure is self-contained: the disconnected computer does not need a pack The provider needs DataLab installed in a Python environment, with the ``datalab-plugin`` command available. The standalone application does not supply this authoring command; its users only need the prepared files. Use a reference environment matching the target's DataLab version, Python version, operating system, architecture and host package versions. **? > Installation and configuration...** identifies the target installation. A Windows 64-bit CPython 3.11 wheel is not interchangeable with a Linux, 32-bit or CPython 3.12 wheel. -Create a folder, place the plugin's ``.whl`` file at its root, then run: +Create a folder, place the plugin's ``.whl`` or ``datalab_*.py`` file at its root, then run: .. code-block:: powershell datalab-plugin catalog prepare D:\approved_plugins datalab-plugin catalog check D:\approved_plugins -``prepare`` needs Internet access: it resolves the missing requirements against the reference host, downloads their wheels, including transitive dependencies, into ``dependencies``, and writes ``catalog.json``. It does not install these packages in the reference host. No source distribution is compiled. If it reports a missing wheel or a conflict, fix the reference catalog before transferring it. +For wheels, ``prepare`` resolves missing requirements against the reference host, downloads their wheels and transitive dependencies into ``dependencies``, then writes ``catalog.json``. It does not install them in the reference host or compile source distributions. Modules are included without dependency downloads. Use ``build`` for a purely file-based catalog; if preparation reports a missing wheel or a conflict, fix the catalog before transferring it. Keep all files produced, including the dependency wheels. Do not copy just ``catalog.json`` or just the plugin wheel. If serving several target environments, prepare their wheels on matching reference environments, combine the needed files, then run ``datalab-plugin catalog build D:\approved_plugins`` followed by ``check``. A pure-Python wheel may also restrict the supported Python versions in its metadata. @@ -323,7 +382,7 @@ Keep all files produced, including the dependency wheels. Do not copy just ``cat 2. Transfer the folder, preserving its layout """"""""""""""""""""""""""""""""""""""""""" -Copy the complete folder to a USB drive, local disk or network share accessible from the target computer. Preserve relative paths and file names. A mirror of the public catalog may store files under ``wheels\\...`` instead of ``dependencies``: preserve those directories too, because ``catalog.json`` references them. The JSON file alone is not a mirror, and DataLab does not automatically export an online catalog for offline use. +Copy the complete folder to a USB drive, local disk or network share accessible from the target computer. Preserve relative paths and file names. A public mirror may store artifacts under ``wheels\\...`` and ``modules\\...``: preserve those directories too, because ``catalog.json`` references them. The JSON file alone is not a mirror, and DataLab does not automatically export an online catalog for offline use. For a strictly offline JSON catalog, the installation ``url`` of every release and dependency artifact must resolve to an existing local file. An absolute ``https://`` URL is still downloaded from its server even if the catalog itself is local; placing a wheel beside that JSON does not override its URL. Informational links such as ``repository``, ``documentation`` and ``source_url`` do not replace installation files. Catalogs generated by ``build`` use relative file URLs; published mirrors must retain the matching relative directory tree. @@ -367,9 +426,9 @@ Restart DataLab if dependencies were added or if replacing an already loaded whe 5. Maintain and troubleshoot the offline catalog """"""""""""""""""""""""""""""""""""""""""""""" -When ``catalog.json`` exists, it is authoritative: DataLab lists and copies only its declared releases and dependency artifacts. For a catalog managed from plugin wheels at its root, run ``build`` and ``check`` again on the preparation computer after adding or changing a plugin **or dependency** wheel, then redistribute the complete folder. Simply copying a new wheel beside an existing JSON file does not update that file. ``prepare`` may require Internet; ``build`` and ``check`` do not fetch missing packages. +When ``catalog.json`` exists, it is authoritative: DataLab lists and copies only its declared releases and dependency artifacts. For a catalog managed from files at its root, run ``build`` and ``check`` again after adding or changing a module, plugin wheel or dependency wheel, then redistribute the complete folder. Copying a new file beside an existing JSON does not update that index. ``prepare`` may require Internet for wheel dependencies; ``build`` and ``check`` never fetch missing packages. -Do not run ``build`` on an unchanged mirror whose plugin wheels live only under ``wheels\\...``: the command scans the catalog root, not that recursive tree, and would remove unrepresented entries from the generated catalog. Update such a mirror by transferring a complete, consistent published catalog and its files. To curate it with the local authoring commands instead, first arrange the plugin wheels at the root and the additional packages in ``dependencies``. +Do not run ``build`` on an unchanged mirror whose plugins live only under ``wheels\\...`` or ``modules\\...``: it scans the catalog root, not the recursive tree, and would remove unrepresented entries. Update a mirror by transferring a complete, consistent published catalog and its files. To manage it with the local authoring commands instead, first arrange plugin wheels and modules at the root and additional packages in ``dependencies``. Users refresh **Available plugins** to discover updates, then explicitly click **Update to...**. Updates are not installed automatically. Plugins installed with ``pip`` in the Python environment are identified but not updated by this dialog. Shared additional dependencies are retained when a plugin is uninstalled, and their existing versions remain protected on subsequent installations. @@ -394,9 +453,9 @@ This tab installs a plugin shared as a file, without any Python tool: this is th - a wheel (``.whl``) built from a plugin project, for example with ``python -m build``: it must be pure Python (``py3-none-any``), declare its plugin class in the ``datalab.plugins`` entry-point group, and have its requirements provided by the host or the adjacent dependency wheelhouse (see :ref:`plugin_dependencies`); - a single Python module named ``datalab_.py``. -Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows the module name, file, size and SHA-256 digest, without claiming to know its version, classes or dependencies. A plugin runs with the same rights as DataLab: install only plugins from authors you trust. +Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows its import name, file, size, SHA-256 and any statically readable or explicit catalog metadata. An unknown version stays unknown and is displayed as a revision; no class or dependency list is guessed. A plugin runs with the same rights as DataLab: install only plugins from authors you trust. -Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Installations not adding dependencies retain the plugin reload workflow; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded plugins are only activated at the next start of DataLab. +Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Modules can be reloaded after installation or replacement; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded wheels are only activated at the next start of DataLab. When reloading plugins, DataLab performs the following steps: @@ -796,7 +855,7 @@ Options adapt the generated file: An existing file is never overwritten. To share the plugin, send its file: other users install it with **Plugins > Configure plugins... > Install plugins** (see :ref:`plugin_install_from_file`). -The catalogs currently require a wheel, not a particular source architecture (see :ref:`plugin_forms`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`). +Both the raw single-file plugin and a wheel can be published in a Desktop catalog (see :ref:`plugin_module_catalog`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`). .. _plugin_project: diff --git a/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po b/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po index c8d9d75cd..bbdc39765 100644 --- a/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po +++ b/doc/locale/fr/LC_MESSAGES/features/advanced/plugins.po @@ -7,7 +7,7 @@ msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" -"POT-Creation-Date: 2026-10-22 14:49+0200\n" +"POT-Creation-Date: 2026-10-22 22:19+0200\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language: fr\n" @@ -72,9 +72,6 @@ msgstr "**Plugins d'entrée/sortie** Définissent de nouveaux formats de fichier msgid "**HDF5 plugins** Special plugins that support HDF5 files with domain-specific tree structures. These allow DataLab to interpret signals or images organized in non-standard ways." msgstr "**Plugins HDF5** Plugins spéciaux qui prennent en charge les fichiers HDF5 avec des structures d'arbre spécifiques au domaine. Ceux-ci permettent à DataLab d'interpréter des signaux ou des images organisés de manière non standard." -msgid "Live preview compatibility" -msgstr "Compatibilité avec l'aperçu interactif" - msgid "Source organization and distribution" msgstr "Organisation du code source et distribution" @@ -90,12 +87,48 @@ msgstr "**Paquet ou projet :** des modules Python et, éventuellement, des resso msgid "**Wheel distribution:** an installable ``.whl`` archive with standardized metadata and plugin entry points. A wheel can contain a single module or a larger package; it is a distribution format, not a functional plugin category." msgstr "**Distribution wheel :** une archive ``.whl`` installable, contenant des métadonnées normalisées et des points d'entrée de plugins. Un wheel peut contenir un seul module ou un paquet plus important ; c'est un format de distribution, pas une catégorie fonctionnelle de plugin." -msgid "At present, a single-file plugin can be loaded from a plugin directory or installed with **Install from file...**, but online and local catalogs accept only wheels. Publishing an existing single-module implementation as a wheel does not require reorganizing it into the generated layered project, although packaging metadata must be supplied." -msgstr "Actuellement, un plugin en fichier unique peut être chargé depuis un répertoire de plugins ou installé avec **Installer depuis un fichier...**, mais les catalogues en ligne et locaux n'acceptent que les wheels. Distribuer une implémentation existante en un seul module sous forme de wheel n'impose pas de la réorganiser selon le projet en couches généré, mais nécessite des métadonnées de packaging." +msgid "Both single-file modules and wheels can be installed with **Install from file...** or distributed through online and local catalogs. The catalogs do not require the layered project generated by ``--project``. Packaging a single module as a wheel remains possible, but is optional for Desktop." +msgstr "Les modules en fichier unique et les wheels peuvent être installés avec **Installer depuis un fichier...** ou distribués dans les catalogues en ligne et locaux. Les catalogues n'imposent pas le projet en couches généré par ``--project``. Empaqueter un module unique en wheel reste possible, mais est facultatif pour Desktop." + +msgid "Development forms and distribution" +msgstr "Formes de développement et distribution" + +msgid "Development form" +msgstr "Forme de développement" + +msgid "Direct source loading" +msgstr "Chargement direct du code source" + +msgid "Managed distribution" +msgstr "Distribution gérée" + +msgid "Single Python file" +msgstr "Fichier Python unique" + +msgid "``datalab_.py`` in a plugin directory" +msgstr "``datalab_.py`` dans un répertoire de plugins" + +msgid "Raw ``.py`` file, or an optional wheel" +msgstr "Fichier ``.py`` brut, ou wheel facultatif" + +msgid "Python package or project" +msgstr "Paquet ou projet Python" + +msgid "Package folder, including a project's ``src`` layout" +msgstr "Dossier du paquet, y compris l'organisation ``src`` d'un projet" + +msgid "Wheel containing the package and its resources" +msgstr "Wheel contenant le paquet et ses ressources" + +msgid "Both managed formats work in local and online Desktop catalogs. The historical single-file form is not a separate functional plugin category, and its source does not need to be changed merely to list it in a catalog. DataLab-Web support is not implied by a Desktop catalog entry." +msgstr "Les deux formats gérés fonctionnent dans les catalogues locaux et en ligne de Desktop. La forme historique en fichier unique n'est pas une catégorie fonctionnelle distincte et son code source n'a pas besoin d'être modifié pour être référencé dans un catalogue. Une entrée dans un catalogue Desktop n'implique pas la prise en charge de DataLab-Web." msgid "A directly loaded source file remains in its development directory. A file installed through DataLab is copied to its managed user storage. A plugin is not a macro: a plugin registers persistent application features, whereas a macro is a script run to automate a task. Supporting the historical single-file form does not mean that obsolete APIs from DataLab v0.20 are automatically supported." msgstr "Un fichier source chargé directement reste dans son répertoire de développement. Un fichier installé par DataLab est copié dans son stockage utilisateur géré. Un plugin n'est pas une macro : un plugin enregistre des fonctions persistantes dans l'application, tandis qu'une macro est un script exécuté pour automatiser une tâche. La prise en charge de la forme historique en fichier unique n'implique pas celle des API obsolètes de DataLab v0.20." +msgid "Live preview compatibility" +msgstr "Compatibilité avec l'aperçu interactif" + msgid "Standard parameterized functions registered with ``register_1_to_1`` can participate in live preview without a custom renderer. They must return a ``SignalObj`` or ``ImageObj``, be importable in a spawned process, and accept serializable source/parameter copies. Preview uses a private process and does not publish results to the workspace. It is not a sandbox for filesystem, network or other external side effects. Register functions that must not run speculatively with ``preview_enabled=False``. Serialization or computation failures are reported in the preview; the normal OK path remains available. Unknown custom ``DataSet.edit`` implementations and alternate guidata backends are not replaced." msgstr "Les fonctions paramétrées standard enregistrées avec ``register_1_to_1`` peuvent utiliser l'aperçu interactif sans moteur de rendu personnalisé. Elles doivent retourner un ``SignalObj`` ou un ``ImageObj``, être importables dans un processus démarré par spawn et accepter des copies sérialisables des sources et paramètres. L'aperçu utilise un processus privé et ne publie aucun résultat dans l'espace de travail. Il ne protège pas des effets externes sur les fichiers, le réseau ou d'autres ressources. Enregistrez les fonctions qui ne doivent pas être exécutées provisoirement avec ``preview_enabled=False``. Les erreurs de sérialisation ou de calcul sont signalées dans l'aperçu ; le chemin normal via OK reste disponible. Les implémentations personnalisées de ``DataSet.edit`` non prises en charge et les backends alternatifs de guidata ne sont pas remplacés." @@ -192,8 +225,8 @@ msgstr "Plugins disponibles" msgid "This tab lists the plugins of the `DataLab plugin catalog `_ and of the local plugin catalogs (see :ref:`plugin_local_catalogs`), with their description, license and source code link. Plugins maintained by the DataLab team are marked **Official**; the others are **Community** plugins, maintained by their authors, or **Local catalog** plugins, provided by your organization (the tooltip of such a plugin shows the folder of its catalog). A search field filters the list." msgstr "Cet onglet liste les plugins du `catalogue de plugins DataLab `_ et des catalogues locaux de plugins (voir :ref:`plugin_local_catalogs`), avec leur description, leur licence et un lien vers leur code source. Les plugins maintenus par l'équipe DataLab sont marqués **Officiel** ; les autres sont des plugins de la **Communauté**, maintenus par leurs auteurs, ou des plugins d'un **Catalogue local**, fournis par votre organisation (l'infobulle d'un tel plugin indique le dossier de son catalogue). Un champ de recherche filtre la liste." -msgid "**Install** downloads the newest release available for DataLab desktop and for the running Python version, checks that its size and SHA-256 digest match the catalog, then installs it as described in :ref:`plugin_install_from_file`, after the same confirmation. When the catalog offers a newer version of a plugin installed this way, the button becomes **Update to...**. Plugins installed with ``pip`` in the Python environment are shown as such and are not modified." -msgstr "**Installer** télécharge la version la plus récente disponible pour DataLab desktop et pour la version de Python utilisée, vérifie que sa taille et son empreinte SHA-256 correspondent au catalogue, puis l'installe comme décrit dans :ref:`plugin_install_from_file`, après la même confirmation. Lorsque le catalogue propose une version plus récente d'un plugin installé ainsi, le bouton devient **Mettre à jour vers...**. Les plugins installés avec ``pip`` dans l'environnement Python sont signalés comme tels et ne sont pas modifiés." +msgid "**Install** copies or downloads the wheel or single-file module available for Desktop and the running Python version, checks its size and SHA-256 digest, then installs it as described in :ref:`plugin_install_from_file`, after confirmation. Versioned releases may offer **Update to...**. For a module without a comparable version, changed content offers **Replace...** instead; an identical digest is already installed. Plugins installed with ``pip`` or supplied by unmanaged source directories are identified but are not modified." +msgstr "**Installer** copie ou télécharge le wheel ou le module en fichier unique disponible pour Desktop et la version de Python utilisée, vérifie sa taille et son empreinte SHA-256, puis l'installe comme décrit dans :ref:`plugin_install_from_file`, après confirmation. Les versions numérotées peuvent proposer **Mettre à jour vers...**. Pour un module sans version comparable, un contenu modifié propose plutôt **Remplacer...** ; une empreinte identique indique qu'il est déjà installé. Les plugins installés avec ``pip`` ou fournis par des répertoires sources non gérés sont identifiés, mais ne sont pas modifiés." msgid "The catalogs are read only when the tab is shown or refreshed, and again when the local catalog folders change. A catalog that cannot be read, for example a network share that is not connected, is reported below the plugin count without hiding the plugins of the other catalogs. The address of the DataLab plugin catalog is the ``plugins_catalog_url`` option of the DataLab configuration file: it may point to a private copy of the catalog, or be emptied to disable it, for example on computers without Internet access. For a catalog stored in a folder, including a folder shared on the network, use a local plugin catalog instead." msgstr "Les catalogues ne sont lus que lorsque l'onglet est affiché ou actualisé, ainsi que lorsque les dossiers des catalogues locaux changent. Un catalogue illisible, par exemple un partage réseau non connecté, est signalé sous le nombre de plugins sans masquer les plugins des autres catalogues. L'adresse du catalogue de plugins DataLab est l'option ``plugins_catalog_url`` du fichier de configuration de DataLab : elle peut désigner une copie privée du catalogue, ou être vidée pour le désactiver, par exemple sur des ordinateurs sans accès à Internet. Pour un catalogue stocké dans un dossier, y compris un dossier partagé sur le réseau, utilisez plutôt un catalogue local de plugins." @@ -216,20 +249,44 @@ msgstr "par la variable d'environnement ``DATALAB_PLUGINS_CATALOG``, par exemple msgid "A local catalog folder holds either:" msgstr "Un dossier de catalogue local contient :" -msgid "**plugin wheels only**: DataLab lists every pure-Python wheel of the folder (subfolders are not scanned) declaring a ``datalab.plugins`` entry point, and groups the versions of each plugin. Other files are ignored and reported. The plugin name is its distribution name; its description, license, keywords and source code link come from the wheel metadata. Copying a new version to the folder is enough to offer it as an update;" -msgstr "**soit uniquement des wheels de plugins** : DataLab liste chaque wheel en Python pur du dossier (les sous-dossiers ne sont pas parcourus) déclarant un point d'entrée ``datalab.plugins``, et regroupe les versions de chaque plugin. Les autres fichiers sont ignorés et signalés. Le nom du plugin est son nom de distribution ; sa description, sa licence, ses mots-clés et le lien vers son code source proviennent des métadonnées de la wheel. Copier une nouvelle version dans le dossier suffit pour la proposer en mise à jour ;" +msgid "**plugin files without an index**: DataLab scans pure-Python wheels declaring a ``datalab.plugins`` entry point and source files named ``datalab_.py`` at the folder root, without scanning subfolders. Wheels are grouped by distribution and version; each module is one installable file, even if it declares several plugin classes. Module metadata is read statically where possible; unknown names use the module name, and unknown versions remain empty and are tracked by digest. Invalid artifacts are reported. Copying a changed module or a new wheel version is enough to offer it;" +msgstr "**des fichiers de plugins sans index** : DataLab recherche les wheels en Python pur déclarant un point d'entrée ``datalab.plugins`` et les fichiers source nommés ``datalab_.py`` à la racine du dossier, sans parcourir les sous-dossiers. Les wheels sont regroupés par distribution et version ; chaque module constitue un fichier installable, même s'il déclare plusieurs classes de plugins. Les métadonnées des modules sont lues statiquement lorsque c'est possible ; un nom inconnu est remplacé par le nom du module et une version inconnue reste vide, avec un suivi par empreinte. Les fichiers invalides sont signalés. Copier un module modifié ou une nouvelle version d'un wheel suffit pour le proposer ;" msgid "**or a** ``catalog.json`` **file**, in the format of the DataLab plugin catalog, whose release URLs are relative to the folder: only the plugins it lists are shown. It may be a copy of the DataLab plugin catalog with its wheels, or a catalog built from the wheels of the folder with ``datalab-plugin catalog build``, then edited to give a display name to the plugins, deprecate or revoke them (``status`` and ``status_reason`` fields), or withdraw a release (``yanked`` field)." msgstr "**soit un fichier** ``catalog.json``, au format du catalogue de plugins DataLab, dont les URL des versions sont relatives au dossier : seuls les plugins qu'il liste sont affichés. Il peut s'agir d'une copie du catalogue de plugins DataLab avec ses wheels, ou d'un catalogue construit à partir des wheels du dossier avec ``datalab-plugin catalog build``, puis modifié pour donner un nom d'affichage aux plugins, les déprécier ou les révoquer (champs ``status`` et ``status_reason``), ou retirer une version (champ ``yanked``)." -msgid "Whatever the catalog, DataLab checks the size and SHA-256 digest of the wheel before installing it, then inspects it without running it and asks for confirmation; the confirmation dialog shows the catalog folder as the source of the plugin. Being listed in a local catalog does not mean that a plugin was audited: the organization managing the folder is responsible for its content, and write access to the folder should be restricted accordingly." -msgstr "Quel que soit le catalogue, DataLab vérifie la taille et l'empreinte SHA-256 de la wheel avant de l'installer, puis l'inspecte sans l'exécuter et demande une confirmation ; la boîte de dialogue de confirmation indique le dossier du catalogue comme source du plugin. Figurer dans un catalogue local ne signifie pas qu'un plugin a été audité : l'organisation qui gère le dossier est responsable de son contenu, et l'accès en écriture au dossier doit être restreint en conséquence." +msgid "Whatever the catalog, DataLab checks the size and SHA-256 digest of the artifact before installing it, inspects it without running it and asks for confirmation; the dialog shows the catalog folder as the source. A module is checked for a valid importable file name, Python encoding and syntax, with a 4 MiB size limit. Inspection does not import its code or certify its behavior. The organization managing a shared catalog is responsible for its content and should restrict write access." +msgstr "Quel que soit le catalogue, DataLab vérifie la taille et l'empreinte SHA-256 du fichier avant son installation, l'inspecte sans l'exécuter et demande confirmation ; la boîte de dialogue indique le dossier du catalogue comme source. Pour un module, la validité du nom importable, l'encodage Python et la syntaxe sont vérifiés, avec une taille maximale de 4 Mio. L'inspection n'importe pas le code et ne certifie pas son comportement. L'organisation responsable d'un catalogue partagé doit en contrôler le contenu et limiter les droits d'écriture." msgid "The ``datalab-plugin`` command maintains the ``catalog.json`` file of a folder:" msgstr "La commande ``datalab-plugin`` gère le fichier ``catalog.json`` d'un dossier :" -msgid "``build`` checks the plugin wheels (pure Python, plugin entry point, requirements available from the host or the catalog wheelhouse, supported Python version) and does not write the file if a wheel is refused. It keeps the fields edited in an existing ``catalog.json`` file (plugin ID, name, tier, status and its reason, repository, documentation, capabilities and keywords, and the reasons why releases are withdrawn), and revoked plugins remain listed. The file is replaced atomically, so that users may read the catalog meanwhile. ``check`` reports wheels that are missing, changed, refused or not listed, including the dependency artifacts. Run ``build`` again after adding wheels. Both commands exit with a non-zero status when they find a problem." -msgstr "``build`` vérifie les paquets wheel des plugins (Python pur, point d'entrée de plugin, exigences satisfaites par l'environnement hôte ou le dépôt de dépendances du catalogue, version de Python prise en charge) et n'écrit pas le fichier si un paquet est refusé. Il conserve les champs modifiés dans un ``catalog.json`` existant (identifiant, nom, catégorie, statut et motif, dépôt, documentation, capacités et mots-clés des plugins, ainsi que les motifs de retrait des versions), et les plugins révoqués restent listés. Le fichier est remplacé de manière atomique, pour que les utilisateurs puissent lire le catalogue pendant cette opération. ``check`` signale les paquets manquants, modifiés, refusés ou non listés, y compris les fichiers de dépendances. Relancez ``build`` après avoir ajouté des paquets. Les deux commandes renvoient un code de sortie non nul si elles détectent un problème." +msgid "``build`` checks wheels and source modules without executing them and does not write the index if an artifact is refused. Wheels retain their pure-Python, entry-point and dependency checks; modules follow the rules below. Curated metadata, withdrawal reasons and revoked entries are preserved. The JSON file is replaced atomically. ``check`` reports missing, changed, refused or unlisted plugin files and dependency artifacts. Run ``build`` again after changing the root files. Both commands exit with a non-zero status when they find a problem." +msgstr "``build`` vérifie les wheels et les modules source sans les exécuter et n'écrit pas l'index si un fichier est refusé. Les wheels conservent leurs vérifications de Python pur, de points d'entrée et de dépendances ; les modules suivent les règles ci-dessous. Les métadonnées administrées, les motifs de retrait et les entrées révoquées sont conservés. Le fichier JSON est remplacé de manière atomique. ``check`` signale les fichiers de plugins et de dépendances manquants, modifiés, refusés ou non listés. Relancez ``build`` après modification des fichiers à la racine. Les deux commandes renvoient un code de sortie non nul si elles détectent un problème." + +msgid "Single-file modules in a catalog" +msgstr "Modules en fichier unique dans un catalogue" + +msgid "For a simple offline catalog, put the unchanged ``datalab_.py`` file at its root and add that folder under **Local plugin catalogs**. No ``pyproject.toml``, wheel or Internet access is needed. To generate an index that an administrator can curate, run ``datalab-plugin catalog build ``, then ``check``. Mixed folders may contain both modules and wheels:" +msgstr "Pour un catalogue hors ligne simple, placez le fichier ``datalab_.py`` inchangé à sa racine et ajoutez ce dossier dans **Catalogues locaux de plugins**. Aucun ``pyproject.toml``, wheel ou accès à Internet n'est nécessaire. Pour générer un index administrable, exécutez ``datalab-plugin catalog build ``, puis ``check``. Un dossier mixte peut contenir des modules et des wheels :" + +msgid "The generated JSON uses ``kind: \"module\"`` and the real import name in ``module``; it does not give the script a fictitious Python distribution. Its release contains ``filename``, a relative ``url``, ``size``, ``sha256``, ``targets: [\"desktop\"]`` and ``version`` (possibly an empty string). ``kind`` is omitted or ``\"wheel\"`` for existing wheel entries. Add an optional ``requires_python`` constraint to a module release when its compatibility is known." +msgstr "Le JSON généré utilise ``kind: \"module\"`` et le vrai nom d'importation dans ``module`` ; il n'attribue pas de distribution Python fictive au script. Sa version contient ``filename``, une ``url`` relative, ``size``, ``sha256``, ``targets: [\"desktop\"]`` et ``version`` (éventuellement une chaîne vide). ``kind`` est omis ou vaut ``\"wheel\"`` pour les entrées wheel existantes. Ajoutez une contrainte facultative ``requires_python`` à une version de module lorsque sa compatibilité est connue." + +msgid "An administrator may edit the display name, summary, license and other descriptive fields in the JSON, and supply a catalog identity and version when they are not statically available in the source. ``build`` preserves these fields. A known source identity or version must agree with the catalog. For dynamic metadata or several plugin classes, the catalog describes the complete module without executing it. Its catalog identity does not rewrite the runtime ``PluginInfo`` or require editing a historical file." +msgstr "Un administrateur peut modifier le nom d'affichage, le résumé, la licence et les autres champs descriptifs du JSON, et fournir une identité de catalogue et une version lorsqu'elles ne sont pas accessibles statiquement dans le code source. ``build`` conserve ces champs. Une identité ou une version connue dans le code source doit correspondre au catalogue. En présence de métadonnées dynamiques ou de plusieurs classes de plugins, le catalogue décrit le module complet sans l'exécuter. Son identité de catalogue ne réécrit pas le ``PluginInfo`` utilisé à l'exécution et n'impose pas de modifier un fichier historique." + +msgid "If a module has no usable version, leave ``version`` empty: DataLab compares SHA-256 fingerprints instead. The confirmation and installed-files list distinguish a known version from a revision. A changed unversioned file, or different contents with the same declared version, requires explicit **Replace...** consent; it is not labelled as a newer version. Existing modules installed manually through DataLab are recognized too. Files in development directories are not overwritten; resolve a name conflict or remove the unmanaged copy before using the managed installation." +msgstr "Si un module n'a pas de version exploitable, laissez ``version`` vide : DataLab compare alors les empreintes SHA-256. La confirmation et la liste des fichiers installés distinguent une version connue d'une révision. Un fichier sans version modifié, ou un contenu différent portant la même version déclarée, demande un accord explicite via **Remplacer...** ; il n'est pas présenté comme une version plus récente. Les modules déjà installés manuellement par DataLab sont aussi reconnus. Les fichiers des répertoires de développement ne sont pas écrasés ; résolvez un conflit de nom ou retirez la copie non gérée avant d'utiliser l'installation gérée." + +msgid "Single-file modules use packages already available to DataLab. Their ``import`` statements are not interpreted as package requirements, no dependencies are downloaded, and a ``dependencies`` subfolder is not installed for them. ``prepare`` includes modules in the catalog but performs dependency downloads only for wheels. Imports that cannot be checked statically may still fail when the plugin is loaded, and are then shown in **Enable/disable plugins**." +msgstr "Les modules en fichier unique utilisent les paquets déjà disponibles dans DataLab. Leurs instructions ``import`` ne sont pas interprétées comme des exigences de paquets, aucune dépendance n'est téléchargée et un sous-dossier ``dependencies`` n'est pas installé pour eux. ``prepare`` inclut les modules dans le catalogue, mais ne télécharge de dépendances que pour les wheels. Les importations qui ne peuvent pas être vérifiées statiquement peuvent encore échouer au chargement du plugin ; ces erreurs sont alors affichées dans **Activer/désactiver les plugins**." + +msgid "For the public catalog, publish the unchanged file as a named GitHub release asset and provide its version, license, description and SHA-256 in the YAML entry. The source is not required to expose those values statically. An entry has this structure (replace the example repository, file and fingerprint with the actual release information):" +msgstr "Pour le catalogue public, publiez le fichier inchangé comme fichier joint nommé d'une release GitHub et indiquez sa version, sa licence, sa description et son SHA-256 dans la fiche YAML. Le code source n'est pas obligé d'exposer ces valeurs statiquement. Une fiche a la structure suivante (remplacez le dépôt, le fichier et l'empreinte d'exemple par les informations de la release réelle) :" + +msgid "The public catalog verifies and mirrors the script under ``modules//...``. To use that published catalog offline, copy its JSON and complete relative tree, including both ``modules`` and ``wheels`` when present. Do not rebuild an unchanged mirror whose artifacts are in these subdirectories: the local authoring commands scan the root. Source URLs, revision checks, withdrawals and local-catalog priority follow the same rules for both formats. Module entries target Desktop only, not DataLab-Web or PyPI." +msgstr "Le catalogue public vérifie le script et le met en miroir dans ``modules//...``. Pour utiliser ce catalogue publié hors ligne, copiez son JSON et toute son arborescence relative, y compris ``modules`` et ``wheels`` lorsqu'ils sont présents. Ne reconstruisez pas un miroir inchangé dont les fichiers se trouvent dans ces sous-dossiers : les commandes locales de préparation parcourent la racine. Les URL sources, les vérifications de révision, les retraits et la priorité des catalogues locaux suivent les mêmes règles pour les deux formats. Les modules ciblent uniquement Desktop, pas DataLab-Web ni PyPI." msgid "Additional dependencies and offline installation" msgstr "Dépendances supplémentaires et installation hors ligne" @@ -264,8 +321,8 @@ msgstr "Sous Windows, les chemins des bibliothèques natives doivent également msgid "Offline deployment, step by step" msgstr "Déploiement hors ligne, pas à pas" -msgid "This procedure is self-contained: the disconnected computer does not need a package index, a Python development environment, or access to the public plugin website. A local network share may be used without Internet access; a USB drive or an ordinary local folder works as well. Preparation takes place on a connected computer, installation on the disconnected computer." -msgstr "Cette procédure est autonome : l'ordinateur déconnecté n'a besoin ni d'un index de paquets, ni d'un environnement de développement Python, ni d'un accès au site public des plugins. Un partage sur le réseau local peut être utilisé sans accès à Internet ; une clé USB ou un simple dossier local convient également. La préparation se fait sur un ordinateur connecté, l'installation sur l'ordinateur déconnecté." +msgid "This procedure is self-contained: the disconnected computer does not need a package index, a Python development environment, or access to the public plugin website. A local network share, USB drive or ordinary local folder works. Preparing missing wheel dependencies requires a connected reference computer; a catalog containing only source modules can be assembled entirely offline (see :ref:`plugin_module_catalog`)." +msgstr "Cette procédure est autonome : l'ordinateur déconnecté n'a besoin ni d'un index de paquets, ni d'un environnement de développement Python, ni d'un accès au site public des plugins. Un partage réseau local, une clé USB ou un simple dossier local convient. Préparer les dépendances manquantes des wheels nécessite un ordinateur de référence connecté ; un catalogue ne contenant que des modules source peut être constitué entièrement hors ligne (voir :ref:`plugin_module_catalog`)." msgid "A plugin catalog is a **source of installation files**, not a plugin search directory. Add it under **Local plugin catalogs**, not **Additional plugin directories**. ``DATALAB_PLUGINS_CATALOG`` supplies catalogs; ``DATALAB_PLUGINS`` supplies code loaded directly. Installing copies the approved files into the user's DataLab configuration directory, so the source folder or network share is not needed to run the installed plugin afterwards." msgstr "Un catalogue de plugins est une **source de fichiers d'installation**, pas un répertoire de recherche de plugins. Ajoutez-le dans **Catalogues locaux de plugins**, et non dans **Répertoires supplémentaires de plugins**. ``DATALAB_PLUGINS_CATALOG`` indique des catalogues ; ``DATALAB_PLUGINS`` indique du code chargé directement. L'installation copie les fichiers approuvés dans le répertoire de configuration DataLab de l'utilisateur : le dossier source ou le partage réseau n'est donc plus nécessaire pour exécuter ensuite le plugin installé." @@ -276,11 +333,11 @@ msgstr "1. Préparer un catalogue complet sur un ordinateur connecté" msgid "The provider needs DataLab installed in a Python environment, with the ``datalab-plugin`` command available. The standalone application does not supply this authoring command; its users only need the prepared files. Use a reference environment matching the target's DataLab version, Python version, operating system, architecture and host package versions. **? > Installation and configuration...** identifies the target installation. A Windows 64-bit CPython 3.11 wheel is not interchangeable with a Linux, 32-bit or CPython 3.12 wheel." msgstr "Le fournisseur doit disposer de DataLab installé dans un environnement Python, avec la commande ``datalab-plugin`` disponible. L'application autonome ne fournit pas cette commande de préparation ; ses utilisateurs n'ont besoin que des fichiers préparés. Utilisez un environnement de référence correspondant à la version de DataLab, à la version de Python, au système d'exploitation, à l'architecture et aux versions des paquets de l'ordinateur cible. **? > Installation et configuration...** permet d'identifier l'installation cible. Un paquet pour CPython 3.11 sous Windows 64 bits n'est pas interchangeable avec un paquet pour Linux, une architecture 32 bits ou CPython 3.12." -msgid "Create a folder, place the plugin's ``.whl`` file at its root, then run:" -msgstr "Créez un dossier, placez le fichier ``.whl`` du plugin à sa racine, puis exécutez :" +msgid "Create a folder, place the plugin's ``.whl`` or ``datalab_*.py`` file at its root, then run:" +msgstr "Créez un dossier, placez le fichier ``.whl`` ou ``datalab_*.py`` du plugin à sa racine, puis exécutez :" -msgid "``prepare`` needs Internet access: it resolves the missing requirements against the reference host, downloads their wheels, including transitive dependencies, into ``dependencies``, and writes ``catalog.json``. It does not install these packages in the reference host. No source distribution is compiled. If it reports a missing wheel or a conflict, fix the reference catalog before transferring it." -msgstr "``prepare`` nécessite Internet : la commande résout les exigences manquantes par rapport à l'environnement hôte de référence, télécharge leurs paquets wheel, y compris les dépendances transitives, dans ``dependencies`` et écrit ``catalog.json``. Elle n'installe pas ces paquets dans l'environnement de référence. Aucune distribution source n'est compilée. Si elle signale un paquet manquant ou un conflit, corrigez le catalogue de référence avant de le transférer." +msgid "For wheels, ``prepare`` resolves missing requirements against the reference host, downloads their wheels and transitive dependencies into ``dependencies``, then writes ``catalog.json``. It does not install them in the reference host or compile source distributions. Modules are included without dependency downloads. Use ``build`` for a purely file-based catalog; if preparation reports a missing wheel or a conflict, fix the catalog before transferring it." +msgstr "Pour les wheels, ``prepare`` résout les exigences manquantes par rapport à l'environnement hôte de référence, télécharge leurs wheels et dépendances transitives dans ``dependencies``, puis écrit ``catalog.json``. La commande ne les installe pas dans l'environnement de référence et ne compile pas de distributions source. Les modules sont inclus sans téléchargement de dépendances. Utilisez ``build`` pour un catalogue reposant uniquement sur des fichiers ; si la préparation signale un wheel manquant ou un conflit, corrigez le catalogue avant de le transférer." msgid "Keep all files produced, including the dependency wheels. Do not copy just ``catalog.json`` or just the plugin wheel. If serving several target environments, prepare their wheels on matching reference environments, combine the needed files, then run ``datalab-plugin catalog build D:\\approved_plugins`` followed by ``check``. A pure-Python wheel may also restrict the supported Python versions in its metadata." msgstr "Conservez tous les fichiers produits, y compris les paquets de dépendances. Ne copiez pas uniquement ``catalog.json`` ou uniquement le paquet wheel du plugin. Pour desservir plusieurs environnements cibles, préparez leurs paquets dans des environnements de référence correspondants, rassemblez les fichiers nécessaires, puis exécutez ``datalab-plugin catalog build D:\\approved_plugins``, suivi de ``check``. Un paquet en Python pur peut lui aussi restreindre les versions de Python prises en charge dans ses métadonnées." @@ -291,8 +348,8 @@ msgstr "``check`` vérifie les fichiers du catalogue, leur intégrité et leurs msgid "2. Transfer the folder, preserving its layout" msgstr "2. Transférer le dossier en conservant son organisation" -msgid "Copy the complete folder to a USB drive, local disk or network share accessible from the target computer. Preserve relative paths and file names. A mirror of the public catalog may store files under ``wheels\\\\...`` instead of ``dependencies``: preserve those directories too, because ``catalog.json`` references them. The JSON file alone is not a mirror, and DataLab does not automatically export an online catalog for offline use." -msgstr "Copiez le dossier complet sur une clé USB, un disque local ou un partage réseau accessible depuis l'ordinateur cible. Conservez les chemins relatifs et les noms de fichiers. Un miroir du catalogue public peut stocker les fichiers dans ``wheels\\\\...`` plutôt que dans ``dependencies`` : conservez également ces répertoires, car ``catalog.json`` les référence. Le seul fichier JSON n'est pas un miroir, et DataLab n'exporte pas automatiquement un catalogue en ligne pour un usage hors ligne." +msgid "Copy the complete folder to a USB drive, local disk or network share accessible from the target computer. Preserve relative paths and file names. A public mirror may store artifacts under ``wheels\\\\...`` and ``modules\\\\...``: preserve those directories too, because ``catalog.json`` references them. The JSON file alone is not a mirror, and DataLab does not automatically export an online catalog for offline use." +msgstr "Copiez le dossier complet sur une clé USB, un disque local ou un partage réseau accessible depuis l'ordinateur cible. Conservez les chemins relatifs et les noms de fichiers. Un miroir public peut stocker les fichiers dans ``wheels\\\\...`` et ``modules\\\\...`` : conservez aussi ces répertoires, car ``catalog.json`` les référence. Le seul fichier JSON n'est pas un miroir et DataLab n'exporte pas automatiquement un catalogue en ligne pour un usage hors ligne." msgid "For a strictly offline JSON catalog, the installation ``url`` of every release and dependency artifact must resolve to an existing local file. An absolute ``https://`` URL is still downloaded from its server even if the catalog itself is local; placing a wheel beside that JSON does not override its URL. Informational links such as ``repository``, ``documentation`` and ``source_url`` do not replace installation files. Catalogs generated by ``build`` use relative file URLs; published mirrors must retain the matching relative directory tree." msgstr "Pour un catalogue JSON strictement hors ligne, l'URL d'installation (champ ``url``) de chaque version et de chaque fichier de dépendance doit désigner un fichier local existant. Une URL absolue ``https://`` entraîne toujours un téléchargement depuis son serveur, même si le catalogue est local ; placer un paquet wheel à côté de ce JSON ne remplace pas son URL. Les liens informatifs tels que ``repository``, ``documentation`` et ``source_url`` ne remplacent pas les fichiers d'installation. Les catalogues générés par ``build`` utilisent des URL de fichiers relatives ; les miroirs publiés doivent conserver l'arborescence relative correspondante." @@ -336,11 +393,11 @@ msgstr "Redémarrez DataLab si des dépendances ont été ajoutées ou si vous r msgid "5. Maintain and troubleshoot the offline catalog" msgstr "5. Maintenir et dépanner le catalogue hors ligne" -msgid "When ``catalog.json`` exists, it is authoritative: DataLab lists and copies only its declared releases and dependency artifacts. For a catalog managed from plugin wheels at its root, run ``build`` and ``check`` again on the preparation computer after adding or changing a plugin **or dependency** wheel, then redistribute the complete folder. Simply copying a new wheel beside an existing JSON file does not update that file. ``prepare`` may require Internet; ``build`` and ``check`` do not fetch missing packages." -msgstr "Lorsque ``catalog.json`` existe, il fait autorité : DataLab ne liste et ne copie que les versions et fichiers de dépendances qui y sont déclarés. Pour un catalogue géré à partir de paquets de plugins placés à sa racine, relancez ``build`` et ``check`` sur l'ordinateur de préparation après ajout ou modification d'un paquet de plugin **ou de dépendance**, puis redistribuez le dossier complet. Copier simplement un nouveau paquet à côté d'un JSON existant ne met pas ce fichier à jour. ``prepare`` peut nécessiter Internet ; ``build`` et ``check`` ne récupèrent pas les paquets manquants." +msgid "When ``catalog.json`` exists, it is authoritative: DataLab lists and copies only its declared releases and dependency artifacts. For a catalog managed from files at its root, run ``build`` and ``check`` again after adding or changing a module, plugin wheel or dependency wheel, then redistribute the complete folder. Copying a new file beside an existing JSON does not update that index. ``prepare`` may require Internet for wheel dependencies; ``build`` and ``check`` never fetch missing packages." +msgstr "Lorsque ``catalog.json`` existe, il fait autorité : DataLab ne liste et ne copie que les versions et fichiers de dépendances qui y sont déclarés. Pour un catalogue géré à partir de fichiers à sa racine, relancez ``build`` et ``check`` après ajout ou modification d'un module, d'un wheel de plugin ou de dépendance, puis redistribuez le dossier complet. Copier un nouveau fichier à côté d'un JSON existant ne met pas cet index à jour. ``prepare`` peut nécessiter Internet pour les dépendances des wheels ; ``build`` et ``check`` ne récupèrent jamais les paquets manquants." -msgid "Do not run ``build`` on an unchanged mirror whose plugin wheels live only under ``wheels\\\\...``: the command scans the catalog root, not that recursive tree, and would remove unrepresented entries from the generated catalog. Update such a mirror by transferring a complete, consistent published catalog and its files. To curate it with the local authoring commands instead, first arrange the plugin wheels at the root and the additional packages in ``dependencies``." -msgstr "N'exécutez pas ``build`` sur un miroir inchangé dont les paquets de plugins sont uniquement dans ``wheels\\\\...`` : la commande parcourt la racine du catalogue, pas cette arborescence récursive, et supprimerait du catalogue généré les entrées non représentées. Mettez à jour un tel miroir en transférant un catalogue publié complet et cohérent avec ses fichiers. Pour le gérer avec les commandes locales de préparation, disposez d'abord les paquets des plugins à la racine et les paquets supplémentaires dans ``dependencies``." +msgid "Do not run ``build`` on an unchanged mirror whose plugins live only under ``wheels\\\\...`` or ``modules\\\\...``: it scans the catalog root, not the recursive tree, and would remove unrepresented entries. Update a mirror by transferring a complete, consistent published catalog and its files. To manage it with the local authoring commands instead, first arrange plugin wheels and modules at the root and additional packages in ``dependencies``." +msgstr "N'exécutez pas ``build`` sur un miroir inchangé dont les plugins se trouvent uniquement dans ``wheels\\\\...`` ou ``modules\\\\...`` : la commande parcourt la racine du catalogue, pas l'arborescence récursive, et supprimerait les entrées non représentées. Mettez à jour un miroir en transférant un catalogue publié complet et cohérent avec ses fichiers. Pour le gérer avec les commandes locales de préparation, disposez d'abord les wheels et modules des plugins à la racine et les paquets supplémentaires dans ``dependencies``." msgid "Users refresh **Available plugins** to discover updates, then explicitly click **Update to...**. Updates are not installed automatically. Plugins installed with ``pip`` in the Python environment are identified but not updated by this dialog. Shared additional dependencies are retained when a plugin is uninstalled, and their existing versions remain protected on subsequent installations." msgstr "Les utilisateurs actualisent **Plugins disponibles** pour découvrir les mises à jour, puis cliquent explicitement sur **Mettre à jour vers...**. Les mises à jour ne sont pas installées automatiquement. Les plugins installés avec ``pip`` dans l'environnement Python sont identifiés, mais ne sont pas mis à jour par cette boîte de dialogue. Les dépendances supplémentaires partagées sont conservées lorsqu'un plugin est désinstallé, et leurs versions existantes restent protégées lors des installations suivantes." @@ -381,11 +438,11 @@ msgstr "un paquet wheel (``.whl``) construit à partir d'un projet de plugin, pa msgid "a single Python module named ``datalab_.py``." msgstr "un module Python unique nommé ``datalab_.py``." -msgid "Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows the module name, file, size and SHA-256 digest, without claiming to know its version, classes or dependencies. A plugin runs with the same rights as DataLab: install only plugins from authors you trust." -msgstr "Avant l'installation, DataLab vérifie le fichier sans l'exécuter. Pour un wheel, la confirmation affiche la distribution, la version, les points d'entrée et les exigences issus de ses métadonnées. Pour un module en fichier unique, elle affiche le nom du module, le fichier, la taille et l'empreinte SHA-256, sans prétendre connaître sa version, ses classes ou ses dépendances. Un plugin s'exécute avec les mêmes droits que DataLab : n'installez que des plugins provenant d'auteurs de confiance." +msgid "Before installing, DataLab checks the file without running it. For a wheel, the confirmation shows the distribution, version, entry points and requirements from its metadata. For a single-file module, it shows its import name, file, size, SHA-256 and any statically readable or explicit catalog metadata. An unknown version stays unknown and is displayed as a revision; no class or dependency list is guessed. A plugin runs with the same rights as DataLab: install only plugins from authors you trust." +msgstr "Avant l'installation, DataLab vérifie le fichier sans l'exécuter. Pour un wheel, la confirmation affiche la distribution, la version, les points d'entrée et les exigences issus de ses métadonnées. Pour un module en fichier unique, elle affiche son nom d'importation, le fichier, la taille, le SHA-256 et les métadonnées lisibles statiquement ou explicites du catalogue. Une version inconnue reste inconnue et est affichée comme une révision ; aucune liste de classes ou de dépendances n'est devinée. Un plugin s'exécute avec les mêmes droits que DataLab : n'installez que des plugins provenant d'auteurs de confiance." -msgid "Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Installations not adding dependencies retain the plugin reload workflow; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded plugins are only activated at the next start of DataLab." -msgstr "Les plugins installés sont conservés dans le dossier ``installed_plugins`` du répertoire de configuration DataLab (par exemple ``~/.DataLab_v1/installed_plugins``) et listés dans l'onglet, où **Désinstaller** les supprime. Les installations sans ajout de dépendances conservent le mécanisme de rechargement des plugins ; un plugin nouvellement installé est activé. Les dépendances supplémentaires et les nouvelles versions de plugins déjà chargés ne sont activées qu'au prochain démarrage de DataLab." +msgid "Installed plugins are kept in the ``installed_plugins`` folder of the DataLab configuration directory (e.g. ``~/.DataLab_v1/installed_plugins``) and listed in the tab, where **Uninstall** removes them. Modules can be reloaded after installation or replacement; a newly installed plugin is enabled. Additional dependencies and new versions of already loaded wheels are only activated at the next start of DataLab." +msgstr "Les plugins installés sont conservés dans le dossier ``installed_plugins`` du répertoire de configuration DataLab (par exemple ``~/.DataLab_v1/installed_plugins``) et listés dans l'onglet, où **Désinstaller** les supprime. Les modules peuvent être rechargés après installation ou remplacement ; un plugin nouvellement installé est activé. Les dépendances supplémentaires et les nouvelles versions de wheels déjà chargés ne sont activées qu'au prochain démarrage de DataLab." msgid "When reloading plugins, DataLab performs the following steps:" msgstr "Lors du rechargement des plugins, DataLab effectue les étapes suivantes :" @@ -621,8 +678,8 @@ msgstr "``--package`` et ``--plugin-id`` : à défaut, le nom du module et l'ID msgid "An existing file is never overwritten. To share the plugin, send its file: other users install it with **Plugins > Configure plugins... > Install plugins** (see :ref:`plugin_install_from_file`)." msgstr "Un fichier existant n'est jamais écrasé. Pour partager le plugin, envoyez son fichier : les autres utilisateurs l'installent avec **Plugins > Configurer les plugins... > Installer des plugins** (voir :ref:`plugin_install_from_file`)." -msgid "The catalogs currently require a wheel, not a particular source architecture (see :ref:`plugin_forms`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`)." -msgstr "Les catalogues exigent actuellement un wheel, pas une architecture particulière du code source (voir :ref:`plugin_forms`). Pour développer un plugin plus important avec le packaging et les tests déjà configurés, créez plutôt un projet structuré en couches avec ``--project`` (voir :ref:`plugin_project`)." +msgid "Both the raw single-file plugin and a wheel can be published in a Desktop catalog (see :ref:`plugin_module_catalog`). To develop a larger plugin with packaging and tests already configured, create a layered project with ``--project`` instead (see :ref:`plugin_project`)." +msgstr "Le plugin brut en fichier unique et le wheel peuvent tous deux être publiés dans un catalogue Desktop (voir :ref:`plugin_module_catalog`). Pour développer un plugin plus important avec le packaging et les tests déjà configurés, créez plutôt un projet structuré en couches avec ``--project`` (voir :ref:`plugin_project`)." msgid "Creating a layered plugin project" msgstr "Créer un projet de plugin structuré en couches" diff --git a/doc/locale/fr/LC_MESSAGES/release_notes/release_1.04.po b/doc/locale/fr/LC_MESSAGES/release_notes/release_1.04.po index 02e847453..d2f3b1487 100644 --- a/doc/locale/fr/LC_MESSAGES/release_notes/release_1.04.po +++ b/doc/locale/fr/LC_MESSAGES/release_notes/release_1.04.po @@ -7,7 +7,7 @@ msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" -"POT-Creation-Date: 2026-10-22 14:31+0200\n" +"POT-Creation-Date: 2026-10-22 22:19+0200\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language: fr\n" @@ -117,6 +117,15 @@ msgstr "Les plugins peuvent déclarer une icône, affichée dans le catalogue ** msgid "**Plugin installation:**" msgstr "**Installation de plugins :**" +msgid "Online and offline Desktop catalogs now accept single-file `datalab_*.py` plugins alongside wheels. Historical files do not need a packaging project or source changes to be listed; catalog inspection never imports or executes them (implements [Issue #419](https://github.com/DataLab-Platform/DataLab/issues/419))" +msgstr "Les catalogues Desktop en ligne et hors ligne acceptent désormais les plugins en fichier unique `datalab_*.py` à côté des wheels. Les fichiers historiques n'ont besoin ni de projet de packaging ni de modification du code source pour être référencés ; l'inspection du catalogue ne les importe ni ne les exécute jamais (implémente [l'issue #419](https://github.com/DataLab-Platform/DataLab/issues/419))" + +msgid "Known script versions can be updated; unversioned files are identified by SHA-256 and offer explicit replacement when their content changes. Scripts previously installed from a file are recognized, and existing development copies are not overwritten" +msgstr "Les versions connues des scripts peuvent être mises à jour ; les fichiers sans version sont identifiés par SHA-256 et proposent un remplacement explicite lorsque leur contenu change. Les scripts précédemment installés depuis un fichier sont reconnus et les copies de développement existantes ne sont pas écrasées" + +msgid "The local catalog commands handle mixed folders of scripts and wheels. Single-file plugins use libraries already available to DataLab and do not trigger dependency downloads. The documentation now explicitly distinguishes source organization, distribution formats and functional capabilities, including in the French offline guide" +msgstr "Les commandes de catalogue local gèrent les dossiers mêlant scripts et wheels. Les plugins en fichier unique utilisent les bibliothèques déjà disponibles dans DataLab et ne déclenchent pas de téléchargement de dépendances. La documentation distingue désormais explicitement l'organisation du code source, les formats de distribution et les capacités fonctionnelles, y compris dans le guide hors ligne français" + msgid "Plugins can now be installed from a file, without any Python tool, including in the standalone version of DataLab: the new **Install plugins** tab of **Plugins > Configure plugins...** accepts a pure-Python wheel (`.whl`) or a single `datalab_.py` module" msgstr "Les plugins peuvent désormais être installés depuis un fichier, sans aucun outil Python, y compris dans la version autonome de DataLab : le nouvel onglet **Installer des plugins** de **Plugins > Configurer les plugins...** accepte un wheel en Python pur (`.whl`) ou un module unique `datalab_.py`" @@ -138,8 +147,8 @@ msgstr "Le nouvel onglet **Plugins disponibles** liste les plugins du [catalogue msgid "Organizations can distribute their own plugins, or the versions of public plugins they have approved, from local plugin catalogs: folders, typically shared on the network, declared in the **Plugin settings** tab or with the `DATALAB_PLUGINS_CATALOG` environment variable. Their plugins are listed in the **Available plugins** tab before those of the DataLab plugin catalog, and hide them when both offer the same plugin" msgstr "Les organisations peuvent diffuser leurs propres plugins, ou les versions de plugins publics qu'elles ont validées, depuis des catalogues locaux de plugins : des dossiers, généralement partagés sur le réseau, déclarés dans l'onglet **Paramètres des plugins** ou par la variable d'environnement `DATALAB_PLUGINS_CATALOG`. Leurs plugins sont listés dans l'onglet **Plugins disponibles** avant ceux du catalogue de plugins DataLab, et les masquent lorsque les deux proposent le même plugin" -msgid "A local catalog folder may simply hold plugin wheels, listed as they are copied to it, or a `catalog.json` file giving display names, deprecations and withdrawn releases; `datalab-plugin catalog build` writes this file from the wheels of the folder, keeping the edited fields, and `datalab-plugin catalog check` reports wheels that are missing, changed, refused by DataLab or not listed" -msgstr "Un dossier de catalogue local peut simplement contenir des wheels de plugins, listées dès qu'elles y sont copiées, ou un fichier `catalog.json` indiquant les noms d'affichage, les dépréciations et les versions retirées ; `datalab-plugin catalog build` écrit ce fichier à partir des wheels du dossier en conservant les champs modifiés, et `datalab-plugin catalog check` signale les wheels manquantes, modifiées, refusées par DataLab ou non listées" +msgid "A local catalog folder may hold plugin wheels and single-file modules, listed as they are copied to it, or a `catalog.json` index giving display names, deprecations and withdrawn releases; `datalab-plugin catalog build` writes this index while keeping curated metadata, and `datalab-plugin catalog check` reports missing, changed, refused or unlisted artifacts" +msgstr "Un dossier de catalogue local peut contenir des wheels et des modules en fichier unique, listés à mesure qu'ils y sont copiés, ou un index `catalog.json` indiquant les noms d'affichage, les dépréciations et les versions retirées ; `datalab-plugin catalog build` écrit cet index en conservant les métadonnées administrées, et `datalab-plugin catalog check` signale les fichiers manquants, modifiés, refusés ou non listés" msgid "Authors can prepare an offline catalog with `datalab-plugin catalog prepare`: it downloads the additional dependency wheels needed by the reference host without installing them, then writes the catalog. The complete folder can be distributed on a network share or copied to computers without Internet access" msgstr "Les auteurs peuvent préparer un catalogue hors ligne avec `datalab-plugin catalog prepare` : la commande télécharge les paquets de dépendances supplémentaires nécessaires à l'environnement hôte de référence sans les installer, puis écrit le catalogue. Le dossier complet peut être distribué sur un partage réseau ou copié sur des ordinateurs sans accès à Internet" diff --git a/doc/release_notes/release_1.04.md b/doc/release_notes/release_1.04.md index fc75f34d9..17660131e 100644 --- a/doc/release_notes/release_1.04.md +++ b/doc/release_notes/release_1.04.md @@ -43,6 +43,9 @@ **Plugin installation:** +* Online and offline Desktop catalogs now accept single-file `datalab_*.py` plugins alongside wheels. Historical files do not need a packaging project or source changes to be listed; catalog inspection never imports or executes them (implements [Issue #419](https://github.com/DataLab-Platform/DataLab/issues/419)) +* Known script versions can be updated; unversioned files are identified by SHA-256 and offer explicit replacement when their content changes. Scripts previously installed from a file are recognized, and existing development copies are not overwritten +* The local catalog commands handle mixed folders of scripts and wheels. Single-file plugins use libraries already available to DataLab and do not trigger dependency downloads. The documentation now explicitly distinguishes source organization, distribution formats and functional capabilities, including in the French offline guide * Plugins can now be installed from a file, without any Python tool, including in the standalone version of DataLab: the new **Install plugins** tab of **Plugins > Configure plugins...** accepts a pure-Python wheel (`.whl`) or a single `datalab_.py` module * Before installing, DataLab checks the file without running it and shows its name, version, plugin classes, dependencies and SHA-256 digest * Plugins may now include additional dependencies as separate wheels, including compatible native libraries, without changing packages already supplied by DataLab. A complete local catalog can be installed without Internet access; missing files and version conflicts are reported before any partial installation is activated @@ -50,7 +53,7 @@ * Installed plugins are listed in the same tab, where they can be uninstalled; a newly installed plugin is enabled after a plugin reload when no additional packages are needed, or after restarting when dependencies were added. A new version of an already loaded plugin is used at the next start * The new **Available plugins** tab lists the plugins of the [DataLab plugin catalog](https://datalab-platform.com/plugins/), official or from the community, with a search field: **Install** and **Update** download the wheel, check it against the catalog and install it after confirmation. The catalog address may be changed to use a private copy * Organizations can distribute their own plugins, or the versions of public plugins they have approved, from local plugin catalogs: folders, typically shared on the network, declared in the **Plugin settings** tab or with the `DATALAB_PLUGINS_CATALOG` environment variable. Their plugins are listed in the **Available plugins** tab before those of the DataLab plugin catalog, and hide them when both offer the same plugin -* A local catalog folder may simply hold plugin wheels, listed as they are copied to it, or a `catalog.json` file giving display names, deprecations and withdrawn releases; `datalab-plugin catalog build` writes this file from the wheels of the folder, keeping the edited fields, and `datalab-plugin catalog check` reports wheels that are missing, changed, refused by DataLab or not listed +* A local catalog folder may hold plugin wheels and single-file modules, listed as they are copied to it, or a `catalog.json` index giving display names, deprecations and withdrawn releases; `datalab-plugin catalog build` writes this index while keeping curated metadata, and `datalab-plugin catalog check` reports missing, changed, refused or unlisted artifacts * Authors can prepare an offline catalog with `datalab-plugin catalog prepare`: it downloads the additional dependency wheels needed by the reference host without installing them, then writes the catalog. The complete folder can be distributed on a network share or copied to computers without Internet access * A catalog that cannot be read, such as a network share that is not connected or the online catalog on a computer without Internet access, is reported in the **Available plugins** tab without hiding the plugins of the other catalogs