wxGUI: Extract menu strings with an ITS definition

.flake8

@@ -33,8 +33,6 @@ per-file-ignores =
     gui/wxpython/mapdisp/test_mapdisp.py: E501
     gui/wxpython/mapswipe/g.gui.mapswipe.py: E501
     gui/wxpython/timeline/g.gui.timeline.py: E501
-    # Generated file
-    gui/wxpython/menustrings.py: E501
     # C wrappers call libgis.G_gisinit before importing other modules.
     # TODO: Is this really needed?
     python/grass/jupyter/__init__.py: E501

.gitignore

@@ -22,7 +22,6 @@ macos/cache/*
 *.dox.org
 
 # Ignore specific files generated during make
-gui/wxpython/menustrings.py
 gui/wxpython/xml/menudata.xml
 gui/wxpython/xml/module_tree_menudata.xml
 include/Make/Doxyfile_arch_html

doc/development/README.md

@@ -22,3 +22,7 @@ maintaining GRASS.
 ## C API
 
 - [GRASS Programmer's manual](https://grass.osgeo.org/programming8/)
+
+## wxGUI
+
+- [wxGUI menus and their translation](wxgui_menus.md)

doc/development/wxgui_menus.md

@@ -0,0 +1,145 @@
+# wxGUI menus and their translation
+
+This note describes how the menus of the wxGUI are defined, built and
+translated. It covers the main menu, the tool tree (Tools tab), and the menus
+of the Graphical Modeler, the Cartographic Composer and the Python editor.
+Paths of wxGUI files are relative to `gui/wxpython/`, other paths to the
+repository root.
+
+## Source files
+
+| File | Content |
+| --- | --- |
+| `xml/main_menu.xml` | Root of the main menu: the top-level toolboxes. |
+| `xml/module_tree.xml` | Root of the tool tree, with the `<user-toolboxes-list/>` and `<addons/>` placeholders. |
+| `xml/toolboxes.xml` | Toolboxes (menus) with their labels and items: `module-item` (a tool), `wxgui-item`, `subtoolbox`, `separator`. |
+| `xml/wxgui_items.xml` | GUI items: label, handler or command, description, keywords. |
+| `xml/menudata_modeler.xml`, `xml/menudata_psmap.xml`, `xml/menudata_pyedit.xml` | Complete menus in the final `menudata` format, read as they are. |
+| `xml/*.dtd` | Structure of each file type. |
+
+## Build and runtime
+
+At build time, with the C locale (English):
+
+1. `tools/build_modules_xml.py` writes `module_items.xml` with the
+   description and keywords of every tool, taken from its interface
+   description. For a tool with a label, the description is the label and
+   the description joined with a space.
+2. `core/toolboxes.py` combines `main_menu.xml` (or `module_tree.xml` with
+   the `module_tree` argument), `toolboxes.xml`, `wxgui_items.xml` and
+   `module_items.xml` into `menudata.xml` (or `module_tree_menudata.xml`).
+   It expands subtoolboxes, fills each item with the data of the item of the
+   same name (elements already in the item take precedence), adds handlers,
+   and converts the tags to the `menudata` format (`toolbox` to `menu`,
+   `description` to `help`, `module` to `command`, and so on).
+
+All three files are generated and not committed. Autotools writes
+`module_items.xml` directly to the `xml/` directory of the distribution and
+the other two to `xml/` in the source tree (ignored by Git), from where the
+distribution gets copies. CMake writes all of them to the output tree.
+
+At runtime, `core/menutree.py` (`MenuTreeModelBuilder`) reads a `menudata`
+file into a tree model, passing each menu label, item label and item help
+through `_()`. Classes in `lmgr/menudata.py`, `gmodeler/menudata.py`,
+`psmap/menudata.py` and `gui_core/pyedit.py` choose the file. When the user
+config directory has `toolboxes/toolboxes.xml`, `toolboxes/main_menu.xml` or
+`toolboxes/module_tree.xml`, `core/toolboxes.py` regenerates the menu data in
+that directory when it is missing or older than these files. Each time the
+tool tree is loaded, its `<addons/>` placeholder is replaced by an Addons
+menu listing the installed addons.
+
+```mermaid
+flowchart LR
+    TOOLS["tools<br>--interface-description"] --> BMX["tools/build_modules_xml.py"]
+    BMX --> MI["module_items.xml"]
+    MM["main_menu.xml<br>module_tree.xml"] --> TBX["core/toolboxes.py"]
+    TB["toolboxes.xml<br>wxgui_items.xml"] --> TBX
+    MI --> TBX
+    USER["user toolboxes/*.xml"] -. runtime .-> TBX
+    TBX --> MD["menudata.xml<br>module_tree_menudata.xml"]
+    MD --> MT["core/menutree.py"]
+    MDX["menudata_modeler.xml<br>menudata_psmap.xml<br>menudata_pyedit.xml"] --> MT
+    MT --> GUI["menus and tool tree"]
+```
+
+To print a menu as a tree or as a list of tools, run in a GRASS session from
+the installed `gui/wxpython` directory:
+
+```bash
+PYTHONPATH=. python core/menutree.py tree manager  # or module_tree, modeler, psmap
+PYTHONPATH=. python core/menutree.py commands manager
+```
+
+## Translation
+
+`make pot` in `locale/` writes the templates in `locale/templates/`. For the
+`grasswxpy` domain, it extracts the `_()` strings from the Python files of the
+wxGUI, then joins (`-j`) the strings of the committed menu XML files listed in
+`WXPY_XMLFILES`. xgettext finds the rules for these files through
+`GETTEXTDATADIRS=locale/`: `locale/its/grass-wxgui.loc` maps the root
+elements `toolboxes`, `wxgui-items` and `menudata` to
+`locale/its/grass-wxgui.its`, which marks `label`, `help` and `description`
+as translatable and adds translator comments built from the XML itself.
+
+This part reads no generated file, so it needs no build. The help of a tool
+item is the tool description, which is translated from `grassmods` for a tool
+without a label, because `_()` in
+`python/grass/__init__.py` looks a message up in `grasslibs`, then
+`grassmods`, then `grasswxpy`.
+
+```mermaid
+flowchart LR
+    PY["gui/wxpython/**/*.py"] --> XG1["xgettext"]
+    XML["xml/toolboxes.xml<br>xml/wxgui_items.xml<br>xml/menudata_*.xml"] --> XG2["xgettext +<br>locale/its/grass-wxgui.*"]
+    XG1 --> WXPOT["grasswxpy.pot"]
+    XG2 -- "-j" --> WXPOT
+    C["C tools,<br>locale/scriptstrings"] --> XG3["xgettext"] --> MODPOT["grassmods.pot"]
+    LIB["lib/"] --> XG4["xgettext"] --> LIBPOT["grasslibs.pot"]
+    WXPOT & MODPOT & LIBPOT -- "make update-po" --> PO["locale/po/*.po<br>(Weblate)"]
+    PO -- msgfmt --> MO["GISBASE/locale/LANG/<br>LC_MESSAGES/*.mo"]
+    MO --> RT["_(): grasslibs, grassmods,<br>grasswxpy"]
+```
+
+## How to add or change a menu entry
+
+- Tool: add `<module-item name="r.example"><label>...</label></module-item>`
+  to a toolbox in `xml/toolboxes.xml`. The help text comes from the tool.
+- GUI item: define a `wxgui-item` in `xml/wxgui_items.xml` with a handler
+  method of the main window frame (`lmgr/frame.py` and `main_window/frame.py`)
+  or a command, and reference it by name from `xml/toolboxes.xml`.
+- Menu: add a `toolbox` to `xml/toolboxes.xml` and a `subtoolbox` reference
+  to its parent toolbox, to `xml/main_menu.xml` or to `xml/module_tree.xml`.
+- Modeler, Cartographic Composer or Python editor: edit its
+  `xml/menudata_*.xml`.
+- New XML file with translatable strings: add it to `WXPY_XMLFILES` in
+  `locale/Makefile` and, for a new root element, a rule to
+  `locale/its/grass-wxgui.loc`.
+
+Rebuild to regenerate the menu data. Mark an accelerator key by `&` (written
+`&amp;` in XML) before its letter.
+
+## How translators see it
+
+Each menu string appears once in `grasswxpy.pot` with references to the
+committed XML files and lines and a comment in a `key=value` form, for
+example:
+
+```text
+#. type=menu-item-help (description of a menu entry); label="Georectify"
+#: ../gui/wxpython/xml/wxgui_items.xml:8
+msgid "Manage Ground Control Points for Georectification"
+```
+
+The `&` of an accelerator key stays in the message as it is.
+
+## Known limitations
+
+- The help of a tool with a label (label and description joined) is not in
+  any template, so it is shown untranslated.
+- Keywords are neither extracted nor passed through `_()`.
+- Labels from user toolboxes and descriptions of addons are not in any
+  template.
+- Messages have no context: identical strings share one translation, and the
+  first domain containing a message wins.
+- The descriptions of Python tools reach `grassmods.pot` through
+  `locale/scriptstrings/*_to_translate.c`, which only a build generates.

gui/wxpython/CMakeLists.txt

@@ -147,6 +147,7 @@ if(NOT WITH_FHS)
 
     add_custom_target(
         build_module_tree_menudata
+        ALL
         COMMAND
             ${grass_env_command} ${PYTHON_EXECUTABLE}
             ${CMAKE_CURRENT_SOURCE_DIR}/core/toolboxes.py "module_tree" >
@@ -163,26 +164,6 @@ if(NOT WITH_FHS)
     )
 
     set_target_properties(build_module_tree_menudata PROPERTIES FOLDER GUI)
-
-    add_custom_target(
-        build_menustrings
-        ALL
-        COMMAND
-            ${grass_env_command} ${PYTHON_EXECUTABLE}
-            ${CMAKE_CURRENT_SOURCE_DIR}/core/menutree.py "manager" >>
-            ${CMAKE_CURRENT_SOURCE_DIR}/menustrings.py
-        COMMAND
-            ${grass_env_command} ${PYTHON_EXECUTABLE}
-            ${CMAKE_CURRENT_SOURCE_DIR}/core/menutree.py "module_tree" >>
-            ${CMAKE_CURRENT_SOURCE_DIR}/menustrings.py
-        COMMAND
-            ${grass_env_command} ${PYTHON_EXECUTABLE}
-            ${CMAKE_CURRENT_SOURCE_DIR}/core/menutree.py "psmap" >>
-            ${CMAKE_CURRENT_SOURCE_DIR}/menustrings.py
-        DEPENDS build_module_tree_menudata gui_images
-    )
-
-    set_target_properties(build_menustrings PROPERTIES FOLDER GUI)
 endif() # NOT WITH_FHS
 
 install(DIRECTORY "${OUTDIR}/${WXPYTHON_DIR}/xml" DESTINATION "${WXPYTHON_DIR}")

gui/wxpython/Makefile

@@ -1,7 +1,7 @@
 MODULE_TOPDIR = ../..
 
 SUBDIRS = docs animation datacatalog mapswipe gmodeler rlisetup psmap dbmgr vdigit rdigit iclass gcp timeline tplot photo2image image2target
-EXTRA_CLEAN_FILES = menustrings.py build_ext.pyc xml/menudata.xml xml/module_tree_menudata.xml */*.pyc
+EXTRA_CLEAN_FILES = build_ext.pyc xml/menudata.xml xml/module_tree_menudata.xml */*.pyc
 
 include $(MODULE_TOPDIR)/include/Make/Dir.make
 include $(MODULE_TOPDIR)/include/Make/Python.make
@@ -33,7 +33,8 @@ ifndef CROSS_COMPILING
 	-$(MAKE) $(DSTDIR)/xml/module_items.xml
 	-$(MAKE) xml/menudata.xml
 	-$(MAKE) xml/module_tree_menudata.xml
-	-$(MAKE) menustrings.py
+	-$(MAKE) $(DSTDIR)/xml/menudata.xml
+	-$(MAKE) $(DSTDIR)/xml/module_tree_menudata.xml
 endif
 	$(MAKE) parsubdirs
 
@@ -49,13 +50,6 @@ xml/module_tree_menudata.xml: core/toolboxes.py
 	$(call run_grass,$(PYTHON) $< "module_tree" > $@)
 	$(call run_grass,$(PYTHON) $< "validate" $@)
 
-menustrings.py: core/menutree.py $(DSTDIR)/xml/menudata.xml $(DSTDIR)/xml/module_tree_menudata.xml $(DSTDIR)/xml/menudata_modeler.xml $(DSTDIR)/xml/menudata_psmap.xml
-	@echo "# This is a generated file.\n# pylint: skip-file\n" > $@
-	$(call run_grass,$(PYTHON) $< "manager" >> $@)
-	$(call run_grass,$(PYTHON) $< "module_tree" >> $@)
-	$(call run_grass,$(PYTHON) $< "modeler" >> $@)
-	$(call run_grass,$(PYTHON) $< "psmap" >> $@)
-
 $(DSTDIR)/xml/module_items.xml: tools/build_modules_xml.py
 	@echo "Generating interface description for all modules..."
 	$(call run_grass,$(PYTHON) $< > $@)

gui/wxpython/README.md

@@ -55,15 +55,14 @@ clear the display and start from scratch.
 Notes:
 
 - Help part of menu entries is coming from the module descriptions
-- The menu is generated by `./tools/build_modules_xml.py`
-
-Update of module description strings in `./xml/menudata.xml`:
-
-- in a GRASS session, run `./tools/update_menudata.py`
-
-From this (updated) `./xml/menudata.xml`, the gettext strings are generated
-via Makefile and stored into the file `menustrings.py`.
+- The menu data (`./xml/menudata.xml` and `./xml/module_tree_menudata.xml`)
+  is generated by `./core/toolboxes.py`
+- How the menus are built and translated is described in
+  [wxGUI menus and their translation](../../doc/development/wxgui_menus.md)
 
 While generating the po files in `../../locale/po/` by `../../locale/Makefile`,
 all `.py` files are parsed and the strings are stored in
-`../../locale/po/grasswxpy_XX.po`.
+`../../locale/po/grasswxpy_XX.po`. The labels, help texts and descriptions in
+the menu definitions (`./xml/toolboxes.xml`, `./xml/wxgui_items.xml` and
+`./xml/menudata_*.xml`) are extracted with the ITS rules in
+`../../locale/its/`.

gui/wxpython/core/menutree.py

@@ -12,8 +12,7 @@
 @endcode
 
 where <i>action</i>:
- - strings (default, used for translations)
- - tree (simple tree structure)
+ - tree (default, simple tree structure)
  - commands (command names and their place in tree)
  - dump (tree structure with stored data)
 
@@ -38,8 +37,6 @@
 import copy
 import xml.etree.ElementTree as ET
 
-from typing import TYPE_CHECKING
-
 import wx
 
 from core.treemodel import TreeModel, ModuleNode
@@ -52,9 +49,6 @@
 if not os.getenv("GISBASE"):
     sys.exit("GRASS is not running. Exiting...")
 
-if TYPE_CHECKING:
-    from typing_extensions import Writer
-
 # TODO: change the system to remove strange derived classes
 
 
@@ -158,17 +152,6 @@ def PrintTree(self, fh):
         for child in self.model.root.children:
             printTree(node=child, fh=fh)
 
-    def PrintStrings(self, fh: Writer[str]):
-        """Print menu strings to file (used for localization)
-
-        :param fh: file descriptor
-        """
-        className = self.__class__.__name__
-        fh.write("menustrings_%s = [\n" % className)
-        for child in self.model.root.children:
-            printStrings(child, fh)
-        fh.write("    '']\n")
-
     def PrintCommands(self, fh):
         printCommands(self.model.root, fh, itemSep=" | ", menuSep=" > ")
 
@@ -192,20 +175,6 @@ def printTree(node, fh, indent=0):
         printTree(node=child, fh=fh, indent=indent + 2)
 
 
-def printStrings(node, fh: Writer[str]):
-    # node.label  - with module in brackets
-    # node.data['label'] - without module in brackets
-    if node.label and not node.data:
-        fh.write("    _(%r),\n" % str(node.label))
-    if node.data:
-        if node.data.get("label"):
-            fh.write("    _(%r),\n" % str(node.data["label"]))
-        if node.data.get("description"):
-            fh.write("    _(%r),\n" % str(node.data["description"]))
-    for child in node.children:
-        printStrings(node=child, fh=fh)
-
-
 def printCommands(node, fh, itemSep, menuSep):
     def collectParents(node, parents):
         parent = node.parent
@@ -227,11 +196,11 @@ def collectParents(node, parents):
 
 
 if __name__ == "__main__":
-    action = "strings"
+    action = "tree"
     menu = "manager"
 
     for arg in sys.argv:
-        if arg in {"strings", "tree", "commands", "dump"}:
+        if arg in {"tree", "commands", "dump"}:
             action = arg
         elif arg in {"manager", "module_tree", "modeler", "psmap"}:
             menu = arg
@@ -264,9 +233,7 @@ def collectParents(node, parents):
 
         gscore.fatal("Unknown value for parameter menu: %s" % menu)
 
-    if action == "strings":
-        menudata.PrintStrings(sys.stdout)
-    elif action == "tree":
+    if action == "tree":
         menudata.PrintTree(sys.stdout)
     elif action == "commands":
         menudata.PrintCommands(sys.stdout)

gui/wxpython/docs/wxgui_sphinx/src/wxgui_libraries.rst

@@ -14,7 +14,6 @@ Libraries
   location_wizard
   mapdisp
   mapwin
-  menustrings
   modules
   nviz
   vnet