Skip to content

i18n: Extract tool header strings without a build, with context for translators - #727

Open
echoix wants to merge 7 commits into
mainfrom
claude/string-translation-collection-kmjr2b
Open

echoix wants to merge 7 commits into
mainfrom
claude/string-translation-collection-kmjr2b

Conversation

@echoix

@echoix echoix commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Description

Replaces the locale/scriptstrings workaround for the # %module / # %flag / # %option headers of script tools.

  • New utils/grass_tool_header_pot.py (Python standard library only) reads the headers directly and writes a standard POT: references to the real source lines, #. comments saying what each string is (tool description, keyword, option label, GUI section, ...). make pot merges it into grassmods.pot with xgettext -j.
  • Tools are found by their Makefiles as in the build: $(PGM).py, $(PGM).sh or an extension-less $(PGM) (Script.make, ShScript.make), g.gui.*.py (GuiScript.make), plus gui/scripts/*.py (d.rast3d, d.wms), which were not extracted before. Other files can be given as arguments.
  • g.parser now translates each description in descriptions separately, with the message context option value description, so the values are no longer part of translatable strings. A translation containing ; is ignored. Without a translation, the output is unchanged.
  • g.parser no longer translates guidependency (a list of option keys).
  • Removed the scriptstrings rules from Make and CMake, and the unused STRINGDIR variable passed by g.extension. The Windows launcher previously created by cmake/locale_strings.cmake is now created at configure time, as for GUI tools.
  • grassmods.pot regenerated in separate commits; they can be dropped if preferred.

Motivation and context

The entries pointed to generated locale/scriptstrings/*_to_translate.c files, so Weblate could not link to the source, a full build was needed, and translators had no context. Value lists were one string, so values could be translated by mistake (24 of the 44 existing translations of these strings did). The g.gui.* and gui/scripts tools were not extracted. With CMake, strings containing ; were also split.

xgettext has no plugin mechanism for new languages, and ITS rules only apply to XML input, so a separate extractor producing a standard POT merged by gettext is the standard way to extend it.

The extractor and its tests are in utils/, not locale/: with --import-mode=importlib, tests under locale/ make pytest import that directory as a package named locale, which replaces the standard library module for the following tests.

How has this been tested?

  • make pot (Autotools): compared to the current grassmods.pot, the only changes are the split value descriptions, removed guidependency values, and the added g.gui.* and gui/scripts strings; no C references changed.
  • Runtime check with a test .mo in a French locale: a description with the context is translated, a translation with ; is rejected, and an entry without the context is not used.
  • New pytest tests in utils/tests/ (including shell, extension-less, and Perl files, and capitalized header commands) and general/g.parser/tests/ (Python and shell scripts); CMake build with and without docs; pre-commit.
  • The extractor reads the headers of all 366 script tools in grass-addons (including the two shell scripts) without errors, and msgfmt accepts the result.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing
    functionality to not work as before)

Checklist

  • PR title provides summary of the changes and starts with one of the
    pre-defined prefixes
  • My code follows the code style
    of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

The extractor and the g.parser changes were written with assistance from Claude (Anthropic).

claude added 4 commits October 4, 2026 20:18
The guidependency field is a list of option keys used by the GUI (e.g.,
input,maps), so a translation would break the dependency. C tools
already set it without translation.

Developed with assistance from Claude (Anthropic).
The descriptions field (value;description;value;description...) was
translated as a whole, so the values were part of the translatable string
and translators could change them, breaking the tool.

Now each description is translated separately with the message context
"option value description", looked up the same way as pgettext() from
GNU gettext.h. The values are kept as written. A translation containing a
semicolon is not used because it would split into more items. Without a
translation, the result is the same as before.

Developed with assistance from Claude (Anthropic).
The strings of the # %module, # %flag, and # %option headers of Python
tools were collected by running g.parser -t during the build and writing
them to locale/scriptstrings/*_to_translate.c files for xgettext. The
resulting entries in grassmods.pot referred to these generated files, so
Weblate could not link to the source, and translators had no context.

Add locale/grass_tool_header_pot.py, which reads the headers directly
using only the Python standard library and writes a standard POT file:

- references point to the real source file and line,
- extracted comments tell translators what each string is (tool
  description, keyword, label of an option, GUI section, ...),
- descriptions of option values are separate entries with the
  "option value description" context, as g.parser now translates them.

make pot merges this POT into grassmods.pot with xgettext. Tools are found
by their Makefiles as in the build, which now also includes the g.gui.*
tools that were not extracted before.

Remove the scriptstrings rules from the Make and CMake builds. The
Windows launcher, which the CMake script also created, is now created
when configuring as for the GUI tools. With CMake, the scriptstrings
lines containing a semicolon were also split because CMake treats
semicolons as list separators.

Developed with assistance from Claude (Anthropic).
Generated with make pot. The entries from tool headers now refer to the
source files instead of locale/scriptstrings, have comments for
translators, and descriptions of option values are separate entries with
the "option value description" context. The g.gui.* tools are included.

Developed with assistance from Claude (Anthropic).
claude added 2 commits October 4, 2026 20:53
The tools in gui/scripts (d.rast3d, d.wms) are installed by a plain
Python Makefile without the script rules, so their interface strings were
not extracted. Add them to the extractor.

The script build rules also accepted shell scripts (ShScript.make, used
by addons) and files without an extension. Test that the extractor finds
and reads them, that header commands ignore case as in g.parser, and
that g.parser gives the same interface for a shell script as for a
Python script.

Developed with assistance from Claude (Anthropic).
STRINGDIR was passed so that building an addon with make would write
the strings for translation from its interface header into the temporary
source directory instead of the GRASS installation. The files were never
used and the build rules using STRINGDIR are now removed.

Developed with assistance from Claude (Anthropic).
pytest runs with --import-mode=importlib, which imported the locale/tests
directory as part of a package named locale. That package replaced the
standard library locale module for the following tests in the same
process, which then failed (module 'locale' has no attribute
'normalize'). Move the extractor and its tests to utils, next to the
other build helpers.

Developed with assistance from Claude (Anthropic).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants