Repository navigation
Conversation
Proposal to add the option to point at an alternative addon server for MS-Window users.
|
You can test it using https://ecodiv.earth/share for 8.5.0 and 8.6.0dev |
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
echoix
left a comment
There was a problem hiding this comment.
If it’s something that essential to have, why would it be limited to Windows? Couldn’t it work for other platforms (to set a base url), even if they are t using pre-compiled addons?
Is tweaking around with env vars how we want this to work? It’s easier to implement, like done here.
For Linux it is already possible to provide an alternative URL, so this adds an option that is already available for Linux (for Mac, I don't know).
That is a good question. I don't think it is ideal, it is just the only way I could think of that would not require changes in the current interface, and which I thought would still be easy enough for users. |
Can windows be adapted to do it too if that’s enough?
|
You're right that this isn't fundamentally Windows-only. The Linux flow also has a default location (grass.osgeo.org/addons/ for the listing, the OSGeo grass-addons GitHub repo for sources) that could be redirected the same way using the ADDONS_BASE_URL. The main reasons this PR focusses on Windows is that the binary-distribution layout makes the implementation trivial there. A Linux equivalent has to deal with multiple source mechanisms (git/zip/svn/local), which makes it (for me at least) more complicated.
I am not sure if it is the best way, but it was the best way I could think of that does not mess with the current interface and is still easy enough for users. It is still not as easily discoverable as I would like, so perhaps a dedicated parameter in g.extension itself would be better? It still should, in my opinion, expose the list of addons available on the selected server when using g.extension through the menu. |
Yes, and this PR already does that, it also extends url= to Windows (for full .zip URLs, matching Linux's existing ZIP-URL handling). Beyond that, the reason I think the base-URL mechanism matters specifically for Windows is operational: addon provision depends on precompiled binaries built by essentially one maintainer (as far as I understand), and there have historically been extended periods where addons weren't being updated. An alternative-server option gives Windows users a real fallback in those situations, which is less of a concern on Linux, where builds are from source.
|
On Windows, a pre-compilation is needed for C/C++ addons. For python addons, "only" a related windows batch file (.bat) is needed. Would it make sense to integrate such a bat-file creation function e.g. e.g. in g.extension? then, at least, python addons would be available from source also for windows; only C addons would need a pre-compilation on a server. |
|
@echoix @hellik, what about limiting the scope to making the The idea of hellik would complement this. I still would like the option to set a different base-server (I agree it would be great to have this for Linux as well) but I can move that perhaps to an addon? |
Sure, go ahead with your approach! For the mid term, we should look at additional options. |
|
I propose to go ahead for now with this solution (I'll check the conflicts), so at least Window users have access to some of the addons (Python). Reason I am picking this up again and would like a solution, even if temporary and not ideal, is that teaching that depend on addons is basically impossible. |
|
If having this merged allows you to unlock your use case, now, let's just merge it (when conflicts are solved) |
echoix
left a comment
There was a problem hiding this comment.
Some contradictions in the docs, and the precedence of the variable vs settings the agent found (opus 5.5)
Details
Branch pr-7383 is checked out; reading the surrounding code in g.extension.py now to check how the new URL handling flows.
I've checked out PR #7383 locally as branch pr-7383. It changes 3 files under scripts/g.extension/ (.py, .md, .html). The idea is sound and the code change is small. But the PR has a file-mode regression, the documentation contradicts the code, the official server doesn't have the layout the docs promise, and the .md and .html files disagree.
I tested the server claims against the live servers. I didn't run the tool on Windows, and I haven't posted anything to GitHub.
Must fix
-
The script loses its executable bit.
g.extension.pygoes from mode100755to100644(upstreammainhas100755). This was probably caused by editing on Windows and needs to be reverted. -
The official WinGRASS server has no
modules.xmlortoolboxes.xmlat the documented path.https://wingrass.fsv.cvut.cz/grass84/addons/grass-8.4.1/modules.xmlreturns 404, and so doestoolboxes.xml. The directory has only ZIPs,.md5sumfiles andlogs/.- The docs say an alternative server should use "the same directory layout as the official WinGRASS server … containing … a
modules.xmllisting". That isn't true, so an exact mirror of the official server would install fine but fail ong.extension -l. - The docs should state that
modules.xmlis an extra requirement. - With the override set,
-l -tand-tinstalls also needtoolboxes.xml, which the docs don't mention.
- The docs say an alternative server should use "the same directory layout as the official WinGRASS server … containing … a
-
The
url=documentation contradicts the code.- The code (
install_extension_win) now requiresurlto end in.zip, so the docs' own exampleurl=http://example.com/.../r.example?format=zipis rejected on Windows. - The new text after that example says
url=works on Windows "when pointing at a server with pre-compiled addons and … the same directory layout". That's wrong: the code downloadsurldirectly as a single precompiled ZIP and never treats it as a server base. - The text should say that on Windows
urlmust be a ZIP of a precompiled addon. A source ZIP from GitHub won't work.
- The code (
-
The
.mdand.htmlfiles disagree on the alternative server URL.- The
.mduseshttps://ecodiv.earth/share/AddonServer, which works:…/AddonServer/grass85/addons/grass-8.5.0/modules.xmlreturns 200. - The
.htmluseshttps://ecodiv.earth/sharein the server list and in theg.gisenvexample. That's wrong:…/share/grass85/addons/returns 404. - The
cmd.exeexample uses the wrongshareURL in both files.
- The
Should fix
- The AUTHORS entry renders on the same line as the one before it. The
.mdline before the new author is missing its two trailing spaces (the line break), while the.htmlhas a<br>. - Typos and inconsistencies in the docs:
- "settings explained in this section are is honoured" ("are is" is a typo).
- "redirect the addon server … once per session" contradicts "persists across GRASS sessions" two lines later.
- There's a Unicode em dash ("—") where the style guide prefers ASCII.
- The NOTES section now says both "only the official repository is working" and "it is now possible to point at an alternative server". The old sentence should be rewritten rather than appended to.
- The precedence order is questionable. The persistent gisenv setting overrides the shell variable
GRASS_ADDONS_BASE_URL. Usually it's the reverse: the short-lived, more specific environment variable wins, which is what you'd want for the CI and one-off cases the docs describe. Worth asking the author why. except Exceptioninget_addons_base_url_override.g.extensionalways runs inside a session, so this broad catch (with anoqa) mostly hides real errors. Either catch only the specific error or drop thetry.get_addons_base_url()adds little. It has one caller and is a one-lineor, so it could be inlined.
Minor / for discussion
- Behaviour change: on Windows,
url=used to be silently ignored, so the official addon was installed. Now a GitHub or source URL fails with an error. That's better than silently ignoring it, but it should be mentioned in the PR description. - With
-tandurl=on Windows, every addon in the toolbox downloads the same ZIP. Toolbox installs on Windows were already broken before this PR (xmlurlis never passed), so this is low priority. - No tests:
addons_server_url()and the precedence logic could get a small pytest intests/, for example usingmonkeypatchonGRASS_ADDONS_BASE_URLwithsys.platformpatched. - The commit history includes "some tweaks" and two merges from
main, so it should be squash-merged.
Do you want me to post this as a review comment on the PR, or prepare fixup commits for items 1–6?
- Clarified that an alternative server needs the ZIPs plus modules.xml (and toolboxes.xml for -t) - Rewrote the url= section: on Windows url must point at a .zip of a single precompiled addon; use - ADDONS_BASE_URL to switch servers. - Fixed the .html to use https://ecodiv.earth/share/AddonServer. - few other fixes In code: - Flipped precedence, now GRASS_ADDONS_BASE_URL (env var) wins over the ADDONS_BASE_URL gisenv setting. - Dropped the broad except Exception in get_addons_base_url_override() and inlined the one-line get_addons_base_url() wrapper. - Behaviour change: on Windows url= was previously ignored; it now errors on a non-.zip URL. - File-mode regression reverted
|
I think/hope I addressed all points @echoix |
| Markus Neteler (original shell script) | ||
| Markus Neteler (original shell script) | ||
| Martin Landa, Czech Technical University in Prague, Czech Republic | ||
| (Python rewrite) | ||
| (Python rewrite) |
There was a problem hiding this comment.
Doesn't this put the lines together?
There was a problem hiding this comment.
@wenzeslaus @echoix the automatic check suggest to remove the spaces, but it does indeed put the lines together. So keeping the spaces (as is was)?
- Changed wording in code comment to reflect better that this new option only works for GRASS on MS-Windows - added back double space after each author so they end up on a separate line.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Motivation
On MS-Windows, g.extension downloads precompiled addon ZIPs from a hardcoded URL (
http://wingrass.fsv.cvut.cz). There is no way to point Windows users at a different server. The url= option is documented as not supported on Windows, and the WinGRASS host is baked intoinstall_extension_win(). The listing URL is similarly hardcoded tohttps://grass.osgeo.org/addons/<branch>/. Consequence:This PR adds two settings that redirect both listing and install to a custom server, plus a per-call
url=for one-off ZIP installs. No new parser options are introduced, and the default behavior is unchanged. For Linux users, there should be no change at all.Changes
New:
get_addons_base_url()helper, which resolves the base URL of the addon server with the following precedence:ADDONS_BASE_URLset viag.gisenv set="ADDONS_BASE_URL=..."(persists across sessions, settable from the GUI's variables panel).GRASS_ADDONS_BASE_URLshell environment variable (useful for CI / one-off shell sessions).Built-in default
http://wingrass.fsv.cvut.cz.Guards against
gs.gisenv()failing outside an active GRASS session.install_extension_win()honors the resolver andurl="http://wingrass.fsv.cvut.cz/"base is replaced byget_addons_base_url().urlparameter for Window users. Ifurl=is given, it must be a full ZIP URL ending in.zip. The zip file must reside on a server with pre-compiled add-ons with the same structure as the default server.ADDONS_BASE_URL/GRASS_ADDONS_BASE_URL. This avoids platform-dependent reinterpretation of the same url= value.What does not change
Server-side
A custom server is expected to mirror the official WinGRASS layout:
I have an example at
https://ecodiv.earth/share. This contains Python scripts only. These were created by pulling the grass-addon repository and using a custom script to create the zipped addon files.Testing and AI
I used AI to help me write and test the proposed changes. I tested the script on Linux (Ubuntu 24.04) and Windows 11. On Linux, no changes in behavior or options. On Windows, I installed addons setting
ADDONS_BASE_URL=https://ecodiv.earth/shareand by using theurl=..pointing directly at the zip file.