Improve the surface of pixi integration

[francbartoli]

Complete the migration to pixi for packages management, see #510

[justb4]

Quite some changes, I am working on many of the same files to get Docker working, but not pushed yet. Have you tested your changes? I see some that may cause failures:

• pyproject.toml: so folks can also use conda etc for local/non-Docker install?
• tasks.py: the setup function calls c.run('pip3 install -r requirements.txt') - not required anymore. Also call python i.s.o. python3.
• at some point invoke may be replaced by pixi tasks IMHO
• .gitignore: exclude .pixi dir
• I am trying to have the Docker image run as user ghc i.s.o. root
• pixi.lock needed?
• was thinking of moving install.sh directly to Dockerfile for clarity.
• my Docker image is still 1.3GB uncompressed, like to bring this down
• would like to investigate Multi-staged Docker build with pixi like in this article

We have quite some overlapping changes now, but I am glad things are moving in a modern direction! I'll give it some thinking what the best strategy is. If I merge your changes there will be quite some merge conflicts later with my local (non-pushed) version...

[francbartoli]

Quite some changes, I am working on many of the same files to get Docker working, but not pushed yet. Have you tested your changes? I see some that may cause failures:

• pyproject.toml: so folks can also use conda etc for local/non-Docker install?

yes, pixi run would be the common installation method from lock file

• tasks.py: the setup function calls c.run('pip3 install -r requirements.txt') - not required anymore. Also call python i.s.o. python3.

indeed task for installing requirements.txt is commented, I will remove. Call of python is already managed and wrapped by same pixi run -e prod (or dev) with the binary brought by that dependencies chain which can become a task as you point out in the next point to finally get rid of invoke. I would leave that change out of this PR.

• at some point invoke may be replaced by pixi tasks IMHO

agree, see above

• .gitignore: exclude .pixi dir

it is recommended to ignore .pixi dir unless its config.toml

• I am trying to have the Docker image run as user ghc i.s.o. root
• pixi.lock needed?

the lock file is the source of truth for pixi reproducibility

• was thinking of moving install.sh directly to Dockerfile for clarity.

feel free

• my Docker image is still 1.3GB uncompressed, like to bring this down

ok

• would like to investigate Multi-staged Docker build with pixi like in this article

+1 it was intentionally missed in this PR

We have quite some overlapping changes now, but I am glad things are moving in a modern direction! I'll give it some thinking what the best strategy is. If I merge your changes there will be quite some merge conflicts later with my local (non-pushed) version...

np, my intention was to push the usage of pixi with some useful patterns

[justb4]

I have pushed new changes to https://github.com/geopython/GeoHealthCheck/tree/510-pixi-deploy , taking some of @francbartoli conventions:

• using pyproject.toml (i.s.o pixi.toml)
• version pinning to get e.g. gunicorn+gevent working
• define pixi tasks from invoke tasks
• README changes
• .dockerignore with .pixi (caused weird errors...)
• not yet doc changes
• get rid of make dependency in Dockerfile

Main focus was:

• get Dockerfile building and running
• slimmer Docker Image
• get full app running with Docker Compose

The docker/scripts/install.sh file is integrated directly in Dockerfile, also for clarity.

Docker Image is now 772MB uncompressed, works with Compose and executing tests.

Only error is when calling DB Upgrade, e.g. pixi run -e prod upgrade-db, this looks like a version-clash with old Flask-SQLalchemy-Alembic versions with Python 3.12:

ghc_web      | Traceback (most recent call last):
ghc_web      |   File "/GeoHealthCheck/GeoHealthCheck/manage.py", line 53, in <module>
ghc_web      |     manager.run()
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/site-packages/flask_script/__init__.py", line 417, in run
ghc_web      |     result = self.handle(argv[0], argv[1:])
ghc_web      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/site-packages/flask_script/__init__.py", line 356, in handle
ghc_web      |     app_namespace, remaining_args = app_parser.parse_known_args(args)
ghc_web      |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 1914, in parse_known_args
ghc_web      |     return self._parse_known_args2(args, namespace, intermixed=False)
ghc_web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2188, in _parse_known_args
ghc_web      |     stop_index = consume_positionals(start_index)
ghc_web      |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2141, in consume_positionals
ghc_web      |     take_action(action, args)
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2018, in take_action
ghc_web      |     action(self, namespace, argument_values, option_string)
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 1272, in __call__
ghc_web      |     subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
ghc_web      |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 1914, in parse_known_args
ghc_web      |     return self._parse_known_args2(args, namespace, intermixed=False)
ghc_web      |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2188, in _parse_known_args
ghc_web      |     stop_index = consume_positionals(start_index)
ghc_web      |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2141, in consume_positionals
ghc_web      |     take_action(action, args)
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 2018, in take_action
ghc_web      |     action(self, namespace, argument_values, option_string)
ghc_web      |   File "/GeoHealthCheck/.pixi/envs/prod/lib/python3.12/argparse.py", line 1274, in __call__
ghc_web      |     setattr(namespace, key, value)
ghc_web      | AttributeError: 'NoneType' object has no attribute 'x_arg'
ghc_web      | Upgrading database...

[justb4]

Ok, the main.yml CI now also succeeds!