Skip to content

Migrate dependency management from Pipenv to uv - #5417

Draft
letitz wants to merge 16 commits into
google:masterfrom
letitz:uv
Draft

Migrate dependency management from Pipenv to uv#5417
letitz wants to merge 16 commits into
google:masterfrom
letitz:uv

Conversation

@letitz

@letitz letitz commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

This PR completes the migration of ClusterFuzz's Python dependency management system from Pipenv to uv. pipx is used to install uv in developer and CI/bot environments.

Architecture & Dependency Management

  • Centralized pyproject.toml and uv.lock: Consolidated dependencies from root and App Engine Pipfiles into a single root pyproject.toml with [project.dependencies], [dependency-groups], and [project.optional-dependencies] (for vendored core, appengine, and platform-specific extras).
  • Public PyPI Configuration: Configured [[tool.uv.index]] with PyPI (https://pypi.org/simple) as the default index to ensure lockfile references and wheels point to public PyPI rather than internal mirrors.
  • Deterministic Tooling: Pinned pipx==1.10.0 and uv==0.12.3 across all setup scripts, container images, and CI configurations for setup determinism.
  • Compatibility & Standards:
    • Maintained package versions matching the original Pipfiles.
    • Added setuptools==80.9.0 for pkg_resources API compatibility with legacy packages.
    • Pinned isort~=5.13.2 and pylint~=2.17.7 in dev dependencies for consistent linting across local and CI environments.
    • Removed deprecated Pipfile, Pipfile.lock, src/Pipfile, src/Pipfile.lock, and redundant generated requirements.txt files.

Butler & Build Tooling

  • Vendoring with uv: Updated _install_third_party and _install_platform_pip in src/local/butler/common.py to use uv export --extra directly, eliminating intermediate requirements.txt generation.
  • App Engine Unit Testing: Updated src/local/butler/py_unittest.py to preserve src/third_party in sys.path while adding src/appengine/third_party, ensuring both App Engine and core libraries are cleanly importable during App Engine unit tests.
  • Environment Guards: Updated virtual environment guards in src/local/butler/guard.py and CLI runner in cli/casp/src/casp/utils/container.py to check for .venv and support uv run.

Setup Scripts, Bots & Container Images

  • Robust Path & Fallback Handling: Dynamically query $(pipx environment --value PIPX_BIN_DIR 2>/dev/null || echo "$HOME/.local/bin") to ensure uv is on PATH across all setup scripts (local/install_python_deps_linux.bash, local/install_deps_macos.bash, local/install_python_from_source.bash) and bot startup scripts (configs/test/bot/setup/*.bash).
  • Debian/Ubuntu Packaging: Added python3-venv to install_deps_linux.bash so Python virtual environments can be built without relying on global pip.
  • Windows GCE Setup: Updated configs/test/gce/windows-init.ps1 to install uv via pipx and append the pipx binary directory to $env:Path.
  • Base Docker Images: Updated base Dockerfiles (docker/base/Dockerfile, docker/base/ubuntu-20-04.Dockerfile, docker/base/ubuntu-24-04.Dockerfile) to install uv via pipx and symlink uv into /usr/local/bin.

CI Pipelines

  • Deduplicated CI Execution: Extracted the container-based CI test pipeline into local/tests/ci_tests_inner.bash, shared between Cloud Build (local/tests/cloudbuild_ci.bash / deploy/cloudbuild.yaml) and GitHub Actions (local/tests/ci_tests.bash / .github/workflows/build.yaml).
  • Fresh Checkout Integration Test: Added local/tests/install_deps_test_linux.bash to verify end-to-end repository installation and testing from a clean checkout.

Documentation

  • Updated AGENTS.md, local/README.md, .devcontainer/devcontainer.json, and developer documentation to reflect uv workflows (uv run butler.py ...).

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6

@letitz letitz left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass at self-review.

Comment thread configs/test/gce/windows-init.ps1
Comment thread configs/test/gce/windows-init.ps1 Outdated
Comment thread docker/base/Dockerfile Outdated
Comment thread docker/base/Dockerfile Outdated
Comment thread local/tests/ci_tests.bash Outdated
Comment thread local/install_python_deps_linux.bash
Comment thread src/local/butler/py_unittest.py Outdated
Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 10, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
Comment thread local/install_deps_linux.bash Outdated
Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 10, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
Comment thread local/README.md Outdated
Comment thread AGENTS.md Outdated
Comment thread local/tests/kubernetes_e2e_test.bash Outdated
Comment thread .devcontainer/devcontainer.json Outdated
Comment thread pyproject.toml Outdated
Comment thread pyproject.toml Outdated
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 10, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
@letitz
letitz force-pushed the uv branch 2 times, most recently from 98c2257 to 735f644 Compare August 10, 2026 20:38
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 11, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 14, 2026
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 14, 2026
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
letitz added a commit to letitz/clusterfuzz that referenced this pull request Aug 14, 2026
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
Comment thread local/install_python_deps_linux.bash
Comment thread configs/test/bot/setup/android.bash Outdated
Comment thread configs/test/bot/setup/linux.bash Outdated
Comment thread configs/test/bot/setup/mac.bash Outdated
Comment thread configs/test/gce/windows-init.ps1 Outdated
Comment thread docker/base/Dockerfile Outdated
Comment thread docker/base/ubuntu-20-04.Dockerfile Outdated
Comment thread docker/base/ubuntu-24-04.Dockerfile Outdated
Comment thread local/install_deps_macos.bash Outdated
Comment thread local/install_python_deps_linux.bash Outdated
Comment thread local/install_python_from_source.bash Outdated
@letitz
letitz force-pushed the uv branch 2 times, most recently from 5ea2bc4 to ac6bf28 Compare August 14, 2026 03:45
letitz added 15 commits August 14, 2026 15:48
This commit completes the migration of ClusterFuzz's Python dependency
management system from Pipenv to uv. pipx is used to install uv locally.

Architecture & Configs:
- Created root pyproject.toml with 5 dependency groups, centralizing
  dependency definitions in a single file (from two Pipfiles) and removing
  the need for the hack where appengine packages were modelled as dev
  dependencies in src/Pipfile.
- Added setuptools==80.9.0 to project dependencies for pkg_resources API support.
- Pinned isort~=5.13.2 and pylint~=2.17.7 in dev dependencies to support line length options.
- Shared pinned Google client dependencies in vendor-appengine to avoid version mismatches with vendor-core.
- Deleted legacy Pipfile* and requirements.txt files.

Butler & Runtime Improvements:
- Updated _install_third_party in src/local/butler/common.py to use uv
  and avoid materializing requirements.txt into the repo, which are
  duplicative of `uv.lock` contents anyway.
- Updated src/local/butler/py_unittest.py to preserve src/third_party in
  sys.path while adding src/appengine/third_party, ensuring both App Engine
  and core libraries are importable during App Engine unit tests. This
  fixes an issue where the previous setup assumed third-party packages
  were not only vendored into third_party directories, but also
  installed into the virtual environment.
- Updated virtual environment guards in src/local/butler/guard.py and CASP
  CLI wrapper in cli/casp/src/casp/utils/container.py.

Scripts, CI, Docker & Docs:
- Updated setup scripts in local/ to install pipx and uv and activate .venv.
- Updated CI workflows to do the same.
- Updated Docker build scripts and Dockerfiles similarly.
- Updated bot setup scripts (configs/test/bot/setup/*.bash, windows-init.ps1).
- Updated developer and agent documentation.
- Updated local/tests/install_deps_test_linux.bash to verify fresh checkout
  environment setup with uv (.venv, third_party vendoring, bower components,
  and uv run butler.py lint / py_unittest).
TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
- pyproject.toml: Removed duplicate root dependencies already in vendor-core (cryptography, future, google-cloud-ndb, protobuf, psutil, crcmod); added conditional pywinauto dependency for win32 under dev group.
- windows-init.ps1: Reverted redundant PATH modification (pipx ensurepath handles PATH); updated uv sync to use --frozen.
- Dockerfiles: Removed redundant --upgrade and --no-deps flags from uv pip install -e /data.
- local/install_deps_linux.bash: Made optional gcloud apt packages conditional on rodete and added b/414408644 and b/484368884 references.
- local/install_deps_macos.bash: Removed redundant pipx check (handled by Brewfile); added --frozen to uv sync.
- local/install_python_deps_linux.bash: Removed extra source .venv/bin/activate call after nodeenv.
- src/local/butler/py_unittest.py: Removed unnecessary site.addsitedir call.
- CI/Bot setup scripts: Added --frozen flag to all automated uv sync invocations.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
- pyproject.toml: Moved setuptools==80.9.0 to [dependency-groups] dev.
- local/tests/ci_tests.bash & setup scripts: Reverted --frozen flag from uv sync.
- local/install_deps_linux.bash: Removed || true from apt-get install on rodete.
- Dockerfile: Added explanatory comment for uv and python3.11 symlinks.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
- pyproject.toml: Moved vendor-core, vendor-appengine, and vendor-platform-specific to [project.optional-dependencies] extras: core, appengine, platform-specific. Added comment explaining special handling for platform-specific binary wheel downloads via pip download in common.py _install_platform_pip.
- src/local/butler/common.py: Updated _install_third_party and _install_platform_pip to use uv export --extra.

TAG=agy
CONV=e3e8364e-3aff-4a79-9c78-17378c583b06
- local/README.md, AGENTS.md, kubernetes_e2e_test.bash, devcontainer.json: Removed redundant explicit pipx/uv installation steps as local/install_deps.bash handles uv setup.
- pyproject.toml: Removed redundant packages from appengine optional dependencies group that are already included in core. Updated comment on platform-specific extras to clarify vendoring behavior.
- uv.lock: Updated lockfile.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
…ests.bash

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
- Configured [[tool.uv.index]] with PyPI (https://pypi.org/simple) as the default index in pyproject.toml.
- Regenerated uv.lock so all package sources and wheel URLs point to PyPI/files.pythonhosted.org rather than Google internal pkg.dev.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
The pre-built ci docker image does not have uv installed. Install
pipx and use it to install uv before executing the CI test pipeline.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
Avoid Cloud Build substitution parsing errors on shell environment variables
by extracting the CI setup and test invocation into a dedicated script.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
Parameterize local/tests/ci_tests_inner.bash to accept the test runner
command (defaulting to local/tests/run_tests) and reuse it in cloudbuild.yaml.

TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
TAG=agy
CONV=1906a716-4a06-47e6-b2fb-fc2038c78ef6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant