Skip to content

Use shared localization submission and pull-only release imports - #651

Merged
Bill Schnurr (bschnurr) merged 5 commits into
mainfrom
users/bschnurr/shared-localization
Nov 5, 2026
Merged

Bill Schnurr (bschnurr) merged 5 commits into
mainfrom
users/bschnurr/shared-localization

Conversation

@bschnurr

Copy link
Copy Markdown
Member

Summary

  • Use the shared Python localization core and composite action rather than copying helper scripts into this extension.
  • Keep GitHub English submission separate from pull-only DevDiv stable/pre-release imports.
  • Use drop branch dev and canonical extension ID ms-python.mypy-type-checker; remove tracked generated locale snapshots where present and keep generated files in VSIX selection.
  • Preserve existing runtime metadata, extraction sources, feeds, platform matrices, and version/validation behavior.
  • Validate readiness independently in every required real-language manifest/runtime table; allow individual untranslated entries and reject missing/wholly untranslated tables before packaging.

Shared dependencies

The GitHub action and Azure tooling checkout are pinned to d1877b84bb10ff7caa6bd12ab4e75498ba760790. Azure templates temporarily reference the shared feature branch for verification; switch to Pylance-Eng main after the dependency merges, before merging this caller.

Verification

Executable guards read the actual YAML and preserve existing triggers, build steps, publication defaults, validation contacts, version flags, and matrices. Both release YAMLs expanded with publishing explicitly disabled (environments used the enabled validation definition because its native definitions are disabled). Actual English export used the pinned CLI through the internal feed. Changed YAML/JSON formatting and whitespace checks passed.

Required setup and release blockers

Current canonical ms-python.mypy-type-checker handback lacks required runtime tables. Do not select the legacy ms-python.mypy alias.

No repository-level PYTHON_LOCALIZATION_APP_ID variable or PYTHON_LOCALIZATION_PRIVATE_KEY secret is currently visible here. Organization-level access could not be verified with current permissions. Configure or expose them through the owners; do not copy credentials between repositories. Authorize Github-Pylance for both release resources/pipelines.

Keep this PR draft while dependencies/setup/handbacks are pending. No merge or publication is requested.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr Bill Schnurr (bschnurr) added the debt Technical debt or repo cleanup label Oct 30, 2026
Eduardo Villalpando Mello (edvilme) pushed a commit to microsoft/vscode-python-ci that referenced this pull request Oct 30, 2026
…lation checkout support

## Shared pull-only release localization
This PR replaces the unused unmerged Azure English extraction/submission machinery with an opt-in pullLocalization interface on both public release wrappers. It invokes the same Python core as the public GitHub composite action in microsoft/vscode-common-python-lsp#125 rather than maintaining a second importer.

- Consumers declare translations (private drop dev) and localizationTools (reviewed immutable common-LSP SHA), with literal Github-Pylance endpoints.
- Source stays at workspace/s; tools and translations are sibling checkouts outside extension source.
- Run the shared importer after caller build/pre-built overlays and before final VSIX packaging.
- Keep the default self SDL scan, explicitly include executable tooling, and exclude translation data. Additional caller exclusions remain supported; sourceAnalysisPool preserves original caller pools.
- Derive canonical IDs and optional runtime metadata. Require all locales and per-table real-language translation evidence before copying; allow individual untranslated strings.
- No release-side English extraction, app write-token acquisition, Git pushes, or translation-update PRs. Existing signing/setup PowerShell is intentionally unchanged.
- Preserve opt-out callers, platform/version parameters, stable manual validation, and publication gates. Azure expansion can persist checkout credentials; no claim is made otherwise.

## Verification and rollout
Actual YAML-reading guards and shared Python/Git fixtures passed. Both release entry points for seven enabled repositories expanded successfully with publishing explicitly disabled. Both environments definitions are disabled; their exact YAML also expanded through the enabled isort definition without enabling or modifying those definitions. All expanded platform jobs preserve import order, source paths, SDL scanning, and publication gates. Shared core CI and the actual isort GitHub submission action passed.

Shared dependency: microsoft/vscode-common-python-lsp#125
Pilot: microsoft/vscode-isort#777
Sibling draft PRs:
- debugger: microsoft/vscode-python-debugger#1134
- environments: microsoft/vscode-python-environments#1971
- pylint: microsoft/vscode-pylint#915
- flake8: microsoft/vscode-flake8#609
- autopep8 (including runtime l10n): microsoft/vscode-autopep8#415
- mypy: microsoft/vscode-mypy#651
- black-formatter: microsoft/vscode-black-formatter#833

Keep this PR and callers draft pending dependencies, credential setup, and real handbacks. Current drop blocks isort (English-only), environments (missing canonical output), and flake8/mypy/black (missing runtime tables). Debugger, pylint, and autopep8 passed actual manifest/runtime imports from current drop. Autopep8 runtime ...
@bschnurr
Bill Schnurr (bschnurr) marked this pull request as ready for review November 2, 2026 23:33
@rchiodo

Rich Chiodo (rchiodo) commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

🔒 Automated review in progress — Rich Chiodo (@rchiodo) is auto-reviewing this PR.

@rchiodo

Copy link
Copy Markdown
Contributor

Result: 🔴 could-not-verify

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: [unavailable] Container verification could not start and local execution was not authorized for this PR HEAD: C:\Program Files\RedHat\Podman\podman.EXE --connection pyrx-automation failed: stderr: Traceback (most recent call last): File "/opt/automation-tools/sandbox_sources.py", line 107, in <module> _main() ~~~~~^^ File "/opt/automation-tools/sandbox_sources.py", line 103, in _main restore_submodules(arguments.cache, arguments.root, json.load(sys.stdin)) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/automation-tools/sandbox_sources.py", line 79, in restore_submodules raise ValueError( ...<2 lines>... ) ValueError: Submodule &#x27;external/vscode-common-python-lsp&#x27; at ea47d519798d121fea262aaae23f0571e8f3d6da is not in the trusted image cache. Publish an image containing that protected-base commit; network fetching is disabled.

Test runs: none recorded.

@rchiodo Rich Chiodo (rchiodo) added the review-auto:changes-requested Automated review: posted blocking findings to address. label Nov 3, 2026
@rchiodo

Copy link
Copy Markdown
Contributor

Result: 🔴 could-not-verify

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: [unavailable] Container verification could not start and local execution was not authorized for this PR HEAD: C:\Program Files\RedHat\Podman\podman.EXE --connection pyrx-automation failed: stderr: Traceback (most recent call last): File "/opt/automation-tools/sandbox_sources.py", line 107, in <module> _main() ~~~~~^^ File "/opt/automation-tools/sandbox_sources.py", line 103, in _main restore_submodules(arguments.cache, arguments.root, json.load(sys.stdin)) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/automation-tools/sandbox_sources.py", line 79, in restore_submodules raise ValueError( ...<2 lines>... ) ValueError: Submodule &#x27;external/vscode-common-python-lsp&#x27; at ea47d519798d121fea262aaae23f0571e8f3d6da is not in the trusted image cache. Publish an image containing that protected-base commit; network fetching is disabled.

Test runs: none recorded.

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@rchiodo Rich Chiodo (rchiodo) added review-auto:approved Automated review: no blocking findings (approval posted). and removed review-auto:changes-requested Automated review: posted blocking findings to address. labels Nov 3, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rchiodo

Copy link
Copy Markdown
Contributor

Result: 🔴 could-not-verify

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: [unavailable] Container verification could not start and local execution was not authorized for this PR HEAD: C:\Program Files\RedHat\Podman\podman.EXE --connection pyrx-automation failed: stderr: Traceback (most recent call last): File "/opt/automation-tools/sandbox_sources.py", line 107, in <module> _main() ~~~~~^^ File "/opt/automation-tools/sandbox_sources.py", line 103, in _main restore_submodules(arguments.cache, arguments.root, json.load(sys.stdin)) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/automation-tools/sandbox_sources.py", line 79, in restore_submodules raise ValueError( ...<2 lines>... ) ValueError: Submodule &#x27;external/vscode-common-python-lsp&#x27; at ea47d519798d121fea262aaae23f0571e8f3d6da is not in the trusted image cache. Publish an image containing that protected-base commit; network fetching is disabled.

Test runs: none recorded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rchiodo

Copy link
Copy Markdown
Contributor

Result: ⚠️ partially-verified

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: All 39 existing setup/feed tests passed in the Linux container. Targeted checks also passed for localization YAML wiring, formatting, and VSCE inclusion of all 28 generated locale files. The PR adds no tests. Shared dependency restoration hit an offline cache miss, and the pinned localization tooling, GitHub submission, Azure expansion, and real-handback readiness checks could not be exercised. Caller configuration is partially verified; end-to-end release localization remains unverified. **Reviewer dependency follow-up (nonblocking):** Suggest a separate PR adding `serialize-javascript@7.1.1 (locked)` to the official reviewer dependency declarations, using `external/vscode-common-python-lsp/typescript/package-lock.json` as the dependency/version evidence. Do not automatically create that PR. This is reviewer environment maintenance, not a required change to the reviewed PR.

Test runs: 5 passed, 2 not run

  • ⚠️ Not run | Pinned localization tooling and release integration
  • ⚠️ Not run | Offline Node dependency preparation | npm ci --offline --ignore-scripts > /tmp/mypy-npm-restore.log 2>&1; root_status=$?; tail -n 18 /tmp/mypy-npm-restore.log; printf '\nRoot npm restore exit=%s\n' "$root_status"; (cd external/vscode-common-python-lsp/typescript && npm ci --offline --ignore-scripts) > /tmp/mypy-shared-restore.log 2>&1; shared_status=$?; tail -n 18 /tmp/mypy-shared-restore.log; printf '\nShared npm restore exit=%s\n' "$shared_status"; test "$root_status" -eq 0 && test "$shared_status" -eq 0
  • ✅ Passed | Setup, registry and lockfile regression tests | npm run test:setup
  • ✅ Passed | Environment and test discovery | printf 'PROFILE=%s\n' "$AUTOMATION_SANDBOX_PROFILE"; uname -s; git status --short; git log -3 --oneline; git diff --stat HEAD^ HEAD; printf '\nTools and dependency probes\n'; command -v node npm python uv; node --version; npm --version; python -c "import sys, importlib.util; print('interpreter:', sys.executable); print({m: importlib.util.find_spec(m) is not None for m in ['pytest','yaml']})"; printf '\nDependency manifests and localization test discovery\n'; git ls-files 'requirements' 'lock' 'testlocal*' 'localtest*' 'pyproject'; git submodule status; ls external/vscode-common-python-lsp; test ! -d node_modules || printf 'node_modules exists\n'; git grep -n -i 'localization|l10n|handback' -- build 'test' ':!package-lock.json' || true
  • ✅ Passed | Localization caller contract checks | python - <<'PY'
    import json
    from pathlib import Path
    import yaml
    pin = 'd1877b84bb10ff7caa6bd12ab4e75498ba760790'
    w = yaml.safe_load(Path('.github/workflows/localization.yml').read_text())
    triggers = w.get('on', w.get(True))
    assert set(triggers) == {'push', 'workflow_dispatch'}
    assert triggers['push']['branches'] == ['main']
    assert w['permissions'] == {'contents': 'read'}
    assert w['concurrency']['cancel-in-progress'] is False
    job = w['jobs']['localization']
    assert job['if'] == "github.repository == 'microsoft/vscode-mypy'"
    step, = job['steps']
    assert step['uses'] == 'microsoft/vscode-common-python-lsp/.github/actions/localization@' + pin
    assert json.loads(step['with']['source-paths']) == ['src', 'external/vscode-common-python-lsp/typescript/src']
    assert step['with']['node-version'] == Path('.nvmrc').read_text().strip().lstrip('v')
    pkg = json.loads(Path('package.json').read_text())
    assert pkg['publisher'] + '.' + pkg['name'] == 'ms-python.mypy-type-checker'
    assert pkg['l10n'] == './l10n'
    assert Path('package.nls.json').is_file()
    ignore = Path('.gitignore').read_text().splitlines()
    assert '/package.nls..json' in ignore and '/l10n/' in ignore
    for channel in ('stable', 'pre-release'):
    p = Path(f'build/azure-devdiv-pipeline.{channel}.yml')
    doc = yaml.safe_load(p.read_text())
    repos = {r['repository']: r for r in doc['resources']['repositories']}
    assert repos['translations']['name'] == 'microsoft/vscode-python-extensions-loc'
    assert repos['translations']['ref'] == 'refs/heads/dev'
    assert repos['localizationTools']['ref'] == pin
    assert all(repos[r]['endpoint'] == 'Github-Pylance' for r in ('translations', 'localizationTools'))
    assert doc['extends']['template'] == f'azure-pipelines/extension/{channel}.yml@templates'
    params = doc['extends']['parameters']
    assert params['pullLocalization'] is True
    assert params['standardizedVersioning'] is True
    assert params['publishExtension'] == '${{ parameters.publishExtension }}'
    assert all(params[k] is True for k in ('ghCreateTag','ghCreateRelease','ghReleaseAddChangeLog'))
    assert params['customNPMRegistry'] == '$(NPM_CONFIG_REGISTRY)'
    assert params['sourceAnalysisPool'] == 'VSEng-MicroBuildVSStable'
    declared = {p['name']: p for p in doc['parameters']}
    assert declared['publishExtension']['default'] is True
    scripts = [s.get('script','') for s in declared['buildSteps']['default']]
    expected = 'python ./build/update_ext_version.py ' + ('--release ' if channel == 'stable' else '') + '--for-publishing'
    assert expected in scripts and scripts[-1] == 'npm run package'
    if channel == 'stable':
    assert params['validationNotifyUsers'] == 'plseng@microsoft.com'
    assert doc['trigger']['branches']['include'] == ['release/
    ']
    else:
    assert doc['trigger'] == 'none'
    assert doc['schedules'][0]['cron'] == '0 10 * * 1-5'
    print(f'PASS {p}: pull-only wiring, pins, version flags, publication and validation parameters')
    print('PASS GitHub submission wiring, extraction paths, canonical ID, English source and generated-file ignores')
    shared = Path('external/vscode-common-python-lsp')
    print('Shared localization-related files:', [str(p) for p in shared.rglob('*') if p.is_file() and ('localization' in p.name.lower() or 'l10n' in p.name.lower())][:50])
    for p in (shared/'typescript/package.json', shared/'python/pyproject.toml'):
    if p.exists():
    print(f'--- {p} ---')
    print(p.read_text())
    PY
  • ✅ Passed | Changed YAML formatting | node ./node_modules/prettier/bin/prettier.cjs --check .github/workflows/localization.yml build/azure-devdiv-pipeline.stable.yml build/azure-devdiv-pipeline.pre-release.yml
  • ✅ Passed | Generated localization files survive VSCE selection | node <<'JS'
    const fs = require('node:fs');
    const path = require('node:path');
    const assert = require('node:assert/strict');
    const { spawnSync } = require('node:child_process');
    const locales = ['cs','de','es','fr','it','ja','ko','pl','pt-br','qps-ploc','ru','tr','zh-cn','zh-tw'];
    const originals = new Map();
    const hadL10n = fs.existsSync('l10n');
    const expected = [];
    try {
    fs.mkdirSync('l10n', {recursive:true});
    for (const locale of locales) {
    for (const file of [package.nls.${locale}.json, l10n/bundle.l10n.${locale}.json]) {
    originals.set(file, fs.existsSync(file) ? fs.readFileSync(file) : null);
    fs.writeFileSync(file, JSON.stringify({'packaging-verification':'translated fixture'}));
    expected.push(file);
    }
    }
    const result = spawnSync(process.execPath, ['node_modules/@vscode/vsce/vsce', 'ls', '--no-dependencies'], {encoding:'utf8', timeout:45000});
    if (result.status !== 0) {
    console.error(result.stderr, result.stdout);
    throw new Error(vsce ls failed: ${result.status}; ${result.error || &#x27;&#x27;});
    }
    const files = new Set(result.stdout.split(/\r?\n/).map(f => f.replaceAll('\', '/').replace(/^.//,'')));
    assert(files.has('package.nls.json'), 'English manifest omitted');
    for (const file of expected) assert(files.has(file), ${file} omitted by VSCE);
    console.log(PASS actual VSCE file selection includes English plus ${expected.length} generated locale files across ${locales.length} locales despite Git ignores.);
    console.log('Selection-only check: fixture translations; no VSIX build, release import or publication performed.');
    } finally {
    for (const [file, original] of originals) {
    if (original === null) fs.unlinkSync(file); else fs.writeFileSync(file, original);
    }
    if (!hadL10n) fs.rmdirSync('l10n');
    }
    JS
⚠️ Pinned localization tooling and release integration diagnostic output
The restored submodule contains no localization-related files. The separately pinned GitHub action and Azure templates were unavailable for execution in the offline container. No English submission, Azure template expansion, real-handback validation, or localized release packaging was performed.
⚠️ Offline Node dependency preparation diagnostic output
Selected Node: v24.17.0; npm: 11.13.0. Inspected shared package metadata before setup; its checked-in package-lock.json governs restoration.
Root restore succeeded: added 650 packages; exit=0.
Shared restore failed: npm error code ENOTCACHED
request to https://packagefeedproxy.microsoft.io/npm/serialize-javascript/-/serialize-javascript-7.1.1.tgz failed: cache mode is 'only-if-cached' but no cached response is available.
Shared npm restore exit=1. No network retry or package substitution attempted.

@bschnurr
Bill Schnurr (bschnurr) enabled auto-merge (squash) November 5, 2026 00:11
@bschnurr
Bill Schnurr (bschnurr) merged commit 9d74859 into main Nov 5, 2026
21 checks passed
@bschnurr
Bill Schnurr (bschnurr) deleted the users/bschnurr/shared-localization branch November 5, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debt Technical debt or repo cleanup review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants