Skip to content

Share Python extension localization tooling and submission action - #125

Merged
Bill Schnurr (bschnurr) merged 1 commit into
mainfrom
users/bschnurr/shared-localization
Nov 3, 2026
Merged

Bill Schnurr (bschnurr) merged 1 commit into
mainfrom
users/bschnurr/shared-localization

Conversation

@bschnurr

@bschnurr Bill Schnurr (bschnurr) commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Centralize English export/submission and staged release imports in one standard-library Python CLI.
  • Expose a pinned composite action for thin public extension workflows; Pylance-Eng invokes the same importer from an explicit tooling checkout.
  • Derive canonical IDs and optional runtime metadata. Support manifest-only extensions generically; the autopep8 caller now explicitly enables runtime localization as requested.
  • Require translation evidence independently in every required real-language manifest/runtime table; allow individual untranslated entries and validate everything before placement.

Verification

  • 29 dependency-free regression tests, including actual bare-Git dev-only submission/no-op fixtures, failure cases, and bounded push retry.
  • Black, isort, flake8; Python 3.10 syntax and executable guards reading the actual action/Azure YAML.
  • Executed export/submission commands read from the composite action against a real bare remote.
  • Actual pinned-CLI English exports succeeded for all eight extensions (debugger 75 runtime strings, environments 456; autopep8 now extracts 3 runtime strings).

Dependencies and release blockers

This is the shared dependency for Pylance-Eng PR 801431, vscode-isort PR 777, and seven sibling migrations. Consumers pin the same reviewed tooling/action revision; no runtime library adoption is required.

Keep the rollout draft until credentials/onboarding and handbacks are ready. Current drop validation rejects English-only isort, missing environments output, and missing runtime tables for flake8/mypy/black. Debugger, pylint, and autopep8 pass current manifest/runtime handback readiness checks. Autopep8 now enables its three existing runtime code strings. Do not merge or publish as part of this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member Author

Live non-publishing release verification is complete for the three extensions whose current handbacks are ready:

All three exact-feature builds succeeded with publishing explicitly disabled and the publication stage skipped. Four actual VSIXs were inspected: universal artifacts for all three and debugger win32-x64. Each contains all 28 localization files and genuine German manifest/runtime translations. Debugger's full eight-platform build succeeded; autopep8's new runtime l10n wiring is verified.

Release blockers remain for isort (English-only handbacks), environments (missing canonical handback; native definitions disabled), and flake8/mypy/black (missing runtime tables). Seven siblings' GitHub localization credential access still needs owner confirmation/setup. Autopep8 also has the documented pre-existing Python 3.10 hashed-requirements CI failure.

Merge order: common-LSP #125, Pylance-Eng #801431, then update all eight consumer PRs to merged refs and merge only after their prerequisites are ready. No merges or extension publication were performed.

@bschnurr
Bill Schnurr (bschnurr) marked this pull request as ready for review October 30, 2026 18:03
@rchiodo

Rich Chiodo (rchiodo) commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

🔒 Automated review in progress — Rich Chiodo (@rchiodo) is auto-reviewing this PR.

@rchiodo

Copy link
Copy Markdown
Contributor

Result: ⚠️ partially-verified

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: All 29 localization regression tests passed in the container, including real bare-Git submission and no-op fixtures. Flake8, Black, and isort passed; three additional checks passed for genuine competing pushes, retry limits, and action/Python 3.10 syntax guards. No PR-caused failure was observed. Verification is partial because the pinned localization CLI was unavailable in the offline cache, preventing real extraction/import verification and end-to-end action execution. **Reviewer dependency follow-up (nonblocking):** Suggest a separate PR adding `@vscode/l10n-dev@0.0.35` to the official reviewer dependency declarations, using `scripts/localization.py` as the dependency/version evidence. Do not automatically create that PR. This is reviewer environment maintenance, not a required change to the reviewed PR.

Test runs: 4 passed, 1 not run

  • ⚠️ Not run | Pinned localization CLI offline preparation | npm exec --offline --yes --package=@vscode/l10n-dev@0.0.35 -- l10n-dev --help
  • ✅ Passed | Localization regression suite | python -m unittest discover -s scripts -p '*_test.py' -v
  • ✅ Passed | Localization lint and formatting checks | python -m flake8 scripts/localization.py scripts/localization_test.py && python -m black --check scripts/localization.py scripts/localization_test.py && python -m isort --check scripts/localization.py scripts/localization_test.py
  • ✅ Passed | Dependency and test discovery | printf 'Sandbox profile: %s\n' "$AUTOMATION_SANDBOX_PROFILE"; python --version; git --version; node --version; npm --version; git status --short; git log -1 --oneline; python - <<'PY'
    import importlib.util
    import pathlib
    for name in ('unittest', 'black', 'isort', 'flake8', 'yaml'):
    print(f'{name}: {importlib.util.find_spec(name) is not None}')
    print('Declared Python environment:')
    print(pathlib.Path('python/pyproject.toml').read_text())
    print('Localization test methods:')
    import ast
    tree = ast.parse(pathlib.Path('scripts/localization_test.py').read_text())
    for cls in tree.body:
    if isinstance(cls, ast.ClassDef):
    for method in cls.body:
    if isinstance(method, ast.FunctionDef) and method.name.startswith('test_'):
    print(f'{cls.name}.{method.name}')
    PY
    npm cache ls '@vscode/l10n-dev'
  • ✅ Passed | Additional concurrency, retry, and action guard verification | python - <<'PY'
    import ast
    import os
    import pathlib
    import subprocess
    import sys
    import unittest
    from unittest import mock
    import yaml
    sys.path.insert(0, str(pathlib.Path('scripts').resolve()))
    import localization
    from localization_test import SubmissionTests

class AdditionalVerification(SubmissionTests):
def test_real_competing_push(self):
competitor = self.base / 'competitor'
subprocess.run(['git', 'clone', '--quiet', '--branch', 'dev', str(self.remote), str(competitor)], check=True)
localization.git(competitor, 'config', 'user.name', 'Competitor')
localization.git(competitor, 'config', 'user.email', 'competitor@example.invalid')
original_run = subprocess.run
raced = False
rejected = []
def concurrent_push(command, **kwargs):
nonlocal raced
if command[:3] == ['git', 'push', 'origin'] and kwargs.get('cwd') == self.drop and not raced:
raced = True
(competitor / 'another-extension.xlf').write_text('<xliff/>\n')
localization.git(competitor, 'add', 'another-extension.xlf')
localization.git(competitor, 'commit', '-m', 'Concurrent extension update')
localization.git(competitor, 'push', 'origin', 'dev')
result = original_run(command, **kwargs)
rejected.append(result)
return result
return original_run(command, **kwargs)
with mock.patch.object(localization.subprocess, 'run', side_effect=concurrent_push):
self.submit()
self.assertTrue(raced)
self.assertEqual(len(rejected), 1)
self.assertNotEqual(rejected[0].returncode, 0)
self.assertIn('[rejected]', rejected[0].stderr)
self.assertEqual(self.git('rev-parse', 'origin/dev'), self.git('rev-parse', 'HEAD'))
self.assertTrue((self.drop / 'another-extension.xlf').exists())
self.assertTrue((self.drop / 'src/en/ms-python.tool/ms-python.tool-new.xlf').exists())
self.assertEqual(self.git('rev-parse', 'origin/main'), self.main)

def test_retry_limit_and_non_rejection_failure(self):
    original_run = subprocess.run
    for reason, expected in [(&#x27;! [rejected] HEAD -&gt; dev (fetch first)&#x27;, 3), (&#x27;fatal: authentication failed&#x27;, 1)]:
        with self.subTest(reason=reason):
            attempts = []
            def failed_push(command, **kwargs):
                if command[:3] == [&#x27;git&#x27;, &#x27;push&#x27;, &#x27;origin&#x27;]:
                    attempts.append(command)
                    return subprocess.CompletedProcess(command, 1, &#x27;&#x27;, reason)
                return original_run(command, **kwargs)
            (self.output / &#x27;ms-python.tool-new.xlf&#x27;).write_text(f&#x27;&lt;xliff&gt;{expected}&lt;/xliff&gt;\n&#x27;)
            with mock.patch.object(localization.subprocess, &#x27;run&#x27;, side_effect=failed_push):
                with self.assertRaises(subprocess.CalledProcessError):
                    self.submit()
            self.assertEqual(len(attempts), expected)
            self.git(&#x27;reset&#x27;, &#x27;--hard&#x27;, &#x27;origin/dev&#x27;)

class WiringVerification(unittest.TestCase):
def test_python_310_and_action_guards(self):
for path in ('scripts/localization.py', 'scripts/localization_test.py'):
ast.parse(pathlib.Path(path).read_text(), feature_version=(3, 10))
path = pathlib.Path('.github/actions/localization/action.yml').resolve()
action = yaml.safe_load(path.read_text())
self.assertEqual(action['runs']['using'], 'composite')
self.assertTrue((path.parent / '../../../scripts/localization.py').resolve().is_file())
steps = {step['name']: step for step in action['runs']['steps']}
validation = steps['Validate localization credentials']['run']
for app_id, key in [('', ''), ('123', ''), ('', 'secret')]:
with mock.patch.dict(os.environ, {'LOCALIZATION_APP_ID': app_id, 'LOCALIZATION_PRIVATE_KEY': key}):
with self.assertRaises(ValueError):
exec(validation, {})
with mock.patch.dict(os.environ, {'LOCALIZATION_APP_ID': '123', 'LOCALIZATION_PRIVATE_KEY': 'secret'}):
exec(validation, {})
for name, operation in [('Extract English strings', 'export'), ('Submit changed English XLF', 'submit')]:
command = steps[name]['run']
self.assertIn(f'--operation {operation}', command)
self.assertIn('"$LOCALIZATION_ACTION_PATH/../../../scripts/localization.py"', command)
self.assertNotIn('${{', command)
print('Python 3.10 syntax, action helper path, credential guards, and command wiring validated.')

suite = unittest.TestSuite([
AdditionalVerification('test_real_competing_push'),
AdditionalVerification('test_retry_limit_and_non_rejection_failure'),
WiringVerification('test_python_310_and_action_guards'),
])
result = unittest.TextTestRunner(verbosity=2).run(suite)
sys.exit(not result.wasSuccessful())
PY

⚠️ Pinned localization CLI offline preparation diagnostic output
Selected Node v24.17.0/npm 11.13.0; Python checks used python 3.13.18. scripts/localization.py:15 declares CLI = "@vscode/l10n-dev@0.0.35". Offline preparation exited 1:
npm error code ENOTCACHED
npm error request to https://packagefeedproxy.microsoft.io/npm/@vscode%2fl10n-dev failed: cache mode is 'only-if-cached' but no cached response is available.
Network installation is prohibited by sandbox policy. Real CLI extraction/import checks could not proceed.

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@rchiodo Rich Chiodo (rchiodo) added the review-auto:approved Automated review: no blocking findings (approval posted). label Oct 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

Eduardo Villalpando Mello (edvilme) pushed a commit to microsoft/vscode-python-ci that referenced this pull request Oct 30, 2026
…lation checkout support

## Shared pull-only release localization
This PR replaces the unused unmerged Azure English extraction/submission machinery with an opt-in pullLocalization interface on both public release wrappers. It invokes the same Python core as the public GitHub composite action in microsoft/vscode-common-python-lsp#125 rather than maintaining a second importer.

- Consumers declare translations (private drop dev) and localizationTools (reviewed immutable common-LSP SHA), with literal Github-Pylance endpoints.
- Source stays at workspace/s; tools and translations are sibling checkouts outside extension source.
- Run the shared importer after caller build/pre-built overlays and before final VSIX packaging.
- Keep the default self SDL scan, explicitly include executable tooling, and exclude translation data. Additional caller exclusions remain supported; sourceAnalysisPool preserves original caller pools.
- Derive canonical IDs and optional runtime metadata. Require all locales and per-table real-language translation evidence before copying; allow individual untranslated strings.
- No release-side English extraction, app write-token acquisition, Git pushes, or translation-update PRs. Existing signing/setup PowerShell is intentionally unchanged.
- Preserve opt-out callers, platform/version parameters, stable manual validation, and publication gates. Azure expansion can persist checkout credentials; no claim is made otherwise.

## Verification and rollout
Actual YAML-reading guards and shared Python/Git fixtures passed. Both release entry points for seven enabled repositories expanded successfully with publishing explicitly disabled. Both environments definitions are disabled; their exact YAML also expanded through the enabled isort definition without enabling or modifying those definitions. All expanded platform jobs preserve import order, source paths, SDL scanning, and publication gates. Shared core CI and the actual isort GitHub submission action passed.

Shared dependency: microsoft/vscode-common-python-lsp#125
Pilot: microsoft/vscode-isort#777
Sibling draft PRs:
- debugger: microsoft/vscode-python-debugger#1134
- environments: microsoft/vscode-python-environments#1971
- pylint: microsoft/vscode-pylint#915
- flake8: microsoft/vscode-flake8#609
- autopep8 (including runtime l10n): microsoft/vscode-autopep8#415
- mypy: microsoft/vscode-mypy#651
- black-formatter: microsoft/vscode-black-formatter#833

Keep this PR and callers draft pending dependencies, credential setup, and real handbacks. Current drop blocks isort (English-only), environments (missing canonical output), and flake8/mypy/black (missing runtime tables). Debugger, pylint, and autopep8 passed actual manifest/runtime imports from current drop. Autopep8 runtime ...
@bschnurr
Bill Schnurr (bschnurr) merged commit f6cd2fb into main Nov 3, 2026
24 of 25 checks passed
@bschnurr
Bill Schnurr (bschnurr) deleted the users/bschnurr/shared-localization branch November 3, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debt Code Cleanup, Refactorings and Repo health. Not feature related review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants