Skip to content

Add acceptance test for ssh connect to serverless CPU compute#5889

Merged
anton-107 merged 1 commit into
mainfrom
enable-acceptance-tests-for-ssh-connect-command-17
Jul 10, 2026
Merged

Add acceptance test for ssh connect to serverless CPU compute#5889
anton-107 merged 1 commit into
mainfrom
enable-acceptance-tests-for-ssh-connect-command-17

Conversation

@anton-107

Copy link
Copy Markdown
Contributor

Summary

Adds acceptance/ssh/connect-serverless-cpu/, an acceptance test for the bare databricks ssh connect invocation, which targets serverless CPU compute (no --cluster, no --accelerator).

This complements the existing acceptance/ssh/connect-serverless-gpu/ test. Compared to it:

  • Drops --accelerator → serverless CPU instead of GPU; the connection name is auto-generated (databricks-cpu-<hash>), so no --name is needed.
  • Drops the CloudEnvs.gcp = false exclusion — that exclusion exists only because serverless GPU is unavailable on GCP. Serverless CPU is available there.
  • Keeps RequiresUnityCatalog = true — serverless requires Unity Catalog.

Like both existing ssh tests, it passes --releases-dir=$CLI_RELEASES_DIR so the tunnel binaries come from the artifacts the harness builds on cloud runs rather than a GitHub download, and it keeps the on-failure jobs get-run diagnostic so any CI failure is actionable.

Test plan

  • Local run correctly skips (Local = false).
  • Verified end-to-end against a real workspace (aws-prod-ucws): the command provisions serverless CPU, uploads the CLI binaries, opens the tunnel, and runs the remote command — out.stdout.txt matches Connection successful.
--- PASS: TestAccept/ssh/connect-serverless-cpu/DATABRICKS_BUNDLE_ENGINE=direct

This pull request and its description were written by Isaac.


This PR was created with GitHub MCP.

Adds `acceptance/ssh/connect-serverless-cpu/`, exercising the bare
`databricks ssh connect` invocation, which targets serverless CPU
compute (no `--cluster`, no `--accelerator`).

This complements the existing serverless-GPU test. It drops the
`--accelerator` flag (CPU instead of GPU, with an auto-generated
connection name) and the `CloudEnvs.gcp = false` exclusion, since
serverless CPU is available on GCP. It keeps `RequiresUnityCatalog`
because serverless requires Unity Catalog.

Verified end-to-end against a real workspace (aws-prod-ucws): the
command provisions serverless CPU, uploads the CLI binaries, opens the
tunnel, and runs the remote command, producing "Connection successful".

Co-authored-by: Isaac
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 8f9377e

Run: 29029736360

Env 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 4 4 230 1078 6:05
💚​ aws windows 4 4 232 1076 7:33
💚​ aws-ucws linux 4 4 316 995 6:19
💚​ aws-ucws windows 4 4 318 993 7:12
💚​ azure linux 4 4 230 1077 4:43
💚​ azure windows 4 4 232 1075 7:05
💚​ azure-ucws linux 4 4 318 992 6:20
💚​ azure-ucws windows 4 4 320 990 7:42
💚​ gcp linux 4 4 229 1079 4:53
💚​ gcp windows 4 4 231 1077 6:57
8 interesting tests: 4 RECOVERED, 4 SKIP
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 11 slowest tests (at least 2 minutes):
duration env testname
6:38 aws windows TestAccept
6:07 azure windows TestAccept
6:07 azure-ucws windows TestAccept
6:00 gcp windows TestAccept
5:57 aws-ucws windows TestAccept
2:58 gcp linux TestAccept
2:53 aws linux TestSecretsPutSecretStringValue
2:53 azure linux TestAccept
2:52 aws-ucws linux TestAccept
2:48 azure-ucws linux TestAccept
2:48 aws linux TestAccept

@anton-107 anton-107 requested review from rclarey and rugpanov July 10, 2026 10:58

@rugpanov rugpanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — adds an ssh connect serverless-CPU acceptance test, consistent with the existing connect-serverless-gpu / connection tests. Two non-blocking nits inline; both also apply to the sibling connect-serverless-gpu test, so they may be a pre-existing convention rather than something introduced here.

@@ -0,0 +1,6 @@
errcode $CLI ssh connect --releases-dir=$CLI_RELEASES_DIR -- "echo 'Connection successful'" >out.stdout.txt 2>LOG.stderr

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: under -useversion in a cloud run, the harness doesn't set CLI_RELEASES_DIR (acceptance_test.go only sets it when UseVersion == ""). Since scripts run with bash -u, expanding $CLI_RELEASES_DIR would fail before ssh connect runs, breaking release-version cloud runs. Non-blocking — the sibling connect-serverless-gpu test has the same pattern, so this may be pre-existing.

@@ -0,0 +1,8 @@
Local = false
Cloud = true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: with Cloud = true (not CloudSlow), this runs in -short cloud runs even though ssh connect provisions a serverless job and can wait up to the 10-minute startup timeout. Consider CloudSlow = true so short cloud runs skip it. Non-blocking — connect-serverless-gpu uses the same bare Cloud = true, so this may be intentional.

@anton-107 anton-107 added this pull request to the merge queue Jul 10, 2026
Merged via the queue into main with commit f441c84 Jul 10, 2026
22 checks passed
@anton-107 anton-107 deleted the enable-acceptance-tests-for-ssh-connect-command-17 branch July 10, 2026 14:33
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: f441c84

Run: 29100290780

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 4 4 529 1007 38:31
💚​ aws windows 4 4 486 1020 55:21
❌​ aws-ucws linux 9 1 6 4 2 989 817 115:42
❌​ aws-ucws windows 9 1 6 5 2 930 835 131:41
💚​ azure linux 4 4 523 1008 40:48
💚​ azure windows 4 4 480 1021 51:48
🔄​ azure-ucws linux 4 4 2 896 852 97:50
🔄​ azure-ucws windows 3 5 2 838 870 99:48
💚​ gcp linux 4 4 514 1013 48:57
🔄​ gcp windows 3 3 4 469 1026 59:58
29 interesting tests: 13 flaky, 9 FAIL, 4 RECOVERED, 2 SKIP, 1 KNOWN
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 💚​R 💚​R 🟨​K 🟨​K 💚​R 💚​R 🔄​f 🔄​f 💚​R 🔄​f
🔄​ TestAccept/bundle/deploy/mlops-stacks ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
💚​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
🔄​ TestAccept/bundle/resources/dashboards/change-name ✅​p ✅​p 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/dashboards/change-name/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/change-name/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/replace_existing 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/replace_existing/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/update_autoscaling 🙈​s 🙈​s ❌​F ❌​F 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F
❌​ TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F
🔄​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​s 🙈​s ✅​p 🔄​f 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_projects/update_display_name/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/postgres_synced_tables/recreate/DATABRICKS_BUNDLE_ENGINE=terraform 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🔄​f 💚​R 🙈​S 🙈​S 🔄​f 💚​R 🙈​S 🙈​S
🔄​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name/DATABRICKS_BUNDLE_ENGINE=direct 🔄​f ✅​p 🔄​f ✅​p
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestImportFileUsingContentFormatAuto ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
18:02 aws-ucws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
16:38 azure-ucws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
15:27 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
13:59 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:26 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
12:10 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:48 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:59 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
8:59 aws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:21 aws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
8:18 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:17 aws-ucws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:09 azure-ucws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:55 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:49 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct
7:45 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:40 azure-ucws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:34 aws-ucws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:27 aws-ucws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:23 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:12 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:59 gcp windows TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:59 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:56 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:26 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:04 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:04 azure windows TestAccept
6:04 azure-ucws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:03 aws windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
6:00 aws windows TestAccept
6:00 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:45 aws-ucws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
5:23 azure-ucws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:19 azure-ucws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:14 gcp linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
5:10 gcp linux TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
5:07 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:06 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:02 gcp windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
5:00 aws-ucws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:59 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:57 aws-ucws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:55 aws-ucws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
4:51 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:44 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:37 azure-ucws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
4:30 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:27 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
4:27 azure linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
4:23 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants