feat(gcp): Dataproc Metastore services - #1251
Merged
Merged
Conversation
Add the metastore.googleapis.com/v1 services control plane (GA /v1/): Create/ Get/List/Patch/Delete, all LRO-wrapped and resolved inline via the shared lro registry with the typed Service Any response. Server fills the Computed defaults (port, tier, network, state, uid, endpointUri, artifactGcsUri) and the provider-defaulted database_type/release_channel, minted once at create and stable across reads so terraform stays drift-free. Nested telemetryConfig/ hiveMetastoreConfig are Optional non-Computed blocks, carried through verbatim and never injected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New greenfield GCP control-plane service: Dataproc Metastore (
metastore.googleapis.com/v1, resource typeservices). Track-1 control-plane emulation only (no Hive data plane). Realgoogle.golang.org/api/metastore/v1clients, gcloud, and the Terraformgoogle_dataproc_metastore_serviceresource hit it unchanged.Provider tier: GA.
google_dataproc_metastore_serviceis in the STABLEhashicorp/googleprovider at the default/v1/path (verified live with provider v8.1.0); themetastore/v1SDK confirms the GA surface. No beta path needed.What's covered
POST .../services?serviceId=·GET(item + list) ·PATCH ?updateMask=·DELETE, plus location-scopedoperationspolls.google.longrunning.Operationdone:truesynchronously, resource embedded inresponseas an Any typedtype.googleapis.com/google.cloud.metastore.v1.Service— no LRO hang. Operations flow through the sharedlro.Registry;Matchesyieldsoperationspaths to the shared poller so no sibling GCP service regresses.?serviceId=(bodynamefallback); 409 dup / 404 missing / 400 bad input.updateMasktop-level replace, unmentioned fields survive.Snapshottablecompile-asserted and auto-discovered bypersistcompleteness.Computed-field stability (drift prevention)
name,state(seeded ACTIVE),stateMessage,uid,endpointUri(deterministic Thrift endpoint from name+port),artifactGcsUri(deterministicgs://…/hive-metastore),createTime/updateTimeare minted once at create, stored, and stable across GETs. The Computedport(9083) and provider-defaulteddatabaseType(MYSQL) /releaseChannel(STABLE) /tierare filled when omitted.Per the provider schema (
terraform providers schema -json),telemetryConfigandhiveMetastoreConfigare Optional, non-Computed nested blocks — so they are carried verbatim and deliberately not server-injected; injecting them was the sole plan-drift source and is avoided.Live e2e (Terraform, hashicorp/google STABLE, default path)
Ran a real
cloudemu serve --gcp-port 18050+terraform(provider v8.1.0) against agoogle_dataproc_metastore_servicewithservice_id/location/tier=DEVELOPER, leaving port/database_type/release_channel/telemetry/hive UNSET:apply→ created, LRO did not hang,state = ACTIVE.plan -detailed-exitcode→ exit 0, no drift, specifically on the defaulted/computedport,database_type,release_channel,state,endpoint_uri,artifact_gcs_uri,uid,create_time.labels+maintenance_window) → in-place PATCH applied; re-planexit 0 clean.destroy→ removed; endpoint returns 404.Gates
go build, fullgo vet,gofmt,golangci-lint(0 issues),go test -raceon new packages, fullgo test ./server/gcp/...(no sibling operations regression),./providers/...,./persist/...(completeness), root., andgo generateidempotency — all green. Docs regenerated (metastore-only diff).Deferred
metadataImports, backups, federations subcollections, IAM policy verbs, and the real Hive metastore data plane.