Skip to content

fix: re-parsing an unchanged connector no longer restarts everything downstream - #10048

Merged
nishantmonu51 merged 1 commit into
mainfrom
nishant/sort-connector-templated-properties
Oct 21, 2026
Merged

nishantmonu51 merged 1 commit into
mainfrom
nishant/sort-connector-templated-properties

Conversation

@nishantmonu51

Copy link
Copy Markdown
Collaborator
  • analyzeTemplatedProperties built a connector's TemplatedProperties by ranging over a Go map. A connector with two or more templated properties (for example dsn and write_dsn) got them in a random order on every parse.
  • The project parser compares specs with proto.Equal, which is order-sensitive for repeated fields. So a full re-parse of an unchanged connector file could update its spec, which marks every resource downstream of the connector as pending and cancels the running ones. Long model runs restarted from scratch, and a manual full refresh that takes longer than the gap between parses could never finish.
  • The connector's own reconcile was a no-op, since its execution hash doesn't depend on the order. The damage came from the downstream cascade.
  • The fix sorts the property names. Affected connectors get one last spec update on the first parse after upgrading.

Steps to reproduce:

  1. Deploy a project with a connector that has two or more templated properties, for example dsn: "{{ .env.dsn }}" and write_dsn: "{{ .env.write_dsn }}", plus a model on that connector that takes a few minutes to run.
  2. While the model is running, run rill project refresh --parser a few times.
  3. Expected: nothing is reconciled again and the model run continues.
  4. Actual: on some of the parses, the project logs show the connector being reconciled, followed by every resource downstream of it, and the running model is cancelled and starts over.

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

…downstream

analyzeTemplatedProperties built a connector's TemplatedProperties by ranging over a map,
so a connector with two or more templated properties got them in a random order on every parse.
The project parser compares specs with proto.Equal, which is order-sensitive for lists,
so a full re-parse of an unchanged connector could update its spec.
That marks every resource downstream of the connector as pending and cancels the running ones.

Sort the property names so the parser output is stable.
@nishantmonu51
nishantmonu51 merged commit 1cf5973 into main Oct 21, 2026
16 checks passed
@nishantmonu51
nishantmonu51 deleted the nishant/sort-connector-templated-properties branch October 21, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants