Add plugin unit tests, and playwright workflows. - #9
Conversation
|
Related additions to plugin generator in core: OpenC3/cosmos#3860 |
jmthomas
left a comment
jmthomas
left a comment
There was a problem hiding this comment.
We don't run any github actions on these files?
| await expect(page.locator('[data-test=process-list]')).toContainText( | ||
| complete, | ||
| { timeout: INSTALL_TIMEOUT }, | ||
| ) |
There was a problem hiding this comment.
These are the total values it can be 'Complete|Warning|Error|Crashed|Expired'. I think you should check for any of those here. This causes a fail fast if there is an error. Then assert that the value is actually Complete and maybe also Warning because that isn't a failure case.
There was a problem hiding this comment.
Done in 4f9820d. The spec now waits for any of Complete|Warning|Error|Crashed|Expired, then asserts the state is Complete or Warning, so an error fails right away instead of running out the 10 min timeout. The process list keeps 10 minutes of history in no fixed order, so it also snapshots the finished rows before submitting. That way a Playwright retry isn't judged by a stale row from the first attempt.
| }, | ||
| // Has to leave room for the setup budget (20 min) plus the install test and | ||
| // its one retry, and still finish inside the workflow's 45 minute job timeout | ||
| globalTimeout: 40 * 60 * 1000, |
There was a problem hiding this comment.
This isn't going to be enough since the checkout, build and run stages can take 20 min. I'd move to at least 1hr.
There was a problem hiding this comment.
Raised the job timeout-minutes to 75 in 4f9820d. Playwright's globalTimeout stays at 40 because it only covers the Playwright step. I rewrote the comment to say so, since the old one suggested it had to fit inside the job budget together with everything else.
| // IFRAME and NEW tools point somewhere else, so their console says nothing | ||
| // about this plugin. INLINE tools are the ones COSMOS itself renders. | ||
| const checkable = mine.filter( | ||
| (tool) => tool.window === 'INLINE' && tool.url && tool.url.startsWith('/'), |
There was a problem hiding this comment.
Per tool_model.rb:233 window can take "WINDOW <INLINE | IFRAME | SAME | NEW>". Do we need to support SAME? There's no comment.
I see SAME checkced in AppNav.vue but don't see it used anywhere. What does this do?
There was a problem hiding this comment.
SAME renders as a plain <a :href="item.url"> in AppNav.vue (line 102), so clicking it navigates the current tab away from COSMOS. NEW does the same thing in a new tab. Neither is mounted as a single-spa app, so like IFRAME and NEW there's no COSMOS console to check, and the spec already skips it because it only checks INLINE. I added a comment in 4f9820d listing all four modes and why only INLINE is checked, and updated the skip message.
The install spec now waits for any final process state and fails on Error, Crashed or Expired rather than waiting out the timeout for a Complete that won't come. Warning passes. Rows left from an earlier attempt are ignored so a retry is judged on its own result. The Playwright job timeout goes to 75 minutes since checkout, build and startup can take 20 minutes before Playwright's own 40 minute budget starts. Adds CI for this repo: actionlint on workflows and templates, template metadata checks, and a typecheck and test listing of the Playwright harness. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Re: We don't run any github actions on these files? Added
The real end to end run of the harness still happens through each plugin's Playwright workflow, since it needs COSMOS running. |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2 participants
What changed
Added reusable workflows for every COSMOS plugin.
Also added the option to push to rubygems to the release workflow.
Why it changed
Plugins need consistent and repeatable tests
Testing strategy
Will be first tested with new Kayhan plugin (after merge). We will fix from there.