feat(datasource-pylon): all collections + relations (EXT-8) - #354
Merged
christophebrun-forest merged 9 commits intoAug 18, 2026
Merged
Conversation
Adds search/list/fetch methods for the four new resources, routed through shared private helpers; search_issues and fetch_issue now delegate to the same helpers with unchanged behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extracts the cursor-walk search flow, sort warning and operator maps into BaseCollection and shared modules; declares the four ManyToOne relations on PylonIssue and adds the schema-driven RelationEmbedder. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cursor-paginated read-only collections with server-side filters, free-text search, a single-id fast path on the record endpoint and their relations to issues, contacts and account. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fetch-all read-only collections: the endpoints return the complete dataset, so filtering, sorting and pagination run in memory over operators proven evaluable by the toolkit equivalence machinery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Registers the five collections on the datasource and covers the relation embedding end to end: bulk id search per foreign collection, chunking, dedup and missing-record handling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
19 new issues
|
![qltysh[bot]](https://avatars.githubusercontent.com/in/890766?s=60&v=4){kind=small}
| fetch_resource('issues', id) | ||
| end | ||
|
|
||
| def search_accounts(limit:, cursor: nil, filter: nil, search_text: nil) |
| fetch_resource('accounts', id) | ||
| end | ||
|
|
||
| def search_contacts(limit:, cursor: nil, filter: nil, search_text: nil) |
|
|
||
| private | ||
|
|
||
| def search_resource(path, limit:, cursor: nil, filter: nil, search_text: nil) |
| end | ||
| end | ||
| end | ||
| end |
| 'nor POST /accounts/search takes a sort parameter, so accounts come back in the order the API imposes.' | ||
| end | ||
|
|
||
| def search_page(limit:, cursor:, filter:, search_text:) |
| 'nor POST /contacts/search takes a sort parameter, so contacts come back in the order the API imposes.' | ||
| end | ||
|
|
||
| def search_page(limit:, cursor:, filter:, search_text:) |
| return ascending ? comparison : -comparison | ||
| end | ||
|
|
||
| 0 |
| return 1 if left.nil? | ||
| return -1 if right.nil? | ||
|
|
||
| (left <=> right) || (left.to_s <=> right.to_s) |
| 'POST /issues/search always returns issues from the most recent to the oldest.' | ||
| end | ||
|
|
||
| def search_page(limit:, cursor:, filter:, search_text:) |
| # not it resolved: a null foreign key, and a record the operator can no | ||
| # longer reach, both read as "no related record" rather than as a row | ||
| # missing the field. | ||
| def embed_foreign(foreign_collection, relations, records, rows) |
- refuse an absence filter Pylon cannot answer, with a message naming it, and map missing onto is_unset where the API takes it - report a refused filter as a ValidationError, so the agent answers 400 with the message rather than a 500 "Unexpected error" - refuse a filter on a related field, naming the key to use instead - drop a record answered under an alias of the primary key - declare no column groupable, and answer an aggregation in memory on the two collections holding every record Pylon has - take no custom field on those two, where nothing reads one - leave a blank foreign key out of the relation reads, like a null - serialize only the records a relation asked for - hoist filter_table, api_filters and add_column into BaseCollection Also fixes SortCollectionDecorator#refine_schema, which marked every column of every datasource sortable instead of the ones a sort was registered for, and rewrote the schema of its child while doing so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| # | ||
| # FetchAllCollection, which does hold every record Pylon has, overrides | ||
| # this and answers exactly. | ||
| def aggregate(_caller, _filter, _aggregation, _limit = nil) |
| rescue APIError => e | ||
| raise unless e.status == 404 | ||
|
|
||
| [] |
| # | ||
| # The rows are keyed with strings because that is how the agent reads | ||
| # them, while `Aggregation#apply` hands them back keyed with symbols. | ||
| def aggregate(caller, filter, aggregation, limit = nil) |
|
|
||
| serialized = serialize(record) | ||
| serialized if matches_id?(serialized, id) | ||
| end |
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
page_window no longer slices when the filter carries no page, or a page naming no limit. FetchAllCollection was answering a page-less read with the first 1000 records of a larger set, which left SortCollectionDecorator without a position for the rest. domains and tags no longer advertise contains / not_contains: the columns are typed Json, on which the toolkit refuses every substring operator, so the UI offered a filter that could only error. Typing them as a String array is not a way out either, the validator has no branch for an array column type. A relation:field leaf is resolved into the foreign_key in [...] the endpoint filters, by reading the foreign collection for the keys of the records matching it, instead of being refused while the schema advertises the relation as filterable. Bounded at 500 keys and refused rather than truncated past that, and answered with no record and no request when no foreign record matched. Resolved at the entry of the read, so the primary-key lookup route benefits too and the tree is walked once instead of twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| next page_window(records_by_id(id), query) if id | ||
|
|
||
| search_records(caller, query) | ||
| end |
page_window stopped slicing a page-less read in cee51dd, which fixed FetchAllCollection, where the records are already in hand. The cursor collections take theirs from the walk, and the walk was still bounded: translate_page answered a missing page with MAX_SEARCH_LIMIT, so a read asking for every record travelled as a read asking for a thousand. The walker could not tell the two apart, broke out on records.size >= needed before reaching its cap check, and answered a larger set with its first thousand records without logging a truncation. translate_page now answers a missing page, or a page naming no positive limit, with a nil limit, and walk reads nil as "every record past the offset": it follows the cursor until Pylon reports no page left or a cap stops it, and a cap stopping it is a truncation the warning now names. batch_size bounds itself on the record budget alone when there is no window, and the final slice becomes a drop. A window the caller did ask for keeps its exact cost and its silence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| end | ||
|
|
||
| records[offset.to_i, limit.to_i] || [] | ||
| limit ? (records[offset, limit] || []) : records.drop(offset) |
Brings 28d4da6 in: the gem registered in .releaserc.js and the activesupport runtime dependency. Disjoint from the files this branch touches, so the merge carries no conflict. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
christophebrun-forest
merged commit Aug 18, 2026
4b14086
into
feat/datasource-pylon
52 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Story
EXT-8 — Story 4: All collections + relations
Adds the four remaining read-only collections and wires the relational graph between issues, accounts, contacts, users and teams.
What's inside
Client — search/list/fetch endpoints for the four resources, mutualized in generic private helpers (
search_issues/fetch_issuenow delegate to them, behavior unchanged).Collections
PylonAccount,PylonContact(CursorCollection): cursor-paginated listing (GET, 60 req/min) when browsing,POST /search(20 req/min) as soon as a filter or free-text search is present, and a single-id fast path on the record endpoint for detail views. Their search endpoints filteridserver-side, so unlike issues they need no primary-key short-circuit:idsits inapi_filtersand translates like any other field,orincluded.PylonUser,PylonTeam(FetchAllCollection):GET /users/GET /teamsreturn the complete dataset, so filtering, sorting and pagination run in memory — correct because the snapshot is complete, and every advertised operator is proven in-memory-evaluable through the toolkit's equivalence machinery (guarded by a programmatic spec). Deactivated agents are included so historical assignees stay resolvable.Relations —
issue → account / requester (contact) / assignee (user) / team(ManyToOne) with their OneToMany inverses, pluscontact → accountandaccount → contacts. All reverse sides ride the existing server-side filters (account_id,requester_id,assignee_id,team_id).RelationEmbedder — schema-driven port of the Zendesk pattern: ManyToOne projections are resolved in bulk at read time (one
id in [...]search per foreign collection per page for accounts/contacts, chunked at 100; one indexed fetch-all for users/teams), reusing each foreign collection's own serializer instead of duplicating field lists. A relation the projection does not ask for costs zero requests.Filter allow-lists are transcribed from the Pylon API reference (
static.usepylon.com/openapi.json); fields the API cannot filter advertise no operator, andexternal_idsadvertises none because the API matches bare id strings while the column displays{external_id, label}objects.Tests
🤖 Generated with Claude Code
Note
Add PylonAccount, PylonContact, PylonUser, and PylonTeam collections with ManyToOne relation support
PylonAccount,PylonContact,PylonUser,PylonTeam) in the datasource alongside the existingPylonIssuecollection.CursorCollectionandFetchAllCollectionbase classes to handle cursor-paginated (Account, Contact) and full-dataset (User, Team) retrieval strategies with in-memory filtering, sorting, and pagination.RecordSerializationandRelationEmbeddermixins to flatten nested foreign-key objects and embedManyToOnerelated records into list responses.with_resolved_relationstoBaseCollectionto rewrite relation field filters into local foreign-keyINconditions, short-circuiting to empty results when no foreign records match.OperatorMapsmodule and per-collectionApiFiltersto centralize server-side filter operator declarations.UnsupportedOperatorErrornow inherits fromValidationErrorto surface as HTTP 400.MAX_RELATION_KEYS(500) foreign reads per filter; queries matching more than 500 foreign records raiseUnsupportedOperatorError.Macroscope summarized 3027ce6.