Skip to content

Expose resource attribution in the REST API - #654

Merged
bpepple merged 14 commits into
masterfrom
651-expose-resource-attribution-source-url-in-the-rest-api
Oct 16, 2026
Merged

bpepple merged 14 commits into
masterfrom
651-expose-resource-attribution-source-url-in-the-rest-api

Conversation

@bpepple

@bpepple bpepple commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Closes #651

Summary

Detail responses for arcs, characters, creators, imprints, issues, publishers, series, teams, and universes now include a read-only attribution list. Each entry gives the source name and the URL that the descriptive content was adapted from. API consumers that reuse desc can now credit the source, which some licenses require (Wikipedia's CC BY-SA, for example).

"attribution": [
  {"source": "Wikipedia", "url": "https://en.wikipedia.org/wiki/Spider-Man"},
  {"source": "Marvel", "url": "https://www.marvel.com/characters/spider-man-peter-parker"}
]

The list is empty when there's no attribution. List responses and create/update responses are unchanged. This is an additive, non-breaking change.

Changes

  • Serializers: new AttributionSerializer (source and url), nested read-only on each detail serializer. source uses fixed English names, so the output doesn't change with the request's language and can be cached per object. It's declared in the schema as AttributionSourceEnum. Arc, Creator, and Publisher had no separate read serializer, so this adds ArcReadSerializer, CreatorReadSerializer, and PublisherReadSerializer and uses them for retrieve.
  • Ordering: Attribution gets a stable default ordering (migration 0060), so the list order is deterministic.
  • Cache invalidation: post_save/post_delete handlers on Attribution update the parent's modified field and bump its cache version. Adding, editing, or deleting an attribution (from the web form, an admin inline, or a merge) therefore invalidates the cached detail response and updates Last-Modified. A queryset delete bumps each parent once, and a cascade delete from the parent is skipped.
  • Detail cache key version: detail cache keys now include DETAIL_CACHE_SCHEMA_VERSION (api/cache.py). Without it, responses cached before this deploy, which have no attribution field, would be served for up to DETAIL_CACHE_TTL (3 days). Bump the version whenever a detail response's shape changes.
  • Edit views: the attribution create views (and IssueCreate) no longer save the parent twice. AttributionUpdateMixin now saves the parent inside the same transaction as the attribution formset, so a failed parent save can't leave attribution changes committed without it.
  • Cleanup: removed the unused move_pub_to_imprint management command.
  • Docs: api/README.md has a new Attribution section under Common Patterns, the field is listed for each of the nine resources, and there's a Version 1.12 changelog entry. That entry also covers the cover_date_range filter from Add cover_date_range filter to the issue API #628, which never got one.

Testing

New tests/comicsdb/test_api_attribution.py:

  • For each of the nine resources: the list is empty by default, source names and URLs are returned, and it's absent from list responses.
  • Every Attribution.Source has an API name, the schema names the enum, and an unknown stored code falls back to the raw value instead of erroring.
  • Adding, editing, and deleting an attribution clears the cached detail response. If-Modified-Since returns 200 after a change, and the parent's modified and cache version are bumped. This also covers queryset deletes (each parent bumped once), cascade deletes, raw fixture loads, and parents without a modified field.
  • Create views save the parent only once.

Also added: bumping DETAIL_CACHE_SCHEMA_VERSION turns a cached hit into a miss, and audit_response_cache still categorizes versioned detail keys by model.

Full suite passes. manage.py spectacular --validate reports no errors, and the Attribution component appears in the generated schema.

Deployment

The detail cache key version means old cached detail responses are never served again, but they stay in Redis until their TTL expires (up to 3 days). To free that memory right away, run this in manage.py shell after deploying:

from django.core.cache import cache

client = cache._cache.get_client()
deleted = 0
for key in client.scan_iter(match="*api:detail:*", count=1000):
    # Keep entries already written in the new format.
    if b":v2:" not in key:
        client.delete(key)
        deleted += 1
print(f"Deleted {deleted} old detail cache keys")

It only deletes old-format keys. The detail cache starts cold after the deploy either way, so expect a short rise in database load while it warms back up.

Known limitations

  • Writing attributions through the API is out of scope and likely won't be added in a follow-up.
  • An attribution's parent is never reassigned in the current code. Code that moves an attribution to a different parent must bump the old parent itself.

Add a read-only `attribution` list (source label + URL) to the detail responses for arcs, characters, creators, imprints, issues, publishers, series, teams, and universes, so API consumers can credit where descriptive content came from. List responses are unchanged.

- Add AttributionSerializer, returning `source` as its display label
- Add Arc/Creator/Publisher read serializers for retrieve
- Prefetch `attribution` on each retrieve queryset
- Bump the parent's `modified` and cache version when an Attribution is saved or deleted, so cached detail responses and Last-Modified stay current
- Document the field in api/README.md
Replace the endpoint query-count comparison, which passed with or without the prefetch, with a check that the retrieve queryset returns objects whose attribution is already loaded (zero queries on access).
@bpepple bpepple linked an issue Oct 16, 2026 that may be closed by this pull request
5 tasks
@bpepple bpepple self-assigned this Oct 16, 2026
@bpepple bpepple added enhancement New feature or request api An API bug/feature labels Oct 16, 2026
Add an id tiebreaker to Attribution.Meta.ordering so an object's attributions come back in creation order. Before, Postgres could return them in any order, so the same data could serialize differently after a cache miss.

Serialize `source` from a fixed map of English names instead of the translatable choice label. The detail cache key doesn't include the language, so a translated label could be cached and then served to clients in other languages. The values are the ones the README already documents, and the OpenAPI schema now lists them as an enum.
- Look up the parent model via the cached ContentType.objects.get_for_id() instead of the content_type FK, avoiding a query per save/delete.
- Skip the post_delete bump when the delete cascades from the parent (detected via the signal's `origin`), avoiding no-op UPDATEs and cache bumps when deleting series, issues, etc.
- Ignore raw (loaddata) saves, and record the previous parent in pre_save so reassigning an attribution also bumps the old parent.
- Skip the parent bump for attribution parents without a `modified` field, instead of raising FieldError.
- Add attribution_parent_saved_separately() and use it around the attribution formset saves in the create/update views, whose own parent save already bumps `modified` and the cache version. Move the parent save inside the transaction in the update views so it commits together with the attribution changes.
- Add AttributionRetrieveMixin to prefetch `attribution` on retrieve, replacing the per-viewset prefetches.
- Redirect directly from AttributionCreateMixin and IssueCreate instead of calling super().form_valid(), which re-ran form.save() and wrote a second history record for every new object.
- Bump each parent at most once per attribution queryset delete.
- Name the attribution source enum AttributionSourceEnum in the OpenAPI schema rather than the generic SourceEnum.
- Remove AttributionRetrieveMixin: retrieve serializes a single object, so prefetching attribution saved no queries.
- Revert attribution_parent_saved_separately(): it saved one cheap UPDATE per formset row but every save path had to remember to use it (the admin inlines didn't), and forgetting was silent.
- Fall back to the stored source code in AttributionSerializer instead of raising KeyError on a value outside SOURCE_NAMES.
Record (content_type_id, object_id) in Attribution.from_db() and after each save, and compare against it in post_save to detect reassignment. This replaces the pre_save handler, which ran an extra SELECT before every update of an existing attribution.
AttributionUpdateMixin.form_valid() only called form.save(commit=False) inside transaction.atomic() and left the real save to super().form_valid(), which ran after the block closed. If that save failed, the attribution rows and the parent `modified` bump stayed committed, but the parent edit was lost.

Save the form, including its M2M fields, inside the transaction and redirect directly, the same way AttributionCreateMixin does.
Nothing reassigns an attribution to a different parent: the formset and admin inline fix the parent, and move_pub_to_imprint, the only code that moved attributions, has been removed. Remove Attribution.from_db() and loaded_parent, and only bump the current parent on save.
Record the new attribution field on detail endpoints, and the cover_date_range filter from #628, which never got an entry.
Detail keys depend only on pk and `modified`, so responses cached before a shape change, such as the new `attribution` field, were served for up to DETAIL_CACHE_TTL after deploy. Add DETAIL_CACHE_SCHEMA_VERSION to the key so bumping it orphans entries in the old shape.
@bpepple
bpepple merged commit 03c045b into master Oct 16, 2026
2 checks passed
@bpepple
bpepple deleted the 651-expose-resource-attribution-source-url-in-the-rest-api branch October 16, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api An API bug/feature enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose resource attribution (source + URL) in the REST API

1 participant