Resolve Host last_job filters against the latest job host summary

[Aureliolo]

SUMMARY

Host.last_job and Host.last_job_host_summary are denormalized caches that are no longer written. The serializer, DashboardView and Inventory.update_computed_fields all derive the value from JobHostSummary, but filtering goes to the columns, which are NULL for every host. Those lookups therefore return an empty set rather than an error.

Three visible effects:

• The dashboard's "Failed hosts" tile links to /hosts?host.last_job_host_summary__failed=true (Dashboard.js:107). The count beside it is correct, the list it opens is always empty.
• HostFilterLookup.js:145 offers "Last job" as a host-filter key for smart inventories. A smart inventory built on it silently matches nothing, and unlike the dashboard there is no count to contradict it.
• Any API client filtering last_job or last_job_host_summary gets a silently wrong answer instead of an error. Compare ?has_active_failures=true, which fails loudly with Host has no field named 'has_active_failures'.

Both entry points are covered. HostFieldLookupBackend resolves the two fields for the REST filters, and SmartFilter routes them through the same code so a smart inventory's host_filter agrees with /api/v2/hosts/; SmartFilter validates the lookup with the backend but then builds its own queryset, so it needs handling of its own. __search is included: the parent expands it to a list of lookups relative to the related model, so the OR is collapsed against JobHostSummary rather than applied to Host. A smart inventory cannot express that OR, and SmartFilter rejects such keys as a parse error.

Recency comes from the _latest_summary_id annotation in HostLatestSummaryQuerySet, which every Host list view already carries and which HostSerializer reads last_job and last_job_host_summary through. Reusing it keeps one definition of "latest summary" across the serializer, DashboardView, Inventory.update_computed_fields and the filters, and drives the scan off Host rather than the much larger JobHostSummary. __isnull maps to the annotation being null, so it means "the host has never run". Non-Host models fall through to the parent backend, which matters because UnifiedJobTemplate.last_job is a live column. Registration matches the generic backend by exact dotted path and raises ImproperlyConfigured if it is absent, since a silent miss would leave the derived filters matching nothing. No UI change is needed; the existing link and filter key work as written.

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

ASCENDER VERSION

awx: 25.4.1.dev225+gcc4e6d337

ADDITIONAL INFORMATION

Reproduction on an instance where hosts have run jobs:

GET /api/v2/dashboard/                                -> hosts.failed = 4
GET /api/v2/hosts/?last_job_host_summary__failed=true -> count = 0
GET /api/v2/hosts/?last_job_host_summary__isnull=true -> count = 262 (every host)

A host serializing last_job: 274 is not matched by ?last_job=274, because the serialized value is derived while the filter reads the column.

Full suite, make test in the branch-built ascender_devel image:

3685 passed, 6 skipped, 3 failed in 329s

The three are test_is_not_inventory[bad|bad_encoding|empty.txt], which key off file permissions and fail on any mount that reports every file as executable. They are unrelated to this change.

Negative control, with the derived-field resolution removed:

3 failed, 7 passed
  test_smart_inventory_host_filter             [] != ['failing']
  test_search_across_last_job                  400 != 200
  test_search_across_summary_matches_nothing   400 != 200

awx/main/tests/functional/api/test_host_derived_filters.py   11 passed
black --check, flake8                                        clean

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a Host-specific filter backend so Host derived fields (last_job, last_job_host_summary) resolve against the latest JobHostSummary, and validates the behavior via functional API tests.

Changes:

• Override DRF DEFAULT_FILTER_BACKENDS to use awx.api.filters.HostFieldLookupBackend in place of the generic FieldLookupBackend.
• Add HostFieldLookupBackend implementing lookups against the latest JobHostSummary per host.
• Add functional tests covering derived-field filtering and dashboard consistency.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
awx/settings/defaults.py	Replaces the generic field lookup backend with a Host-aware backend.
awx/api/filters.py	Implements HostFieldLookupBackend resolving Host derived fields via latest JobHostSummary.
awx/main/tests/functional/api/test_host_derived_filters.py	Adds functional tests for the new derived-field filtering behavior.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

awx/api/filters.py:83

• The method returns the lookup as both a string ('pk__in') and a list (['pk__in']) depending on the branch. That inconsistent return type makes non-DRF consumers easy to break (as seen in smart inventory filtering) and adds unnecessary special-casing. Consider normalizing the return type to always use a string ('pk__in') since the derived-host implementation always collapses to a single lookup anyway; this simplifies callers and reduces integration risk.

        elif inner == 'search':
            # A summary holds none of the field names treated as searchable, which the
            # parent expresses as an empty lookup list, and an empty list of ORs matches
            # every row rather than none.
            return Host.objects.none().values('pk'), ['pk__in'], False

        value, inner, _ = super().value_to_python(JobHostSummary, inner, value)
        if isinstance(inner, list):
            # __search expands to several lookups, all relative to JobHostSummary, so the
            # OR collapses here; left to the caller they would be applied against Host.
            if not inner:
                return Host.objects.none().values('pk'), ['pk__in'], False
            condition = functools.reduce(operator.or_, (Q(**{one: value}) for one in inner))
            return hosts_matching_latest_summary(condition), ['pk__in'], False
        return hosts_matching_latest_summary(Q(**{inner: value})), 'pk__in', False

awx/main/tests/functional/api/test_host_derived_filters.py:131

• The smart-inventory path is covered for equality filters, but not for derived-field lookups that expand to multi-lookup behavior (notably last_job__search=...), which is where the updated backend integration is most likely to regress. Adding a smart-inventory host_filter test case for last_job__search=<substring> would exercise the value_to_python() return-shape handling and prevent runtime failures.

@pytest.mark.django_db
def test_smart_inventory_host_filter(inventory_with_hosts, get, admin):
    run_job(inventory_with_hosts, ok=['passing'], failures=['failing'])
    last = run_job(inventory_with_hosts, ok=['recovered'])

    query = urllib.parse.quote('last_job_host_summary__failed=true', safe='')
    response = get(host_list('?host_filter=%s' % query), admin)
    assert names_from(response) == ['failing']

    query = urllib.parse.quote('last_job=%d' % last.id, safe='')
    response = get(host_list('?host_filter=%s' % query), admin)
    assert names_from(response) == ['recovered']

awx/settings/defaults.py:1250

• The dotted path for FieldLookupBackend is hard-coded, which can silently drift from the actual import path used by django-ansible-base (and cause startup failure). If feasible in settings initialization order, derive the string from the imported class (e.g., FieldLookupBackend.__module__ + "." + FieldLookupBackend.__name__) to keep this check aligned with the installed package.

generic_field_lookup_backend = 'ansible_base.rest_filters.rest_framework.field_lookup_backend.FieldLookupBackend'
if generic_field_lookup_backend not in REST_FRAMEWORK['DEFAULT_FILTER_BACKENDS']:
    raise ImproperlyConfigured(
        'Expected {} in REST_FRAMEWORK["DEFAULT_FILTER_BACKENDS"], found {}.'.format(generic_field_lookup_backend, REST_FRAMEWORK['DEFAULT_FILTER_BACKENDS'])
    )

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

awx/settings/defaults.py:1253

• This settings hook assumes dynamic_settings.py always populates REST_FRAMEWORK['DEFAULT_FILTER_BACKENDS']. If that key is absent (or set to a falsy value), the current code will raise a KeyError during settings import rather than the intended ImproperlyConfigured with a clear message. Guard for a missing key explicitly before checking membership so startup failures are actionable.

generic_field_lookup_backend = 'ansible_base.rest_filters.rest_framework.field_lookup_backend.FieldLookupBackend'
if generic_field_lookup_backend not in REST_FRAMEWORK['DEFAULT_FILTER_BACKENDS']:
    raise ImproperlyConfigured(
        'Expected {} in REST_FRAMEWORK["DEFAULT_FILTER_BACKENDS"], found {}.'.format(generic_field_lookup_backend, REST_FRAMEWORK['DEFAULT_FILTER_BACKENDS'])
    )