Skip to content

Exclude noisy fields from collection and file behavior responses - #275

Open
avadhsonagara wants to merge 1 commit into
google:mainfrom
avadhsonagara:main
Open

Exclude noisy fields from collection and file behavior responses#275
avadhsonagara wants to merge 1 commit into
google:mainfrom
avadhsonagara:main

Conversation

@avadhsonagara

@avadhsonagara avadhsonagara commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Several GTI MCP tools return large, noisy response payloads that bloat context and crowd out the fields that actually matter for analysis. This PR trims known-noisy/verbose fields from four tool responses — either via the VirusTotal API's exclude_attributes param (cheaper, filtered server-side) or a new utils.remove_fields helper for endpoints where that param isn't supported or doesn't reach nested sub-fields.

Also included: a rule_type/rule_types comparison bug fix in get_collection_rules, and loosening its rule_types param to Optional[List[str]] so an explicit null validates correctly over the MCP protocol.

Changes

Excluded fields by tool

Tool Fields removed
get_entities_related_to_a_file (relationship behaviours) memory_dumps, processes_terminated, registry_keys_opened, files_written, files_deleted, files_opened, files_dropped, processes_tree, processes_created, signature_matches
get_file_behavior_summary files_opened, modules_loaded, mutexes_created, mutexes_opened, processes_terminated, processes_tree, registry_keys_opened, tags, text_highlighted, registry_keys_deleted, signature_matches, files_written, memory_dumps, http_conversations, files_deleted, files_copied, files_dropped, ids_alerts, registry_keys_set, processes_created, command_executions, ip_traffic, attack_techniques, memory_pattern_urls, memory_pattern_domains, processes_injected
get_collections_commonalities attributes.aggregations.files.{itw_urls, execution_parents, compressed_parents, pcap_parents, dropped_files_sha256, email_parents, tags, main_icon_dhash, main_icon_raw_md5, vhash, imphash, behash, tlshhash, attributions, crowdsourced_ids_results, crowdsourced_yara_results, embedded_domains, embedded_ips, mutexes_created, mutexes_opened, registry_keys_deleted, registry_keys_opened, registry_keys_set, file_types, crowdsourced_sigma_results, debug_codeview_guids, debug_codeview_names, debug_timestamps, dropped_files_path, exiftool_authors, exiftool_create_dates, exiftool_creators, exiftool_last_printed, exiftool_producers, exiftool_subjects, exiftool_titles, filecondis_dhash, netassembly_mvid, office_application_names, office_authors, office_creation_datetimes, office_last_saved, pe_info_imports, pe_info_exports, pe_info_section_md5, pe_info_section_names, sandbox_verdicts, memory_pattern_urls, embedded_urls, parent_contacted_domains}
search_vulnerabilities cpes, vendor_fix_references, sources, version_history, field_sources, tags_details, alt_names_details

Supporting changes

  • utils.py: added remove_fields(data, field_paths) — removes dotted-path fields (e.g. attributes.aggregations.files.tags) from a response dict in place, for endpoints where the API's exclude_attributes can't reach nested sub-fields or isn't supported at all.
  • collections.py: _search_threats_by_collection_type now accepts an optional extra_excluded_attrs list, used by search_vulnerabilities to layer VULNERABILITY_EXCLUDED_ATTRS on top of the shared COLLECTION_EXCLUDED_ATTRS.
  • collections.py: fixed get_collection_rules filtering the wrong loop variable (rule_type not in rule_typerule_type not in rule_types), and widened rule_types to Optional[List[str]].

Test plan

  • Added parametrized test asserting get_entities_related_to_a_file sends exclude_attributes=FILE_BEHAVIOUR_EXCLUDED_ATTRS for the behaviours relationship.
  • Added test asserting get_file_behavior_summary strips FILE_BEHAVIOR_SUMMARY_EXCLUDED_ATTRS from the response.
  • Updated get_collections_commonalities fixture/expected output to reflect stripped fields.
  • Added test for get_collections_commonalities error path when the API response has no data key.
  • Added test asserting search_vulnerabilities sends the combined exclude_attributes list.
  • Added test covering get_collection_rules with an invalid relationship_name and with explicit rule_types: null.
  • Run full test suite (pytest server/gti/tests/test_tools.py) locally before merge.

Also fixes a rule_types comparison bug and loosens its type to Optional[List[str]].
@google-cla

google-cla Bot commented Jul 13, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@avadhsonagara
avadhsonagara marked this pull request as ready for review July 14, 2026 10:28
@avadhsonagara
avadhsonagara requested a review from a team July 14, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant