[DG47]

Fixes OHDSI/Atlas#2644

The Data Sources condition "by type" drilldown (cdmresults/sql/report/condition/drilldown/byType.sql and the matching drilldownsummary/byType.sql) joins Achilles analysis 405 to condition type concepts filtered with lower(vocabulary_id) = 'condition type'. The 'Condition Type' vocabulary was deprecated in the 2020 vocabulary release that consolidated all type concepts into the 'Type Concept' vocabulary, so a CDM that uses current condition_type_concept_id values (for example 32817 EHR, 32810 Claim, or 32831 from the linked issue) gets an empty "Condition by type" chart. The other domains' by type queries join concept without a vocabulary filter and are not affected.

Changes:

• Also accept concepts from the 'Type Concept' vocabulary in both queries.
• Apply the existing inpatient/outpatient primary/secondary diagnosis grouping only to 'Condition Type' concepts. Without this, new type concepts such as 32855 "Inpatient claim header" would be relabelled "Claim- Inpatient: Secondary diagnosis". Type Concept rows keep their own concept ID and name.
• Add CDMResultsConditionByTypeTest, which runs the condition drilldown and drilldown summary queries through CDMResultsAnalysisRunner.getDrilldown against the embedded Postgres database with old and new condition type concepts.

The new SQL is plain nested CASE. I checked that SqlRender translates it for Snowflake, SQL Server, BigQuery, Spark, Redshift and Oracle.

Testing:

• mvn -B test -Dtest=CDMResultsConditionByTypeTest: both tests fail on master (only the two 'Condition Type' rows come back) and pass with this change.
• mvn -B test: 156 unit tests, 0 failures, 5 skipped. In the integration test run, 3 VocabularyServiceIT tests fail (404). They fail the same way on master without this change.

Note: drilldown results already stored in the Achilles cache stay as they are until that cache is cleared or warmed again.

[chrisknoll]

Vocabulary changes are fun! Personally, the nature of the CDM vocab change to me is a 'major version' type of change (due to breaking previous queries). But I understand the challenges with that, but at the same time I wish they would take more care in these types of changes.

[DG47]

Agreed. It's a quiet break, too: the old 'Condition Type' concepts are still in the vocabulary, so nothing errors. The report just comes up empty for any CDM that uses the newer type concepts. OHDSI/Atlas#2644 is a screenshot of exactly that, from December 2021.

To make sure there wasn't more of this, I grepped src/main/resources for a vocabulary_id filter on any of the other old * Type vocabularies. These two byType.sql files are the only ones, so this PR should cover WebAPI's side of it.