Repository navigation
t.list: Add support for listing multiple dataset types - #7731
Conversation
ninsbl
left a comment
There was a problem hiding this comment.
Looks in general good to me. I have only minor comments inline below...
|
@petrasovaa it would be good if you could give your opinion on how JSON output should be handled / look like. Where should mapset or type go for example and should the json structure be consistent for single an multiple requested types I probably thonk yes... |
So, to avoid different sets of keys in each item in JSON (raster vs strsds), how about allowing user to just combine the same type (e.g. strds, stvds) and not allow combining e.g. raster and strds. I would like to keep the flat structure of the JSON. |
ninsbl
left a comment
There was a problem hiding this comment.
I tried to think through the PR on a bus once more. Sorry for bringing up some new stuff. I have no strong opinion there, but I would be glad if you could consider my comments and address them as you see fit.
|
I've tried to cover all the suggested changes:
I hope I haven't missed anything! Also, apologies for the late response, I was a bit occupied with other things. |
ninsbl
left a comment
There was a problem hiding this comment.
In other, similar tools, JSON output defaults to all columns in the output, if I remember correctly?!?
How is this handled here? Different types can have different columns...
Yes, In Here is an example output of [
{
"id": "nc_lst_daily@PERMANENT",
"name": "nc_lst_daily",
"mapset": "PERMANENT",
"temporal_type": "absolute",
"semantic_type": "mean",
"creation_time": "2026-06-03 18:15:06",
"modification_time": "2026-06-03 18:18:05",
"creator": "saket",
"start_time": "2002-01-01 00:00:00",
"end_time": "2004-10-05 00:00:00",
"granularity": "1 day",
"north": 279073.975466,
"south": 113673.975466,
"east": 798143.311797,
"west": 595143.311797,
"bottom": 0.0,
"top": 0.0,
"proj": "XY",
"raster_register": "raster_map_register_9ca4f56710e5453291c0c830aa8f9a92",
"number_of_maps": 1008,
"nsres_min": 200.0,
"ewres_min": 200.0,
"nsres_max": 200.0,
"ewres_max": 200.0,
"min_min": -1.49,
"min_max": 30.63,
"max_min": -0.29,
"max_max": 48.23,
"aggregation_type": null,
"title": "NC Daily LST",
"description": "Standard MODIS LST Time Series for North Carolina",
"command": "...",
"type": "strds"
},
{
"id": "schools_stds@PERMANENT",
"name": "schools_stds",
"mapset": "PERMANENT",
"temporal_type": "absolute",
"semantic_type": "mean",
"creation_time": "2026-06-10 17:50:37",
"modification_time": "2026-06-10 17:52:46",
"creator": "saket",
"start_time": "2024-01-01 00:00:00",
"end_time": "2027-01-01 00:00:00",
"granularity": "1 year",
"north": 248159.844411,
"south": 203559.011362,
"east": 671714.551108,
"west": 619215.133883,
"bottom": 0.0,
"top": 0.0,
"proj": "XY",
"vector_register": "vector_map_register_e0d7ff31f8dc41de82128f9a9c1e237b",
"number_of_maps": 3,
"title": "Wake County Schools",
"description": "Test dataset for GSoC",
"command": "...",
"points": 501,
"lines": 0,
"boundaries": 0,
"centroids": 0,
"faces": 0,
"kernels": 0,
"primitives": 501,
"nodes": 0,
"areas": 0,
"islands": 0,
"holes": 0,
"volumes": 0,
"type": "stvds"
}
] |
|
Some issues found (they might be pre-existing):
|
|
Regarding the different keys for the different types, that is indeed a problem I didn't realize. Currently the CSV with mixed types won't work at all, the header won't match. So unfortunately this is getting more complicated with the "columns=all" option. I think we need to select only keys that intersect for the different groups. |
|
I was going back to the "type" option we had previously as one of the options in the columns option. I am now thinking this might be a better option. So if user doesn't request it (e.g. columns=id) it won't show up (which might be more expected). Now when user requests multiple types and columns is not explicitly specified, the type would show up in JSON (default is columns=all behavior) and in CSV (the default is columns=id,type). When single type is requested, I am now leaning to omit it (unless explicitly requested with columns option). Sorry for keep changing the design, this has been somewhat more complicated than I hoped... |
|
I also realized the output of the function changed, basically breaking the API. Could we return the previous output if single type was selected? So if type is a str, return exactly the old shape (raw row objects, no injected key), if it's a list, return the new dict form. |
|
Thanks for the update, found several issues:
|
|
This is Claude's suggestion about the implementation, hopefully this will be helpful:
|
Done
Actually, I did that intentionally thinking it would be better in plain format. Reverted now.
Yeah, I thought it was only needed for CSV. |
ninsbl
left a comment
There was a problem hiding this comment.
Thanks for addressing the issues. It looks gererally good to me.
When testing, I noticed however, that with all possible combinations of options and flags, it is not easy to predict how expected results will look like.
Still, OK from my point of view.
|
@petrasovaa, are any changes needed here? |
This PR implements the feature requested in #7695 to support listing multiple dataset types simultaneously, along with improvements in t.list tests.
@ninsbl @petrasovaa