feat(state): Enable :set tables

[runkelcorey]

Summary of changes

🐞 State.StopEvent times out on refresh

This PR adds the option of making a :set table instead of a :bag. For conservatism, I opted for only allowing :set and :bag.

Insert and lookup times in tables of type set are constant, regardless of the table size. For table types bag and duplicate_bag time is proportional to the number of objects with the same key.

After deploying to dev-blue, Splunk showed that the init_table log time (which approximates the full reload) dropped 27% against dev-green, the baseline:

measure	dev-green	dev-blue	𝚫
5th percentile	6,442ms	5,952ms	-8%
median	13,347ms	9,695ms	-27%
95th percentile	16,336ms	14,589ms	-10%

This change won't eliminate the issue entirely—9.6s is still much longer than it takes to reload Prediction—so I'm working on more PRs to address the issue.

[lemald]

Interesting! So were there a lot of StopEvents with the same key? That would explain the poor performance with a :bag table type, but then wouldn't that imply that there will be a problem with the :set table type due to :set only allowing one value per key?

[runkelcorey]

Interesting! So were there a lot of StopEvents with the same key? That would explain the poor performance with a :bag table type, but then wouldn't that imply that there will be a problem with the :set table type due to :set only allowing one value per key?

No, the id is unique and enforced in lamp. I think the insert performance is not only proportional to the number of objects with the same key but, by implication, the number of objects. Also, perhaps other performance is also affected by table type?

[lemald]

Okay, makes sense. I wish we had a slightly more certain story about what's improving the performance, but since this is a very targeted change that only applies to StopEvents I'm okay with merging it.