Look at human and model accuracy within each arg class for each scenario

[danielbear]

For example, compare humans to DPI to FitVid

We know they have similar overall accuracy on some of the scenarios.

What is the "per-arg class" accuracy for each? For instance, there are different classes/batches of Dominoes trials -- some with 2 dominoes, some with 4, some with occluders, some with missing dominoes, etc. Are there particular arg classes where people/models are doing especially well, or differently from each other?

This analysis is critical for improving the benchmark: we want to make sure that models are not doing well by essentially classifying which arg set was used to generate a trial, and then guessing the average outcome for that class (which may not be 50% since we didn't "microbalance" the scenarios.)