Improve snapshot testing experience

[zanieb]

Snapshot testing is a really useful way for us to develop diagnostics and prevent regressions. However, there are a few issues with snapshot testing today:

• Test source files use inconsistent commenting to indicate expected "OK" and "ERROR" cases
• A snapshot diagnostic for a single test case often includes test cases on other lines, making snapshots harder to review
• Snapshots are done per source file rather than per test case, this means that adding a test case will break all of the following cases
• Snapshot diffs are hard to review unless test cases are appended to end the source file

We'd love to improve the experience here but building out testing utilities will be a significant amount of effort. Here, I'd like to discuss possible solutions! Please start a new thread for each independent idea.

[zanieb]

We can improve the diff review experience by anonymizing line numbers as described in #6097

There is additional discussion on that issue on why this may not be the ideal solution. Further discussion can occur here though :)

[konstin]

I've been thinking about adding separators to the tests files, either comments (e.g. # ---) or something that is not valid python syntax to split on that and run each test individually. Separators also give us an option to annotate or separate ok and failing examples.