Non-static / dependency injection friendly api.

[BenjaBobs]

As far as I can tell, Verify only provides a static api for doing the actual verifications.

// TestSetup.cs
public static class Setup
{
    [ModuleInitializer]
    public static void Init()
    {
        // this is just an example, it could be any amount of plugins/settings being registered
        VerifyImageMagick.RegisterComparers(0.015);
        VerifyQuestPdf.Initialize();
    }
}

// Test.cs
public class MyTest
{
    [Fact]
    public async Task Test()
    {
        var result = /* some data */

        await Verifier.Verify(result);
    }
}

But what I'm looking for is something that's DI friendly, so I can

• Have different settings/plugins per test
• Not have to rely on [ModuleInitializer] which makes it difficult to make utility libraries that contain wrappers around the verification logic to specify verify with specific settings/plugins

Essentially something like

public class MyTest
{
	private readonly VerifySettings _sensitiveSettings = new VerifySettings().ImageMagic(new() { Threshold = 0.001d });
	private readonly VerifySettings _relaxedSettings = new VerifySettings().ImageMagic(new() { Threshold = 0.02d });

    [Fact]
    public async Task SensitiveTest()
    {
        var result = /* some data */

        await Verifier.Verify(result, _sensitiveSettings);
    }

    [Fact]
    public async Task RelaxedTest()
    {
        var result = /* some data */

        await Verifier.Verify(result, _relaxedSettings);
    }
}

In this example, Verifier is still static api, but to me it would be even nicer if it was also non-static (maybe with a static singleton like Random has, for easy access) because that would allow me to write extension methods.

I realize this project has 28 major versions (thanks for the hard work!!), so I imagine that this is maybe a design decision.
To me there is no reason that a non-static api could not co-exist with the static one if there's a shared singleton that the static methods just forward the calls to.
Although it would of course require a bunch of work, also for plugin maintainers.
I'm curious as to why the current design is as it is (maybe I missed some obvious reason why the current one is actually ideal?), and if there's any chance of it changing in the future?

[SimonCropp]

To me there is no reason that a non-static api could not co-exist with the static one if there's a shared singleton that the static methods just forward the calls to.

plugins are statically added. They sensible defaults that can be overriden in a moduleinit. but you can override any of a specific plugins settings using either a VerifySettings instance or the fluent API. If your want to use DI then you can do that with instances of VerifySettings.

here are some some scenarios that may work for you

[TestFixture]
public class Samples
{
    static VerifySettings _sensitiveSettings = new();
    static VerifySettings _relaxedSettings = new();

    static Samples()
    {
        _sensitiveSettings.ImageMagickComparer(threshold: 0.001d);
        _relaxedSettings.ImageMagickComparer(threshold: 0.02d);
    }

    [Test]
    public Task SensitiveTest() =>
        VerifyFile("sample.jpg", _sensitiveSettings);

    [Test]
    public Task RelaxedTest() =>
        VerifyFile("sample.jpg", _sensitiveSettings);
}

or this

TestFixture]
public class Samples
{
    [Test]
    public Task SensitiveTest() =>
        VerifyFile("sample.jpg")
            .ImageMagickComparer(threshold: 0.001d);

    [Test]
    public Task RelaxedTest() =>
        VerifyFile("sample.jpg")
            .ImageMagickComparer(threshold: 0.02d);
}

or this

[TestFixture]
public class Samples
{
    [Test]
    public Task SensitiveTest()
    {
        VerifySettings settings = new();
        settings.ImageMagickComparer(threshold: 0.001d);
        return VerifyFile("sample.jpg", settings);
    }

    [Test]
    public Task RelaxedTest()
    {
        VerifySettings settings = new();
        settings.ImageMagickComparer(threshold: 0.02d);
        return VerifyFile("sample.jpg", settings);
    }
}

[BenjaBobs]

Yes, that seems very nice, but the issue here is that the plugin relies on adding comparers and converters, and these do not live in the VerifySettings, but instead the static context of Verifier.
This makes it very hard to isolate the verification logic.

If Verifier was just the engine, with none of the settings, or if it was non-static, this would not be the case, but as it is currently, how would one create a plugin, e.g. for comparing json-files, solely by using VerifySettings?

EDIT:
An example of this way of implementating plugins/extensions is the System.Text.Json library with the JsonSerializerOptions where you can add new converters and contracts and handling to the options.

[SimonCropp]

This makes it very hard to isolate the verification logic.

you will need to show me a concrete scenario where this is a problem

An example of this way of implementating plugins/extensions is the System.Text.Json

I have tried several time to move to System.Text.Json, and the API is simple not complete enough to work for Verify

If Verifier was just the engine, with none of the settings, or if it was non-static, this would not be the case, but as it is currently, how would one create a plugin, e.g. for comparing json-files, solely by using VerifySettings

interesting scenario. but i cant see a reason why you would want different json comparing per test

[BenjaBobs]

This makes it very hard to isolate the verification logic.

you will need to show me a concrete scenario where this is a problem

Here's an example scenario.
Let's say I have a pdf builder library that wraps some other pdf library (QuestPdf, PdfSharp, etc) in case I want to switch it out later.
Now I want to provide both pdf facilities, and ways to test those pdfs in my two domain projects that each need to create some different pdfs.

src/
├── Pdf.csproj
│   └── IPdf.cs
│
├── Pdf.Testing.csproj
│   └── Init.cs // registers stuff required to test IPdf
│    // ^^^^ maybe contains extension methods on IPdf to Verify with a specific threshold or other customization
│
├── DomainA.csproj  // refs Pdf
├── DomainA.Tests.csproj // refs DomainA and Pdf.Testing
│   └── Tests.cs
│       ├── Test_using_0.05_threshold()
│       └── Test_using_0.002_threshold()
│
├── DomainB.csproj  // refs Pdf
└── DomainB.Tests.csproj // refs DomainB and Pdf.Testing
    └── Tests.cs
        ├── Test_using_0.05_threshold()
        └── Test_using_0.002_threshold()

In this case, I have to have a lot of module initializers in all my projects to register the pdf testing mechanics, instead of just being able to rely on Pdf.Testing.csproj to supply the things I need to test IPdf.
Here as an example, if Verifier was non-static, Pdf.Testing.csproj could have an instance of Verifier specific to Pdf.Testing.csproj which could be used, and would eleminate the need for dependent test projects to have to initialize things.
E.g.

public static class PdfTestingExtensions
{
    private static Verifier _verifier005 = new();
    private static Verifier _verifier0002 = new();

    public static PdfTestingExtensions()
    {
        // in this example I'm using ImageMagic and QuestPdf, but it could be anything
        VerifyImageMagick.RegisterComparers(_verifier005, 0.05);
        VerifyQuestPdf.Initialize(_verifier005);

        VerifyImageMagick.RegisterComparers(_verifier0002, 0.002);
        VerifyQuestPdf.Initialize(_verifier0002);
    }

    public static async Task VerifyAt005(this IPdf pdf) => _verifier005.Verify(pdf);
    public static async Task VerifyAt0002(this IPdf pdf) => _verifier0002.Verify(pdf);

    public static async Task Verify(this IPdf pdf, double threshold)
    {
        var verifier = new Verifier();
        VerifyImageMagick.RegisterComparers(_verifier005, 0.05);
        VerifyQuestPdf.Initialize(_verifier005);

        await verifier.Verify(pdf);
    }
}

Or if the VerifySettings were in charge of plugins/extensions:

public static class PdfTestingExtensions
{
    private static VerifySettings _defaultSettings = new();

    public static PdfTestingExtensions()
    {
        // in this example I'm using ImageMagick and QuestPdf, but it could be anything
        VerifyImageMagick.RegisterComparers(_defaultSettings);
        VerifyQuestPdf.Initialize(_defaultSettings);
    }

    public static async Task Verify(this IPdf pdf, double threshold)
    {
        // create a copy of the default settings to not cause issues in parallel tests
        var settings = new VerifySettings(_defaultSettings);

        // set the ImageMagick threshold specifically in the settings copy
        settings.SetImageMagickThreshold(threshold);

        await verifier.Verify(pdf, settings);
    }
}

---

An example of this way of implementating plugins/extensions is the System.Text.Json

I have tried several time to move to System.Text.Json, and the API is simple not complete enough to work for Verify

If Verifier was just the engine, with none of the settings, or if it was non-static, this would not be the case, but as it is currently, how would one create a plugin, e.g. for comparing json-files, solely by using VerifySettings

interesting scenario. but i cant see a reason why you would want different json comparing per test

System.Text.Json was just an example, since while JsonSerializer has some static methods, all behaviour is configurable through the non-static JsonSerializerOptions.
It could also just be the case that I want to define my own custom comparer for pdf to use for one test, and use one of the other existing ones for another test.
The point is that this is currently a bit hard due to the static nature.

Anyway I hope the pdf example above makes more sense. 🙂

[SimonCropp]

Although it would of course require a bunch of work, also for plugin maintainers.

also note that "plugin maintainers" is me for all but 2 of the plugins