Allow tests to run in parallel

[anderseknert]

With almost 700 unit tests in the Regal codebase, and more being added, we've now reached a point where the test execution time is... noticeable. Not slow, mind you, but a couple of seconds on our fastest hardware, and many more in contexts like GitHub runners.

One obvious improvement would be to allow running tests in parallel in order to make use of more than a single CPU core where available. Rego tests should be fairly easy to parallelize given how they can't easily depend on state getting mutated between tests, etc. But I can't say I have looked into this much yet, so feel free to raise your voice if there are some things a parallel implementation would need to consider.

Having an option / opt-in to set the level of parallelism would be a great first step. Later I suppose we could either make this default to the number of cores available (like opa run) or perhaps some smart heuristic based on the number of tests to run.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.

[anderseknert]

Great work @sspaink 👏 Did you have some code for integrating this in Regal laying around? :)

[sspaink]

No integration changes required! tester.NewRunner() will use the number of CPUs by default and run the tests in parallel. Will need to add a new flag to Regal to adjust the number of goroutines though.

[anderseknert]

Awesome! Thanks @sspaink 👍 I guess you can already adjust this via GOMAXPROCS? And I don't see many reasons for why you would want to change the default, so unless someone asks for it, we may just skip that flag.