Introduce “noarch” tests to the test suite

Currently, we have slow/fast tests in the test suite, which are intended to allow people to add long running tests to the test suite without gumming up wait times for PR CI turnaround.

I propose to add another new category to the test suite: boring noarch tests. Noarch tests don’t exercise any interesting aspects about the underlying CI configuration; in other words, if the noarch tests pass on one configuration, they probably pass everywhere, no need to burn CPU hours in other configurations. A single test configuration will be anointed the “noarch” test runner and we will only run noarch tests there.

Alternative: mark noarch tests as slow tests. I don’t like this alternative because it implies that the noarch tests are slow, which they are not; the tests I intend to add in #53682 individually run very fast; there are just a lot of them so it takes time to get through all of them.

cc @mruberry @VitalyFedyunin @walterddr @malfet @seemethere

1 possible answer(s) on “Introduce “noarch” tests to the test suite

  1. @walterddr p sure Ed means @slowTest decorator, as that allows developers to mark up any test, pretty much, where as the SLOW_TESTS list (now TARGET_DET_LIST) is applied every PR to only run tests that seem to have been affected.