[tests] failing test only when run in a group

If someone wants to solve a puzzle, this test:

RUN_SLOW=1 pytest examples/seq2seq/test_finetune_trainer.py::TestFinetuneTrainer::test_finetune_trainer_slow

works on its own, but fails if it’s run in the group with other tests:

RUN_SLOW=1 pytest examples/seq2seq/test_finetune_trainer.py

it doesn’t learn anything – eval_blue remains 0.0

The only small issue is that the test is being renamed and moved to use run_seq2seq.py, so if you’re reading this in a few days, most likely it will be the following case instead – which has the exact same problem:

RUN_SLOW=1 pytest examples/tests/trainer/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_slow

works on its own, but fails if it’s run in the group with other tests:

RUN_SLOW=1 pytest examples/tests/trainer/test_trainer_ext.py

it doesn’t learn anything – eval_blue remains 0.0

Thanks.

1 possible answer(s) on “[tests] failing test only when run in a group