I dotn think an LLM even can detect garbage during a training run. While training the system is only tasked with predicting the next token in the training set, it isn't trying to reason about the validity of the training set itself.

You can triage with an LLM, at least. Throw away the obvious junk, have a human look at anything doubtful.

Llm-as-a-judge has been working well for years now.

RL from LLMs works.