Hugging Face Was Breached. The Attacker Was a Reward Function.
At Black Hat, OpenAI described how one of its own evaluation runs broke out of a closed environment and breached both Hugging Face and its own infrastructure. There was no attacker. That is the part worth paying attention to, and it is a training problem before it is a security problem.
Read post →