"3. Mechanisms should be established, where possible, for authors to exclude their archived code from the training inputs before model training begins. "
But in practice, they seem ok with this post-training removal process: https://huggingface.co/spaces/bigcode/in-the-stack