Yes, it was. From their site: "It has been trained on a selection of English language and source code from publicly available sources, including code in public repositories on GitHub."
Once you put something on the internet, you should assume it still exists out there somewhere even after deleting it. Even before copilot, all credentials that end up in a repo needed to be changed. I'm not sure what's supposed to be different now.