Not trained on MS closed source

GitHub does not require uploaded code to be FOSS, as long as it allows the use of GitHub's "fork" button (loosely equivalent to git clone), and one or two other pieces of site functionality. In theory, they could have limited their training set to only include FOSS repositories, and not proprietary or no-license repositories (in most jurisdictions, no license means the same thing as "all rights reserved").

But I couldn't find any statement in their FAQ one way or the other - it just refers to "public repositories on GitHub," a category including both FOSS and proprietary code. It's entirely possible that they are using all of that code, and IMHO that seems like the most straightforward way to read the sentence (which doesn't mean that it is the intended meaning, of course).