I can't help but feeling like if we're just going to be talking about copyright, we've jumped the shark.

As an aside, that reminds me of an interesting bit of history in the famous Sony Betamax US Supreme Court case. In Justice Stevens first draft opinion, he wrote, "It would plainly be unconstitutional to prohibit a person from singing a copyrighted song in the shower or jotting down a copyrighted poem he hears on the radio". That second scenario, transcribing a poem, gets black letter copyright law dead wrong, at least in the sense he meant--his original opinion argued the Copyright Act didn't apply at all to such private, personal use. IIRC, the other justices had to correct him that that is the very definition of copyright infringement, and the definition of infringement per se never distinguished private from commercial. Fair Use would protect a person from liability (and the First Amendment would likely demand it), but it's nonetheless infringement, per se; Fair Use is a *defense* to infringement.

> What if I do what a lot of musicians do, and extemporise a piece where I have no idea where the inspiration came from?

In the US, at least, you generally prove infringement by showing 1) substantial similarity (to copyrighted material) and 2) access to the copyrighted work. It doesn't matter your intentions or whether you even knowingly copied or made a derivative. Access can be proven by showing that, e.g., a work is effectively universally encountered, e.g. Beatles' songs.

As regards AI or anything else, substantial similarity and access are really all you need to show infringement. Showing access in the case of AI is basically a slam dunk for any published work, given large models have been trained on effectively everything that has been digitized. Copyright cases usually turn on the degree of similarity, or other issues, like whether a work was actually protected by copyright, whether copied material was copyrightable, establishing the rights holder, application of Fair Use, etc. But the question whether infringement could occur is the simplest aspect to copyright law, at least in countries where copyright law derives from common law copyright. Your hypotheticals don't actually pose any sort of challenge to the basic logic of infringement in this regard. The keystone to copyright law is the concept of originality, and answering the question of originality is fundamentally just 1) substantial similarity and 2) access.