You are definitely right about the whole desktop being an IDE.
However I would like to add that there is actual IDEs (in the Windows sense) available such as the excellent KDevelop, Anjuta, Eclipse and a handful of others. Its annoying that so many people do not bother to actually look at alternatives. Do they really expect Microsoft Word to be installed in the Linux desktop by default? It's like expecting your kitchen microwave to browse the internet.
Going out further, the CLI is also a great IDE and if you set it up well and learn tricks it can absolutely blitz any graphical setup in terms of productivity, but this is a setup that should be approached willingly and not kicking and screaming.