I don't see how we are ever going to agree, because we see something like "running apache on a high port on an internal desktop for internal business purposes" differently.
You view it as developer Joe just wanting to be able to run the server for his own use, and it cannot possibly harm anyone so long as the corporate firewall works.
Management may see it as (a) something that others may come to rely upon and when Joe leaves for another firm someone else has to take over requiring that the code be brought up to the appropriate standards or (b) an internal application that leaks data across business lines and fails to integrate with the standard security policies managed by the firm.
Similarly with installing Inkscape, it could become something that is integrated into a process without approval, such that nobody knows how to manage it when the installer leaves, or could expose the firm to legal risk down the road (those automated charts Joe creates with inkscape are deemed to be deceptive according to Regulation 142.6(a) subsection (iv) paragraph 3.14 which requires that all bar charts have a width of at least 22px). I personally don't like this attitude, but after working in a regulated industry I recognize that it exists.
You are also implicitly suggesting that every machine on the internal network run a firewall that blocks incoming packets on high ports. How many companies actually do this, vs just having a firewall at gateway? It could be a lot of work for network admins to customize the firewall rules to the individuals machine.
I'd also be curious to know how a company like Google handles this kind of situation. Certainly their staff is skilled enough to be able to run personal web-servers, but at the same time someone with a misconfigured server could leak data across the google network.
A lot of this audit stuff is ridiculous, and I think there is a tacit recognition that it is absurd. Saying that an employee went off an did something on his own without approval makes it possible for the corporation to avoid liability, whereas if you all them to apt-get install something its much less clear that they were violating policy in doing so.