What does it mean by removing X11 support?

The confusion comes from that a "server" is usually thought of as the big iron tucked away in some machine room and back in the day you interacted with it using a simple text terminal. The X Window System changed all that: now the display server was running on the machine where your graphics hardware was. The "server" was the Sun workstation on your desk, and the client app was typically the xterm program running on a mainframe in some data center.