It's certainly possible to live in Linux without file managers, but sometimes it's almost painful to write series of commands which can be done in a few keypresses in a good file manager. Does not happen that often, so it's not a deal-breaker.
I guess, users of vim/emacs feel the same when they are forced to use less powerful editors.