Yes, yes, yes. I started using debian unstable many years ago, and like you, found it difficult. However, it was a lot tougher back then, and there are techniques you can learn to avoid problems, which is no different really from knowing how not to cause yourself problems with a release version. Most debian desktop users run unstable, and the tools and the distribution itself copes very well with making sure you don't burn yourself too badly. It's at least as safe as playing with ports, and probably a LOT more safe.
I also use it on servers, most of the time. The availability of debian unstable's modern, huge selection of managed, up-to-date packages, all downloadable at the stroke of a key, together with a little know-how, is much better than relying on something like a release from CDs.