Here's why what you are doing is tricky.
When you change use flags and recompile everything, they are all compiled in a compatible way. The packages built in entropy are built this way and the packages on an individual machine are also compiled this way.
But, if you mix and match, they might not match up nicely. If you decide to recompile a library and you end up removing support for something that a prebuilt (entropy) package relies on, there's a good chance that application will crash. The lower you go, the better chance you will break something.
If, for the most part, you are simply compiling applications differently without compiling their dependencies, and they themselves aren't a dependency, then you'll be perfectly fine. Worst case scenario, that application might not work right, but you won't really harm your system.
If you decide you want to recompile, let's say, all of KDE or gnome to have different use flags, then you are going to have an extremely rough time.
If you are looking to go the other way around, and simply install a few entropy packages on a mostly compiled portage system, those packages will most likely not work right. Entropy, by default, won't even let you do that although it certainly possible to get around it's warnings and pleas for sanity.