This is what we're doing. Costs for running NexentaStor (not Nexenta) on non-Oracle hardware are compareable to what Oracle charges now.
The risk is whether or not Nexenta and the Illumos folks have the resources to support development of drivers for newer hardware going forward... and whether or not they can count on Oracle doing Solaris source code dumps periodically to either re-base or cherry pick...