Speculation is easy, opinions are interesting. Some thoughts on what The Foundry should do with Nuke.
The facelift: Redesign the UI. Nuke feels cramped on one display. Lose the floating window model and adopt panes like Shake and Fusion have. Steal a color palette from a website you like.
The if-you-can't-beat-'em: Provide an option to view nodes' output without manually linking them to a viewer node.
The edumacation: Publish some video tutorials and release demos for all platforms.
The big, wide world: Outside the sanctum sanctorum of DD, little things like the ability to use Quicktime movies matter a lot.
The no-brainer: Integrate Furnace technology like crazy. The not-so-obvious adjunct: Don't raise the price in the process.
The clincher: Lower the price (even just a little). You are still competing against Shake, and what you need most is to convert users who have already made a financial investment in other solutions.
The clincher part two: Continue to support Windows, Linux, and OS X. Nobody's gotten anywhere by only living on one of the three, and nobody else is on all three.
The hard part: All the cool kids have particles.