I am also having these random crashes...sometimes I get the error msg-nvidia driver stopped working and it has recovered, or sometimes there is nothing at all. I have tried running the Nvidia hotfix driver and that didn't help either. Went back to the official Witcher 3 drivers...still happens once in a while... I have used DDU to completely remove my old drivers and install the new one...
I don't get crashes on any other games, I am pretty sure it's the Nvidia Driver though.
My specs:
Asus Z97Pro
Core I7 4790
16GB DDR3 1600Mghz
EVGA GTX 980SC
Samsung 840 Pro SSD 256GB
Windows 7x64 Ultimate