The biggest problem with console VR is the garden walled ecosystem. Just in the way that PC gets far more games and experiences VR does as well.
You're at the behest of Sony and whatever they decide to approve. You'll never get something like VR chat, you'll never get lewd or sexual content if you're into that, you'll never get those obscure Indie games which end up being very popular, you'll never get most of everything.
The other problem is most VR Developers don't have a lot of money. Porting a game to an entirely different platform cost a decent chunk of change and resources. If they want to get a game on PlayStation they have to pay platform costs.
Along with games not being compatible these are just some of the other reasons why a VR ecosystem on a console is not worth investing in.