You need SteamVR set up and running to do most VR things on the Mac (at least for Vive), even if they aren't from Steam.
Connect your headset and run Steam. It should detect the headset and offer SteamVR to you. Once it's installed, you can then run Room Setup.
You may optionally want to switch over to the beta to get the latest updates:
If you encounter problems and want to file bug reports, the SteamVR GitHub project is:
https://github.com/ValveSoftware/openvr
Very few SteamVR experiences work on the Mac at the moment. Here's the list:
There was an error while loading. Please reload this page.