I noticed the panning of the background layer (skybox and planet) looked odd when flying around and remembered that I had a MatchRotation script on the background camera that matched the player ships rotation. This worked great when the main camera was fixed to the player ship but looked weird when using a Cinemachine follow cam. So I changed the target of the MatchRotation script to be the follow cam which fixed the issue. 

I did some more game balance tweaking.