I know you can't fly there, you can still ride. Most of the time on my server, people don't bother you unless you mess with them first. During the actual battle however, everybody is either in the battle or crowding at the gate, so there's barely anybody out an about on the peninsula.