So I went back and looked at some photos of him riding to work on his first day, and sure enough he's got the reins in his right hand.
Riding an english saddle too, but that's probably all the Park Police had.
The reins are held in your non-dominant hand. So either Zinke is left-handed (and I'm told he's not) or, just like with the cowboy hats, he's trying to portray himself as something he's not. That's not all that unusual for a politician, but these Zinke photo ops are exposing the truth. Maybe he should just walk and wear a cap. His job is tough enough as it is. Drop the image stuff and keep making good decisions.