It really is your right as a spouse to speak with her