Princess Anne was done with Parker Bowles before Charles met Camilla, so no overlap.
I think that was perhaps because Margaret showed "bad taste" by talking about her troubles in her birthday party. It's regarded vulgar to ask for pity.
When Elizabeth earlier visited Margaret, she was sympathic - the difference was that they were alone.
I also don't know if Tony was "openly" cheating on her. Was he bringing his mistress to parties, or even appearing in public with her? If men were discrete, their behavior could be overlooked. Margaret and Elizabeth's father didn't have mistresses, and Margaret appears to have really loved Tony, so she couldn't overlook it. Plus, Tony seemed to want to enjoy their volatile relationship whenever he felt like it, with no thought to what she would be doing or feeling in the interim.
I hope their children were at boarding school, because they didn't seem to get any parental attention. In real life, hadn't Margaret given birth to her younger child very shortly before the time she is shown as having visited President Johnson? She could have felt more useful just by being with her children, but I suppose she hadn't really experienced that growing up.