Victoria’s deeply insecure, and very likely Borderline Personality (which is incredibly difficult to treat and does not miraculously disappear when subjected to “tough love”). The eye contact thing is 100% in line with my personal experiences; some people truly can not make eye contact in stressful situations, and the shakey eye thing when looking away confirms it for me. I truly feel bad for her, but I’m also mature enough to not let somebody like that make their problems into my problems. And for all her problems, she still has enough control over her actions that I'm not excusing any flat-out bad behavior, just understanding when she's temporarily overwhelmed by her emotions.
As for why he keeps her around, I think it’s not just sexual attraction. Really, they genuinely have fun together. They are at the same level as far as maturity and intelligence. Watch their dates when they’re doing goofy stuff like the amusement park (not the concert section with her ex-one night stand) and the Old Timey dress-up. They are so clearly having a ton of fun, especially compared to just about every other situation on the entire show where he & the women are pretending to have fun. With P & V together, it’s completely different. They are on exactly the same page, unlike with the other women who are too good for him. I don’t mean they act like they’re too good for him, I mean they actually are, even if they won’t quite admit it to themselves. Or, if not too good, then at least too mature. VF's girly flirty behavior completely connects for him, however much it would drive most people up the wall. I bet his mother is cutesy the same way. (Ick.)
Add in that Victoria is dramatic exactly like his mother, too, and they just fit. They’d never last as a couple because neither one has the social/communication skills or maturity, but they do fit together. Even the Sharleen criticism cited above (Sharleen wrote "... Victoria F relies heavily on a pattern of defensiveness, playing victim, fleeing the scene, and then apologizing weepily …”) describes Peter exactly as much as it describes Victoria. How many times all season has he run away, cancelled cocktail parties, walked away from dates when he’s overwhelmed, or felt attacked when his bad behavior is pointed out? I hope they end up together because that at least protects the rest of the world from relationships with either of them. Well, until she upsets him and he runs home to mommy, or he upsets her and she runs off and shuts down.
I like Hannah Ann alright and she may be immature enough for him but I don’t think there’s anything there, and he is not a Real Man like Daddy. And if Madison wants someone with any actual religious convictions, Peter’s the wrong guy for her. He’ll play along with whatever, but he’s never going to be a faith leader for a family. He’s a goofy spoiled rich boy who likes airplanes and simple music and pretty girls and just happily floating through life, with occasional big drama and tears over nothing much at all.