Actually, I would absolutely think less of a man who did such a thing. I had an immediate dislike of Joel for S1E1 for the blase way he left, not only his wife, but his children. I find it to be a super-shitty thing for any person to do. I have never believed that "finding oneself" should be accomplished by mistreating the small people we have decided to have and agreed to raise. And expecting other people (i.e. our parents) to take care of our responsibilies is also a terribly selfish idea.