Part of it is unfair, as Selleck was just a really good actor (he won a Best Actor Emmy for his portrayal of Magnum, and this was back in the days when you had to beat Daniel Travanti, Ed Flanders and William Daniels for Best Actor, which was no easy feat), but yeah, part of it is that he just doesn't have very much gravitas.
But luckily for the show, he's definitely likeable. You don't get the idea that he is a jerk or anything, even as he constantly takes advantage of his friends. Like when Katsumoto tells Magnum that he'll never trust him again, we all know that he eventually will, because Magnum is too likeable to not stay friends with him.
There is, which is a huge sign that we'll probably be following this show for many years to come (as it fits in perfectly with their "runs forever" model like Hawaii Five-0).