I think Djokovic just doesn't have the humor or charisma to be a real fun tennis villain. He comes across as so self-serious and sort of a gamebot.
People like Tom Brady, they have a real "I don't give a f__k what y'all think, I'm just going to win" energy. You might not like it, but you respected it.
Djokovic comes across as so whiny and needy.
And this is petty, but the way he and his wife relentlessly try to brand themselves as this SuperCouple on social media is also annoying:
https://www.instagram.com/p/C9O8uaeoMZF/