I feel the exact same way. I got through maybe an episode and a half of Clone Wars - did nothing for me. Yes, I've heard that it gets better and is awesome towards the end. I've seen all of the SW movies and enjoyed them to varying degrees (I'm an original trilogy purist, I guess).
I would probably bail if they move on from our basic premise of Din & the Child. I'm not excited about these other SW characters making an appearance; it narrows what, for me, should be a huge universe. I like the focus on Dad & baby, with new characters popping in once in a while or being revisited. I have zero interest in another Mandalorian replacing Din.
Though Pedro Pascal seems like a great guy, well, there have been other actors who seemed great who turned out to be not so great. So I don't know. Plus, if his movie career is taking off, he might not have as much interest in a time consuming series where his face isn't shown.
It wouldn't be the first time that showrunners have specific plans for their series of which I don't approve, heh! I'll keep watching and enjoying for now.