I don't remember the conversation you're asking about, but, yes, the couple with the baby on the ship were Roger's ancestors--Geillis and Dougal's son, his wife and his child.
They don't really know how the stones work, not exactly, but it does appear there is both a physical (gem stones and certain metals) and mental aspect (thinking of where you want to go) to being able to steer successfully while traveling. Remember Roger's first attempt failed because he was thinking of himself as a lad and started traveling back to that time--which apparently you can't travel back to a time when you already existed. So, IMO, Brianna is wrong that you need someone on the other side, but I do think you need to think of where you want to end up and having someone you think you're returning to might make that easier.
BTW, I'm not so sure it was a total accident Claire ended up where she did. It does seem like 200 years is the default one travels if they aren't focused, but there had been lots of talk about Frank's ancestor over the days before she traveled and the night before Frank had talked about the Highlander ghost who is presumably Jamie's ghost. So, I do think it likely that Claire could've been thinking of these things when she traveled which could've drawn her to that time when both Jamie and Black Jack existed, as well.
It's been a while since I read it, but, as I recall, Roger knew where this person he was stealing the gems from was going to be on a certain date and then this person was supposedly sailing back across the ocean. So, if Roger was going to get the gems, this was his only opportunity. However, I agree with you that it was really silly. It seems it would've been more prudent to go with Brianna and meet the in laws instead of heading off on a foolhardy errand that had little chance of succeeding anyway. More likely he end up dead, IMO, but what do I know! ;)
Don't even get me started... . ;)