Both of the shows filmed in the Czech Republic. I work at a Norbertine college, and tooka tour of Norbertine sites in Germany and the area last spring. We visited Strahov Abbey in Prague, which is where the two libraries of the abbey served as sets in both series. The wood-panelled library is the main library of the abbey, the one where Jamie plays chess. The other one, the Philosophical library was also shown in Outlander - as part of Louis' suites of rooms when Claire visits him to earn Jamie's release from the Bastille. She walks through a beautiful rococco library with on her way in and out.
The town set of the Musketeers is at the convent at Doksany, in the Czech republic. We got a chance to visit there, and walk through the set, which is built among the existing buildings. It was really hard to tell, even close up, which were real stone buildings, and which were only wood facades. Here's where Jamie plays chess.