He said he wanted to be friends with her. When she felt rejected, she basically spurned him and made a sarcastic remark about Roger. It seems like after that, they didn't speak or see each other again so I think he's talking about not having her in his life as a friend.
Also thought it was interesting that Townes said he got his visa so fast thanks to the newspaper and the Russian government - and he speculated it was because they thought he might distract her. This makes me think Cleo WAS trying to sabotage Beth in Paris. Not only was she a distraction, but Beth told Cleo that Townes was