Physical attraction is a strange thing. I've been in the situation where I was really attracted to someone's personality but they had certain physical traits that I just could not deal with. I felt terrible and tried for weeks to see if I could get past it. Even had a make-out session. Based on that experience, I can understand why it could happen on a show like this.
Alternatively, I've had the experience of not finding someone particularly attractive, but then once I got to know them, my feelings did a complete 180 and I found them to be the hottest person in the entire world.
Also had the experience where I gradually got to know a co-worker over many months. Slowly I started to have a massive crush on them. Then one day, I went over to their desk and as I leaned over to show them something on their computer screen, I got close enough that I could smell them. Instantly I was 100% turned off. It was not body odor, but just their natural scent. My crush immediately ended.
It would be too much work for these shows, but it might be nice if they could find a handful of possible matches and then show the candidates the photos and ask which ones they are and aren't attracted to and then choose one of the ones they found attractive. Or... maybe they should let them smell the person's dirty t-shirt and then pair them with the one whose smell they liked best. That would be pretty interesting.