I don't think anyone makes a decision on the island without some consideration as to whether there will be some backlash against them, whether it helps them, whether it hurts them. Its totally clear that Kelley was thinking that, during her conversation with the producer, as stated by @ProfCrash.
I have no doubt Kelley thought Missy was the bigger threat gamewise. but she was happy enough to vote Dan out, even though he was her ally, when she learned that others wanted to vote him out too because he was making others uncomfortable with his touching as well.
She didn't want to be the only one making that decision, because she didn't want her other allies to blame her for taking away their ally. but sure, when her allies are interested in voting him out, she was totally on board.
She didn't think she needed to use her idol because she thought everyone was voting Dan out. should she have played it to 'be safe' and because she had a spare? Of course. But again, if that had happened, she would have been the deciding factor, as opposed to 'one of the group' and that was what she wanted, thought was best for her game at the time.