Oliver went to prison without telling Felicity last season, as part of a deal with the FBI to catch the season long big bad. He escaped but Oliver still went to prison, and Felicity and his son went into witness protection, where the guy found them and tried to kill them. Felicity became obsessed with finding him and killing him and nearly did, when she was talked out of it in order to exchange Oliver's freedom for him. After he got out Oliver realized that Felicity had gotten obsessed with security (because she had a villain try and kill her while she was on her own with his kid), and she pulled a gun and shot someone who broke into their home into the shoulder, and he doesn't like it.