Maybe Mary was just tired from spending the morning with the family and needed some quiet time alone. And then she fell into the pool. Accidental deaths are an unfortunate fact of life. That doesn't mean she was being ignored.
My mother is 82. She has dementia but a few years ago when she was still lucid she needed her quiet time. She lives 10 minutes from my brother, another brother comes to visit for a week every summer with his family. They would bring her over every day, until she asked for a day off because she found all of the commotion exhausting. This was partly because she is hard of hearing and even with the hearing aid when there is a lot of background noise it is difficult to understand what people are saying. She was not being ignored, on the contrary, my brothers were respecting her wishes.
Jim Bob is cheap as hell, but Mary evidently had money to pay for in home help if she had needed it. She appeared to be frail but it seemed she was still able to make decisions for herself. I always got the impression that she was careful and smart with her money and I doubt she would have let Jim Bob take advantage or strong arm her into doing something she did not want to do.