AllAstronauts February 2 Share February 2 The world is full of suck right now and that includes AI bots, including those embedded in major search engines natively now and run by well respected business entities (well... maybe just large nominally well-behaved rather than respected...). They're ignoring standard internet practices because those are merely societal norms and not law and anyways who's going to stop us now so we'll crawl as much as we want, when we want, where we want, in quantities of our own determination, at the random times of our choosing. And yes, that rant is on-point. To wit: search, the site search that not too many people use anyways that should never be interacted with by search bots, is now restricted to members only. For reasons. Server reasons. I'll deal with this tomorrow and put up some hard copulate-off guard rails but it's almost 1am on a Saturday night. What's that you say? You work in IT and you think robots.txt is just misconfigured? Additionally if you find the server offline with a response along the lines of cannot load the page or whatever but its the site giving you the message and not the browser itself, that means the database is down again. I'll catch it up in the morning <including the service restart catches which are apparently not there>. I'd tell you more but its all so horrible and stupid. 2 3 3 Link to comment https://forums.primetimer.com/topic/151688-fyi-server-instability/
Recommended Posts