March 2024 Performance Issues

[Matt K]

Received some reports of sluggishness today. There was a memory leak in the database that I just fixed which should address the issue. Let me know here if anyone is sitll having issues and I'll take a closer look into it today.

[mgladdish]

I literally got a CloudFlare error when trying to view this thread

Also been having intermittent errors connecting to MySQL over the course of the day.

[bitbckt]

Much improved, particularly jumping to the latest posts on longer threads. Thanks!

[ghmerrill]

Some of the "issue" may not be with this site. I was having very sluggish network performance (via our supposedly 40M DSL line) this morning for about half an hour and then it improved. But I'm currently getting half the download speed that we're supposed to -- 25Mbps vs. 40Mbps. How local that is, I haven't looked into -- because life is too short.

[BGuttman]

I had noticed a bunch of duplicate posts (more than normal) today. Probably because the posters experienced the delay and thought it had not posted.

Note that if you double post and I catch it, I'll remove one. If one of the posts has more comment I'll keep that one.

[hyperbolica]

It was bad, then it improved, and now it's bad again. I think it's larger than just this site. There was a cable break of a major line in the middle east. That may be causing traffic to re-route and disrupting us here.

[ghmerrill]

There appear to be jagged spikes in "bandwidth outages" and DNS server issues beginning this morning. The graphs are saw-toothed. Up and down, up and down, ...

[Slidingmyway]

I had noticed a bunch of duplicate posts (more than normal) today. Probably because the posters experienced the delay and thought it had not posted.

Note that if you double post and I catch it, I'll remove one. If one of the posts has more comment I'll keep that one.

That happened to me. I accidentally posted the same thread 5 times. My apologies to whomever had to clean up my mess.

[Posaunus]

Response speed briefly improved this morning (I think). Now again very slow.

[EriKon]

Yeah, takes quite long to load everything since one or two days. For me it's just on this page.

[Posaunus]

Just 5 minutes ago, Website speed magically improved. Almost immediate response!

(Let's hope it remains so!)

[EriKon]

Yes, over here as well!

[Matt K]

Hmmmm interesting. I have a family issue today that cropped up so I haven’t had a chance to look into it. I think the connection stuff is definitely partially to blame. It may not have been a memory leak but if the connections are slow then it’ll leave handles to the DB open which would look like a memory leak.

It’s still a little slow for me as I type. I’ll keep my eye on it as my family stuff hopefully resolves today and see if I can (ideally) permanently fix tomorrow if there are any issues that are fixable by me

[EriKon]

Seems like it's not a constant thing over here. Sometimes it works super fast for a few minutes and then again it takes a minute to load everything (sometimes with error pop-up)

[mgladdish]

Yeah, still very bad. Just got this when loading a topic:



If I can be any help on the tech side, give me a shout. Techie creds: https://www.linkedin.com/in/mgladdish/

[Posaunus]

STILL ... ... VERY ... ... SLOW

General Error
SQL ERROR [ mysqli ]

Lock wait timeout exceeded; try restarting transaction [1205]

An SQL error occurred while fetching this page. Please contact the Board Administrator if this problem persists.

Please notify the board administrator or webmaster: [email protected]

[slideandtraps]

Experienced above SQL error this evening and personal total posts dropped to 1
with "800%" and "300%" of total posted in other topics.

[Matt K]

Okay, think I found the culprit. Everything all good with my family too, one of my relatives had a minor surgery last night but all is good today. I'm going to keep my eye on it today and will need to put a more permanent fix in this weekend.

[EriKon]

Seems to work fine now. Thanks Matt for taking care of that! Much appreciated!

[sacfxdx]

Where's that AI when you need it? It should have fixed it automatically. I guess it's too busy faking nude pictures.

Good job Matt. Thanks for taking care of this.

[Matt K]

Where's that AI when you need it? It should have fixed it automatically. I guess it's too busy faking nude pictures.

Good job Matt. Thanks for taking care of this.

Welllllll about that. Turns out the script I’ve been using to capture spam, bots, and AI has became quite expensive computationally because of some automatic rules compounded with an enormous amount of AI generated garbage. The solution is actually not a big deal, just didn’t expect to have it come to a head this quickly. I was hoping to hold off until the next LTS of Ubuntu came out and switch over to that and fix a bunch of stuff at the same time (next LTS drops in April)

[ghmerrill]

... enormous amount of AI generated garbage.

Dumb AI. Well (in virtue of my 10 years in a previous life as a commercial compiler writer), It takes a very well designed automated code generator (and read that as "often very expensive") to generate optimal code or anything approximating it. And if you're doing some or all of that on the fly at run time, then there's an added efficiency cost as well (in optimizing the optimizer). Other than that, you can only do your best with what you can get.

[Posaunus]

Matt,

All good this morning.

Thanks for your efforts on our behalf.

[Matt K]

I think I have the spam busting script back in place more efficiently now. It just ran and spike performance for a few seconds but it shouldn't be much more noticeable than typical usage and runs every few minutes, but only if there have been posts. LMK if anyone has issues tonight/into tomorrow.

[ithinknot]

Have you noticed that user post counts are displaying weird numbers? Sometimes it's just a bit wrong, in other cases longtime users are showing as 1...

At the time of writing this, mine says 1010 next to a post or when you open my profile, but if you click through to 'Search user's posts' it finds all 1173.

Must be indicative of something.

[Matt K]

I reset one of the sites indexes the other day which has the side effect of recalculating the number of posts. I’m not sure why it thinks you have fewer posts than you do, unless the search results show duplicates for some reason

[harrisonreed]

It might be counting quoted snippets as posts.

[ithinknot]

It might be counting quoted snippets as posts.

Nah, it's just nonsense in some cases.

To pick a couple of posters in this thread:
memberlist.php?mode=viewprofile&u=9927
search.php?author_id=9927&sr=posts
56 vs 152

memberlist.php?mode=viewprofile&u=13350
search.php?author_id=13350&sr=posts
12 vs 32

Obviously this doesn't particularly matter in use; I'm just bringing it up in case it's diagnostically significant...