Forum sloooow again...

[cdbc]

Hi
In the last 2..3 days the forum has gradually gotten slower, to the point of 'Gateway Timeout'.
I've noted that when the 'Guest' count climbs above ca. 1000 it begins to slow down...
@Marc: I think we need another /dose/ of your powerful 'VooDoo' sprinkled over our /overworked/ little forum...

At the time of this writing the 'Guest' count is ~ 2650 and members 11 
Regards Benny

[Handoko]

We are under attack:

[Joanna from IRC]

Who is doing this?
I just looked and it seems more than one guest is reading same thread. Are they just navigating every thread in entire forums? They seem to be scraping ancient threads

[cdbc]

Good morning Joanna 
Isn't it early morning at yours?
eta: It's 2 in the afternoon, here in Denmark  ...and the sun is shining 
It doesn't make much sense to me either...
Regards Benny

[marcov]

Who is doing this?
I just looked and it seems more than one guest is reading same thread. Are they just navigating every thread in entire forums? They seem to be scraping ancient threads

Probably bots harvesting data for AI purposes.  As a moderator I can see IPs on the page that Handoko quotes, and they are mostly from 2-3 ranges in the USA and Asia.

[cdbc]

Hi Marco
That fits, timezone-wise, for scraping at (theirs respectively) night, with when they annoy us...
Regards Benny
edit: typo

[Joanna from IRC]

Good morning cdbc, yes it is quite early.
Marco Maybe there is a way to ban/drastically slow down the server response time to those who aren’t logged in if they are accessing things too fast?
What an annoying problem.

[marcov]

Good morning cdbc, yes it is quite early.
Marco Maybe there is a way to ban/drastically slow down the server response time to those who aren’t logged in if they are accessing things too fast?
What an annoying problem.

Let's leave the decisions to the forum adminstrators.

[Tony Stone]

Is anyone familiar with YaCy?  It is a P2P search engine.  So I am gonna put this out here now...


If the forum is now all of the sudden being indexed a lot by crawlers of a P2P network I may know something about it.

If that is not the case (admins would know) then all I can say is I have also experienced poor performance on the forum several times in the past couple weeks.

[Marc]

I'm still keeping an eye on it. Last friday I happened to catch a attacker hammering the server with 300 requests/sec. those requests didn't reach the forum, so the server itself could handle it well. It only became a bit sluggish.
I don't want to block to much. Today I got Google reports that the site wasn't reachable. Google was blocked....

[cdbc]

Hi
Thank you Marc, for your vigilance and hard work  ...and not to forget your 'VooDoo', that helps our server catch its breath, once in a while when not being hammered...
We very much appreciate it.
Regards Benny

[TRon]

Thank you Marc, for your vigilance and hard work  ...and not to forget your 'VooDoo', that helps our server catch its breath, once in a while when not being hammered...
We very much appreciate it.

+ 👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍👍

[duralast]

What is the hardware of the database and web server? Cores/threads, CPU, RAM? More than one web server? What is the database? Using connection pooling? Apache or nginx or something else?

Cloud hosted?

What's the firewall? Does it have Deep Packet Inspection to detect and block various types of DDoS attacks, like SYN floods, ICMP floods, and other malicious traffic?

300 requests per second isn't much.

[Joanna from IRC]

@duralast I don’t think that  it would be prudent for those responsible for dosing the forums to be provided with the answers to those questions ..

[duralast]

@duralast I don’t think that  it would be prudent for those responsible for dosing the forums to be provided with the answers to those questions ..

Some of it is freely available. I was on mobile at the time and didn't want to look it up.

This website is running Apache 2.4.41 (w3techs.org), which was released 2019-08-14. The latest available is 2.4.63.
Netcraft shows Apache 2.4.58.
Here is one 2.4.41 vulnerability: moderate: mod_http2, DoS attack by exhausting h2 workers. (CVE-2019-9517)
Here is one 2.4.58 vulnerability: low: Apache HTTP Server: DoS in HTTP/2 with initial windows size 0 (CVE-2023-43622)

lazarus-ide.org is running newer Apache, but the forum is a subdomain on freepascal.org.

The operating system is Ubuntu.
Using MediaWiki 1.39.8. Version 1.39.11 is legacy LTS. Current stable is 1.43.0.
Running jquery 1.10.1 Current core 1.x is 1.12.4.
It is using a MySQL database.
OVH and Hetzner are the web hosting and data center providers in France and Germany. A subdomain is running in the Netherlands.

Is it a dedicated server or virtual private server? Caching? CDN?

Still, 300 requets per second is not a lot. That isn't even getting to the database, which could handle that.