Waking up early this morning to the website down, for the second day in a row was a frustrating start to Christmas. We've been incredibly proud to have near perfect uptime over the past 4 years.
In general over the past month we've had to leave up our Cloudflare security check most of the time, with our server being bombarded by high amounts of automated requests. Nothing recently seems to be malicious, though several different parties seem to be trying to run automated scrapes of the website at regular intervals; and in general AI language models are bombarding the website with requests consistently.
The long-term solution is simply rewriting high-resource pages/features to better facilitate the increasing volume of traffic, we already utilise significant caching for performance purposes. However, many of the required changes require rewrites of core functionality, it's a huge overhaul that can only reasonably be completed at the same time as MedBud 2.0 – our mammoth on-going UI redesign, focused on mobile compatibility (expect an announcement in the new year, with major progress made recently).
In the short-term, @GrownHealth has been highlighting issues users are having with Cloudflare security checks which we feel needs solving quickly, so I've decided to brute-force a solution and upgrade our server significantly - something we've wanted to do for months now.
New Server Architecture
___
If you're reading this post, you're already live on our new server. While I have already been told off by @Tim Henley for working on Christmas morning, my excuse is that I was waiting on my roast coming up to room temp before it goes in the oven – and sincerely, MedBud deserved the Christmas present too
Merry Christmas everyone, we're looking forward to the new year.
In general over the past month we've had to leave up our Cloudflare security check most of the time, with our server being bombarded by high amounts of automated requests. Nothing recently seems to be malicious, though several different parties seem to be trying to run automated scrapes of the website at regular intervals; and in general AI language models are bombarding the website with requests consistently.
The long-term solution is simply rewriting high-resource pages/features to better facilitate the increasing volume of traffic, we already utilise significant caching for performance purposes. However, many of the required changes require rewrites of core functionality, it's a huge overhaul that can only reasonably be completed at the same time as MedBud 2.0 – our mammoth on-going UI redesign, focused on mobile compatibility (expect an announcement in the new year, with major progress made recently).
In the short-term, @GrownHealth has been highlighting issues users are having with Cloudflare security checks which we feel needs solving quickly, so I've decided to brute-force a solution and upgrade our server significantly - something we've wanted to do for months now.
New Server Architecture
- We've moved from an AMD Ryzen 5800X (8 core) to an AMD Epyc 4585PX (16 core) processor, with around a 25-30% uplift on individual core performance (effectively the newer server version of the AMD Ryzen 9950X).
- We've doubled our server's RAM from 32GB to 64GB.
- Our public uplink has been upgraded from 1Gb to 2.5Gb (backplane 25Gb).
- We've moved to newer NVMe drives with over twice the I/O performance, upgrading from PCI-E 3 to 5.
___
If you're reading this post, you're already live on our new server. While I have already been told off by @Tim Henley for working on Christmas morning, my excuse is that I was waiting on my roast coming up to room temp before it goes in the oven – and sincerely, MedBud deserved the Christmas present too
Merry Christmas everyone, we're looking forward to the new year.
