February 2024 update
📈 Performance improvements
February was a bit of a stressful month. Between the 3rd and the 4th, and again between the 7th and the 10th, AstroBin suffered severe performance degradation several times a day.
While that was going on, you'd get very slow loading pages, pages that just wouldn't load at all, and the dreaded "Oops, AstroBin is not available right now" error page.
Apparently, a not-so-friendly crawlerbot associated with spamming activity decided to crawl AstroBin at a heightened rate, with requests coming out from Hong Kong, Singapore, and Frankfurt.
This ended up stressing too much a caching server that was not equipped to handle that kind of load, and was being the weak link of the chain, in this case.
Unfortunately, this took me several sleepless nights to figure out, until I started deploying application performance metric tools that led me to understand exactly where the problem was (this is not an ad, but thank you Sentry for being awesome!)
As usual, what doesn't kill us makes us stronger! So once I had better tools to analyze performance issues, I delved down the rabbit hole of optimizing AstroBin's performance. 😀
Without going into technical details, here's the key points:
Overall, the average request response time went from 0.27s to 0.17s, while the slowest 1% of requests went from an average response time of 2.9s to 1.3s. (before you comment that no page on AstroBin opens in 0.17s, this is the average response time from a server as detected by the load balancer, and it include several tiny and fast requests that pull the value down; it does not include network time and render time on your device)
The mean time to respond to an image's page went from 0.7s to 0.35s, while the slowest 1% went from 2.1s to 1.0s.
The biggest improvement went to the Search page, which went from an average of 2.5s to only 0.5s!
The number of database queries was substantially reduced due to lots of tweaks that I did, which will allow me to move a considerable portion of the budget for infrastructure to other parts of AstroBin's complex architecture, therefore further increasing performance and/or reducing cost (this will be later in the year because the database server instances are already paid for in advance).
Just during the last couple of days, I also did a number of stability improvements on the app.astrobin.com server infrastructure (this hosts pages like the image uploader and data editor, the equipment database, the subscriptions...) so if you frequently saw delays, especially in the equipment database, during the past weeks, those should be fixed too.
Besides the story told by the numbers, personally I can see that the website is a lot snappier than before, on average.
I still have a number of improvements that I can do, to cater to the slowest 1% of requests, that are a bit more difficult to implement.
I would like to thank you all for your patience and understanding. AstroBin is a pretty complex system with lots of moving parts, and I'm doing my best to stay on top of it.
Unfortunately, this month of performance related work put a huge dent in the mystery feature I'm working on (that I mentioned in previous newsletters), as I had to put it aside completely to work on all of the above.
I can finally resume working on it, tho, now! 🤩