Sorry about the server outage · 2012-11-23 14:20 by Wladimir Palant
Following the Adblock Plus 2.2 release yesterday our server became pretty much unreachable. This was the result of the new typo correction feature downloading its database. Unfortunately, while I was very focused on ensuring that this feature has the necessary quality I entirely forgot what this one server request per user would mean for our old server which cannot really handle much load. A new server with much more resources is handling this now but that change still took a day to propagate. Sorry about that, again a lesson learned.
On the bright side, we are working on replacing that old server completely and building up a more reliable infrastructure where our fallback doesn’t take a day to kick in. It isn’t quite trivial because that server is running many applications and we want to avoid downtimes whenever possible. Still, we are getting there and should start migrating stuff soon.
In case anybody is interested in the technical details: the plan is to separate all applications from each other properly. We will install each in its own Linux Container which appears to be a very light-weight virtualization approach. And we intend to use Puppet and Vagrant to set up these containers automatically (identical setup for test environments and production servers).
Commenting is closed for this article.