Redis to RabbitMQ Migration in a Web Application Broken Link Maintenance

Abstract

The internet has become the most important means of communication in the last decade, and with it, web applications have become a tool capable of using this resource to transmit information to the largest possible audience. As of January 2020, there were approximately 4.5 billion internet users worldwide, representing a significant number of existing web pages and applications. To support the high level of user interaction, web applications employ technologies that launch processes which run asynchronously on different work nodes, thus avoiding affecting the stability of the main processes.

BYE BYE 404 is a web application dedicated to detecting and repairing broken links in order to maintain the integrity of the information contained on web pages. Currently, the BYE BYE 404 application implements a message broker system responsible for receiving and executing tasks asynchronously, by combining Celeris and Redis technologies. However, the latter technology presents stability issues, causing application crashes. For this reason, it was necessary to update the platform by changing the technology to take advantage of the stability offered by combining Celeris and RabbitMQ as the message broker.

As a result of the message broker upgrade to RabbitMQ, the application's stability was guaranteed, resolving the crashes that occurred when using asynchronous tasks, which were taking the application offline. Additionally, a lower percentage of failed tasks was achieved compared to the previous configuration.