How to deal with technical debt and SOS situations in a digital product

Sometimes, customers come to Evrone for rescue because their product works poorly or not at all, mistakes have been made in the software architecture, or business performance is suffering. When this happens, we have to do serious work on refactoring, legacy reworking, and technical debt analysis.

Management for the project "Let's Eat at Home!" contacted us because their website had begun to crash under the current loads. In the future, this could lead to an outflow of regular users, so it required an urgent solution.

What can go wrong?

Digital products require attention, just like any business does. In every company, management should ensure that production runs smoothly, employees close tasks and have up-to-date knowledge, and customers return again and again.

A digital solution also needs time and resources. Along with the introduction of new functions and features, it is necessary to update the versions of technologies on which the product is written. There should be someone who prepares clear and useful documentation, and infrastructure problems should be resolved quickly and with no interference with scaling.

Most often, the CTO is responsible for development and support, but a company might not have a CTO if the digital product is not a source of profit. As was the case with the project "Let’s Eat at Home!". Their website worked as a channel for brand promotion, so they didn’t have a large IT department. Because of this, there was some confusion with the management overseeing the development of the culinary portal, and then, as a result, technical difficulties.

The site began to crash when the load reached over 100,000 visitors, and updating the functions was difficult due to the accumulated technical debt. The libraries and gems were not kept up to date, so any new function required reworking a large part of the site.

It became impossible to keep the project up-to-date and implement interesting solutions on the front-end.

Correcting errors in a digital product

If you need to gain technical expertise, you have a few options. First, you can hire an entire new staff of programmers to work on the product. Of course, there is a risk of hiring the “wrong” ones or going over budget if more people are required than you had planned for, but in the end, you can get good results with this route. However, this method guarantees all the in-house team issues, including the difficulties of hiring an in-house development team. You’ll have to spend a lot of time recruiting, evaluating candidates, and integrating them.

So, as a more convenient alternative, you also have the option to hire an outsourced team. You will receive the technical expertise you need, but you won't have to figure out where to find and how to evaluate candidates. There will also be no need to organize jobs for them and keep personnel records.

In the case of “Let’s Eat at Home!”, outsourcing was a great fit. They needed a solution urgently, because users left every day while the site was not working. They appreciated our Ruby expertise and experience diagnosing performance issues in Rails applications, so we entered into a contract and started work on correcting the errors in the project the very next day.

First, we audited the software design problems and localized them to three large blocks:

• The site crashes from loads
• The site is open to attacks
• There is no one to support the site after solving the problems

After the initial audit, we fixed the main bugs and relaunched the website. Users stopped leaving, so we gained a bit more time to understand the project in detail and determine how to improve performance of the Rails application in the best way possible.

As a result, we decided to redo the entire backend of the project, because even quick-fix solutions would only add another six months to a year to the project’s functional lifespan. To speed up loading and improve reliability, we reworked the caching scheme, worked on the Rails code optimization, and improved image loading. We also added protection against malicious attacks.

The backend of the product remained in Ruby, but we updated it to the current version. We used PostgreSQL as a database, while Sphinx was used for full-text search on the site (for recipes, ingredients, etc.), and Memcached was used for caching. At each stage of work, we prepared documentation that would be useful for support in the future.

While we were reworking the backend, improving database performance, and helping to do the redesign, our client gained the necessary experience managing a remote team of developers. Using this insight, they were able to gradually recruit their own technical team, which we transferred the site to after the front-end tasks were finished.

Summing up

The choice between outsourcing and hiring your own team can be very important for a digital project. It is very easy to make a mistake at the start, which, in the future, costs a substantial amount of money to fix and can even threaten the existence of the business.

For example, in 2020, when the retail and restaurant industries were forced to go online, companies that were already actively developing their digital channels and investing in infrastructure and technical debt management did the best.

When looking at the causes of technical debt, another example of a mistake that we often see is focusing only on releasing new features. This is a classic example of accumulating technical debt while developing, for example, a portal.

Since business owners often do not have an understanding of the technical nuances, the likelihood of making mistakes in development management is increased. For example, a very small part of the budget and time is spent on updating libraries and tools and introducing new technologies. As a result, the product turns into a solution that consists of patches and "crutches" on the backend. This can affect, not only reliability, but also the turnover of technical staff. Hardly any developer wants to tidy up an unbalanced product architecture.

Fortunately, the “Let’s Eat At Home!” team was aware of the potential risks, so they decided to hire an experienced outstaffed team to quickly resolve issues and support the Ruby on Rails project. After that, they started hiring their own team, once they learned what to expect and demand from candidates. This helped them get rid of the development problems that arose due a lack of management to define parameters, establish deadlines, and set clear goals.

We encountered a similar situation with the Kupikupon project, where we even helped build the process of hiring technical staff and implemented automated code quality assessment. If you are looking for a technical partner who is ready to dive into your project, or you want to find out why you need development management or an outsourced CTO, fill out the form below and we will contact you to discuss how to effectively meet business goals for your company.