How to Optimize Python for High-Load Services

Pros and Cons of Python for Large Projects

Python is an interpreted language, which makes it slower than compiled languages by default. Additionally, CPython has a Global Interpreter Lock (GIL) that prevents multiple Python threads from executing simultaneously, limiting its capabilities as a programming language for high-load systems. However, with custom Python software development, these limitations can often be mitigated by optimizing code and leveraging Python’s flexibility.

Python offers high-level abstractions that simplify development but can introduce performance overhead compared to low-level languages. Moreover, dynamic typing adds extra processing costs for type checking, which slows down code execution.

On the other hand, Python is loved by businesses and developers alike for its simple and concise syntax and convenient memory management. This ease of use makes it a popular choice among programming languages to accelerate coding. However, this combination does impact speed. Writing in Python is quick and easy, making it suitable for a wide range of projects, and it boasts an active community that has developed tools for almost every need. However, when a project initially written in Python begins to scale and encounters high loads, this becomes a problem.

So, what can you do? First, you can use C through extensions or by leveraging Cython, which allows static typing in critical code sections. Second, you can use JIT compilers like PyPy. Third, caching. Properly implemented caching can significantly improve backend performance.

However, these methods are suitable for specific optimizations but not ideal for actual high-load situations, except perhaps for C. But even there, nuances exist. In our work for a client, we found another path, which we will describe below.

The Task

One of our clients, a major tech company, approached us to develop a service concept. This service needed to accept messages, interact with them in real-time, and then transfer them to a long-term storage system for further analysis. The service had to meet two essential requirements: handle 100,000 requests per second and not lose a single message, making real-time data processing critical.

To make sure messages weren’t lost and to avoid overloading the processing component, we chose a solution based on Kafka, which facilitated the delivery of these messages to various microservices within our product. The service had not just to accept requests but also to enrich them, process them, and only then transfer them to long-term storage. Processing included validating HTTP requests and enriching Kafka packages according to configurable rules. Additionally, HTTP handling had to work with blacklists, which could include specific IP addresses and incorrect counter metadata. The most heavily loaded part was receiving requests and sending them to Kafka. Messages arrived via HTTP in JSON format.

Initially, the client envisioned implementing the service in Go because it’s fast and suitable for high-load systems. We were tasked with finding a solution that would provide optimal performance under load and be cost-effective from an infrastructure standpoint.

Finding a Solution

Which language should you choose for the backend of a high-load service? The client’s primary stack was Python. Therefore, we decided not to abandon it immediately and instead compare it with other languages. The shortlist included Go, Python, and Rust.

We needed to select a framework for each to minimize our workload. We tested AIOHTTP, Litestar, and Robyn for Python, but ultimately settled on Granian—a framework written in Rust. After some optimization, this solution helped us achieve the necessary performance, showcasing how effective performance testing with Python can be when combined with the right tools.

To test performance, we built four services: in Go, Rust, native Python, and optimized Python with Granian. This allowed us to conduct thorough Python performance testing and compare the results against other languages.

Testing

To make sure our solution could handle 100,000 requests per second, we needed to create a test environment that could generate such a load. For load testing, we chose Locust, which, by the way, is also written in Python.

To dynamically scale the load resources, we deployed the infrastructure in a Kubernetes cluster in the cloud and automated everything with Terraform.

For each test service, we prepared Docker images and wrote testing scenarios. The scenarios included different numbers of users—from 100 to the maximum that Locust could generate. This allowed us to effectively carry out load testing using Python to make sure that our solution could meet the demanding requirements.

The results surprised us quite a bit. On a single core, Rust managed to handle 14,000 requests per second, Go handled 5,700 requests per second, while optimized Python with Granian handled 5,500 requests per second. We didn’t conduct full-scale performance testing on native Python since even local tests showed significantly lower performance. These findings highlighted the strengths and limitations of each language, contributing to our understanding of backend languages statistics.

Defending the Concept

During a demo call with the client, we demonstrated the tests of all four solutions in real time and recommended using the optimized Python solution. Here are its main advantages:

• Performance on par with Go.

• The client can remain within their familiar stack.

• There is no need to complicate the architecture by introducing strictly typed languages for dynamic rules.

• The solution can be maintained and developed by mid-level Python developers.

• Cost-effective due to the optimal setup and the availability of Python specialists in the market.

We immediately ruled out the Rust solution because there is almost no commercial Rust development in the market and, accordingly, very few specialists. Globally, Rust is primarily used for crypto projects or significant cloud platforms like Google Cloud and AWS. The situation with Go is similar: qualified specialists are scarce and more expensive, and overall development would be relatively costly.

Conclusion

The client agreed with our proposal and decided to implement the solution using Python with Granian. However, the client requested that we provide all the developments we made on the other technologies as well. We are currently finalizing the project’s architecture and detailing the requirements to make sure the final concept fully meets the client’s goals.

Evrone offers analytics and consulting services tailored to the needs and capabilities of our clients. Our experienced specialists will propose a technically sound and economically advantageous solution that will enable your business to grow. Whether it involves data analysis with Python, real-time data processing, or leveraging our outsource Python development services, we are here to help you achieve your goals efficiently and effectively.