Optimizing Python with Numba library
As we discussed in the previous post, Python is a widely used programming language, among other reasons because it is easy to program at a high level, it has many good libraries and it enables easily readable and maintainable code. Despite all these advantages, it should be noted that it is notoriously slow, so it is convenient to know some tools that enable us to speed up the Python code.
Python vs Numba
One of the ways Python can be sped up is by using the Numba library. To illustrate this, we will take as an example the calculation of the number π using the Monte Carlo method, similar to our previous post (example taken from the Numba page).
Sped up Phyton
The code is very simple. The number π is calculated by sampling a random number of points and using the ratio of the area of a square to a circle. The area of the circle is estimated as the number of points within the circle, and that of the square as the total of points.
Montecarlo
Using 100,000 points, the Python function takes 30.1ms, while using Numba it takes 896μms, that is, 33.55x times faster.
But… What is Numba?
Numba is a Python library that gives the user the ability to compile the code as soon as it is executed (it is a Just-in-Time compiler), thus achieving speeds of C code without leaving the simplicity of Python.
And all this by making minimal modifications to the Python code, as you can see in the example. In addition to compiling the code, Numba has the ability to detect the type of CPU it is running on and make use of hardware-specific optimizations, as well as using different threads (which Python by itself can’t do).
As if that weren’t enough, Numba is designed to work with Numpy, one of the most famous Python libraries for numerical computing.
One of the drawbacks of Numba is that it is not friendly with dynamic data structures (that is, data structures that can increase or reduce their size arbitrarily) such as adding to lists or dictionaries, and it can be difficult to debug. Furthermore, all code written with Numba requires a system that supports its JIT compiler.
How to use Numba?
Four things are needed for a Python program that uses Numba to work, besides having Numba installed of course.
(1) Import the Numba library from the script, like this: import numba.
(2) The code to be accelerated needs to be clearly defined in a function. Above this function the decorator @numba.njit is added. This decorator tells Numba that it is the function that it should try to compile. If for some reason Numba can’t compile it, it will return an error. If you don’t want it to return an error and you prefer to fallback to normal Python, then the decorator should be changed to @numba.jit.
(3) The code does not contain dynamic data structures, and preferably the data structures that do exist are Numpy ones. That is, things such as .append() to a list should be avoided.
(4) The loops are not of type for element in elements, but iterate through the indices, such as for i in range (len (elements)).
As it has been observed, using Numba for parts of the code that can be bottlenecks from a computational point of view can provide great speed increases, and with little effort.
We hope this article has been useful to you. If you have an engineering project in your hands and you think we can help you, here is the link where you can contact us and explain more about it.