Morioh
Login
Sofia Kelly

Sofia Kelly

4 years ago

Hackspeed with Cython

Python is one of the most popular and popular programming languages today. However, when working with Python, you will see or hear about one of the weaknesses to this point: Python is slow.

There are several ways to speed up your Python code. Maybe you read it somewhere:

  • Using multi-processing libraries
  • Using asynchronous

As above, you will approach 2 sides to speed up Python code: parallel programmingand asynchronous programming. Now, I will introduce a different approach. That is Cython.

What’s Cython?

Can understand Cython is an intermediate step between Python and C / C ++. It allows you to write pure Python with some minor modifications, then translate it directly into C.

Cython will bring you the combined power of Python and C:

  • Python code calls back and forth C or C ++ native code at any time.
  • Easily modify Python code for performance like C code simply by adding static type declarations, also in Python syntax.
  • Interact effectively with the big data set.
  • Integrate native with existing code, low-level or high-performance libs / apps.
  • …

Some other information:

You can easily the Cython via pip

pip install cython

Compared to Python code, you need to add type information to every variable. Usually, to declare a Python variable, very simple:

x = 1

With Cython, you need to add the type for that variable:

cdef int x = 1

Just like in C, the type declaration for a variable in Cython is required.

Types in Cython

When using Cython, there are two different points for variables and functions.

For variable:

cdef int a, b, c
cdef char *s
cdef float x = 0.5 (single precision)
cdef double x = 60.4 (double precision)
cdef list images
cdef dict user 
cdef object card_deck

All of these types are derived from C / C ++.

For function:

def function1...
cdef function2...
cpdef function2...

With:

  • def: Function pure python, only called from Python.
  • cdef: Cython only functions. Only called from Cython.
  • cpdef: C and Python function. Can be called from C and Python.

How to speedup your code with Cython

First, I will create a pure Python code with for-loop.

#run_test_python.py
def test_python(x):
    y = 1
    for i in range(1, x+1):
        y *= i
    return y

Applying what I have understood above, I will write the code in Cython with a meaning:

#run_test_cython.pyx
cpdef int test_cython(int x):
    cdef int y = 1
    cdef int i
    for i in range(1, x+1):
        y *= i
    return y

When coding Cython, make sure that all of your variables are set.

Next, we need to create a file to compile from Cython -> C code:

# setup.py
from distutils.core import setup
from Cython.Build import cythonize

setup(ext_modules = cythonize('run_test_cython.pyx'))

After putting run_test_cython.pyxand setup.pywith dir, we start to compile:

% python setup.py build_ext --inplace                                        
Compiling run_test_cython.pyx because it changed.
[1/1] Cythonizing run_test_cython.pyx
running build_ext
building 'run_test_cython' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/ha.hao.minh/.pyenv/versions/viblo-venv/include -I/home/ha.hao.minh/.pyenv/versions/3.6.8/include/python3.6m -c run_test_cython.c -o build/temp.linux-x86_64-3.6/run_test_cython.o
gcc -pthread -shared -L/home/ha.hao.minh/.pyenv/versions/3.6.8/lib -L/home/ha.hao.minh/.pyenv/versions/3.6.8/lib build/temp.linux-x86_64-3.6/run_test_cython.o -o /home/ha.hao.minh/workspace/viblo/112019/run_test_cython.cpython-36m-x86_64-linux-gnu.so

Result:

% ls
build  run_test_cython.c  run_test_cython.cpython-36m-x86_64-linux-gnu.so  run_test_cython.pyx  run_test_python.py  setup.py

You will see in this folder that contains all the files needed to run C code. If you’re curious what the other Cython code will compile into is what C code you can cat to see that file:

cat run_test_cython.c

Come on, it’s time to show the power of C code. The following code compares the speed of Python pure and Cython:

# speedtest.py
import run_test_python
import run_test_cython
import time

def speedtest_python(number):
    start = time.time()
    run_test_python.test_python(number)
    end = time.time()

    py_time = end - start
    print(f"Python time = {py_time}")

    return py_time

def speedtest_cython(number):
    start = time.time()
    run_test_cython.test_cython(number)
    end =  time.time()

    cy_time = end - start
    print("Cython time = {}".format(cy_time))

    return cy_time

if __name__=="__main__":
    for number in [10, 100, 1000, 10000, 100000]:
        print(f"Speedtest with number = {number}")
        py_time = speedtest_python(number)
        cy_time = speedtest_cython(number)
        print("Speedup = {}".format(py_time / cy_time))

Results after running speedtest.py

% python speedtest.py
Speedtest with number = 10
Python time = 3.5762786865234375e-06
Cython time = 7.152557373046875e-07
Speedup = 5.0
Speedtest with number = 100
Python time = 7.867813110351562e-06
Cython time = 2.384185791015625e-07
Speedup = 33.0
Speedtest with number = 1000
Python time = 0.0002810955047607422
Cython time = 9.5367431640625e-07
Speedup = 294.75
Speedtest with number = 10000
Python time = 0.02144336700439453
Cython time = 7.62939453125e-06
Speedup = 2810.625
Speedtest with number = 100000
Python time = 3.1171438694000244
Cython time = 8.630752563476562e-05
Speedup = 36116.70994475138

With the current device configuration for my test:

  • CPU: Intel® Core ™ i5-4460 CPU @ 3.20GHz × 4
  • Ram: 8G

Fill the results in the table for easy viewing:

NumberPython timeCython timeSpeedupten

36116- Seems like an unthinkable number

Clearly, Cython gives you very good performance. This is also a viable solution if you want to improve your code.

#python #cython

Hackspeed with Cython
12.35 GEEK