{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Cython\n", "\n", "* Example from [here](https://github.com/shekhar270779/Learn_Pandas/blob/master/Cython%20in%20Jupyter-notebook.ipynb) and [this tutorial](https://www.youtube.com/watch?v=CC0IkiNByV4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Cython** is a superset of python programming language that gives access into c/c++ constructs. \n", "- Cython acts as a bridge between Python and C/C++." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Load cython notebook extension (and i have already installed cython )" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: Cython in /srv/conda/envs/notebook/lib/python3.7/site-packages (0.29.24)\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "pip install Cython" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%load_ext cython" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "%%cython\n", "'''Inside this cell is Cython Code only\n", "'''\n", "\n", "def first_cython_function(int i):\n", " return i * 3.1415926;\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "31.415926" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# use cython function in a separate (python) cell\n", "first_cython_function(10)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\u001b[0;31mDocstring:\u001b[0m\n", "::\n", "\n", " %cython [-a] [-+] [-3] [-2] [-f] [-c COMPILE_ARGS]\n", " [--link-args LINK_ARGS] [-l LIB] [-n NAME] [-L dir] [-I INCLUDE]\n", " [-S SRC] [--pgo] [--verbose]\n", "\n", "Compile and import everything from a Cython code cell.\n", "\n", "The contents of the cell are written to a `.pyx` file in the\n", "directory `IPYTHONDIR/cython` using a filename with the hash of the\n", "code. This file is then cythonized and compiled. The resulting module\n", "is imported and all of its symbols are injected into the user's\n", "namespace. The usage is similar to that of `%%cython_pyximport` but\n", "you don't have to pass a module name::\n", "\n", " %%cython\n", " def f(x):\n", " return 2.0*x\n", "\n", "To compile OpenMP codes, pass the required `--compile-args`\n", "and `--link-args`. For example with gcc::\n", "\n", " %%cython --compile-args=-fopenmp --link-args=-fopenmp\n", " ...\n", "\n", "To enable profile guided optimisation, pass the ``--pgo`` option.\n", "Note that the cell itself needs to take care of establishing a suitable\n", "profile when executed. This can be done by implementing the functions to\n", "optimise, and then calling them directly in the same cell on some realistic\n", "training data like this::\n", "\n", " %%cython --pgo\n", " def critical_function(data):\n", " for item in data:\n", " ...\n", "\n", " # execute function several times to build profile\n", " from somewhere import some_typical_data\n", " for _ in range(100):\n", " critical_function(some_typical_data)\n", "\n", "In Python 3.5 and later, you can distinguish between the profile and\n", "non-profile runs as follows::\n", "\n", " if \"_pgo_\" in __name__:\n", " ... # execute critical code here\n", "\n", "optional arguments:\n", " -a, --annotate Produce a colorized HTML version of the source.\n", " -+, --cplus Output a C++ rather than C file.\n", " -3 Select Python 3 syntax.\n", " -2 Select Python 2 syntax.\n", " -f, --force Force the compilation of a new module, even if the\n", " source has been previously compiled.\n", " -c COMPILE_ARGS, --compile-args COMPILE_ARGS\n", " Extra flags to pass to compiler via the\n", " `extra_compile_args` Extension flag (can be specified\n", " multiple times).\n", " --link-args LINK_ARGS\n", " Extra flags to pass to linker via the\n", " `extra_link_args` Extension flag (can be specified\n", " multiple times).\n", " -l LIB, --lib LIB Add a library to link the extension against (can be\n", " specified multiple times).\n", " -n NAME, --name NAME Specify a name for the Cython module.\n", " -L dir Add a path to the list of library directories (can be\n", " specified multiple times).\n", " -I INCLUDE, --include INCLUDE\n", " Add a path to the list of include directories (can be\n", " specified multiple times).\n", " -S SRC, --src SRC Add a path to the list of src files (can be specified\n", " multiple times).\n", " --pgo Enable profile guided optimisation in the C compiler.\n", " Compiles the cell twice and executes it in between to\n", " generate a runtime profile.\n", " --verbose Print debug information like generated .c/.cpp file\n", " location and exact gcc/g++ command invoked.\n", "\u001b[0;31mFile:\u001b[0m /srv/conda/envs/notebook/lib/python3.7/site-packages/Cython/Build/IpythonMagic.py\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%%cython?" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "def foo(a, b):\n", " return a + b" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 2 0 LOAD_FAST 0 (a)\n", " 2 LOAD_FAST 1 (b)\n", " 4 BINARY_ADD\n", " 6 RETURN_VALUE\n" ] } ], "source": [ "from dis import dis \n", "# disassebles compiled python objects\n", "dis(foo)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "%%cython\n", "def cyfoo(a, b):\n", " return a + b" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "120 ns ± 3.57 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n" ] } ], "source": [ "%timeit foo(1000000, 2000000)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "110 ns ± 2.29 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n" ] } ], "source": [ "%timeit cyfoo(1000000, 2000000)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "139 ns ± 3.65 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n" ] } ], "source": [ "%timeit foo('x', 'y')" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "138 ns ± 5.23 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)\n" ] } ], "source": [ "%timeit cyfoo('x', 'y')" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "def pyfac(n):\n", " if n <= 1:\n", " return 1\n", " return n * pyfac(n -1)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "3.51 µs ± 151 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n" ] } ], "source": [ "%timeit pyfac(20.0)" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "%%cython\n", "\n", "def cyfac(n):\n", " if n <= 1:\n", " return 1\n", " return n * cyfac(n-1)\n", "\n", "def cyfac_double(double n):\n", " if n <= 1:\n", " return 1.0\n", " return n * cyfac_double(n-1)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.46 µs ± 47.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n" ] } ], "source": [ "%timeit cyfac(20.0)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "914 ns ± 71 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)\n" ] } ], "source": [ "%timeit cyfac_double(20.0)" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", " \n", "Generated by Cython 0.29.24
\n", "\n",
" Yellow lines hint at Python interaction.
\n",
" Click on a line that starts with a \"+\" to see the C code that Cython generated for it.\n",
"
1: \n",
"+2: cpdef double cyfac_double_fast(double n):\n", "
static PyObject *__pyx_pw_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_1cyfac_double_fast(PyObject *__pyx_self, PyObject *__pyx_arg_n); /*proto*/\n",
"static double __pyx_f_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_cyfac_double_fast(double __pyx_v_n, CYTHON_UNUSED int __pyx_skip_dispatch) {\n",
" double __pyx_r;\n",
" __Pyx_RefNannyDeclarations\n",
" __Pyx_RefNannySetupContext(\"cyfac_double_fast\", 0);\n",
"/* … */\n",
" /* function exit code */\n",
" __pyx_L0:;\n",
" __Pyx_RefNannyFinishContext();\n",
" return __pyx_r;\n",
"}\n",
"\n",
"/* Python wrapper */\n",
"static PyObject *__pyx_pw_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_1cyfac_double_fast(PyObject *__pyx_self, PyObject *__pyx_arg_n); /*proto*/\n",
"static PyObject *__pyx_pw_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_1cyfac_double_fast(PyObject *__pyx_self, PyObject *__pyx_arg_n) {\n",
" double __pyx_v_n;\n",
" PyObject *__pyx_r = 0;\n",
" __Pyx_RefNannyDeclarations\n",
" __Pyx_RefNannySetupContext(\"cyfac_double_fast (wrapper)\", 0);\n",
" assert(__pyx_arg_n); {\n",
" __pyx_v_n = __pyx_PyFloat_AsDouble(__pyx_arg_n); if (unlikely((__pyx_v_n == (double)-1) && PyErr_Occurred())) __PYX_ERR(0, 2, __pyx_L3_error)\n",
" }\n",
" goto __pyx_L4_argument_unpacking_done;\n",
" __pyx_L3_error:;\n",
" __Pyx_AddTraceback(\"_cython_magic_88dfdb3a94d86c859cfd2127a1590a68.cyfac_double_fast\", __pyx_clineno, __pyx_lineno, __pyx_filename);\n",
" __Pyx_RefNannyFinishContext();\n",
" return NULL;\n",
" __pyx_L4_argument_unpacking_done:;\n",
" __pyx_r = __pyx_pf_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_cyfac_double_fast(__pyx_self, ((double)__pyx_v_n));\n",
" int __pyx_lineno = 0;\n",
" const char *__pyx_filename = NULL;\n",
" int __pyx_clineno = 0;\n",
"\n",
" /* function exit code */\n",
" __Pyx_RefNannyFinishContext();\n",
" return __pyx_r;\n",
"}\n",
"\n",
"static PyObject *__pyx_pf_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_cyfac_double_fast(CYTHON_UNUSED PyObject *__pyx_self, double __pyx_v_n) {\n",
" PyObject *__pyx_r = NULL;\n",
" __Pyx_RefNannyDeclarations\n",
" __Pyx_RefNannySetupContext(\"cyfac_double_fast\", 0);\n",
" __Pyx_XDECREF(__pyx_r);\n",
" __pyx_t_1 = PyFloat_FromDouble(__pyx_f_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_cyfac_double_fast(__pyx_v_n, 0)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error)\n",
" __Pyx_GOTREF(__pyx_t_1);\n",
" __pyx_r = __pyx_t_1;\n",
" __pyx_t_1 = 0;\n",
" goto __pyx_L0;\n",
"\n",
" /* function exit code */\n",
" __pyx_L1_error:;\n",
" __Pyx_XDECREF(__pyx_t_1);\n",
" __Pyx_AddTraceback(\"_cython_magic_88dfdb3a94d86c859cfd2127a1590a68.cyfac_double_fast\", __pyx_clineno, __pyx_lineno, __pyx_filename);\n",
" __pyx_r = NULL;\n",
" __pyx_L0:;\n",
" __Pyx_XGIVEREF(__pyx_r);\n",
" __Pyx_RefNannyFinishContext();\n",
" return __pyx_r;\n",
"}\n",
"+3: if n <= 1:\n", "
__pyx_t_1 = ((__pyx_v_n <= 1.0) != 0);\n",
" if (__pyx_t_1) {\n",
"/* … */\n",
" }\n",
"+4: return 1.0\n", "
__pyx_r = 1.0;\n",
" goto __pyx_L0;\n",
"+5: return n * cyfac_double_fast(n - 1)\n", "
__pyx_r = (__pyx_v_n * __pyx_f_46_cython_magic_88dfdb3a94d86c859cfd2127a1590a68_cyfac_double_fast((__pyx_v_n - 1.0), 0));\n",
" goto __pyx_L0;\n",
"