python 集成cython 简单测试

 

实际开发中我们可能需要集成c/c++ 编写的模块，我们可以通过cython 解决类似的问题

以下测试一个简单的c add 方法, 使用venv 同时构建为一个pip 包

环境准备

• venv 初始化

python3 -m venv .

• 添加项目依赖包

pip install click  cython

• 代码结构

├── Makefile
├── README.md
├── cli
│ ├── __init__.py
│ └── app.pyx
├── ext
│ ├── Makefile
│ ├── add.c
│ ├── add.h
├── pyvenv.cfg
└── setup.py

• c 项目代码说明
  ext 目录为c add 函数同时make 进行项目构建
  add.h

int add(int first,int second);

add.c

#include "add.h"
int add(int first,int second){
    return first+second;
}

Makefile: 主要是进行静态库的构建

CC = gcc
default: libadd.a
libadd.a: add.o
 ar rcs $@ $^
add.o: add.c add.h
 $(CC) -c $<
clean:
 rm *.o *.a

• cython 包装c 静态库
  lib/app.pyx

cdef extern from "../ext/add.h":
    int add(int first,int second)
def py_add(first: int,second: int) -> int:
    return add(first,second)

• 配置代码生成

  主要是通过setup.py 定义，注意需要安装cyhton 包

import setuptools
from distutils.extension import Extension
from Cython.Build import cythonize
with open("README.md", "r") as fh:
    long_description = fh.read()

// 配置依赖的c 静态库
add_extension = Extension(
    name="add_app",
    sources=["./cli/app.pyx"],
    libraries=["add"],
    library_dirs=["ext"],
    include_dirs=["ext"]
)

setuptools.setup(
    name="dalongrong_cythoncli",
    version="0.0.4",
    author="dalongrong",
    author_email="1141591465@qq.com",
    description="a simple cli project",
    long_description=long_description,
    install_requires=['click'],
    ext_modules= cythonize([add_extension]),
    long_description_content_type="text/markdown",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    project_urls={
        'Documentation': 'https://github.com/rongfengliang/cython-c-pip-demo.git',
        'Say Thanks!': 'https://github.com/rongfengliang/cython-c-pip-demo.git',
        'Source': 'https://github.com/rongfengliang/cython-c-pip-demo.git',
        'Tracker': 'https://github.com/rongfengliang/cython-c-pip-demo.git',
    },
    entry_points={
        'console_scripts': [
            'podcli=cli:apply',
        ],
    }
)

• cython make 构建

  主要是简化了依赖项目的构建

LIB_DIR = ext
CLI_DIR = cli
default: pyadd
pyadd: setup.py $(CLI_DIR)/app.pyx $(LIB_DIR)/libadd.a
    python3 setup.py build_ext --inplace && rm -f add_app.c && rm -Rf build && cp *.so cli/ && rm *.so
$(LIB_DIR)/libadd.a:
    make -C $(LIB_DIR) libadd.a
clean:
    rm *.so

构建&&测试

• 构建

make 

效果

python3 setup.py build_ext --inplace && rm -f add_app.c && rm -Rf build && cp *.so cli/ && rm *.so
running build_ext
building 'add_app' extension
creating build
creating build/temp.macosx-10.13-x86_64-3.7
creating build/temp.macosx-10.13-x86_64-3.7/cli
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I./cli -Iext -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/Users/dalong/mylearning/py-cli-demo/cpython-demo/include -I/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c ./cli/app.c -o build/temp.macosx-10.13-x86_64-3.7/./cli/app.o
creating build/lib.macosx-10.13-x86_64-3.7
clang -bundle -undefined dynamic_lookup -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk build/temp.macosx-10.13-x86_64-3.7/./cli/app.o -Lext -L/usr/local/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/sqlite/lib -ladd -o build/lib.macosx-10.13-x86_64-3.7/add_app.cpython-37m-darwin.so
copying build/lib.macosx-10.13-x86_64-3.7/add_app.cpython-37m-darwin.so -> 

• 安装

  使用pip 本地安装

pip install .

• 测试调用代码

  cli/init.py,集成了一个click 的cli 开发包

import click
import add_app
@click.command()
@click.option("--scale", default=1, help="Number to scale.")
@click.option("--pod", prompt="pod name",
              help="The Pod counts.")
def apply(scale, pod):
    """Simple program that scale pod."""
    results = add_app.py_add(scale,10)
    print("pod scale with counts",pod,results)

if __name__ == '__main__':
    apply()

• 使用

podcli --pod demo

效果

podcli --pod demo
pod scale with counts demo 11

说明

这个只是一个简单的学习，因为在学习一个开源框架中使用了 cython ，所以研究下，方便阅读源码，同时代码很简单，深入的还得多看看
官方文档

参考资料

https://medium.com/@shamir.stav_83310/making-your-c-library-callable-from-python-by-wrapping-it-with-cython-b09db35012a3
https://github.com/stavshamir/cython-c-wrapper/
https://cython.readthedocs.io/en/latest/src/tutorial/external.html
https://cython.readthedocs.io/en/latest/src/tutorial/clibraries.html
http://pages.cs.wisc.edu/~yezheng/post/cython/
https://github.com/rongfengliang/cython-c-pip-demo