[转]Converting a C library to gyp
Source:http://n8.io/converting-a-c-library-to-gyp/
Converting a C library to gyp
Published: 2012.10.02
gyp
, short for "Generate Your Projects", is a build system written by Google that was specifically designed to build their Chromium web browser.gyp
makes it easy to develop such a large project because the developers can create and test each piece individually by defining self-contained "targets". Then at Release-time, gyp
makes it easy to stitch the "targets" together into a single statically linked executable. Ryan Dahl once called it "the module-system of C".
Node.js recently adopted gyp
as its build system because V8 was already using it (through Chromium), and because of its explicit Windows support. node-gyp
was created to help the community create node.js native addons by writing simple gyp files. This article is targeted towards Node.js native module authors, so hopefully you are somewhat familiar will all of this already.
gyp-ify all the things!
One thing that I want to see more of in the node community from native module authors is the additional work done to convert the dependencies of the native module to use gyp
to build, and bundle them with the native module. This allows the library to be statically linked to your native module, rather than dynamically linked. There are pros and cons to both kinds of linking, but node.js has gone with static linking, mostly because of portability and to ease Windows support.
Now converting the build system of some odd C library that you haven't written can be a daunting task, so that's what this article hopes to address. I'll explain my method of converting a C library originally using autotools
to build using gyp
, and hopefully you will do the same (or improve on my method!) in order to make your native module more portable. Your users will thank you for the easier installation process (i.e. not having toapt-get install
some external libraries).
libmp3lame, I choose you!
Our test subject for this article will be libmp3lame
, a library that does MP3 encoding and decoding. This is a nice, medium complexity library that will be a good demonstration of my method.
Baseline gyp file
We're gonna start with a baseline gyp
file to use. Let's call it libmp3lame.gyp
in this case:
{
'target_defaults': {
'default_configuration': 'Debug',
'configurations': {
'Debug': {
'defines': [ 'DEBUG', '_DEBUG' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 1, # static debug
},
},
},
'Release': {
'defines': [ 'NDEBUG' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 0, # static release
},
},
}
},
'msvs_settings': {
'VCLinkerTool': {
'GenerateDebugInformation': 'true',
},
},
},
'targets': [
]
}
The msvs_settings
configuration stuff at the top ensures that we end up with statically built libraries and executables on Windows. This boilerplate isn't strictly necessary, but it matches up with some defaults set in node.js' common.gypi
file, and I like to stay consistent. The emptytargets
array will end up with the definitions to build libmp3lame
, as well as a simple test program.
Adding the platform-specific files
Most C libraries have a ./configure
step, and usually that step will create some kind of "config.h" file of some sort, which has appropriate defines for your current platform. So our strategy for converting to gyp
is to pre-generate these "config.h" files on all your supported platforms and architectures, and save them for use when building with gyp
.
So for example, the first platform + architecture combo I will get working is Mac OS X 64-bit. Therefore, the resulting "config.h" file will be placed atconfig/mac/x64/config.h
. In the end, we want to support if possible this baseline of platforms and architectures:
OS | Arch | config.h path |
---|---|---|
Linux | ARM | config/linux/arm |
Linux | 32-bit | config/linux/ia32 |
Linux | 64-bit | config/linux/x64 |
Mac OS X | 32-bit | config/mac/ia32 |
Mac OS X | 64-bit | config/mac/x64 |
Solaris | 32-bit | config/solaris/ia32 |
Solaris | 64-bit | config/solaris/x64 |
Windows | 32-bit | config/win/ia32 |
Windows | 64-bit | config/win/x64 |
Some of the combinations might not be supported by the library you are working with, but (except for linux-arm) this list matches up with the releases of the official node.js binary tarballs, and what node supports, you should try to support too!
Running ./configure
With C libraries configured using autotools
, I've been using a standard set of switches, and adding library-specific ones as necessary to try to get as close of a static build as gyp
is going to make. The basic set of flags I use are:
$ ./configure --enable-static --disable-shared --with-pic
These flags are basically saying, build the static version of the library (the .a
/.lib
files), and don't bother with the shared version (the.so
/.dylib
/.dll
files). Also, build "position independent code".
In some cases, that will be all that you need. But for libmp3lame
, I'm also gonna need to specify a few additional flags to disable some unnecessary stuff like the lame
program frontend.
$ ./configure --enable-static --disable-shared --with-pic \
--disable-rpath --disable-frontend --disable-gtktest
After running configure on my 64-bit Mac OS X machine, I'm going to copy the resulting config.h
file to config/mac/x64/config.h
. The"config.h" file could be called something else for your library, like "expat_config.h" for libexpat
for example. In some cases there could be more than 1 ".h" file generated, like libffi
had 3 per platform+arch combo. So you kinda just need to take a look around, visually inspect the configure
output, and see what gets generated.
The first change we're going to make to the gyp file is to make sure that these platform-specific header files are included:
diff --git a/libmp3lame.gyp b/libmp3lame.gyp
index d30085c..cdc5b9b 100644
--- a/libmp3lame.gyp
+++ b/libmp3lame.gyp
@@ -1,4 +1,6 @@
{
+ 'variables': { 'target_arch%': 'ia32' },
+
'target_defaults': {
'default_configuration': 'Debug',
'configurations': {
@@ -24,6 +26,10 @@
'GenerateDebugInformation': 'true',
},
},
+ 'include_dirs': [
+ # platform and arch-specific headers
+ 'config/<(OS)/<(target_arch)'
+ ],
},
'targets': [
So you essentially need to repeat these "configure & copy to config
" steps for all of the platforms + arch combinations listed in the table above. I usually do things a little out of order though, and move on to making the library build (the next section) before worrying about the other platforms...
Defining the targets
So usually the next step when building these libraries is to run the make
command. Sometimes you will need something additonal like overwriting a specific variable, but in this case we'll just invoke it straight up.
$ make
The key to defining proper targets is, again, visually inspecting the make output and seeing exactly which files are compiled, and defines are added, and which flags are used. You can also get part of the picture by looking at the Makefile.am
files in the library.
So based off the make output in the case of libmp3lame
, there's going to be 3 total "targets" that will be defined:
mpglib
- libmp3lame's decoder (an old version of libmpg123 apparently)liblamevectorroutines
- some internal routines used by libmp3lamelibmp3lame
the API frontend for libmp3lame
Each of these "targets" are like "modules" in node.js, they're their own independent functional component that make up part of the library.
Ok, so let's define those targets. This is a lot of new gyp
code here, so be sure to study it until it makes sense to you:
diff --git a/libmp3lame.gyp b/libmp3lame.gyp
index cdc5b9b..f30319b 100644
--- a/libmp3lame.gyp
+++ b/libmp3lame.gyp
@@ -27,11 +27,88 @@
},
},
'include_dirs': [
+ 'include',
+ 'mpglib',
+ 'libmp3lame',
+ 'libmp3lame/vector',
# platform and arch-specific headers
'config/<(OS)/<(target_arch)'
],
+ 'defines': [
+ 'PIC',
+ 'HAVE_CONFIG_H'
+ ],
},
'targets': [
+ # mpglib
+ {
+ 'target_name': 'mpgdecoder',
+ 'product_prefix': 'lib',
+ 'type': 'static_library',
+ 'sources': [
+ 'mpglib/common.c',
+ 'mpglib/dct64_i386.c',
+ 'mpglib/decode_i386.c',
+ 'mpglib/interface.c',
+ 'mpglib/layer1.c',
+ 'mpglib/layer2.c',
+ 'mpglib/layer3.c',
+ 'mpglib/tabinit.c',
+ ],
+ },
+
+ # liblamevectorroutines
+ {
+ 'target_name': 'lamevectorroutines',
+ 'product_prefix': 'lib',
+ 'type': 'static_library',
+ 'sources': [
+ 'libmp3lame/vector/xmm_quantize_sub.c'
+ ],
+ },
+
+ # libmp3lame
+ {
+ 'target_name': 'mp3lame',
+ 'product_prefix': 'lib',
+ 'type': 'static_library',
+ 'sources': [
+ 'libmp3lame/VbrTag.c',
+ 'libmp3lame/bitstream.c',
+ 'libmp3lame/encoder.c',
+ 'libmp3lame/fft.c',
+ 'libmp3lame/gain_analysis.c',
+ 'libmp3lame/id3tag.c',
+ 'libmp3lame/lame.c',
+ 'libmp3lame/newmdct.c',
+ 'libmp3lame/presets.c',
+ 'libmp3lame/psymodel.c',
+ 'libmp3lame/quantize.c',
+ 'libmp3lame/quantize_pvt.c',
+ 'libmp3lame/reservoir.c',
+ 'libmp3lame/set_get.c',
+ 'libmp3lame/tables.c',
+ 'libmp3lame/takehiro.c',
+ 'libmp3lame/util.c',
+ 'libmp3lame/vbrquantize.c',
+ 'libmp3lame/version.c',
+ 'libmp3lame/mpglib_interface.c',
+ ],
+ 'dependencies': [
+ 'lamevectorroutines',
+ 'mpgdecoder',
+ ],
+ 'direct_dependent_settings': {
+ 'include_dirs': [
+ 'include',
+ 'mpglib',
+ 'libmp3lame',
+ 'libmp3lame/vector',
+ # platform and arch-specific headers
+ 'config/<(OS)/<(target_arch)'
+ ],
+ },
+ },
]
}
One important thing to note is how we make the "mp3lame" target depend on "lamevectorroutines" and "mpgdecoder". This allows us to only have to specify "mp3lame" as a dependency in the node module's binding.gyp
file. Also note how the direct_dependent_settings
are defined within that target as well, for the same reason.
Try it out
At this point, we should be able to build libmp3lame
, at least on 64-bit OS X. I like to add a very quick little test program of some sort to the gyp file, to make sure that things get built and statically linked like we expect, and that it works at all.
We have to add an executable
target to the gyp file:
diff --git a/libmp3lame.gyp b/libmp3lame.gyp
index f30319b..de2984d 100644
--- a/libmp3lame.gyp
+++ b/libmp3lame.gyp
@@ -110,5 +110,13 @@
],
},
},
+
+ # test program that prints the version number
+ {
+ 'target_name': 'test',
+ 'type': 'executable',
+ 'dependencies': [ 'mp3lame' ],
+ 'sources': [ 'test.c' ]
+ },
]
}
diff --git a/test.c b/test.c
new file mode 100644
index 0000000..1fb3dbf
--- /dev/null
+++ b/test.c
@@ -0,0 +1,7 @@
+#include <stdio.h>
+#include "lame.h"
+
+int main () {
+ printf("get_lame_version(): %s\n", get_lame_version());
+ return 0;
+}
This super basic test program simply prints out the result of the get_lame_version()
function. Let's download gyp
, configure our project, and run this test
program:
$ svn co http://gyp.googlecode.com/svn/trunk gyp # download gyp
$ ./gyp/gyp -f make --depth=. libmp3lame.gyp # run gyp for the `make` target
$ make V=1 # verbose build
$ ./out/Debug/test # run the `test` program
get_lame_version(): 3.99.5
Success!
Windows support
So after generating all the config files for the other platforms (mac, linux, solaris), the last one is Windows (win). Windows usually takes some special dedication to get working. At the very least, a Google search asking "how to build library xxxx on Windows" should yield something.
If there are any major differences, you can introduce a conditions
block in your gyp file targeting Windows like so:
'conditions': [
['OS=="win"', {
# windows-specific gyp stuff goes here...
}],
],
The config files
In the case of libmp3lame
, the developers have included a hand-crafted configMS.h
file, which is the config.h file for Windows, both 32-bit and 64-bit. So all we have to do this time is copy this file to config/win/ia32
and config/win/x64
. Piece of cake!
In some cases though you might have to actually run the ./configure
script using version of bash
compiled for Windows. This was the case forlibffi
.
The source files and defines
It's ideal to actually run the build process for your library, ideally using MSVC, doing the method that the library authors suggest. That way you can inspect that build output and be able to determine any differences for the build on Windows.
For libmp3lame
, there's a Makefile.MSVC
file that is compatible with the nmake
command on Windows. If we run it and inspect the output, we can see that there are a few different defines used on Windows, but all the same sources files.
Let's adjust the gyp file to reflect that:
diff --git a/libmp3lame.gyp b/libmp3lame.gyp
index de2984d..bb4b010 100644
--- a/libmp3lame.gyp
+++ b/libmp3lame.gyp
@@ -38,6 +38,16 @@
'PIC',
'HAVE_CONFIG_H'
],
+ 'conditions': [
+ ['OS=="win"', {
+ 'defines': [
+ 'TAKEHIRO_IEEE754_HACK',
+ 'FLOAT8=float',
+ 'REAL_IS_FLOAT=1',
+ 'BS_FORMAT=BINARY',
+ ]
+ }]
+ ],
},
'targets': [
Sometimes additional sources files are used (or not used) so in that case you would add a sources
block within that conditional.
At this point, Windows should work as well:
C:\Users\Nathan\Desktop\lame> svn co http://gyp.googlecode.com/svn/trunk gyp
C:\Users\Nathan\Desktop\lame> gyp\gyp.bat -f msvs -G msvs_version=2010 --depth=. libmp3lame.gyp
C:\Users\Nathan\Desktop\lame> msbuild libmp3lame.sln
C:\Users\Nathan\Desktop\lame> Debug\test.exe
get_lame_version(): 3.99.5
Excellent!
The finished gyp
file
So we're done! At this point, we have our completed gyp file:
{
'variables': { 'target_arch%': 'ia32' },
'target_defaults': {
'default_configuration': 'Debug',
'configurations': {
'Debug': {
'defines': [ 'DEBUG', '_DEBUG' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 1, # static debug
},
},
},
'Release': {
'defines': [ 'NDEBUG' ],
'msvs_settings': {
'VCCLCompilerTool': {
'RuntimeLibrary': 0, # static release
},
},
}
},
'msvs_settings': {
'VCLinkerTool': {
'GenerateDebugInformation': 'true',
},
},
'include_dirs': [
'include',
'mpglib',
'libmp3lame',
'libmp3lame/vector',
# platform and arch-specific headers
'config/<(OS)/<(target_arch)'
],
'defines': [
'PIC',
'HAVE_CONFIG_H'
],
'conditions': [
['OS=="win"', {
'defines': [
'TAKEHIRO_IEEE754_HACK',
'FLOAT8=float',
'REAL_IS_FLOAT=1',
'BS_FORMAT=BINARY',
]
}]
],
},
'targets': [
# mpglib
{
'target_name': 'mpgdecoder',
'product_prefix': 'lib',
'type': 'static_library',
'sources': [
'mpglib/common.c',
'mpglib/dct64_i386.c',
'mpglib/decode_i386.c',
'mpglib/interface.c',
'mpglib/layer1.c',
'mpglib/layer2.c',
'mpglib/layer3.c',
'mpglib/tabinit.c',
],
},
# liblamevectorroutines
{
'target_name': 'lamevectorroutines',
'product_prefix': 'lib',
'type': 'static_library',
'sources': [
'libmp3lame/vector/xmm_quantize_sub.c'
],
},
# libmp3lame
{
'target_name': 'mp3lame',
'product_prefix': 'lib',
'type': 'static_library',
'sources': [
'libmp3lame/VbrTag.c',
'libmp3lame/bitstream.c',
'libmp3lame/encoder.c',
'libmp3lame/fft.c',
'libmp3lame/gain_analysis.c',
'libmp3lame/id3tag.c',
'libmp3lame/lame.c',
'libmp3lame/newmdct.c',
'libmp3lame/presets.c',
'libmp3lame/psymodel.c',
'libmp3lame/quantize.c',
'libmp3lame/quantize_pvt.c',
'libmp3lame/reservoir.c',
'libmp3lame/set_get.c',
'libmp3lame/tables.c',
'libmp3lame/takehiro.c',
'libmp3lame/util.c',
'libmp3lame/vbrquantize.c',
'libmp3lame/version.c',
'libmp3lame/mpglib_interface.c',
],
'dependencies': [
'lamevectorroutines',
'mpgdecoder',
],
'direct_dependent_settings': {
'include_dirs': [
'include',
'mpglib',
'libmp3lame',
'libmp3lame/vector',
# platform and arch-specific headers
'config/<(OS)/<(target_arch)'
],
},
},
# test program that prints the version number
{
'target_name': 'test',
'type': 'executable',
'dependencies': [ 'mp3lame' ],
'sources': [ 'test.c' ]
},
]
}
Are we done yet?
No still not done... now we have to add it as a dependency to our node.js native addon.
Following the model of node.js core, I'm gonna place libmp3lame with our gyp stuff in deps/lame
of the node-lame
repo. After that I can add it as a dependency to the main binding.gyp
file like so:
{
'targets': [
{
'target_name': 'bindings',
'sources': [
'src/bindings.cc',
'src/node_lame.cc'
],
'dependencies': [
'deps/lame/libmp3lame.gyp:mp3lame'
]
}
]
}
The binding.gyp
file remains nice and simple. NOW we're done. Grab a pint!
[转]Converting a C library to gyp的更多相关文章
- Writing Reentrant and Thread-Safe Code(译:编写可重入和线程安全的代码)
Writing Reentrant and Thread-Safe Code 编写可重入和线程安全的代码 (http://www.ualberta.ca/dept/chemeng/AIX-43/sha ...
- The Java library for converting Wikipedia wikitext notation to HTML
https://code.google.com/p/gwtwiki/ The Java Wikipedia API (Bliki engine) is a parser library for con ...
- Gyp语法规则参考 & 工具的使用
转自:http://www.cnblogs.com/nanvann/p/3913880.html 翻译自 https://code.google.com/p/gyp/wiki/GypLanguageS ...
- Calibre - book library management application
http://calibre-ebook.com/ Library Management E-book conversion Syncing to e-book reader devices Down ...
- JSBinding / Plugins & Build Mozjswrap Library
There are 2 libraries in Plugins: mozjs-31. This is SpiderMonkey library, built from https://github. ...
- .net Framework Class Library(FCL)
from:http://msdn.microsoft.com/en-us/library/ms229335.aspx 我们平时在VS.net里引用的那些类库就是从这里来的 The .NET Frame ...
- Multi-Targeting and Porting a .NET Library to .NET Core 2.0
Creating a new .NET Standard Project The first step for moving this library is to create a new .NET ...
- Error:Error converting bytecode to dex: Cause: com.android.dex.DexException: Multiple dex files define Lcom/lidroid/xutils/task/TaskHandler;
Error:Error converting bytecode to dex: Cause: com.android.dex.DexException: Multiple dex files defi ...
- c# Use NAudio Library to Convert MP3 audio into WAV audio(将Mp3格式转换成Wav格式)
Have you been in need of converting mp3 audios to wav audios? If so, the skill in this article prov ...
随机推荐
- hdoj 1226 超级password 【隐图BFS】
称号:hdoj 1226 超级password 分析:这题属于隐式图搜索,状态不是非常明显,须要自己建立. 事实上搜索说白了就是暴力. 这个题目就是,首先对给出的能够组成的全部的数依次枚举.长度从小到 ...
- MySQL存储引擎差异化实验
本篇把MySQL最常用的存储引擎给大家做一个介绍,然后通过插入.修改和并发实验来了解和验证一下它们之间的一些差异. 一.MySQL存储引擎简介 存储引擎在MySQL结构里占据核心的位置,是上层抽象接口 ...
- ZOJ 2675 Little Mammoth(计算几何)
圆形与矩形截面的面积 三角仍然可以做到这一点 代码: #include<stdio.h> #include<string.h> #include<stdlib.h> ...
- POJ 2777 Count Color(线段树+位运算)
题目链接:http://poj.org/problem?id=2777 Description Chosen Problem Solving and Program design as an opti ...
- ASP.NET 5 Hello World
ASP.NET 5系列教程 (二):Hello World 本篇文章内容比较基础,主要是向大家展示如何创建一个 ASP.NET 5 工程,主要包含内容如下: 创建ASP.NET 5 工程 添加 T ...
- Visual Studio Contact
Visual Studio Contact(); 直播笔记 昨天微软干了几件了不起的事:.NET开发环境将开源.跨平台支持(Mac OS X和Linux).多设备支持(WP.Android和iOS ...
- 探索Android该Parcel机制上
一.先从Serialize说起 我们都知道JAVA中的Serialize机制.译成串行化.序列化……,其作用是能将数据对象存入字节流其中,在须要时又一次生成对象.主要应用是利用外部存储设备保存对象状态 ...
- word-wrap同word-break的区别
兼容 IE 和 FF 的换行 CSS 推荐样式 最好的方式是 以下是引用片段: word-wrap:break-word; overflow:hidden; 而不是 以下是引用片段: word-wra ...
- 读书笔记—CLR via C#章节1-2
这本书这几年零零散散读过两三遍了,作为经典书籍,应该重复读反复读,既然我现在开始写博了,我也准备把以前觉得经典的好书重读细读一遍,并且将笔记整理到博客中,好记性不如烂笔头,同时也在写的过程中也可以加深 ...
- C注意,使用的语言字符串
转载请注明出处! 在C语言没有具体的字符串数据类型,字符串的字符串常量和字符数组的形式. 实际上该字符串是零个或更多字符的字符串.并在整个位模式0NUL字节结束.因此,字符串所包括的字符内部不能出现N ...