Cross compiling ruby to arm processors – the not so zen way

Compiling ruby for linux running under another processor can be quite an adventure. Considering the limited setups you will find, not only small processing power and limited memory, I consider the building process to be the hardest nut to crack.

When cross-compiling ruby to run under a xscale arm SoC, I ran in a couple of road blocks, not all of them documented or easily found by googling around. I wanted a complete ruby distribution, including openssl and socket capabilities.

From the stock source distribution to a complete binary set, I had about 7 or 8 interations until reach a clean build. I still not satisfied with some workarounds, so I will let comments open to see if someone can come with a more elegant solution.

My cross-compiling environment was all setup, using arm-linux-gcc 3.x, a static openssl installation and ruby source code. Also, you will need a working ruby interpreter on the host machine, which will be used by ruby’s building system to generate makefiles and such.

Ruby’s building system can be fairly complex, and y ou may keep in mind that you wont be able to hack it so easily, as changing variables and parameters. The main config file for the entire build system is rbconfig.rb.

First, untar the source code for Ruby and apply the following patch. I’ve used ruby-1.8.6-p111.tar.gz, along with a static build of openssl-0.9.8g.tar.gz. The patch is needed because in the final linking part, ruby symbols will conflict with static openssl symbols of the same name.

Mainly, the digest module will cause problems. I think that in a runtime environment, these problems wont show up at linking time, but as I used a static library I had to rename these symbols. Basically, I put rb in front of them in ext/digest/sha2.c. The functions area: SHA256_Transform and SHA512_Transform.

So, after untar’ing the source code, apply the following patch using cd ext/Setup/digest ; cat sha2_static_openssl.patch | patch -p1 sha2.c.

— sha2-orig.c 2007-12-14 18:19:16.000000000 -0200
+++ sha2.c 2007-12-14 17:56:51.000000000 -0200
@@ -176,8 +176,8 @@
* only.
*/
void SHA512_Last(SHA512_CTX*);
-void SHA256_Transform(SHA256_CTX*, const sha2_word32*);
-void SHA512_Transform(SHA512_CTX*, const sha2_word64*);
+void rb_SHA256_Transform(SHA256_CTX*, const sha2_word32*);
+void rb_SHA512_Transform(SHA512_CTX*, const sha2_word64*);

/*** SHA-XYZ INITIAL HASH VALUES AND CONSTANTS ************************/
@@ -329,7 +329,7 @@
(h) = T1 + Sigma0_256(a) + Maj((a), (b), (c)); \
j++

-void SHA256_Transform(SHA256_CTX* context, const sha2_word32* data) {
+void rb_SHA256_Transform(SHA256_CTX* context, const sha2_word32* data) {
sha2_word32 a, b, c, d, e, f, g, h, s0, s1;
sha2_word32 T1, *W256;
int j;
@@ -387,7 +387,7 @@

#else /* SHA2_UNROLL_TRANSFORM */

-void SHA256_Transform(SHA256_CTX* context, const sha2_word32* data) {
+void rb_SHA256_Transform(SHA256_CTX* context, const sha2_word32* data) {
sha2_word32 a, b, c, d, e, f, g, h, s0, s1;
sha2_word32 T1, T2, *W256;
int j;
@@ -489,7 +489,7 @@
context->bitcount += freespace << 3;
len -= freespace;
data += freespace;
– SHA256_Transform(context, (sha2_word32*)context->buffer);
+ rb_SHA256_Transform(context, (sha2_word32*)context->buffer);
} else {
/* The buffer is not yet full */
MEMCPY_BCOPY(&context->buffer[usedspace], data, len);
@@ -501,7 +501,7 @@
}
while (len >= SHA256_BLOCK_LENGTH) {
/* Process as many complete blocks as we can */
– SHA256_Transform(context, (const sha2_word32*)data);
+ rb_SHA256_Transform(context, (const sha2_word32*)data);
context->bitcount += SHA256_BLOCK_LENGTH << 3;
len -= SHA256_BLOCK_LENGTH;
data += SHA256_BLOCK_LENGTH;
@@ -541,7 +541,7 @@
MEMSET_BZERO(&context->buffer[usedspace], SHA256_BLOCK_LENGTH – usedspace);
}
/* Do second-to-last transform: */
– SHA256_Transform(context, (sha2_word32*)context->buffer);
+ rb_SHA256_Transform(context, (sha2_word32*)context->buffer);

/* And set-up for the last transform: */
MEMSET_BZERO(context->buffer, SHA256_SHORT_BLOCK_LENGTH);
@@ -557,7 +557,7 @@
*(sha2_word64*)&context->buffer[SHA256_SHORT_BLOCK_LENGTH] = context->bitcount;

/* Final transform: */
– SHA256_Transform(context, (sha2_word32*)context->buffer);
+ rb_SHA256_Transform(context, (sha2_word32*)context->buffer);

#ifndef WORDS_BIGENDIAN
{
@@ -624,7 +624,7 @@
(h) = T1 + Sigma0_512(a) + Maj((a), (b), (c)); \
j++

-void SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) {
+void rb_SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) {
sha2_word64 a, b, c, d, e, f, g, h, s0, s1;
sha2_word64 T1, *W512 = (sha2_word64*)context->buffer;
int j;
@@ -679,7 +679,7 @@

#else /* SHA2_UNROLL_TRANSFORM */

-void SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) {
+void rb_SHA512_Transform(SHA512_CTX* context, const sha2_word64* data) {
sha2_word64 a, b, c, d, e, f, g, h, s0, s1;
sha2_word64 T1, T2, *W512 = (sha2_word64*)context->buffer;
int j;
@@ -779,7 +779,7 @@
ADDINC128(context->bitcount, freespace << 3);
len -= freespace;
data += freespace;
– SHA512_Transform(context, (const sha2_word64*)context->buffer);
+ rb_SHA512_Transform(context, (const sha2_word64*)context->buffer);
} else {
/* The buffer is not yet full */
MEMCPY_BCOPY(&context->buffer[usedspace], data, len);
@@ -791,7 +791,7 @@
}
while (len >= SHA512_BLOCK_LENGTH) {
/* Process as many complete blocks as we can */
– SHA512_Transform(context, (const sha2_word64*)data);
+ rb_SHA512_Transform(context, (const sha2_word64*)data);
ADDINC128(context->bitcount, SHA512_BLOCK_LENGTH << 3);
len -= SHA512_BLOCK_LENGTH;
data += SHA512_BLOCK_LENGTH;
@@ -826,7 +826,7 @@
MEMSET_BZERO(&context->buffer[usedspace], SHA512_BLOCK_LENGTH – usedspace);
}
/* Do second-to-last transform: */
– SHA512_Transform(context, (const sha2_word64*)context->buffer);
+ rb_SHA512_Transform(context, (const sha2_word64*)context->buffer);

/* And set-up for the last transform: */
MEMSET_BZERO(context->buffer, SHA512_BLOCK_LENGTH – 2);
@@ -843,7 +843,7 @@
*(sha2_word64*)&context->buffer[SHA512_SHORT_BLOCK_LENGTH+8] = context->bitcount[0];

/* Final transform: */
– SHA512_Transform(context, (const sha2_word64*)context->buffer);
+ rb_SHA512_Transform(context, (const sha2_word64*)context->buffer);
}

void SHA512_Finish(SHA512_CTX* context, sha2_byte digest[]) {
Note that you only need this patch if you are using a static openssl lib. You will know if you need it in another situation if in the final part of the make process, the linker yield a message about symbol redefinition or symbol size changed. If the patch is messy, you may edit sha2.c and change the functions signature yourself, by pre-pending rb_ to their names and looking where they are used inside the same file.

Next step is selecting which extensions you want to build. Edit ext/Setup and uncomment what you want. As I am compiling everything static, I uncommented the following options: option nodynamic, all digest options, etc, fcntl, iconv, openssl, socket, stringio, strscan, syck, thread and zlib.

Run autoconf and the follwing script:

— ruby_comp.sh —

export CC=/usr/local/arm-linux/bin/arm-linux-gcc
export LD=/usr/local/arm-linux/bin/arm-linux-gcc
export AR=/usr/local/arm-linux/bin/arm-linux-ar
export RANLIB=/usr/local/arm-linux/bin/arm-linux-ranlib

export ac_cv_func_getpgrp_void=yes
export ac_cv_func_setpgrp_void=yes

./configure –target=arm-linux –host=arm-linux –enable-wide-getaddrinfo –with-openssl-dir=/PATH/TO/OPENSSL/STATIC/INSTALLATION/usr

— end —

Besides setting up the target processor, there are some issues that show up when cross-compiling. One of these are set/getpgrp library functions which make your configure script fail. The other is a IPV6/getaddrinfo issue, which will make socket compilation fail. By using these environment variables and configure flags everything will build nice. Try them if you are stuck cross-compiling for other plataforms than arm.

After configure finish its job, run make and it should build to the end. Then, the trick part I want some input for a elegant way to do. If you setup –prefix when configuring to install in another PATH, the libraries path inside ruby will be registered to that PATH.

I wanted to install under /tmp to package the whole binary distribution and copy to my arm board, but the first time I issued a –prefix=/tmp, it would not load any require’d library inside source code. I ran ‘strings’ in the resulting ruby binary and saw that /tmp/lib was built-in the binary as the place to look for its stuff. Also, it was built on every helper script as irb, erb, ri, in the first line, to look for ruby in /tmp/bin. Definately a no-go.

Then I cleaned everything and rebuild without setting up the –prefix tag, and before issuing a make install, I checked into rbconfig.rb to see if there was something to do to help it. As a side note, when compiling for x86 path modifiers in configure worked as expected…

There was a variable called DESTDIR, so I decided to put /tmp there and see what happens. Well, it installed under /tmp and I managed to build a package to install in my arm board. The only issue I had was that I still had to change bin/irb, bin/ri, bin/rdoc, bin/testrb, bin/erb to point to the ruby binary, but the binary itself contained no references do tmp.

After all this work, irb and the other helpers worked ok. Now, on to get gtk and glade gems compiled under arm.

Good luck and leave a comment if you have any pointers to improve this guide.