Using distcc to compile FatdogArm applications with Fatdog64
Thanks to the guys from the Samba team, there is another way: distcc
.
What is distcc?
According to its manpage, distcc is:
distcc
distributes compilation of C code across several machines on a
network. distcc
should always generate the same results as a local compile,
it is simple to install and use, and it is often much faster than a local
compile.So from the point of program being compiled, using distcc is (almost) identical to a "local compile", that is native compiling, which eliminates most of the cross-compile complications. But because distcc distributes (=sends) the compilation work elsewhere (=the more powerful machines), it will be a lot faster than normal native compiling.
How much faster? The compilation phase of Seamonkey 2.20 took 15 hours on Mele. Using distcc, the time taken was just a tad more over 3 hours.
Using distcc on FatdogArm and Fatdog64
- On Fatdog64
- Install
distcc
package - Install
cross-gcc-4.6.2-armv7
package - Start distccd as follows:
DISTCCD_PATH=/cross-tools/bin distccd --daemon --allow 0.0.0.0/0
- On FatdogArm
- Install
distcc
package - Install
distcc-helper
package - Before you compile anything, just ensure that you export the following
two environment variables:
export PATH=/usr/lib/distcc/bin:$PATH export DISTCC_HOSTS="--localslots=1 --localslots_cpp=2 fatdog64-hostname,lzo"
Note: lzo means that the files are compressed with LZO first before sent to the remote server, this causes overheads on the native machine but this is usually more than offset by the resulting reduced network latency. Remove this if you don't want to use compression at all.
- Run "make" with "-j" option (parallel make). As a rule of thumb, the number of concurrent jobs should be about twice the number of available server CPUs combined.
- For more options, please read
distcc
anddistccd
manual pages, here: http://distcc.googlecode.com/svn/trunk/doc/web/man/
Do the above steps for all the Fatdog64 machines you want to use. Make sure that your Fatdog64 is reachable from your FatdogArm machine. Distcc uses port 3632 by default, make sure your firewall is configured to let inbound connections on this port.
Note: distccd cannot run as root (for good reasons - distccd is in fact a passwordless remote shell!). By default it will try to run as user "distcc", failing that it will try "nobody". You can add "--user spot" to tell to run as "spot", but in either case please make sure that user "spot" (or "distcc" or "nobody") has access to the cross-compiler using its primary group. If you don't like this restriction, simply "su spot" first and then run distcc from there; this way it will use the access provided by the auxiliary groups as well.
About the cross-compiler
If you don't want (or can't) use the provided gcc-cross-4.6.2-armv7
package
from Fatdog64 repository, you can always roll on your own. Follow the steps
on BuildingTheToolChain to create your own cross-compiler. The constraint is
that the compiler you use must be identical (or almost identical) to the one
supplied with FatdogArm or otherwise you will experience subtle failures.
The compiler triplet must be armv7l-unknown-linux-gnueabihf
(that's the
name of FatdogArm's native compiler), you can change this to something
else by editing /usr/lib/distcc/bin/gcc
and /usr/lib/distcc/bin/g++
scripts. The scripts have (commented out) examples where the remote compiler
has arm-lfs-linux-gnueabi
triplet (this is what you get if you follow
Cross-LFS 2.0 guide without modification
(it won't work as is, you need to change gcc specs to make it default to
use hardfloat ABI).
Distcc mode of operation
distcc (from version 3.x onwards) has two modes of operation: standard mode and "pump" mode.
Standard mode is relatively straightforward and it is what has just been explained. In this mode the source file is pre-processed locally, compiled and assembled remotely, and then linked locally.
In "pump" mode, the pre-processing is also done remotely, thus speeding up the process even more. However it also demands more stricter cross-compiler requirements (identical system headers; identical location of system headers, etc); which is a lot more difficult to achieve in cross-compilation environments.
The setup described in this article operates in standard mode.
Further references