Building Linux for the nanoEngine

How to build Linux for a Bright Star Engineering nanoEngine. This board is a tiny, embeddable StrongARM computer. This web page is for propeller-heads like me who want everything built from pristine sources. If you don't want to spend two hours and 800 Meg of disk on this process, you can probably use the the CD-ROM from BSE (which cost you about US$1100).

While this documentation and kit are customized, tuned, and debugged for the nanoEngine, nearly all of it applies to any embedded Linux target, and especially other StrongARM embedded Linux targets. Contact me if you adapt this kit for something other than the nanoEngine, I'd love to make this more general.
Previous releases were labed a "WORK IN PROGRESS". While that is obviously still true, this snapshot is also fully functional, and the basis for serious application deployment. The todo list is still there, but there are no known show-stoppers.

The nanoEngine hardware brings out the StrongARM pins that can implement a PCMCIA bus. Miguel Freitas posted instructions for hooking that up and making it work; its software is a follow-on to this work.

Step 0

Throw away the CD that Bright Star sent. It's almost all binaries, and doesn't explain how they were created. When bug fixes or enhanced versions of any of the upstream software comes out, the techniques on this page will help you make the updates on your own terms.

To start, you need a working general-issue GNU-based development system, with a lot of disk space (about 800 Meg should do it). A fast CPU would be nice, too: the build steps take about 80 minutes on my 400 MHz Pentium-II. I used a Red Hat 5.2 system, although I did have to update GNU make (it was 3.76.1, and I went all the way to 3.79.1; the documentation says 3.77 is enough). This version of the kit was tested with gcc upgraded to 2.95.3 (as the host native compiler); earlier versions used the stock gcc-2.7.2.3. There's no reason to think this process would not work on Solaris, AIX, etc. I hope to test that Real Soon Now.

The information on this page should be roughly consistent with other ARM-Linux toolchain documentation on the net:

as well as the general CrossGCC FAQ.

Step 1

Download, unpack, and patch to get: See the downloads page for exact locations of all the upstream files, and a script for making all the patches. You are welcome to try with other (more recent!) copies of these programs, but be prepared to fall back to these known good versions (especially the binutils/gcc pair) if you run into trouble.

This script will do all the unpacking and patching from the above listed reference files. All scripts referenced on this page are included in the nanoengine-kit-0.8, listed above. If you're serious about performing the builds described on this page, you will download that file, and carefully read the enclosed README. That gives additional context and example usage of all the scripts.

As a security-minded individual, I strongly recommend that you take all steps as a non-root (mortal) user. That includes this download step, unpacking, and the building steps as well. That minimizes the security exposure of your development machine. If many people on one development machine want to use the tools, the traditional approach is to install as root. Instead, I recommend dedicating a userid for the build process, and then have all users direct their path to the resulting tools.

Of course, the security of your embedded target is intimately tied to the quality of the source code that goes into it; nothing can avoid that. At least this build-from-source process can let you document all the people who have their "finger in the pie."

The source directories for most of the packages (e.g., binutils, gcc, glibc, strace, busybox, and ntpclient) are not touched by the installation process, and belong in a "pristine" area. Occasional program trees have to be built in the same directory as the sources. The primary offender here is the Linux kernel itself. People recognize the problem, and have tackled it in the linux-2.5 tree. If and when that effort stabilizes, I will happily change over. For now, this meta-kit puts up with linux-2.4. Also, note there is a bug in the pristine-ness of glibc or gcc, I forget which. Try building a toolchain based on sources owned by a different user, you'll see the problem.

Step 2

Build the StrongARM development system, consisting of binutils, gcc, the Linux kernel, and glibc. This is all scripted. Here is a 6.9 Mbyte log of the build process (all the way through to the end of step 4) on my machine. This is a touchy step to get exactly right. Fortunately, now that I have debugged it (I hope), you only have to do it once.

I didn't enable a gcc (GNU Compiler Collection) build of Objective-C, Fortran, or Java, because I don't plan on testing them. You can enable them yourself with a one-line change in the script. Testing is then up to you, of course.

Step 3

Build all the user space programs and files needed to get the embedded system off the ground. Programs that are built but not installed (because of Flash and RAM limitations) can be used via NFS. Again, this is all scripted. Since this gets into application-specific issues, don't be afraid to tune and re-run this step. It's both less tricky and a lot quicker than the last one!

Step 4

Make a ramdisk image out of the file system created in the previous step. This step is also scripted (two files, make_image_arm calls write_devs).

Step 5

Combine (cat) the 4kB fixup.bin, ~490kB zImage, and ~980kB rdisk.gz to get a mondo file in your /tftpboot directory. This new boot process is described in README.booting. From the nanoEngine boot prompt, load and go! 
bload mondo c0800000; go c0800000
I made a "screen shot" of a successful nanoEngine boot, using the results of the build process exactly as described.

Assuming you want persistent storage using the Flash chip on your nanoEngine, you need to erase a section of Flash, and mount a JFFS2 system on it. As you can see by the output of "cat /proc/mtd", I have set aside 1 Mbyte for this purpose (if you want to change this allocation, see linux/drivers/mtd/maps/sa1100-flash.c). Do "eraseall /dev/mtd2", then "mount -t jffs2 /dev/mtdblock2 /mnt/flash". This second step could be added to the boot process. Please read the important notice in the README file about how using the Flash can prevent a hardware reset from rebooting properly. If your console gets cluttered up with "MTD_open" and similar messages, you can silence them with "echo 5 >/proc/sys/kernel/printk".

Congratulations!

Now the fun begins -- The above is basic infrastructure. Now you get to add features and your application-specific code. You have the cross-compiler, and you can easily test new code by NFS mounting a directory from your workstation. If you bring up programs of general interest, mail me and I will add them to this kit. I personally have built and run the following:

At some point, you should probably review the kernel configuration that I picked out for this board. Go to the build/linux-* directory, and "make xconfig". If you change anything, you should play it safe and "make dep && make zImage" to get a new kernel to try out.

Many thanks to George France, whose document "HOW-To build a cross Tool Chain in brief" started me in the right direction, and truly forms the basis for this web page and kit. Stefan Berger gave helpful comments based on his testing of this process.

June 26, 2002
Larry Doolittle