Benabik Alvar (benabik) wrote,
Benabik Alvar

  • Mood:
  • Music:

Compiling Linux From Scratch

I'm going to blog about compiling Linux From Scratch... It's likely to be rambling and long and technical. You have been warned.

Linux From Scratch is two things. First, and most literal, is that I'm building Linux from first principles. Download everything needed for a "real" Linux system and build it myself. Secondly, it's a book. The book is "Copylefted" via the Creative Commons and periodically updated to keep up with the latest and greatest. The website has a load of information including a set of books to help move beyond a basic Linux installation, a set of patches to help tweak things the way you want, and a pile of hints on how you might want to do things. It's a great resource for a roll-your-own Linux "distribution".

The LFS book does a great job of explaining why you're typing in each step. It doesn't just say type to in

CC="$LFS_TGT-gcc -B/tools/lib/" \
 ../gcc-4.4.3/configure --prefix=/tools \
 --with-local-prefix=/tools --enable-clocale=gnu \
 --enable-shared --enable-threads=posix \
 --enable-__cxa_atexit --enable-languages=c,c++ \
 --disable-libstdcxx-pch --disable-multilib \
It also bothers to tell you what each of those options is and usually why you need it. Very very useful stuff. For instance, skipping --enable-__cxa_atexit would change the C++ ABI and make the new system binary incompatable with C++ programs from most Linuxen.

The next thing a rational person might ask is "For the love of penguins, WHY?" The LFS website says a bit about this, but that doesn't say why I'm doing it. Well, because I want to know what makes it tick. I know a lot about Linux. Heck, most of my friends consider me an expert. But I let Debian and Ubuntu tell me what goes where and how things work. This should help me get an understanding of what each piece does, how they hook together, and what's actually needed to make it work.

Also, I've done a little embedded work. I was mostly working off a framework someone else made, but doing this would let me build it myself. I'm also hoping that after this I can make a simple little uClibc/busybox VM. Might need to keep around a "host" system that does things like compiling pacakages, but hopefully I can even write a basic "package management" system. Reinventing the wheel, I know. But as a programmer, knowing the deep magic is important.

I was inspired to look into this by GoboLinux, a Linux distro that has a bit of the "do it from scratch" mentality, but mostly a "let's do it right, not just how it's always been done." It's an interesting setup, and apparently one that causes them some issues from time to time (autoconf is easy, things like CMake, rubygems, and CPAN give them headaches). But I like the concept, it's a lot like what Apple tries to do and what I've thought of from time to time. Put each "package" into it's own little bin rather than installing everything on top of each other. Compelling. I'm going to try it out after my LFS experiment.

(Stream of consciousness follows...)
3/21 evening (written after the fact):
  • Okay, found an LFS LiveCD. Seems appropriate to follow the LFS book using the system created from it. Should also give me everything they expect to be around.
  • Hm. It's a LiveCD for LFS 6.3 and I'm using 6.6. Crud. Means all the packages are out of date...
  • Ooh, found a 6.6-rc1 LFS package. Yay, piles of sources.
  • Hm. -rc1 means it doesn't have everything. Let's go through the list by hand...
  • Got everything. Starting the process... Wow, this is arcane, although not much I haven't seen before.
  • binutils, check. GCC, check. Glibc, check. binutils pass 2, check. Hm, it's late. I'll set GCC pass 2 going and go to bed.
  • 3/22 morning
  • What? Failure? Can't find stdio.h? But it's right there! Try again!
  • Still not working? Buh. I'm not far in, let's start compiling everything again.
  • A few hours later (after breakfast and errands)
  • GAH! What the...? Okay, let's see if we can... Screw it, it's start over and follow instructions more carefully...
  • Okay, rather than finding the package again, let's download the book into the Linux VM and use the source to grab files.
  • No networking?? It worked the first time! *poke, poke* Okay, it's not happy with VirtualBox's NAT setup. Whatever, Bridged Connection works.
  • Hey! The HTML version of the book gives a nice easy list of URLs. Nice.
  • No nice easy list of MD5 hashes. Well, HTML + VIM = expected output from md5sum.
  • Missing zlib? Huh? It's in the list! Hm, it's an old version. Use the new one? Hm. Book says "Follow book, book good." Bugfixes don't matter on a test platform anyway.
  • Gah! It's hard to find old versions of zlib. Finally got it from libpng's SF page.
  • Starting over. Apparently the thing (or one thing) I did wrong was not wiping out the build and source trees. Maybe I had left over config stuff.
  • Binutils, GCC, Glibc...
  • Huh, this is where it screwed up last time. Let's back up.
  • Binutils pass 2, check. GCC... *crosses fingers* Hey! It worked!
  • Hm... "The Tcl test suite may experience failures under certain host conditions that are not fully understood. Therefore, test suite failures here are not surprising, and are not considered critical." Okay, it said running test suites in this section isn't important anyway. Let's just skip them. We'll run them for the final installations.
  • Moving on, compiling lots of stuff now. And hey, I now think I know what it's doing...
  • Although it presents itself as two main compiling steps, it's actually three.
    1. Create a cross-compiling toolchain from Linux to Linux. Seems silly, but it has to be done to isolate the new system from the host. Don't want the new tools we'll be using in a chroot to refer to things outside it.
    2. Create a build system. There are a large number of tools that we need to compile everything for the final system. It starts with glibc and a new toolchain, but also including every tool we need to make everything else.
    3. Build the final system. Use the new self-contained build enviroment, chrooting into it to ensure that nothing outside can possibly affect it.
    I had originally thought that step 2 was to form the core of the final system, but this is a multi-step bootstrap. Interesting.
  • Okay, back to compiling. These packages are quick compared to things like GCC.
  • Hey! Looks like compiling the build system is done! Much faster than I expected. Next step: stripping. (Not that, perverts.) Not required, but let's see how much space I save by stripping out debugging stuff.
  • Oops. Forgot the "before". Well, after I have: 241 MB. Mind you, this is everything I'll need to build and compile the final system (excluding the sources), not the final size of the system.
  • Okay, Part II done. (Part I was the intro.) Now onto Part III "Building the LFS System". Wow. About 5 hours down just doing the prep work. Not an easy recipe.
  • Bed time.
  • 3/22
  • Chroot step 1: device files. Device files are some of the most crazy bits of Linux. Files that represent hardware. Crazy, but crazy-awesome. Also, difficult to maintain. Fun fun fun. LFS claims we just need console and null to make things work, but other things are useful to. It uses mount --bind for the chroot and udev for the full system to give us a full set of device files.
  • Step 2: Virtual file systems. PuedoTTYs, shared memory, and a couple of "access the kernel" directories are handled by storing it all in memory. I knew that, but still interesting. (/dev/pts, /dev/shm, /proc, and /sys)
  • Step 3: Package Manager. Hm. I usually like apt-get, but that's for Debian and I'm not building a Debian. Let's poke around LFS/hints...
  • Hm. Well, let's use " Symlink Style Package Management". It's what GoboLinux does, so it should work. Let's grab my old friend stow. I usually use it to manage /usr/local. Shouldn't be all that different to use it for /usr. I think I'll keep / for boot-needed, /usr/ for stow-managed, and /opt for "funny stuff".
  • Looking around for stow's source led me to XStow. I don't think I'll ditch stow for most uses, but for a minimalist system XStow will let me ditch perl. Awesome.
  • Basic directory structure and files... BORING. But needed. I already see bits that could be stripped out, but first time is by the book.
  • Hmm... Looking at it, I probably should skip installing Xstow until after I get the final glibc in place and point the toolchain at it. Using stow would let me do it now, probably.
  • stow installed. Whee.
  • Linux headers, man pages, glibc... Okay, wait. glibc has to install things outside of /usr. I should manage those too. Reverse earlier decision! Stow everything!
  • Neat. Stow has... issues... dealing with /. Time to patch...
  • --- stow        2010-03-22 06:00:15.000000000 -0400
    +++    2010-03-22 06:00:01.000000000 -0400
    @@ -40,6 +40,13 @@
     $Target = undef;
     $Restow = 0;
    +# Exclude the following when unstowing in /
    +%RootExcludes = (
    +  dev => 1,
    +  proc => 1,
    +  sys => 1,
     # FIXME: use Getopt::Long
     while (@ARGV && ($_ = $ARGV[0]) && /^-/) {
    @@ -222,6 +229,9 @@
         warn "Warning: $ProgramName: Cannot read directory \"$dir\" ($!). Stow might leave some links. If you thing, it does. Rerun Stow with appropriate rights.\n";
       @contents = readdir(DIR);
    +  if (&JoinPaths($Target, $targetdir) eq '/') {
    +    @contents = grep { !$RootExcludes{$_} } @contents
    +  }
       foreach $content (@contents) {
         next if (($content eq '.') || ($content eq '..'));
    @@ -509,7 +319,7 @@
       local($path) = join('/', @_);
       local(@elts) = split(/\/+/, $path);
    -  join('/', @elts);
    +  join('/', @elts) || '/';
     sub usage {
  • (Heh, copying patchfiles by hand is "fun")
  • Ooops. Didn't finish instructions for glibc before deleting the build dir. Let's extract it and run configure again..
  • Oops. Didn't need to bother. *sigh*
  • GAH. A lot of these projects need to learn to clean up their build scripts. It's difficult to find the test output amongst the huge and busy compilation lines.
  • I have eight stow-managed packages installed. So far so good. glibc doesn't take DESTDIr, but it will accept install_root. We're back up to my perennial non-favorite: GCC. It's supposed to take 44 SBU. SBU is the time it took to build the first package. Clever system. For me, it's about 6.5 min so GCC should take... Nearly 5 hours. Oh, goody.
  • Buh. Should have written down when I started the testsuite for GCC...
  • GCC finished, a few more packages down. Going smoothly so far.
  • End of day 2 (of take 2): On section 20 of 62. So, nearly a third of the base packages are installed. I see why people write package managers to do this for them... And then just copy around the resultant binaries. :-D
  • 3/23
    Compiled a few more things. Nothing really interesting except for a few mistypes. Seems to be working well. *knock on wood.*

    Okay, have to do work and life stuff for a bit. I'll post again when I get back to compiling... To note so far, this is getting boring. Notably, I'm devolving to "type what the book says" instead of actually figuring stuff out. *sigh* Maybe a few days break will help.

    Tags: linux, programming
    • Post a new comment


      default userpic

      Your IP address will be recorded 

      When you submit the form an invisible reCAPTCHA check will be performed.
      You must follow the Privacy Policy and Google Terms of use.