10 (Easy) Steps to Generate an ISO
The following are the general steps that need to be taken in order to create an ISO from scratch. These are not necessarily the steps that have to be followed, but this is how I generate ISOs. Example command sequences are provided for each step. Step 10 is more necessary if you are building an official ISO, but is recommended even for home use. Before doing anything in the recipe, you will need a working kitchen. Get yourself a cauldron to brew in (and check out the test branch):
git clone git://scm.sourcemage.org/smgl/cauldron.git cd cauldron git checkout -b test origin/test
Definitions
- build directory
This is (usually) the chroot or basesystem directory you start out with. cauldron/scripts/spellcaster.sh uses this to generate the Sorcery cache files which are later used to populate the system and iso directories (see below).
- system directory
This is the directory which will eventually become system.tar.bz2 (the base system that you install when you actually use the generated ISO) via cauldron/scripts/mksystem.sh. It is populated from basesystem Sorcery caches from the build directory (see above) by cauldron/scripts/spellcaster.sh. The list of spells used to populate the system directory is equal to the contents of rspells.$ARCH, where $ARCH is the (supported) architecture to generate for.
- iso directory
This is the ISO itself. It is to be composed of a basic system (minus build tools like gcc) plus required ISO-specific spells (spells needed during the install itself for end-user functionality to install their system, e.g., filesystem spells like xfs or reiserfs) and optional spell caches (like lilo or grub, which aren't needed during the install itself but may be needed after the ISO (reboot) or are just nice to have). The spells are installed using the Sorcery caches in the build directory (see above) generated by cauldron/scripts/spellcaster.sh. The list of spells used to populate the iso directory is equal to the contents of ispells.$ARCH plus ospells.$ARCH, where $ARCH is the (supported) architecture to generate for.
Important Files
There are three different spell listing files that directly influence the composition of system.tar.bz2 and the ISO itself. All of these files are located in cauldron/data/. Replace $ARCH below with the (supported) architecture for which you will generate an ISO.
- rspells.$ARCH - any spell which is to become a required part of the installed system (this should be as minimal as possible, preferably only basesystem itself unless some other spell is really needed)
- ispells.$ARCH - any spell needed on the ISO during the runtime of the install, including filesystem tools
- ospells.$ARCH - optional spells, only to be provided on the ISO as spell caches (not available during runtime of the install)
Instructions
All cauldron ISO scripts (for the stable and test ISOs) are located in the cauldron/scripts dir. Replace any occurrence of $ARCH in the instructions below with the (supported) architecture you will generate an ISO for.
Get a fresh basesystem tarball, unpack it into a directory, and rename that directory "build". Set up sorcery as needed here (architecture, etc.). For x86, nothing is needed since sorcery defaults to i486.
wget http://download.sourcemage.org/image/official/smgl-stable-0.39-basesystem-x86.tar.bz2 tar xf smgl-stable-0.39-basesystem-x86.tar.bz2 mv smgl-stable-0.39-basesystem-x86 build
Build a linux kernel and modules, however you choose to do so: manually, or by adding the linux spell to rspells.$ARCH, where $ARCH is the (supported) architecture you are building for. We have a kernel config that you can use in our git repo (cauldron/data/config-2.6). The kernel config in the cauldron repo is currently for 2.6.27.10. If you build the kernel manually, don't forget to make sure it is available in usr/src/linux of the build directory by copying it there, as well as adding all the built modules to the build directory's lib/modules/$RELEASE/ path, where $RELEASE is the version of the kernel you built.
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.27.10.tar.bz2 cp linux-2.6.27.10.tar.bz2 build/usr/src/ cp /path/to/cauldron/data/config-2.6.$ARCH build/usr/src/ /path/to/cauldron/scripts/cauldronchr.sh -d build cd /usr/src tar xf linux-2.6.27.10.tar.bz2 ln -sfT linux-2.6.27.10 linux mv config-2.6.$ARCH linux/.config cd linux make && make modules_install exit
Cast all the spells that will go on the ISO (required and optional) via cauldron/scripts/spellcaster.sh build x86. This will default to creating /tmp/cauldron/iso and /tmp/cauldron/sys to put the respective iso and sys content into (i.e., the iso and system dirs referred to later). You can change what directories it uses with the -i and -s option flags. The list of standard x86 spells to cast for the official x86 isos can be found in the repo in cauldron/data/rspells.x86 (for sys), cauldron/data/ispells.x86 (iso), and cauldron/data/ospells.x86 (optional). Note: the terms "required" and "optional" here refer to what is required vs. optional for the user to install onto the target system. All of these spells (or their caches) are required to be present on official ISOs. Also, only the required spells should be in the final system.tar.bz2, and optional spells should only be present on the ISO itself.
/path/to/cauldron/scripts/spellcaster.sh build x86
Make any necessary adjustments to the system tree (/tmp/cauldron/sys). This includes running cauldron/scripts/add-sauce.sh -o -s /tmp/cauldron/sys, copying linux source tarball used to build the ISO (the one from step 2) to /tmp/cauldron/sys/var/spool/sorcery, and creating a symlink from the tarball in the spool dir into usr/src/ e.g., ln -sf /var/spool/sorcery/linux-VERSION.tar.bz2 /tmp/cauldron/sys/usr/src/ for easy access. Remember that all paths just mentioned are relative to the sys dir (except for the first argument to ln, which is not a typo).
/path/to/cauldron/scripts/add-sauce.sh -o -s /tmp/cauldron/sys cp linux-2.6.27.10.tar.bz2 /tmp/cauldron/sys/var/spool/sorcery/ ln -sf /var/spool/sorcery/linux-2.6.27.10.tar.bz2 /tmp/cauldron/sys/usr/src/
Prune the iso tree (/tmp/cauldron/iso). This can be done by running cauldron/scripts/cleaniso.sh /tmp/cauldron/iso. This will remove unnecessary things like sorcery tablet files and logs, and unnecessary documentation in usr/share/doc for example.
/path/to/cauldron/scripts/cleaniso.sh /tmp/cauldron/iso
Make adjustments to the iso tree (/tmp/cauldron/iso) by running cauldron/scripts/add-sauce.sh -o -i /tmp/cauldron/iso.
/path/to/cauldron/scripts/add-sauce.sh -o -i /tmp/cauldron/iso
Make a compressed tarball of the system tree, and copy it to /tmp/cauldron/iso/system.tar.bz2. Creation of the tarball can be done easily by running cauldron/scripts/mksystem.sh, which will use /tmp/cauldron/sys and /tmp/cauldron/iso by default, and will also by default drop the finished tarball into the iso dir. Don't forget to copy the tarball to the iso tree after creating it if you save it to a path other than the root of the iso tree!
/path/to/cauldron/scripts/mksystem.sh
Make an initrd, and copy it to /tmp/cauldron/iso/boot/initrd.gz. This can be done via cauldron/scripts/mkinitrd.sh. This uses /tmp/cauldron/iso by default for the iso dir, and if you don't specify the kernel version as the optional first argument, it will try to get it automatically from the iso dir's lib/modules/VERSION directory (assuming it exists, and that there is only one). It will also by default drop the finished initrd into boot/initrd.gz in the iso dir (however that's defined: the default, or something you specify).
/path/to/cauldron/scripts/mkinitrd.sh
Generate the iso (i.e. mkisofs) and compress. If you are making a custom ISO, then you may find cauldron/scripts/mkiso.sh useful. However, for official releases, you can do cauldron/scripts/mkrelease.sh VERSION (e.g., cauldron/scripts/mkrelease.sh 0.10.0-test5). This uses /tmp/cauldron/iso by default. This will give you both a compressed and uncompressed finished product in the current working directory. You can use the uncompressed ISO for immediate testing, and if it works, you can go ahead and ship the compressed version.
/path/to/cauldron/scripts/mkrelease.sh 0.10.1-test1
TEST the ISO (minimally in a VM). Go through a full install, following all of the steps, and try to catch anything out of the ordinary. Make sure the directions for the steps make sense and produce something that works, to the best of your ability. Assuming everything worked, it's time for a release. Go through and make sure (again) that all the files are as they should be (especially in /etc and /var). If everything is as it should be and the iso worked as expected during testing, release it. Congratulations.