This page serves as a highlight of the new features and capabilities of the devel work (master branch) of Cauldron. It should give you an idea of what is being worked on, what can be done using the newer Cauldron (once the implementations are completely functional), or how it can be further extended to do even more interesting things than what has already been thought of.
Cauldron now contains 3 major components and 2 common libraries:
Contents
Each of the above components will be discussed in turn below. Note that capitalized "Cauldron" always refers to the Source Mage project component, while lowercase "cauldron" always refers to the specific set of sources for generating installation media within the larger Cauldron collection.
Note: these are the features/implementations that the Cauldron Team is currently working on. If you would like something additional to be implemented, please feel free to request it. If you are willing to implement something yourself, then please by all means do so, show it to us, and we will try to include/integrate it into the main repository (unless we have a very good reason not to, which we will try to explain clearly; even in the event that we feel we have a reason not to include it in the main repository, we might still include it in a separate branch or provide an official link to your repository [and/or any releases of the installation media if you make releases] on the wiki). Also, if you would like to make regular releases of installation media (custom ISOs that you will maintain), such as ISOs that come with X Windows, please feel free to contact the Cauldron Team (via email or IRC -- see the Source Mage contact page for information).
cauldron
The cauldron component is responsible for generating installation media. It is meant to be extremely configurable, while still having very sane defaults. This means that you should be able to just install cauldron, run a single command without configuring anything, and you will get an up-to-date installation ISO that you can then immediately test and, assuming there's no problems, release. Meanwhile, you can adjust everything that makes sense to play with (and even some things that normally might not make sense, in case you really want to hack something) to produce an entirely custom installation media.
The implementation of cauldron is not yet ready for use, but is nearing completion. The current elements being implemented are:
- robustness of the entire system (everything should follow a fallback chain -- if something fails, drop to a fallback, and if the fallback fails, drop to its fallback, down the chain until we simply error out to the user and die)
- uniform colorized (or not) output based on libcolor
- uniform error code and message handling via liberror
- automated creation of initrd/initramfs using configurable list files (binaries, libraries, etc.) and a busybox config file
- the system only uses what is available, so if a given list file is missing or empty then nothing of that type is added
- automated chrooting (setup, execution inside the chroot, and tearing down/cleaning up)
- integration with sorcery - high-level means of ensuring specific configurations of sorcery (for example, setting the architecture or guaranteeing archiving of cast spells)
- automated generation of a base build toolchain, which can be created from different source material:
- resurrecting a spell cache from the host system if it can find one (i.e., if you cast gcc on your host system and have the archive of the install, it can use that directly)
- creating a spell cache using the install log on the host system, in the event that you have a spell installed and the install log is intact, but you are missing the spell cache itself (if you had ARCHIVE off in sorcery when casting the spell, or you deleted the cache)
- manually casting and then dispelling in a chroot to generate a fresh spell cache
- in the future, automated cross-compilation will be integrated in, so that you could generate installation media for an architecture other than what your host system is capable of (i.e., generate ARM media from x86 systems, etc.)
- automated installation and setup of all installers (see enchantment below) that are selected for in the cauldron config file
- all functionality is implemented as an accessible and extensible API
- in both the API and the frontend cauldron command, higher level calls/functions are implemented as bundles of lower-level calls/functions with possibly some additional processing (each function should do one and only one thing, but do it well)
- all configuration variables are read from a single file (though because that file is simply a bash file which is sourced, you can easily separate the content and include it by sourcing other files)
- after the configuration file is loaded (if it doesn't exist defaults will be used), cauldron will look for a functions file in the config directory so you can provide custom function overrides, without having to modify the cauldron sources directly
After the above features are fully implemented, work will begin on implementing the generation of chroots from a running filesystem and then installing from those minimal chroots. In other words, an automated form of what Source Mage refers to as a "possession" install. Once all the above parts are implemented, this actually shouldn't be too difficult, though of course care will need to be taken to ensure proper compatibility across different distros of the individual binaries used to generate the initial chroot. Once the initial chroot is generated and verified, though, it should definitely be possible to transition to a full Source Mage system.
enchantment
The enchantment component is responsible for the "installers". All of the installation functionality is implemented in libraries as an extensible API. Individual installers are then simply front-ends that hook into the libraries. This results in all installers having a uniform backend, which means they can share data. This allows a user to seamlessly switch from one installer front-end to another, continue in the install, and then switch again without losing any of their process.
Currently, only the guided shell installer is implemented (similar to what is used on the 0.10 series of the Source Mage ISOs, but reimplemented). The implementation for the shell installer is mostly complete at present. Additional installers are planned, but work is focused on finishing the basic cauldron implementation (see above) before the other installers will be implemented.
The current features of the enchantment backend are:
- state tracking of install information for automated installs and (partial) undo
- better support for internationalization (not yet finished, but the skeleton framework is there)
- improved installation granularity - can install arbitrary files or directories, not just spell caches
- this means that spell sources can now be installed easily, as long as they are provided, giving better control over exactly what is installed and how, before booting and doing a sorcery rebuild
- includes better control over installing kernel images, as now any arbitrary kernel image and associated modules can easily be installed (including from another distro residing on another hard drive partition, for example)
- install steps are now implemented as modules, each having instructions but also potentially scripted elements as well, so that complex installation procedures can be handled and tied into the back-end, providing greater flexibility and extensibility of the installation system (if specific files need to be installed given that others were selected, this can be implemented in that particular installation module)
- installation modules can be architecture-specific or have architecture-specific components, or can be architecture agnostic (not yet finished)
- different front-ends can select different module sets as needed
- module instructions are not hardcoded with presentational elements (such as console bold escape chars), as the presentation is now handled by the front-ends
- linked to liberror for better robustness and problem reporting
The current features of the shell installer are:
- unified command enchantment, with different sub-commands (like git pull, git stash, etc. all have "git" as the main command)
- all functionality of 0.10 guided shell
- new features:
- potion - state tracking during the install for later automated installs, but also enables undo! (not yet finished)
- can install spell caches, sources, or both (depends on what's available on the installation medium though)
- can install any arbitrary kernel image and modules, in addition to the choices in the 0.10 series
- presentation of different elements is hooked into libcolor so there can be colored output, which is also uniform with other Cauldron components
- linked to liberror for better robustness and reporting of problems
The following additional installers are planned for implementation:
- curses-based menu installer (sooner)
- web-ui based installer (later)
If there is enough clamour for a gui installer, it will be considered, but it will be less likely that the Cauldron Team will implement it ourselves (hint: if you want to see a gui installer, we won't necessarily be against it, but implementers wanted!).
ward
The ward component is a test harness for the other Cauldron components. Currently it consists of static analysis tools for the bash sources themselves (that's right, static analysis of bash, in bash!). There are general library-like components that can be easily extended to check other sources, as well as specific front-ends for checking over the Cauldron sources (both cauldron and enchantment).
The following types of static analysis tests are currently implemented (the front-end which runs the test over the cauldron and enchantment sources follows in parentheses):
- function naming and definitions (ward-functions)
- global variable naming and definitions (ward-configvars)
- error code definitions and matching messages (ward-errorcodes)
Currently the implementation of a local variable static analysis is being worked on.
All the ward tests are linked with libcolor and liberror. This means that the tests report in colored output (unless specified not to), and that those colors are uniform with the colored reporting of the other Cauldron components. The error messages themselves are also handled in a uniform way, though the error messages of the ward component are more rudimentary than of either cauldron or enchantment, due to the more straightforward nature of ward (if a test fails, it fails, and since individual tests are fairly specific, it should be obvious why it failed; thus there is less need for individuated error codes than for the other components where a function may fail for different reasons).
In the near future, the framework will be expanded to include run-time tests as well, to guarantee that final products are generated correctly.
liberror
This is not a separate component per se, but rather a general library to provide a common error checking and reporting interface. This unifies the error handling of all the Cauldron components, and helps to make them more robust by providing an easy-to-use API (thus encouraging more use of error handling by developers). All of the error codes, and their corresponding messages, can be overridden as needed. Liberror itself only defines a very minimal set of error codes and messages (currently 2 of each, one being for fatal errors and the other being for "OK", meaning that there was no error). Functions are also provided for outputting different error messages, which will make use of libcolor if libcolor has been sourced, and the output locations (typically STDERR or STDOUT) can also be overridden with environment variables, making logging to a file or silencing all error output easy.
libcolor
Like liberror, this is also not a separate component, but a general library. Libcolor provides a common interface for generating colored output of different messages. Message types are defined, as well as different colors, so that all messages of a given type are colored the same across all applications that link against libcolor. This provides a uniform appearance, which makes the use of color more meaningful. All of the colors can be overridden through environment variables, and it can also be themed easily. Libcolor also understands monocolor, normal console color, and no color, and each of these three can be selected via environment variables. Support may be added in the future for additional color capabilities as needed.