4/21/2004: Adding a build split howto: Build_Split_HOWTO
4/14/2004: Im creating 3 documents: the first defines the old spell build phase api: Build_Api_1 the second defines the new spell build phase api: Build Api 2 the third describes the grimoire API_VERSION, and how it will work: Grimoire_Api
Introduction
When casting a spell the current logical pipeline of actions is: PREPARE -> DETAILS -> CONFIGURE -> DEPENDS -> PRE_BUILD -> BUILD -> POST_INSTALL While there are a number of steps at the begining of this pipeline, the BUILD phase is a bit overloaded. BUILD doesn't just BUILD, it also dispels the old spell and installs the new one. The proposed plan has 3 phases:
1) Have a way to have BUILD split into two or more files, COMPILE and INSTALL :Sorcery should allow either behavior to exist while we swap the grimoire :A script will be provided to split BUILD reliably to ease the transition
2)Take functions related to casting spells in cast, and move them to a new library in preparation for phase 3.
3)Add a tool to allow key spell scripts to be run at users request (ie COMPILE/INSTALL/PRE_BUILD) this utility will use many of the same functions as cast, and so phase 2 is a necessary prerequisite to this step.
Motivation
The current setup of BUILD only has worked for us, however it has some crucial flaws which make dealing with failures annoying at best.
Here is a hypothetical situation: I decide one day to cast OpenOffice. Say for example something special about my box or my compile options, or _something_ causes this spell to fail during compilation. At this point I can frown and try over, or hack up the spell to let me start up where I left off. This process involves copying the old sorce directory somewhere safe, replacing PRE_BUILD with 'true', and commenting out things in BUILD that have already happened, and finally putting a prompt in somewhere to give me time to copy the old source tree back (presumably after I've gotten the thing to build by hand). After I do all that cast will (hopefully) finish and OpenOffice will not only be installed, but sorcery will now track it.
This solution is kludgy at best, try explaining it to a newbie 
Another common problem most of us have experienced is the glibc changover. For those of us who dont remember, if you are upgrading glibc from 2.2.5 to 2.3 then there is a certain part of the install where the install hoses itself and cast died miserably because it was unable to run anything. The only solution we had, as no one was able to find a better one, was to ctrl-c the dying cast, run ldconfig, run make install in the source directory, then run cast -c glibc again.
Exceedingly annoying. At best. Im not blaming anyone, but these are problems I hope this will solve, and create a more clean and flexible sorcery.
The proposed plan at 10,000 feet involves first a split of BUILD into its two logical constituants, COMPILE and INSTALL, this change when accompanied by a new sorcery script to run individual spell scripts will allow these problems to be solved. The working name for the script will be "mutter"
Refering back to the first example: OpenOffice's build failed. So I cd to the source directory and do whatever it is I need to do in order to make the spell build. After that I run "mutter INSTALL OpenOffice". This script ensures that the source directory is there, then runs the INSTALL script for OpenOffice in it, replicating the environment that sorcery had. After that I can run "mutter POST_INSTALL OpenOffice" to run the final bit of casting. No bending over backwards to get OpenOffice installed in the face of problems.
Refering to the glibc issue: I ctrl-c out of the dying cast, run ldconfig, and make install to get a working system again. Then rather than running cast -c glibc again to get sorcery to track the files, I just run "mutter INSTALL POST_INSTALL glibc" and all the right things happen.
The addition of this script allows users to customize the way a spell builds if they desire without forcing them to make a custom spell.
Implementation
The work will be divided into three main phases:
Split BUILD
Support in cast needs to be added to allow for BUILD to optionally exist as two files, COMPILE and INSTALL. In order to ease the transition, a special flag will be temporarily added to DETAILS to indicate which BUILD_STYLE to use. Later the default behavior of cast can be changed to support the new system, and the tag can be removed. An alternative method is to test for the existance of COMPILE and INSTALL, and run those in preference to BUILD. Either method would work.
The changes will be transparent to spells without BUILD files, and those with BUILD will need to be changed over. I will provide a script for properly splitting BUILD.
In terms of code changes, they will be listed acording to file
cast
the 'cast_spells' function needs to test for the existance of a BUILD_STYLE variable variable (set from run_details), or the existance of executable COMPILE and INSTALL files. If the condition is met two new functions will need to be run:
- run_compile, and run_install.
Alternatively if the condition is not met, run_build should run as normal.
run_compile and run_install should consist of logically split portions of run_build. An enhancement thereof would be to take the code that deals with ./configure out of that run_build (as it will have to be duplicated) and place it in a seperate function.
run_install should also re-run activate_voyeur due to the fact that tee stops transfering from the install fifo into the log at EOF, which would be sent at the end of a COMPILE script. Rerunning activate voyeur should have no visible effect on the compile log.
libcodex
codex_clear_current_spell will need to unset BUILD_STYLE if that method of choosing the build style is used.
libapi
Orthoganal to the file BUILD being split, the default_build function should be split into its constituent parts as well. default_compile and default_install should be added, pointing to their real counterparts. default_build should be left for backwards compatibility.
libgrimoire
just as libapi had its default_build split, so should real_default_build. The split is seemingly obvious, but real_default_compile should be everything up to and not including prepare install. The ending redirection should remain.
Likewise real_default_install should include have prepare_install and make install in it.
Create libcast
libcast is just a working name for all the functions that cast has which will be also needed by mutter. The added bonus for doing this is that cast then becomes a simpler script to read. Ideally everything below cast_spells can be moved.
This step should be straight-forward and have little no to impact on anything else.
Mutter
Mutter will take a list of spell scripts to run in order, and a spell to apply them to. A future enahancement would be to have it start at the begining and work its way to the given script, or start the the given script and work its way to the end.
So one could run mutter -start PRE_BUILD OpenOffice and the user will be given a setup source directory for that particular spell with all the files unpacked in the right places.
Mutter will not, and should not deal with dependencies in so far as compile them. That is casts job, it will assume that they are there. The idea behind mutter is that it should be a backend tool for doing low level things, the high level interface is left to cast.
Mutter may have a feature to run CONFIGURE/DEPENDS and set the appropriate options, but it won't actually ensure that uncast dependencies are cast.
Future work
Split INSTALL even more into PRE_INSTALL and INSTALL scripts. The function of PRE_INSTALL could be to run various tests against the spells, and/or to run prepare_install.
Another enhancement on this same line is to change the INSTALL process to install into a seperate install tree, then manually copy over files to ensure a safer installation and eliminate the need for prepare_install altogether.