* $Id: README,v 1.5 2006/04/02 18:16:07 me Exp $

* Repository

The base and simultaneously the product of BTE activity is repository. It is
a consistent and self-contained set of packages, which form a unified software
product (1). As primary data for BTE already existing sources in form of packages,
tarballs, CVS modules could be used. It is followed by the closure of primary
products meaning the creation of an essential minimum of packets which comply
with the above stated specifications (1). The current BTE state does not allow
to fulfill this task (let us call it bootstrap) automatically, though under
certain conditions such functionality could be gained. Further activities with
repository, namely change of packages, addition of new ones, deletion of obsolete
packages, and also introduction of repository history are carried out only by
BTE. Thus, it is possible to obtain repository slice with every existing
functionality and in any BTE functionality state as a final product.

** The structure of repository

Package is a store unit of repository. All operations with the package are conducted
with an indication of its name and, optionally, version. In the event that
a package as a result of build is subdivided into a number of packages, the name
of a package is always defined by the name of its main package. Version is
understood as a line, consisting of Serial, Version and Release values of 
a source package. There are two possible methods of storing a package in repository:

*** module
This method is meant to store sources of a package, which are constantly in operation
that is chosen by indicating $cvsmodule/modules in a $cvsmodule/modules (see below).
The root directory of that module should contain file of a kind <package_name>.spec
(specfile), and also, Mkefile with target `bte', which defines actions to prepare
package sources  for build, usually it is allocating of the archive with sources
to the location where rpm expects to find them and under the name specified in
Source field: of a rpm specfile. Final stage is to move the specfile content to
the standard output (stdout).

*** spec
This usually more often used method is meant for packages, which are under operation
only occasionally and it is conducted in other location. It is advisable to store
spec under CVS management in a specially defined module, specified in $cvsmodule/modules
and under the name that coincide with name of the package, while sources are stored
in directory system that contains all sources for all the versions of the package
over its existence in BTE.

Besides packages proper, repository comprises also module, which name is set by
cvsmodule parameter and which contains BTE specific information. For details
see below.

* Installation

** Usable external utilities:
chrootuid dd mke2fs mount sudo rpm apt [ccache] tcl tclx tcllib tcl-memchan tcl-trf
Package ccache is optional and is not used when not mentioned in a list of packages
forming the buildsystem.

** sudo tuning
While sandd in a process of creation and deletion of chrooted env's should have a 
possibility to fulfill a restricted set of commands with the superuser rights,
it is essential to permit the fulfillment of those commands without the necessity
to enter the password, for /etc/sudoers sample contents see in sudoers.sample

** cvs
should contain (relative to $CVSROOT):
- $cvsmodule/{release,installer,macros,modules}, see examples;
- $cvsmodule/{release,installer,macros}.generic, may be empty;
- CVSROOT/{users,passwd} with their introduction into CVSROOT/checkoutlist;
- CVSROOT/commitinfo contains entries of the following kind:
---- %< ----
<modulename>/* /usr/bin/sandctl -hook cvs:commit
---- %< ----
for every modulename, defined in $cvsmodule/modules, and also for $cvsmodule

** Configuration files sandd
*** /etc/sandman/sandd contain sandd daemon setups
---- %< ----
[global]
# listen on interfaces
# myaddr = 127.0.0.1
# listen on port
port = 65432
# debug level
debug = 0
# sandctl fifo name
fifo = sandctl
# pidfile name
pidfile = sandd.pid
---- >8 ----

*** /etc/sandman/repos
Contains repository definition in a format of variable = value.
Parameters (values are set by default given no other stipulations):

---- %< ----
[global]
# BTE top 
prefix = /var/lib/sandman
# location of chroot fs creation
cachedir = /var/cache/sandman
# common cache for ccache (different ones could be set)
ccachedir = /var/cache/sandman/ccache
# target arch
target = i386
# CVSROOT, value any admissible in respect to cvs(1)
# + possible to indicate relational (from $prefix) path
# CVSROOT environment variable is not used and not exported 
cvsroot = cvs
# service module in cvs, syntax is similar to that used in 
# $cvsmodule/modules, see modules.sample
cvsmodule = release
# indication of this optional parameter allows to maintain stable 
# repository out of sandman tree, see repos.sample
# default value is specified in accordance 
# with $prefix, $target and repository name
stable = file:/var/ftp/pub/Sisyphus i586 classic
# additional uris for stable repository can be added like this:
stable += file:/var/ftp/pub/Sisyphus noarch classic
# basic set of packages, set up by a number of transactions 
basesystem = setup glibc-core filesystem basesystem
# basic set of packages for building
buildsystem = basesystem rpm-build
# directory with sources
# could be any, relative path - from $prefix
sources = sources
# line for RPM
distribution = Braintwisted Linux
# line for RPM
vendor = Yoyodyne, Inc
---- >8 ----

** creation of initial repository 
It is required to create the following structure of folders:
$prefix
   |----$repos
          |----stable (*)
          |     +----SRPMS.$lrep
          |       +----$target
          |               |----base 
          |               +----RPMS.$lrep
          |----current (like stable) (**)
          |----$iso (*) (**)
          |----$sources (*) (**)
          |----$cvs (*)
          |----$cache (*) (**)
          |----$ccache (*) (**)
where:
$var is a parameter indicated in /etc/sandman/repos
 *  - subfolders, the layout of which could vary 
 ** - subfolders, created automatically
 $lrep - repos value in lower register.

Then it is required to fill stable/$target/RPMS.$lrep up with packets that form
buildsystem, observing all dependencies, as usual they are donor-distributive
packages, and finally, to index stable with the help of genbasedir (apt utility).

* Usage

** Maintenance of repository integrity 
The following operations are conducted in indicated modules during commits:

*** for spec:
- Serial, Name, Version, Release, Group, changelog values are validated;
- it is checked whether suggested to be committed spec file contains SVR,
  that were altered in accordance with the latest CVS revision; with the
  increased values cvs tag of $branch-$serial-$version-$release kind is put
	into last spec revision in CVS, with the decreased values the situation is
	regarded erroneous;
-	for every in spec indicated source file the following actions are performed:
  its existence in the package root directory ($sources/$name) is checked;
	if the file exists, it is moved 3 levels lower (serial/version/release);
	other, its  availability is checked in (name/serial/version/release);
	other, its  availability is checked in (name/serial/version/*;
	other, its  availability is checked in name/serial/*/*;
	other, it is regarded an error;

If any error situation arises commit is rejected, $sources/$name directory
is cleared.

*** for module
-	check whether a spec file that is suggested to be committed  exists on a list;
-	if available and not a single one -> error;
- Serial, Name, Version, Release, Group, changelog values are validated;
- contains altered SVR:
	with the increased values cvs tag of $branch-$serial-$version-$release
	kind is put into current (unlike spec) module state (not spec), with
	the decreased values the situation is regarded erroneous;

If any error situation arises commit is rejected.

** builds

Buildis initiated by a corresponding call of sandcl(1) with an indication
of a target repository, package name, Version (optionally) and compiling
kind (mixin), and also method:

*** build 
Last regarded stable state of sources and/or spec is compiled, last revision
of spec in CVS, which has a $branch-$serial-$version-$release kind tag is
considered as such, if it lacks last revision is used. Similar sequence of
actions is used towards the package that is stored as a module, the same 
spec file acts as tag carrier. If succeeded the results of the compiling
are stored to a special current repository.

*** buildpocket
pocket -- a kind of repository created by user request and is meant for operations
with packages/builds, that could not be moved to a stable repository. Unlike the
first method, here the last state of sources and spec-files is compiled. Thus,
experimental and/or incremental builds are possible.
Compiling undergoes following stages:
- creation of file system image using loop-device;
- installation of packages, that form basic buildsystem;
- installation of the buildrequires packages;
- creation of environment (user who initiates compiling, etc.)
- compiling proper;
- if succeeded the results are stored to a corresponding repository


** specific builds
There are situations, when the package should differ in a source structure,
compile options, etc. Such functionality could be acquired by indicating
required additional steps in files that form $cvsmodule. They are scripts
in the same tcl and are performed directly before compiling. The choice of
this or that script is accomplished by invocating sandcl when indicating 
`mixin' parameter using release/release{,.$mixin}. Mixin default value
is set to `generic'.

** branches
If required to perform operations with branches of several (or all) modules,
it is advisable to do the following:
- to create branches for corresponding modules in CVS;
- to create in /etc/sandman/repos description of a new repository with an
  optional indication of a branch for cvsmodule is (see modules.sample);
- to indicate names and branches of a new repository in a $cvsmodule/modules;
- to create new directory structure for components that contain package stable
	versions for all current branches (and head) and to enter all affected repositories
	into stable parameter;
- to perform all further operations with this repository with an indication of
	`repos' option

** user interface (interaction).
Tuning of a client's part means an (optionally) setting of BTEROOT environment
variable, see sandcl(1)

*  
local variables:
mode: outline
fill-column: 80
tab-width: 2
end:




