Minutes of the Hera-B Software Strategy Meeting
Minutes of the Hera-B Software Strategy Meeting, Heidelberg 9-Dec-1998
Present: Hartwig Albrecht,
One of the goals is to allow quick realeases of the code, where
urgent changes and bug fixes can be implemented very fast. In
general those urgent fixes will be related to the datataking.
Non-urgent changes will refer to Monte-Carlo or offline
processing in general.
Reaching the goal of fast releases requires identifying and
disentangling dependencies between software components, a
homogenuous directory structure for all components and a
central place where everything is stored.
The following is a summary of the issues that were discussed at the
meeting, and the decisions that were taken in order to evolve the
existing software landscape into a uniform common hera-b environment.
A large part of the discussion centered on two basic concepts
that constitute the hera-b software environment: packages and
projects. Although a formal definition was lacking, the
operational definitions are roughly as follows:
package: A package is the smallest entity with its own
independent versioning. It usually is maintained
by a single developer.
Examples are marvin, ranger,...
project: A project is a collection of packages, with an
independent versioning for the collection. It is
maintained by a group of people and may have a librarian
assigned to it. Packages inside the project can depend
on each other as well as on external projects. Important,
dependencies between different projects are such, that a
well defined hierarchie exists. Circular dependencies must
Examples for projects are ARTE, ONLINE, VDSLIB,...
Hera-B Software Projects
The following list of projects was identified during the meeting.
The list is still preliminary in the sense that additional
projects may have to be defined or that some of the existing
projects have to be split.
ONLINE (including RHP)
DATAMGR (GPACK, CM, L4ALOG, EVDIR)
From the requirement that no circular dependencies are allowed
it follows, that some projects have to be split. The VDSLIB for
example should be independent of ARTE. Since it also has to work
inside ARTE, interfaces are needed. Currently those interfaces
form part of the vdslib, which makes it depend on ARTE while ARTE
also needs the vdslib. To break that loop the ARTE-interface
"silarte" will become a package inside ARTE and be removed from
the vdslib. The same scheme has to be applied to other detector
specific projects like for example CARE.
The GLOBAL-project should collect functionalities that are of
interest to all projects. Prime candidates are hbprofile and the
C/C++ Cernlib interfaces.
Directory Structure of Software Projects
Agreement was reached for the layout of the directory structure
of the hera-b software projects. After comparing the solutions
used by ARTE, ONLINE, VDSLIB and MIZZI, the following proposal
has been worked out for the organization of the code that should
be visible to the whole collaboration. No specific rules are imposed
on development areas and private schemes in general. Note that the
directory names and use of upper/lower cases used below is mandatory.
/etc/make.log, iol-files, ...
/etc/make.log, iol-files, ...
In Hamburg HBROOT=/afs/desy.de/group/hera-b. pro and dev are links
to the respective versions. The "extras" could be scripts or examples,
i.e. stuff that might be useful and/or specific for a particular
project. The source should be provided either as a set of source
code directories, as it currently is in ARTE, or simply as a tar file
of the source code that can be consulted when trying to understand
the inner workings of the code. As discussed below, it is also needed
for installation of the project on a remote platform.
One step towards achieving fast releases is the use of pro links
for all stable versions of the software. In case of bug fixes the
pro link can simply be moved to the next/corrected version and
everything rebuilt by re-issuing gmake.
The objective of "pro"-links is to point to versions that work
together correctly with pro-versions of all other projects. If
a piece of software is shown to work correctly with all other
pro-versions, it can also be linked to "pro". Similarly, dev-versions
that work together satisfactorily can be moved to "pro" together.
The whole system should be initialized ASAP and then made to work.
Further development then can follow the scheme outlined above.
All changes of links should be logged at a central place on WWW.
The scheme above is a way to ensure integrity of the whole software
by means of symbolic links, which effectively hides the actual
version numbers. The next step is to apply this scheme to the
For the more remote future one might consider an approach where
dependencies form part of the distribution of a package, so that
each package knows with which versions of other packages it can
work together. A protototype for this kind of organization is the
the RedHat-package manager, which is publicly available for a
large variety of platforms.
Some general issues pertaining to makefiles have been discussed at
the meeting. First of all, it is of course impossible to forsee
all constellations that have to be covered in practice. In general
therefore all packages will have to come with their makefiles
which build the binaries, libraries and executables within the
The compiler should be specified by a variable, so that it is trivial
to switch between different compilers in a transparent way.
All projects that a program has to link to should be defaulted to the
pro-version in the makefile. The version string then has to be
overwritten if a dev-version or a particular version number has to be
The recommended make is "gmake"
The default compiler for all hera-b software should be the egcs compiler.
The problems with incompatible binary formats between egcs-output and the
CERN libraries on the hera-b have to be solved, technically it is known
how to proceed. Of course other compilers are admitted as well, but for
compatibility reasons all code should be written such, that on every platform
for that an egcs compiler is provided, it also compiles with egcs.
(A scheme still needs to be defined that also switches compiler options
when switching compilers.)
Distinction between different Bintypes
The HBBINTYPE variable will be used to distinguish between executables for
different platforms. It is more flexible than the regular BINTYPE-variable
and completely under control of hera-b. Whereever possible the value should
be chosen identical to the value of BINTYPE.
Support for different platforms
There are many different unix-platforms in use in hera-b. In the previous
months it has become evident that the software group, i.e. librarians
and developers of the different software modules, do not have sufficient
manpower to guarantie full support for all platforms.
The following platforms are being used within the collaboration
The agreement is to always write code such that it is independent of
the specific platform it has to run on. Except for DEC-ALPHA platforms this
seems to be feasible without major effort. There is however a certain
priority for those platforms which are involved in the hera-b datataking.
For these platforms, which also are part of the DESY afs-domain, "primary"
support is provided. "Primary support" includes full installation and
debugging of the code. Other platforms get "secondary" support, which
implies that the code in principle compiles and runs on these platforms,
but that the installation has to be done by the user. Bug-fixes which are
required for those platforms have to be communicated to the authors and
implemented in the code.
The following list emerges:
Linux-PCs - Linux_intel
SGI - IRIX_mips4
AIX - AIX_risc
HP - HP-UX
SUN - Solaris Sparc
not recommended, e.g. because no reference machine available from afs and
also because of different (64-bit) processor architecture:
ALPHA - OSF1
It was emphazied that authors of software are asked to code in a way
that their code compiles/links/runs on ALL HERA-B platforms.
Despite its various well known problems, the afs-system is still the
backbone of the hera-b code installation and distribution scheme, since
there, in a transparent way, the code can be installed for the primary
platforms and then immediately is available for distribution.
A simple copy of the afs-directory tree holding the binaries for a
specific version is sufficient for installation of that project. The
distribution is required to also contain at least a tar-file with the
respective sources. Copying and unpacking that tar-file and then typing
(g)make should build the binary distribution on a remote machine. For
primary supported platforms the make is required to complete successfully.
On platforms with secondary support it is expected to work as well, but
maybe only after some bug-fixing.
In addition to the organizational items described above, also some
guidelines were formulated with the aim to improve transparency and
usability of the code:
1) Archive files should use names of the type lib*.a
2) One should aim at one header-file per archive.
In most cases this header file would simply include
the other headers needed by the archive.
3) Recursive includes must be blocked by appropriate
4) Environment variables are needed to define paths to
projects that are outside the control of hera-b, e.g.
the CERNlibs, CLHEP, MIZZI etc. The hera-b specific projects
are accessed by the respective makefiles by exploiting the
uniform naming scheme after the global path variable HBROOT.
Versions are defaulted to pro, but can be overwritten by