HSF Packaging Group Meeting #20, March 7, 2018
Agenda
https://indico.cern.ch/event/708110/
Participants: Ben Morgan, Graeme Stewart, Lucio Anderlini, Patrick Gartung, Oana Boeriu, Giulio Eulisse, Pere Mato, Shazad Malik Muzaffar, Emil Obreshkov, Guilherme Amadio, Geri Ganis, Emil Obreshkov
Introduction - Graeme
- Group invited to Naples WLCG/HSF Software Development session
- All welcome, though time (AM Central Europe) not optimal for all people!
- As always, suggestions for additional talks in future welcome.
Use Cases - Graeme
- Document
at “v0.99”
- Refined to remove (but archive) useful discussion on ease of use by end clients.
- Editing to focus on Use Cases rather than Requirements, though some blurring.
- Any remaining questions to go from 0.99->1.00?
- Look through of document:
- Build section mostly unchanged, some additions on
interoperability between different via build config files
(pkg-config/CMake)
- Want to be clear on separation between responsibility of build system (developer) vs package manager (librarian)
- Comments that this is more on Software Development side.
- Install time relocatability not always required by all experiments, but keep it in to cover everyone.
- Build section mostly unchanged, some additions on
interoperability between different via build config files
(pkg-config/CMake)
Test Stack - Ben
- List of packages now in a reasonable state.
- Ben will send round a mail requesting input on “how to” on installing stack through considered package managers. Only consider install/build from scratch, no CVMFS just yet, just build from scratch on a bare metal Linux/macOS box.
Spack Status/Open Issues Discussion - All
- Starting point using Javier’s presentation, initially three
questions:
- Need a higher/more generic layer of software to configure Spack?
- Patrick: almost certainly if we don’t want spack itself to become a kitchen sink.
- From developer point of view, likely not going to run spack directly.
- Main config:
packages.yamlfile to use/reuse packages from system or LCG. Right now this is awkward. Solution could be, e.g., SpackDev may generate this automatically.
- How do you manage the Spack configuration when building full
stack?
- Patrick: Looking at Lmod to configure used packages, but issues with tracking a large number of dependencies and more than one core compiler. Also, “stack” spack on top of system by linking/softlinking system packages into area known by spack. Jim A looking at better way of doing this to track dependencies better than simple links; idea is to allow spack to read multiple spack-db files (i.e., local and pre-installed).
- Patrick: Lmod seems to be the way to go for setting up an environment portability across systems (local vs supercomputing) compared with say Scram/UPS. Question: what does lmod add vs EnvModules? Patrick: Use Lua rather than TCL, plus extra features such as “axes” for compiler/system etc. (though only three axes at the moment).
- How to distribute software using Spack?
- Patrick: Benedikt Riedel, from IceCube, was trying install/use of spack on CVMFS.
- Question: Is spack not relocatable on CVMFS? Patrick: yes, apart from a few cases where absolute paths are compiled in (e.g., TCL). O.k. if can guarantee CVMFS mount point.
- Patrick: looking at checking/editing binaries (via strings) to see/fix up packages. Giulio: In AliBuild, store all path info hash, and have only use of dynamic loader path. CMSSW: Install part, relocate part, so can handle install not equal to relocate path.
- General discussion:
- Giulio: Good to have generated files to help pick up packages from system. Patrick: Jim A has something in SpackDev to do this.
- Patrick: Will probably find more issues as we go along at FNAL.
- Need a higher/more generic layer of software to configure Spack?
AOB
Next Meeting: 21 March 2018. Look again at Nix from Chris Burr.