The goal of this project is to provide a code infrastructure using Spack pipeline feature to test on LC machines any Spack environment of interest for RADIUSS.

As a side effect, those pipelines can be used to populate a build cache. This usage will also be documented.

Structure¶

There are two main parts in this projects: - The gitlab CI configuration, consisting of YAML files. The parent file is

.gitlab-ci.yml and points to sub-parts in .gitlab using local includes.

The spack environments, each one defined in a subdirectory of spack-environments.

GitLab CI configuration¶

Variables¶

In .gitlab-ci.yml we find some global variables. Those are providing default values that can be modified in GitLab UI. Variables will propagate to all the jobs of the pipeline, but not jobs of child pipelines.

..note: We have child pipelines generated by Spack in this CI configuration. global variables defined in .gitlab-ci.yml are not automatically transmitted to the child pipelines.

Among those variables, the user may want to change the value of: * SPACK_REPO: the repo used to download Spack * SPACK_REF: the ref in Spack repo that will be checked out. * SPACK_DEBUG: a placeholder to turn on/off spack debug traces. * ENV_NAME: the name of the environment to concretize, build and test.

To modify any of these variables, it is recommended to use either CI variables in Gitlab settings UI, or Pipeline variables found in GitLab pipelines and schedules UI.

Stages¶

A pipeline is organized in stages. We start by downloading Spack, then we generate the pipelines for the selected environment, on Quartz and Lassen. Generation jobs will produce another Gitlab CI YAML file that will be interpreted as a child pipeline in the build phase. Once all the builds are complete, we do some clean-up.

Job templates¶

.on-quartz and .on-lassen are job templates. They set basic requirements shared by all the jobs that will extend them. Here any job running on Quartz or on Lassen should extend the appropriate template.

Jobs¶

Jobs are top-level YAML sections that do not have a protected meaning in Gitlab CI syntax. get-spack and rm-spack are jobs, as well as generate-on-quartz in .gitlab/generate.yml.

build-on-quartz in particular is a trigger job (it has a trigger sub-key). It takes a GitLab YAML file describing a pipeline and will trigger it as child pipeline of the current one.

Spack environments¶

In the spack-environments directory, each directory contains a Spack environment.

Environment configuration¶

The core file is spack.yaml, but we can split the environment configuration using local “includes” (not to confuse with the GitLab CI includes).

A typical use of include is to provide the Spack configuration per machine: In spack.yaml we find include: [$SYS_TYPE], where $SYS_TYPE will be evaluated before the concretization. Depending on the machine we run on, the environment will include the associated configuration, if an appropriate sub-directory exists.

On Quartz, $SYS_TYPE will evaluate to toss_3_x86_64_ib while on Lassen it will evaluate to blueos_3_ppc64le_ib_p9.

Configuration can also be provided directly in the environment:

packages::

Environment spec¶

We refer to Spack documentation on environments <https://spack.readthedocs.io/en/latest/environments.html#manually-editing-the-specs-list>``_ for the ``spec section of an environment.

CI configuration of an environment¶

The gitlab-ci section of an environment is where we can customize the behavior of the generated jobs. This has to be done coherently with the documentation provided by spack on the topic.

Process to add a new environment¶

The goal of this project is to be able to build any environment of interest for RADIUSS, independently, and in a reproducible manner. We aim in particular at building those environments against Spack develop branch to check that RADIUSS packages do not get broken by recent changes in Spack.

The very first step is to make sure your environment can be built completely isolated of any personnal Spack configuration, on LC systems. All the required configuration needs to be embedded in the environment directory.

Then you need to add the gitlab-ci section to the spack.yaml file. This should be doable by inspecting the existing radiuss environement. Create a Pull Request and ask for help if needed.

Finally, adding an environment to the CI is easy. Most of the work being to update in the UI the environment variables mentioned above.