Skip to content

Provide generic way to propagate env vars into startprefix environment#260

Merged
Neves-P merged 2 commits into
EESSI:mainfrom
casparvl:create_mechanism_for_propagating_env_vars_into_prefix
Jun 24, 2026
Merged

Provide generic way to propagate env vars into startprefix environment#260
Neves-P merged 2 commits into
EESSI:mainfrom
casparvl:create_mechanism_for_propagating_env_vars_into_prefix

Conversation

@casparvl

@casparvl casparvl commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

This PR creates a generic way with which to prefix env vars so that they are propagated into the startprefix environment: you simply set EESSI_PROPAGATE_INTO_PREFIX_X=val to get a variable X=val defined within your startprefix environment.

This is useful for workarounds like the one that would be needed here, where we would want to set export UCX_IB_GPU_DIRECT_RDMA=n in the UGent bot's site configuration script. The current issue is that this doesn't get propagated unless it is explicitely in the list of whitelisted variables in run_in_compat_layer_env.sh. However, it does not make any sense to put this variable in that list, since that propagation is only needed for the UGent bot. Moreover, it means that if you ever want to propagate a variable, you always have to change the run_in_compat_layer_env.sh script first.

Having a long and specific enough prefix to the name of the env var allows us to specify such environment variables in the environment outside of startprefix, yet make the run_in_compat_layer_env.sh automatically export these inside the prefix.

…ropagated into the startprefix environment
@casparvl

Copy link
Copy Markdown
Contributor Author

This PR is easily tested. As we can see, a regular env var doesn't make it into the startprefix environment:

[casparl@tcn36 software-layer-scripts]$ FOO=BAR EESSI_VERSION_OVERRIDE=2025.06 ./run_in_compat_layer_env.sh "env" | grep FOO
[casparl@tcn36 software-layer-scripts]$

But, if we prefix it with PROPAGATE_INTO_PREFIX, it gets exported in the startprefix environment:

[casparl@tcn36 software-layer-scripts]$ PROPAGATE_INTO_PREFIX_FOO=BAR EESSI_VERSION_OVERRIDE=2025.06 ./run_in_compat_layer_env.sh "env" | grep FOO
Running 'export FOO=BAR; export EESSI_INIT_PREFIX=/cvmfs/software.eessi.io/versions/2025.06/init; export EESSI_COMPAT_LAYER_DIR=/cvmfs/software.eessi.io/versions/2025.06/compat/linux/x86_64; export EESSI_VERSION_OVERRIDE=2025.06; export EESSI_CVMFS_REPO_OVERRIDE=/cvmfs/software.eessi.i
o; export SLURM_JOB_ID=24143710; env' in EESSI (/cvmfs/software.eessi.io) 2025.06 compatibility layer environment...
FOO=BAR

@casparvl

Copy link
Copy Markdown
Contributor Author

Ok, made it even more specific:

[casparl@tcn36 software-layer-scripts]$ EESSI_PROPAGATE_INTO_PREFIX_FOO=BAR EESSI_VERSION_OVERRIDE=2025.06 ./run_in_compat_layer_env.sh "env" | grep FOO
Running 'export FOO=BAR; export EESSI_INIT_PREFIX=/cvmfs/software.eessi.io/versions/2025.06/init; export EESSI_COMPAT_LAYER_DIR=/cvmfs/software.eessi.io/versions/2025.06/compat/linux/x86_64; export EESSI_VERSION_OVERRIDE=2025.06; export EESSI_CVMFS_REPO_OVERRIDE=/cvmfs/software.eessi.io; export SLURM_JOB_ID=24143710; env' in EESSI (/cvmfs/software.eessi.io) 2025.06 compatibility layer environment...
FOO=BAR

@Neves-P Neves-P left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, and works as intended when tested

@Neves-P Neves-P merged commit 4309853 into EESSI:main Jun 24, 2026
92 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants