Our team uses Caper/WDL on a SLURM backend with Singularity. We've been encountering a "container collision" issue i.e. when Singularity unpacks the image, there is a chance that it unpacks the image for two or more parallel tasks to the same location. If we can set the $SINGULARITY_TMPDIR environment variable, we should be able to ensure that Singularity always unpacks the image in a location unique to the job (rather than unique to the task name).
Right now, we can set Caper to run in dry-mode, edit the backend configuration to include setting the $SINGULARITY_TMPDIR, then run Caper on the custom backend, but it would be much simpler to be able to pass the $SINGULARITY_TMPDIR value to Caper itself.
Our team uses Caper/WDL on a SLURM backend with Singularity. We've been encountering a "container collision" issue i.e. when Singularity unpacks the image, there is a chance that it unpacks the image for two or more parallel tasks to the same location. If we can set the
$SINGULARITY_TMPDIRenvironment variable, we should be able to ensure that Singularity always unpacks the image in a location unique to the job (rather than unique to the task name).Right now, we can set Caper to run in dry-mode, edit the backend configuration to include setting the
$SINGULARITY_TMPDIR, then run Caper on the custom backend, but it would be much simpler to be able to pass the$SINGULARITY_TMPDIRvalue to Caper itself.