Table of contents

1. Overview
2. Installation of core packages
3. Installation and Compilation of external modules
4. Running Examples
5. Running ML emulators
6. Creating Cosmolike projects (external readme)
7. Integrating external codes as Cobaya theory blocks (external readme)
8. Credits

Common questions raised by developers

9. Appendices about Conda
   1. FAQ: How can users install Conda?
   2. FAQ: How can users install the Cocoa Conda environment via conda-lock (better reproducibility)?
   3. FAQ: How can users deal with Conda conflicts (macOS): a possible solution
10. Appendices about Git
    1. FAQ: How can developers develop from a Git tag?
    2. FAQ: How can developers push changes to the Cocoa main branch?
11. Appendices about Cocoa run
    1. FAQ: How can users debug Cocoa Installation? Suggested steps
    2. FAQ: How can users compile a single external module (not involving Cosmolike)?
    3. FAQ: How can users install Cosmolike projects?
    4. FAQ: How can users run Cocoa with Docker?
    5. FAQ: How can users set the appropriate environment for ML?
12. Advanced Appendices
    1. FAQ: How can users download additional likelihood data? (external readme)
    2. FAQ: Where do users find common FAQs about external modules? (external readme)
    3. FAQ: Where do users find common FAQs about Cosmolike? (external readme)
    4. FAQ: Note on Planck-2018 low-ell SimAll EE and Gibbs TT likelihoods?
    5. FAQ: How can users deal with pip getting stuck during installation (Linux): a possible cause

Overview

Cocoa allows users to run CosmoLike routines inside the Cobaya framework. CosmoLike can analyze data from the Dark Energy Survey and simulate future multi-probe analyses for LSST and the Roman Space Telescope.

Besides integrating Cobaya and CosmoLike, Cocoa introduces shell scripts that allow users to containerize Cobaya, the Boltzmann codes, and multiple likelihoods. The container structure of Cocoa ensures that users can adopt consistent versions for the Fortran/C/C++ compilers and libraries across multiple machines. Such a systematic approach greatly simplifies the debugging process.

Our scripts never install packages or Python modules in a global folder such as $HOME/.local. Here, $HOME denotes a shell environment variable that points to the user's home folder. Doing so would force Cocoa packages to be global to the user, possibly breaking environments. Our scripts enable users to work on multiple Cocoa instances simultaneously, similar to what was possible with CosmoMC.

This README file presents basic and advanced instructions for installing all Cobaya and CosmoLike components on Linux or macOS-arm.

We provide the Docker image whovian-cocoa to facilitate the installation of Cocoa on Windows.

Note

Real-space 3x2pt timings with accuracy-tested sampling (2026-10-02). CPU: Apple M2 Pro, macOS 13.7.5, 8 OpenMP threads.

The timings account for the numerical-convergence target of $\Delta\chi^2 < 0.2$. Here $\Delta\chi^2 = (d_{\rm fine}-d_{\rm base})^T C^{-1}(d_{\rm fine}-d_{\rm base})$ compares two accuracy settings within one code.

Case	CoCoA (s)	DESC-CCL (s)	DESC-CCL / CoCoA
LSST-Y1, NLA	0.0459	10.450	228
LSST-Y1, TATT	0.0528	15.414	292
Roman-Real, NLA	0.0703	17.962	256
Roman-Real, TATT	0.0764	22.747	298

These are means of 20 CoCoA and 10 DESC-CCL evaluations, after two untimed warm-up calls, cycling through five cosmologies. CAMB is excluded from both timings: CoCoA times its likelihood; DESC-CCL times its data-vector script from the supplied CAMB tables, subtracting table-read time. Both use full-sky, bin-averaged transforms. DESC-CCL uses the study's reference sampling and PR #1296 build. The CCL-benchmark study provides the accuracy checks.

Note

Covariance forecasts (2026-10-09). CPU: Apple M2Pro, 8 OpenMP threads.

CLI for production; wrappers for exploration. Run production through the optimized _interface bindings. The notebook _wrapper APIs expose intermediate quantities for exploration but add overhead. In a matched LSST Y1 covariance, the wrapper/CLI runtime ratio was 3.21 (one wrapper run versus the CLI mean), with bitwise-identical results. The timings below use the optimized CLI interfaces.

The galaxy/shear examples include non-Limber clustering and galaxy–shear spectra in the Gaussian covariance. The joint DES cluster example remains Limber. Real-space transforms are full-sky and bin-averaged, retaining the internal cross-bin correlations needed by every covariance block.

Project	Space	Matrix before scale cuts	G + SSC + cNG construction	Runtime / LSST Y1
LSST Y1	Real	1,560 × 1,560	29.5 s	1.00
Roman real	Real	2,115 × 2,115	34.6 s	1.17
Roman Fourier	Fourier	1,485 × 1,485	14.1 s	0.48
Roman KL	Fourier	2,200 × 2,200	23.0 s	0.78
DES Y3	Real	900 × 900	25.8 s	0.87
DES × Planck (galaxy/shear)	Real	1,500 × 1,500	28.7 s	0.97
DES cluster 6×2pt + N	Real (Limber)	2,812 × 2,812	63.3 s	2.15

These are construction times from three sequential runs of each supplied evaluate YAML. The ratios compare these survey workloads on the same machine; they are not predictions for another computer. Project links provide further explanations. Detailed code comparisons are available at OneCov-benchmark and TJPCov-benchmark repositories. The real-space 3x2pt data vectors are separately compared against DESC-CCL in the CCL-benchmark study.

Installation of core packages

Core packages include compilers and numerical libraries that users typically do not modify.

Step 0️⃣ All the commands in this README assume a Bash shell. If your terminal is using zsh, switch to Bash now by typing

bash

Step 1️⃣: Choose the Cocoa version to be installed, and download the corresponding yml file

• Linux

  • Legacy Version

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v4.072/cocoapy310.yml
    

  • Stable Version

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v4.11.8/cocoapy311.yml
    

  • Testing beta release

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v5.09/cocoapy311.yml
    

• macOS (arm)

  Users on macOS may not have wget installed. If that is the case, activate your base Conda environment and install it.

  conda activate
  conda install -y wget
  

  Then, download the yml file matching the chosen version.

  • Legacy Version

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v4.072/cocoapy310-osxarm-base.yml
    

  • Stable Version

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v4.11.8/cocoapy311-osxarm-base.yml
    

  • Testing beta release

    wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v5.09/cocoapy311-osxarm-base.yml
    

Step 2️⃣: Create the Cocoa environment (if the Deprecated or Legacy version was chosen in the previous step, replace cocoapy311 with cocoapy310 below),

• Linux

  conda env create --name cocoa --file=cocoapy311.yml
  

• macOS (arm)

  conda env create --name cocoa --file=cocoapy311-osxarm-base.yml
  

and activate it

conda activate cocoa

Step 3️⃣: When and only when loading the conda cocoa environment for the first time, create the following symbolic links

• Linux

  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-gcc "${CONDA_PREFIX}"/bin/gcc
  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-g++ "${CONDA_PREFIX}"/bin/g++
  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-gfortran "${CONDA_PREFIX}"/bin/gfortran
  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-gcc-ar "${CONDA_PREFIX}"/bin/gcc-ar
  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-gcc-ranlib "${CONDA_PREFIX}"/bin/gcc-ranlib
  ln -s "${CONDA_PREFIX}"/bin/x86_64-conda-linux-gnu-ld "${CONDA_PREFIX}"/bin/ld
  

• macOS (arm)

  ln -s "${CONDA_PREFIX}"/bin/clang "${CONDA_PREFIX}"/bin/gcc
  ln -s "${CONDA_PREFIX}"/bin/clang++ "${CONDA_PREFIX}"/bin/g++
  

Step 4️⃣: When and only when loading the conda cocoa environment for the first time, install git-lfs

git-lfs install

Users can now proceed to the next section.

Tip

If conda create hangs or fails, there are two possible solutions.

1. Install mamba and redo the creation of the cocoa env, explicitly selecting mamba as the dependency solver (e.g., Linux version)

   conda activate base
   conda install -n base -c conda-forge mamba
   mamba env create --name cocoa --file=cocoapy311.yml
   conda activate cocoa
   

2. If the hang happens at the Installing pip dependencies step, see the appendix FAQ: How can users deal with pip getting stuck during installation (Linux): a possible cause.

Installation and Compilation of external modules

In this section, we assume users have previously activated the Cocoa conda environment.

Step 1️⃣: Download Cocoa's latest release and go to its main folder (cocoa/Cocoa),

• Legacy Version

  git clone https://github.com/CosmoLike/cocoa.git --branch v4.072 cocoa
  

• Stable Version

  git clone https://github.com/CosmoLike/cocoa.git --branch v4.11.8 cocoa
  

• Testing beta release

  git clone https://github.com/CosmoLike/cocoa.git --branch v5.09 cocoa
  

Warning

The version cloned here must match the version of the yml file downloaded in the section Installation of core packages.

and

cd ./cocoa/Cocoa

Step 2️⃣: Run the script setup_cocoa.sh via

source setup_cocoa.sh

This script downloads and decompresses external modules, which require internet access (on an HPC, this typically means running setup_cocoa.sh on the login node).

Cocoa does not install all the available external modules by default. If the user needs additional packages, please refer to the appendix FAQ: How can users compile external modules?.

Step 3️⃣: Run the script compile_cocoa.sh by typing

source compile_cocoa.sh

This script compiles external modules selected for installation on set_installation_options.sh (e.g., CAMB) and does not require internet access. Code compilation is CPU-intensive; therefore, running compile_cocoa.sh on a cluster login node may violate HPC policy.

Users can now proceed to the next section.

Note

When rerunning setup_cocoa.sh, Cocoa will not redownload previously installed packages. To force this behavior, use --soft, --hard, --aggressive, --extreme, or --purge flags.

• --soft: force new download of the THEORY and ML script blocks (it spares cosmolike_core and all projects)
• --hard: force new download of THEORY, ML, and LIKELIHOOD script blocks (it spares cosmolike_core and all projects)
• --aggressive: force new download of THEORY, ML, LIKELIHOOD, and CORE script blocks (it spares cosmolike_core and all projects)
• --extreme: force new download of THEORY, ML, LIKELIHOOD, CORE, and DATA script blocks (it spares cosmolike_core and all projects)
• --purge: force new download of all packages (beware of the loss of uncommitted work associated with cosmolike_core and all projects)

Note

When rerunning compile_cocoa.sh, Cocoa will not recompile previously compiled packages, except for cosmolike projects. To force this behavior, use --soft, --hard, --aggressive, --extreme flags.

• --soft: recompile THEORY, ML script blocks and all Cosmolike Projects.
• --hard: recompile THEORY, ML and LIKELIHOOD script blocks and all Cosmolike Projects.
• --aggressive: recompile THEORY, ML, LIKELIHOOD, and CORE script blocks and all Cosmolike Projects.

Running Examples

We assume that users are still in the Conda cocoa environment from the previous conda activate cocoa command, and that users switch to bash, and that you are in the cocoa main folder cocoa/Cocoa.

Step 1️⃣: Activate the private Python environment by sourcing the script start_cocoa.sh

source start_cocoa.sh

Users will see a terminal like this: $(cocoa)(.local). This is a feature, not a bug!

Note

This is a feature, not a bug! Why did we choose to work with two distinct shell environments, (cocoa) and (.local)? Our scripts enable users to work on multiple Cocoa instances. In each instance, our scripts install packages at

 Cocoa/.local/bin
 Cocoa/.local/include
 Cocoa/.local/lib
 Cocoa/.local/share

Step 2️⃣: Select the number of OpenMP cores (below, we set it to 8).

• Linux

  export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \
  export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \
  export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
  

• macOS (arm)

  export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \
  export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \
  export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
  

Examples not involving Cosmolike

Step 3️⃣: The folder projects/example contains a few examples involving different likelihoods. So, run the cobaya-run on the first example following the commands below.

Warning

(Linux only) In some HPC nodes, numa can cause you problems. If that is the case, replace numa with slot

• One model evaluation:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
      cobaya-run ./projects/example/EXAMPLE_EVALUATE1.yaml -f
    

  • macOS (arm)

    mpirun -n 1 --oversubscribe \
      cobaya-run ./projects/example/EXAMPLE_EVALUATE1.yaml -f
    

• MCMC (Metropolis-Hastings Algorithm):

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
      cobaya-run ./projects/example/EXAMPLE_MCMC1.yaml -f
    

  • macOS (arm)

    mpirun -n 4 --oversubscribe \
      cobaya-run ./projects/example/EXAMPLE_MCMC1.yaml -f
    

Examples involving Cosmolike

Cocoa provides several Cosmolike projects, not all of which are installed by default. To activate them, please refer to the appendix FAQ: How can users compile external modules?.

Step 3️⃣: The folder projects/lsst_y1 contains a dozen examples involving different combinations of two-point correlation functions. So, run the cobaya-run on the first example following the commands below.

• One model evaluation:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
      cobaya-run ./projects/lsst_y1/EXAMPLE_EVALUATE1.yaml -f
    

  • macOS (arm)

    mpirun -n 1 --oversubscribe \
      cobaya-run ./projects/lsst_y1/EXAMPLE_EVALUATE1.yaml -f
    

• MCMC (Metropolis-Hastings Algorithm):

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
      cobaya-run ./projects/lsst_y1/EXAMPLE_MCMC1.yaml -f
    

  • macOS (arm)

    mpirun -n 4 --oversubscribe \
      cobaya-run ./projects/lsst_y1/EXAMPLE_MCMC1.yaml -f
    

Running Jupyter Notebooks

Assuming Cocoa is installed on a local (not remote!) machine, type the command below after step 2️⃣ to run Jupyter Notebooks.

 jupyter notebook --no-browser --port=8888

The terminal will then show a message similar to the following template:

 (...)
 [... NotebookApp] Jupyter Notebook 6.1.1 is running at:
 [... NotebookApp] http://f0a13949f6b5:8888/?token=XXX
 [... NotebookApp] or http://127.0.0.1:8888/?token=XXX
 [... NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Now go to the local internet browser and type http://127.0.0.1:8888/?token=XXX, where XXX is the previously saved token displayed on the line

 [... NotebookApp] or http://127.0.0.1:8888/?token=XXX

The project lsst-y1 contains Jupyter notebook examples located at projects/lsst_y1.

Running ML emulators

Cocoa contains a few transformer- and CNN-based neural network emulators capable of simulating the CMB, cosmolike outputs, matter power spectrum, and distances. We provide a few scripts that exemplify their API. To run them, users ensure the following lines are commented out in set_installation_options.sh before running the setup_cocoa.sh and compile_cocoa.sh. By default, these lines should be commented out, but it is worth checking.

  [Adapted from Cocoa/set_installation_options.sh shell script] 
  # insert the # symbol (i.e., unset these environmental keys in `set_installation_options.sh`)
  #export IGNORE_EMULTRF_CODE=1              #SaraivanovZhongZhu (SZZ) transformer/CNN-based emulators
  #export IGNORE_EMULTRF_DATA=1            
  #export IGNORE_LIPOP_LIKELIHOOD_CODE=1     # to run EXAMPLE_EMUL_(EVALUATE/MCMC/NAUTILUS/EMCEE1).yaml
  #export IGNORE_LIPOP_CMB_DATA=1           
  #export IGNORE_ACTDR6_CODE=1               # to run EXAMPLE_EMUL_(EVALUATE/MCMC/NAUTILUS/EMCEE1).yaml
  #export IGNORE_ACTDR6_DATA=1         
  #export IGNORE_NAUTILUS_SAMPLER_CODE=1     # to run PROJECTS/EXAMPLE/EXAMPLE_EMUL_NAUTILUS1.py
  #export IGNORE_POLYCHORD_SAMPLER_CODE=1    # to run PROJECTS/EXAMPLE/EXAMPLE_EMUL_POLY1.yaml
  #export IGNORE_GETDIST_CODE=1              # to run EXAMPLE_TENSION_METRICS.ipynb
  #export IGNORE_TENSIOMETER_CODE=1          # to run EXAMPLE_TENSION_METRICS.ipynb

Now, users must follow all the steps below.

Step 1️⃣: Activate the private Python environment by sourcing the script start_cocoa.sh

source start_cocoa.sh

Step 2️⃣: Ensure OpenMP is OFF.

export OMP_NUM_THREADS=1; export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1

Step 3️⃣ Run cobaya-run on the first emulator example following the commands below.

• One model evaluation:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      cobaya-run ./projects/example/EXAMPLE_EMUL_EVALUATE1.yaml -f
    

  • macOS (arm)

    mpirun -n 1 --oversubscribe \
      cobaya-run ./projects/example/EXAMPLE_EMUL_EVALUATE1.yaml -f
    

• MCMC (Metropolis-Hastings Algorithm):

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      cobaya-run ./projects/example/EXAMPLE_EMUL_MCMC1.yaml -r
    

  • macOS (arm)

    mpirun -n 4 --oversubscribe \
      cobaya-run ./projects/example/EXAMPLE_EMUL_MCMC1.yaml -r
    

Note

The examples below may require a large number of MPI workers. Before running them, it may be necessary to increase the limit of threads that can be created (at UofA/SBU HPC type ulimit -u 1000000), otherwise users may encounter the error libgomp: Thread creation failed

• PolyChord:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      cobaya-run ./projects/example/EXAMPLE_EMUL_POLY1.yaml -r
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      cobaya-run ./projects/example/EXAMPLE_EMUL_POLY1.yaml -r
    

Note

Running on more than one node.

1. Network interface. TCP must use an interface routable between compute nodes. A common exclusion list is --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0; adapt it to the cluster. Keep ib0 if routable IP-over-InfiniBand is the intended network. These examples exchange parameter vectors and scalar scores, so communication volume is small; actual scaling still depends on the machine.
2. Environment. Remote ranks need the same Cocoa paths and libraries: ROOTDIR, PATH, LD_LIBRARY_PATH, PYTHONPATH, CONDA_PREFIX, OpenMP and BLAS settings, and CLIK_PATH/CLIK_DATA/CLIK_PLUGIN when used. Slurm normally exports the submitting environment (--export=ALL). Explicit -x options also forward these variables with SSH launchers. Activate Cocoa before launching; build/download flags do not replace runtime paths. All nodes must see the same files at the same paths.
3. Slurm geometry. Keep ntasks-per-node × cpus-per-task within the allocated physical cores per node. Set OMP_NUM_THREADS to SLURM_CPUS_PER_TASK and use --map-by numa:pe=${OMP_NUM_THREADS}. The minimization, profile and Nautilus pool reserves one MPI rank as coordinator; the remaining ranks evaluate the model.

The Nautilus, Minimizer, Profile, and Emcee scripts below contain an internally defined yaml_string that specifies priors, likelihoods, and the theory code, all following Cobaya Conventions.

• Nautilus:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      python -m mpi4py.futures ./projects/example/EXAMPLE_EMUL_NAUTILUS1.py \
        --root ./projects/example/ --outroot "EXAMPLE_EMUL_NAUTILUS1"  \
        --maxfeval 450000 --nlive 2048 --neff 15000 --flive 0.01 --nnetworks 5
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      python -m mpi4py.futures ./projects/example/EXAMPLE_EMUL_NAUTILUS1.py \
      --root ./projects/example/ --outroot "EXAMPLE_EMUL_NAUTILUS1"  \
      --maxfeval 450000 --nlive 2048 --neff 15000 --flive 0.01 --nnetworks 5
    

Note

What if the user runs an Nautilus chain with maxeval insufficient for producing neff samples? Nautilus creates a checkpoint at chains/outroot_checkpoint.hdf5.

• Emcee:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 21 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      python ./projects/example/EXAMPLE_EMUL_EMCEE1.py --root ./projects/example/ \
        --outroot "EXAMPLE_EMUL_EMCEE1" --maxfeval 80000
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      python ./projects/example/EXAMPLE_EMUL_EMCEE1.py \
      --root ./projects/example/ --outroot "EXAMPLE_EMUL_EMCEE1" --maxfeval 80000
    

  The number of steps per MPI worker is $n_{\mathrm{sw}} = \mathrm{maxfeval}/n_{\mathrm{w}}$, with the number of walkers being $n_{\mathrm{w}}=\mathrm{max}(3n_{\mathrm{params}},n_{\mathrm{MPI}})$. For proper convergence, each walker should traverse 50 times its autocorrelation length ($\tau$), which is provided in the header of the output chain file. A reasonable rule of thumb is to assume $\tau > 200$ and therefore set $\mathrm{maxfeval} > 10,000 \times n_{\mathrm{w}}$.

  With these numbers, users may ask when Emcee is preferable to Metropolis-Hastings? Here are a few numbers based on a Planck CMB (l < 396) + SN + BAO + LSST-Y1 chain with 38 parameters in total.

  1. MH achieves convergence with $n_{\mathrm{sw}} \sim 150,000$, but only requires four walkers.
  2. Emcee has $\tau \sim 300$, so it requires $n_{\mathrm{sw}} \sim 15,000$ when running with $n_{\mathrm{w}}=114$.

  Conclusion: Emcee requires $\sim 3$ more evaluations in this case, but the number of evaluations per MPI worker (assuming one MPI worker per walker) is reduced by $\sim 10$. Therefore, Emcee seems well-suited for cases where the evaluation of a single cosmology is time-consuming (and there is no slow/fast decomposition).

Note

What if the user runs an Emcee chain with maxeval insufficient for convergence? Emcee creates a checkpoint at chains/outroot.h5.

• Sampler Comparison

  The script that generated the plot below is provided at projects/example/scripts/EXAMPLE_PLOT_COMPARE_CHAINS.py.

  

• Global Minimizer:

  Our minimizer is a reimplementation of Procoli, developed by Karwal et al (arXiv:2401.14225)

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 21 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      python ./projects/example/EXAMPLE_EMUL_MINIMIZE1.py \
        --root ./projects/example/ \
        --outroot "EXAMPLE_EMUL_MIN1" \
        --nstw 200
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      python ./projects/example/EXAMPLE_EMUL_MINIMIZE1.py \
      --root ./projects/example/ \
      --outroot "EXAMPLE_EMUL_MIN1" \
      --nstw 200
    

  The number of steps per Emcee walker per temperature is $n_{\mathrm{stw}}$, and the number of walkers is $n_{\mathrm{w}}=\mathrm{max}(3n_{\mathrm{params}},n_{\mathrm{MPI}})$. The minimum number of total evaluations is then $3n_{\mathrm{params}} \times n_{\mathrm{T}} \times n_{\mathrm{stw}}$, which can be distributed among $n_{\mathrm{MPI}} = 3n_{\mathrm{params}}$ MPI processes for faster results. Do maintain $n_{\mathrm{stw}} > 200$ for reliable convergence in LCDM (see plot below). The same rule applies to Profile and Scan codes, as they are all based on the same minimization strategy.

  The script that generated the plot below is provided at projects/example/scripts/EXAMPLE_MIN_COMPARE_CONV.py. The Google Colab notebook Test Minimizer Convergence can also reconstruct a similar version of this figure.

  

  Below we show a case with $n_{\mathrm{param}} = 38$ that illustrates the need for performing convergence tests on a case-by-case basis. In this example, the total number of evaluations for a reliable minimum is approximately $319,200$ ($n_{\mathrm{stw}} \sim 700$), distributed among $n_{\mathrm{MPI}} = 114$ processes for faster results.

  

• Profile:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 21 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      python ./projects/example/EXAMPLE_EMUL_PROFILE1.py \
        --root ./projects/example/ \
        --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \
        --outroot "EXAMPLE_EMUL_PROFILE1" \
        --factor 3 --nstw 200 --numpts 10 --profile 1 \
        --minfile "./projects/example/chains/EXAMPLE_EMUL_MIN1.txt"
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      python ./projects/example/EXAMPLE_EMUL_PROFILE1.py \
      --root ./projects/example/ --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \
      --outroot "EXAMPLE_EMUL_PROFILE1" \
      --factor 3 --nstw 200 --numpts 10 --profile 1 \
      --minfile "./projects/example/chains/EXAMPLE_EMUL_MIN1.txt"
    

    Profile provides the optional argument minfile, as it is significantly faster to run the profile script with a previously provided global minimum. The profile also provides the optional argument cov. Again, it is considerably more efficient to employ a covariance matrix from a converged chain.

    The argument factor specifies the start and end of the parameter being profiled:

      start value ~ minimum value - factor*np.sqrt(np.diag(cov))
      end   value ~ minimum value + factor*np.sqrt(np.diag(cov))
    

    We advise $\mathrm{factor} \sim 3$ for parameters that are well constrained by the data when a covariance matrix is provided. If cov is not supplied, the code estimates one internally from the prior. If a parameter is poorly constrained or cov is not given, we recommend $\mathrm{factor} \ll 1$.

    The script that generated the plot below is provided at projects/example/scripts/EXAMPLE_PLOT_PROFILE1.py.

    

• Profile method 2:

  If the dimensionality of the problem is not large, and the spacing between values of the parameter being profiled is small, it can be considerably faster to use a simple scipy Nelder-Mead to calculate the profile. Here, the minfile and cov options are mandatory.

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 1 \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
      python ./projects/example/EXAMPLE_EMUL_PROFILE_SCIPY1.py \
        --root ./projects/example/ --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \
        --outroot "EXAMPLE_EMUL_PROFILE1M2" --factor 3 --maxfeval 5000 --numpts 10 \
        --profile 1 --minfile "./projects/example/chains/EXAMPLE_EMUL_MIN1.txt"
    

  • macOS (arm)

    mpirun -n 1 python ./projects/example/EXAMPLE_EMUL_PROFILE_SCIPY1.py \
      --root ./projects/example/ --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \
      --outroot "EXAMPLE_EMUL_PROFILE1M2" --factor 3 --maxfeval 5000 --numpts 10 \
      --profile 1 --minfile "./projects/example/chains/EXAMPLE_EMUL_MIN1.txt"
    

  The script that generated the plot below is provided at projects/example/scripts/EXAMPLE_PLOT_PROFILE1_COMP.py

  

• Scan:

  This profile code has a different MPI strategy. It scans one parameter on the entire prior, with each MPI being assigned to one minimization (not an Emcee walker!). This is a strategy when probing beyond-LCDM parameters with oscillatory behavior (e.g., Monodromic Dark Energy).

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \
      -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
      -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \
      -x CLIK_PLUGIN --mca mpi_yield_when_idle 1 \
      --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by slot \
        python -m mpi4py.futures ./projects/example/EXAMPLE_EMUL_SCAN1.py \
          --root ./projects/example/ --outroot "EXAMPLE_EMUL_SCAN1" \
          --nstw 200 --profile 1
    

  • macOS (arm)

    mpirun -n 12 --oversubscribe \
      python -m mpi4py.futures ./projects/example/EXAMPLE_EMUL_SCAN1.py \
      --root ./projects/example/ --outroot "EXAMPLE_EMUL_SCAN1" \
      --nstw 200 --profile 1
    

• Tension Metrics

  We provide the Jupyter Notebook and the SLURM script, located at

  projects/example/EXAMPLE_EMUL_MCMC_TENSION_METRICS/EXAMPLE_TENSION_METRICS.ipynb
  projects/example/scripts/EXAMPLE_EMUL_TM.sbatch
  

  that exemplify how emulators can be used to quickly assess the tension between data sets. All Tension metrics were computed using the package Tensiometer.

  The plot below, taken from EXAMPLE_TENSION_METRICS.ipynb, exemplifies the tension between CMB+BAO and Type Ia SN.

Note

What about CosmoPower? CosmoPower is a collection of popular emulators developed by Prof. Alessio Mancini and collaborators. Our Cocoa port relies on the work done by Hidde Jense @HTJense (kudos to his/their work) as well as the Simons Observatory SOLikeT code. Even though they are not the AI-powered emulators adopted by Cocoa developers, we do provide limited support for running them. To set up and compile Cosmopower and also the corresponding Cobaya Wrapper, comment the following keys before running setup_cocoa.sh and compile_cocoa.sh.

[Adapted from Cocoa/set_installation_options.sh shell script]
#export IGNORE_COSMOPOWER_DATA=1  
#export IGNORE_COSMOPOWER_CODE=1
(...)
export COSMOPOWER_URL="https://github.com/SBU-COSMOLIKE/cosmopower.git"
export COSMOPOWER_GIT_COMMIT="f70fc789426847eed996d707ec67a3a93d74bbc3"
(...)
# Edit below if you have your own set of CosmoPower Emulators
export COSMOPOWER_URL_DATA="https://github.com/cosmopower-organization/jense_2024_emulators.git"
export COSMOPOWER_URL_DATA_COMMIT="4317635eed70289ee1ec6b3df828027173071e36"

Running Hybrid Cosmolike-ML emulators

Our main line of research involves emulators that simulate the entire Cosmolike data vectors, and each project (LSST, Roman, DES) contains its own README with emulator examples. The speed of such emulators is incredible, especially when GPUs are available, and our emulators do take advantage of the CPU-GPU integration on Apple MX chips. Measure the emulator/exact runtime ratio with the same likelihood and numerical settings on the target machine.

While the data vector emulators are incredibly fast, there is an intermediate approach that emulates only the Boltzmann outputs (comoving distance, linear and nonlinear matter power spectrum). This hybrid-ML case can offer greater flexibility, especially in the initial phases of a research project, as changes to the modeling of nuisance parameters or to the assumed galaxy distributions do not require retraining of the network.

Examples in the hybrid case all have the prefix EXAMPLE_EMUL2 (note the 2). The required flags on set_installation_options.sh are similar to what we showed in the previous emulator section.

Now, users must follow all the steps below.

Step 1️⃣: Activate the private Python environment by sourcing the script start_cocoa.sh

source start_cocoa.sh

Step 2️⃣: Select the number of OpenMP cores.

• Linux

  export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \
  export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \
  export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
  

• macOS (arm)

  export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \
  export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \
  export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
  

Step 3️⃣: Remove GPU (idea is to run emulators on the CPU!)

• Linux

  export CUDA_VISIBLE_DEVICES=""
  

Step 4️⃣ Run cobaya-run on the first emulator example, following the commands below (here we only provide lsst-y1 examples).

• One model evaluation:

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
       --bind-to core:overload-allowed --report-bindings \
       --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
       cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_EVALUATE1.yaml -f
    

  • macOS (arm)

    mpirun -n 1 --oversubscribe \
      cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_EVALUATE1.yaml -f
    

• MCMC (Metropolis-Hastings Algorithm):

  • Linux

    "${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \
      --mca pml ob1 --mca btl vader,tcp,self \
      --bind-to core:overload-allowed --report-bindings \
      --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \
      cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_MCMC1.yaml -r
    

  • macOS (arm)

    mpirun -n 4 --oversubscribe \
      cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_MCMC1.yaml -r
    

Details on the matter power spectrum emulator designs will be presented in the emulator_code repository. Basically, we apply standard neural network techniques to generalize the syren-new Eq. 6 of arXiv:2410.14623 formula for the linear power spectrum (w0waCDM with a fixed neutrino mass of $0.06$ eV) to new models, extended ranges, or higher precision. Similarly, we use networks to generalize the syren-Halofit LCDM nonlinear boost fit (Eq. 11 of arXiv:2402.17492).

Note

Users can decide not to correct the syren-new formula for the linear power spectrum (flag in the YAML). Although we have not conducted extensive studies of the caveats of the syren-new approximation, it appears sufficient for w0waCDM forecasts when combined with the Euclid Emulator to compute the nonlinear boost.

For back-of-the-envelope LCDM calculations (e.g., to test cosmolike features), users can also choose not to correct the syren-Halofit formula for the LCDM nonlinear boost (see figure below). In this case, the overhead on top of cosmolike computations is minimal, at the order of $0.01$ seconds on a macOS M2Pro laptop.

Credits

• Acknowledgments:

  The entire CoCoA team is deeply grateful to everyone who has contributed to the development of our code.

  Special Thanks to:

  • Profs. Tim Eifler and Elisabeth Krause for their support of this idea since its inception in 2018 and all Cosmolike-related development.
  • Profs. Antony Lewis and Jesús Torrado for helping me understand Cobaya since its early days in 2019.
  • Jonathan Gordon, Joshua Kable, João Rebouças, Evan Saraivanov, Diogo Souza, Jiachuan Xu, Yijie Zhu, and KunHao Zhong for working on CoCoA on many fruitful projects at Stony Brook Univ. and the Univ. of Arizona.
  • Evan Saraivanov, Yijie Zhu, and KunHao Zhong for developing the emulator interface within the CoCoA framework (and training emulators).
  • Victoria Loyd for developing the hybrid-emulator interface within the CoCoA framework (and training emulators).
  • Haley Bowden, Kali Cao, Nihar Dalal, Yu-Hsiu Huang, Niko, Junzhou Zhang, and members of the Roman HLIS Cosmology PIT for all Roman-specific development and testing.

The following is not an exhaustive list of the codes we use/download/adopt

• Cobaya is a framework developed by Dr. Jesus Torrado and Prof. Anthony Lewis
• Cosmolike is a framework developed by Prof. Elisabeth Krause and Prof. Tim Eifler
• CAMB is a Boltzmann code developed by Prof. Anthony Lewis
• CLASS is a Boltzmann code developed by Prof. Julien Lesgourgues and Dr. Thomas Tram
• Polychord is a sampler code developed by Dr. Will Handley, Prof. Lasenby, and Prof. M. Hobson
• CLIK is the likelihood code used to analyze Planck and SPT data, maintained by Prof. Karim Benabed
• SPT is the official likelihood of the South Pole Telescope 3G Year 1
• MFLike is the official likelihood of the Simons Observatory
• ACTLensing is the official lensing likelihood of the ACT collaboration, developed by Prof. Mathew Madhavacheril
• HiLLiPoP CMB likelihood is a multifrequency CMB likelihood for Planck data.
• Lollipop CMB likelihood is a Planck low-l polarization likelihood.

Following best practices, Cocoa scripts download most external modules from their original repositories. Although our repository includes a few likelihoods in compressed xz file format, we do not want to discourage users from cloning code and data from their original repositories. The work of those authors is extraordinary, and users must cite them appropriately.

Appendices about Conda

⁉️ FAQ: How can users install Conda?

Step 1️⃣: Download and run the Miniforge installation script.

export CONDA_DIR="/gpfs/home/XXX/miniforge" # replace this string!

and

mkdir "${CONDA_DIR:?}"

and

curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"

and

/bin/bash Miniforge3-$(uname)-$(uname -m).sh -f -b -p "${CONDA_DIR:?}"

and

/bin/bash 

Step 2️⃣: After installation, users must source the conda configuration file, as shown below:

source $CONDA_DIR/etc/profile.d/conda.sh \
      && conda config --set auto_update_conda false \
      && conda config --set show_channel_urls true \
      && conda config --set auto_activate_base false \
      && conda config --prepend channels conda-forge \
      && conda config --add allowlist_channels conda-forge \
      && conda config --set channel_priority strict \
      && conda init bash

Step 3️⃣: After running this command, you will see a message in the terminal that ends with the statement For changes to take effect, close and re-open your current shell. Then, type

source ~/.bashrc

After that, the conda command will be available.

⁉️ FAQ: How can users install the Cocoa Conda conda env via conda-lock?

We advise users to maintain exact reproducibility (across time) of the Cocoa conda environment by installing it via conda-lock, following the slightly more convoluted instructions below.

Step 1️⃣ Install the package conda-lock in a private conda environment to avoid conflicts.

 conda create -n lockenv -c conda-forge python=3.11 conda-lock=2.* wget

and

 conda activate lockenv

Step 2️⃣ Download the appropriate conda-lock compatible yml file. As in the section Installation of core packages, replace the tag v5.09 in the URL with the Cocoa version that will be cloned (tags older than v4.11.4 carry the cocoapy310 prefix).

• Linux

  wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v5.09/cocoapy311-linux.yml
  

• macOS (arm)

  wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v5.09/cocoapy311-osxarm.yml
  

Step 3️⃣ Create the conda environment

• Linux

  conda-lock install -n cocoa cocoapy311-linux.yml
  

• macOS (arm)

  conda-lock install -n cocoa cocoapy311-osxarm.yml   
  

and activate it

 conda activate cocoa

⁉️ FAQ: How can users deal with Conda conflicts (macOS): a possible solution

During the Arizona Winter School (January 2026), we noted that some students with macOS struggled to get Conda to work (conflicts). If this is the case for you, try the steps below instead

Step 1️⃣ Create a new base environment, as your base may have legacy packages that can create incompatibilities (note: slightly modified conda command)

 conda create --solver=libmamba --strict-channel-priority --override-channels -c conda-forge --name base2

and

 conda activate base2

Step 2️⃣ Now install wget and download a looser version of the yml file (note here slightly modified conda command). As in the section Installation of core packages, replace the tag v5.09 in the URL with the Cocoa version that will be cloned (tags older than v4.11.4 carry the cocoapy310 prefix).

 conda install -y wget --solver=libmamba --strict-channel-priority --override-channels -c conda-forge

and

 wget https://raw.githubusercontent.com/CosmoLike/cocoa/refs/tags/v5.09/cocoapy311-osxarm-loose.yml

Step 3️⃣ Create the cocoa conda env using a looser yml (note: slightly modified conda command)

conda env create --solver=libmamba --name cocoa -f cocoapy311-osxarm-loose.yml

Appendices about Git

⁉️ FAQ: How can developers develop from a Git tag?

A useful Git hack concerns developing Cocoa from a Git tag.

Git tags set milestones in our development, so they serve as good starting points for coding new features or implementing bug fixes. When you check out a tag, Git places you in a detached HEAD state. So, before making further commits, you must create a local branch from that tag.

Step 1️⃣ (optional) If the repository was cloned with the https URL, switch origin to the SSH-based URL address (assuming your public key is already registered on GitHub)

git remote set-url origin git@github.com:CosmoLike/cocoa.git # that would allow users to push without typing a password

Tip

This step covers the Cocoa repository itself. For the Cosmolike/Emulator repositories, set SWITCH_TO_DEV_MODE=1 on set_installation_options.sh before setup_cocoa.sh, and they are cloned via SSH from the start (see FAQ: How can users install Cosmolike projects?).

Step 2️⃣ (optional) check the names of remote branches so you can create a unique branch name from the checked-out tag

git remote set-branches origin '*'

and

git fetch origin -v

This will display all remote branches in the terminal. The output will resemble the following snippet

[Adapted from the terminal output of git fetch origin -v] 
(...)
From https://github.com/CosmoLike/cocoa
 = [up to date]        CMB_bp        -> origin/CMB_bp
 = [up to date]        cocoa_emu     -> origin/cocoa_emu
 = [up to date]        demu          -> origin/demu
 = [up to date]        dev           -> origin/dev

Step 3️⃣ Create a new local working branch from the checked-out tag:

git switch -c xyzlocdev

Choose a unique name that does not already exist as a branch in the remote repository. A practical convention is to use your initial, followed by suffix such as locdev.

Now, all subsequent commits will be associated with the xyzlocdev local branch.

Step 4️⃣ The developer has two options at the end of development. They can either

• Create a new remote branch

  git push origin xyzlocdev # run on the xyzlocdev branch
  

• Fetch and download the already existing remote xyzdev branch, which will later absorb the changes made on xyzlocdev

  git switch -c xyzdev origin/xyzdev # run on the xyzlocdev branch
  

  Then, squash-merge the changes made on xyzlocdev.

  git merge --squash xyzlocdev # run on the xyzdev branch
  

  If this merge does not create any merge conflicts, then type

  git push origin xyzdev # run on the xyzdev branch
  

⁉️ FAQ: How can developers push changes to the Cocoa main branch?

The main branch is protected: a commit in the main branch should contain an atomic change that takes code from one working state to another working state with meaningful and well-tested improvements. Therefore, developers should propose changes to the main branch in larger chunks (via squash commits), as shown below.

Note

In a developer branch, users are encouraged to make small commits so that work can be tracked easily. Our policy regarding squash atomic changes only applies to the main branch.

• ⁉️ How to apply squash commits?

Step 1️⃣: create a development branch. Do not call the development branch dev, as dev is reserved for work done by the leading Cocoa developers. For concreteness, let's name the new branch xyzdev

git switch -c xyzdev  # run on the main branch

Tip

The use of developers' initials followed by dev helps make the branch easily identifiable.

Tip

If the branch xyzdev already exists, use the git switch command without the -c flag.

Step 2️⃣: develop the proposed changes. We advise developers to commit frequently. In your branch, a commit does not need to be atomic, i.e., changing the code from one working state to another well-tested and meaningful working state. Developers can push to the server via the command.

git push -u origin xyzdev   # run on the xyzdev branch

Step 3️⃣: Once the developers have created an atomic, meaningful, and well-tested improvement to Cocoa, the developer needs to merge any subsequent changes made in main.

git merge main   # run on the xyzdev branch

This step may create conflicts that must be addressed before step four.

Step 4️⃣: Once the developers have merged recent changes made on the main branch, they must push to the main branch the modifications made on the xyzdev branch by first squashing all your changes into a single commit, as shown below

git switch main  # run on the xyzdev branch

and

git merge --squash xyzdev   # run on the main branch

and

git commit -m "merge xyzdev branch"  # run on the main branch

and

git push origin main  # run on the main branch

Step 5️⃣: after the squash commit lands on main, retire the development branch and recreate it fresh. The squash created a brand-new commit on main, so the individual commits of xyzdev are not part of main's history: git sees the same changes twice, and the merge of Step 3️⃣ on the next development cycle would report conflicts on work that main already accepted. Deleting the branch and recreating it from the updated main avoids this without any force push.

git branch -D xyzdev   # run on the main branch

and

git switch -c xyzdev   # run on the main branch

Note

The capital -D (force) flag is required because, due to the squash merge, git cannot detect that the branch was merged, so the lowercase -d would refuse to delete it.

Warning

If the old xyzdev still exists on the server, delete it there before pushing the recreated branch — on the GitHub branches page, or with git push origin --delete xyzdev. Never force-push over a stale branch.

Appendices about Cocoa run

⁉️ FAQ: How can users debug Cocoa Installation? Suggested steps

Step 1️⃣: define the COCOA_OUTPUT_VERBOSE and COSMOLIKE_DEBUG_MODE flags on set_installation_options.sh to obtain more detailed output, as shown below (for even more output, define COCOA_OUTPUT_DEBUG as well)

  [Adapted from Cocoa/set_installation_options.sh shell script] 
  # ------------------------------------------------------------------------------
  # VERBOSE AS DEBUG TOOL --------------------------------------------------------
  # ------------------------------------------------------------------------------
  export COCOA_OUTPUT_VERBOSE=1
  #export COCOA_OUTPUT_DEBUG=1 # turn on bash strict mode (set -exo pipefail) on  
                               # instalation_scripts/setup/compile_x.sh scripts 
  # ------------------------------------------------------------------------------
  # If set, COSMOLIKE will compile with DEBUG flags ------------------------------
  # ------------------------------------------------------------------------------
  export COSMOLIKE_DEBUG_MODE=1
  (...)

Step 2️⃣: restart the Cocoa private environment by rerunning source start_cocoa.sh (every time users edit set_installation_options.sh, they must reload the (.local) environment by rerunning start_cocoa.sh).

Step 3️⃣: rerun setup_cocoa.sh and compile_cocoa.sh.

⁉️ FAQ: How can users compile a single external module (not involving Cosmolike)?

To download and compile only a single module (e.g., CAMB), users must run the associated bash scripts in Cocoa/installation_scripts/ directly (e.g., setup_camb.sh and compile_camb.sh).

Users must also ensure the appropriate keys in set_installation_options.sh are unset, as shown below.

 [Adapted from Cocoa/set_installation_options.sh shell script]
 # ------------------------------------------------------------------------------
 # The flags below allow users to skip downloading specific datasets ------------
 # ------------------------------------------------------------------------------
 #export IGNORE_ACTDR6_DATA=1                  # ACT-DR6 likelihood data
 (...)
 #export IGNORE_SIMONS_OBSERVATORY_CMB_DATA=1  # SO likelihood data
 (...)
 # ------------------------------------------------------------------------------
 # The keys below control which packages will be installed and compiled 
 # ------------------------------------------------------------------------------
 #export IGNORE_SIMONS_OBSERVATORY_LIKELIHOOD_CODE=1     # SO likelihood code
 (...)
 #export IGNORE_ACTDR6_CODE=1                            # ACT-DR6 likelihood code

Whenever users edit set_installation_options.sh, they must reload the Cocoa private environment (.local) by sourcing start_cocoa.sh (even if (.local) is already active). To do that, follow the commands below.

 cd ./cocoa/Cocoa

and

 source start_cocoa.sh # even if (.local) is already active, users must run start_cocoa.sh again to update bash environment values

Similar to setup_cocoa.sh and compile_cocoa.sh, each package contains its own setup script that downloads it from the internet and compile scripts that compile it. For example, the ACT-DR6 likelihood code can be downloaded and installed via the bash commands

 source ./installation_scripts/setup_act_dr6.sh                # download likelihood code

and

 source ./installation_scripts/compile_act_dr6.sh              # compile likelihood code

In addition to setup and compile scripts, Cocoa contains unxv scripts that download large data sets. For instance, to download the ACT-DR6 data, users must source the script

 source ./installation_scripts/unxv_act_dr6.sh                 # download and unpack likelihood data

⁉️ FAQ: How can users install Cosmolike projects?

The script set_installation_options.sh selects which Cosmolike projects are installed. A commented IGNORE_*_CODE key enables a project; an active key skips it; that file is the single source of truth for the selection.

 [Adapted from Cocoa/set_installation_options.sh shell script]
 #export IGNORE_COSMOLIKE_LSST_Y1_CODE=1
 export IGNORE_COSMOLIKE_DES_Y3_CODE=1
 #export IGNORE_COSMOLIKE_DESXPLANCK_CODE=1
 export IGNORE_COSMOLIKE_ROMAN_FOURIER_CODE=1
 #export IGNORE_COSMOLIKE_ROMAN_REAL_CODE=1
 export IGNORE_COSMOLIKE_ROMAN_KL_CODE=1
 export IGNORE_COSMOLIKE_DES_CLUSTER_CODE=1
 # WARNING: des_y6 is not production-ready.
 export IGNORE_COSMOLIKE_DES_Y6_CODE=1
 (...)
 export ROMAN_REAL_URL="https://github.com/CosmoLike/cocoa_roman_real.git"
 export ROMAN_REAL_NAME="roman_real"
 export ROMAN_REAL_GIT_TAG="v5.09"

Each released project is pinned to a tag. To select another revision, set only one of its GIT_COMMIT, GIT_BRANCH or GIT_TAG keys: a commit takes precedence over a branch, and a branch over a tag. DES Y6 is disabled by default and not production-ready; its key block pins a tag like the released projects.

Covariance generation is a separate build option, disabled for all projects by default. To enable it for Roman real, comment out export IGNORE_COSMOLIKE_ROMAN_REAL_COVARIANCE=1, reload start_cocoa.sh and recompile that project. See its covariance guide for the production CLI and notebook examples.

Note

The https URLs are the right choice for almost all users. Developers with write access to the Cosmolike/Emulator repositories can clone them via SSH keys instead by setting the key below on set_installation_options.sh.

[Adapted from Cocoa/set_installation_options.sh shell script]
# ------------------------------------------------------------------------------
# If set, clone Cosmolike/Emulator repos via SSH (developers with write access)
# ------------------------------------------------------------------------------
#export SWITCH_TO_DEV_MODE=1

Once more, anytime set_installation_options.sh is modified, we need to reload (.local) by rerunning start_cocoa.sh. Then, run the following commands:

  cd ./cocoa/Cocoa

and

  source start_cocoa.sh # even if (.local) is already active, users must run start_cocoa.sh again to update bash environment values

Now, to download and compile the enabled Cosmolike projects, type

  source ./installation_scripts/setup_cosmolike_projects.sh   # download enabled cosmolike projects

and

  source ./installation_scripts/compile_all_projects.sh       # compile enabled cosmolike projects

Note

In case users need to rerun setup_cocoa.sh (or setup_cosmolike_projects.sh) , Cocoa will not download previously installed cosmolike projects (this avoids loss of uncommitted work), unless the following key is set on set_installation_options.sh

#export OVERWRITE_EXISTING_COSMOLIKE_CODE=1 # dangerous (possible loss of uncommitted work)

Tip

A stale project folder causes compile errors for files the pinned tag contains, for example compile_des_y6.sh: No such file or directory or a DIR DOES NOT EXIST message naming a folder the tag ships. The no-overwrite protection above keeps the local clone at whatever revision was downloaded first, so moving a pin (or a project gaining new scripts) does not refresh an existing folder. Refresh the one affected project without touching the others (below, the des_y6 project), from the Cocoa main folder cocoa/Cocoa with (.local) active:

rm -rf ./projects/des_y6

and

source ./installation_scripts/setup_cosmolike_projects.sh

and

source ./projects/des_y6/scripts/compile_des_y6.sh

The OVERWRITE_EXISTING_COSMOLIKE_CODE=1 key also fixes this, but it re-downloads every enabled project folder and loses uncommitted work in all of them.

In case users only want to compile a single Cosmolike project (let's say the roman_real project)

  source ./projects/roman_real/scripts/compile_roman_real.sh

⁉️ FAQ: How can users run Cocoa with Docker?

We provide the Docker image whovian-cocoa to facilitate installing Cocoa on Windows and macOS. This appendix assumes that users have already installed the Docker Engine on their local PC. For instructions on installing the Docker Engine on specific operating systems, refer to Docker's official documentation.

Step 1️⃣: Create a folder and go to the location on the host computer where you want to provide access to the Docker container, as shown below.

 mkdir -p cocoa_docker
 cd ./cocoa_docker

Step 2️⃣: Download the Docker image whovian-cocoa, name the associated container cocoa2025 (flag --name cocoa2025 in the command below), and run the container for the first time, type:

docker run --platform linux/amd64 --hostname cocoa --name cocoa2025 -it -p 8888:8888 -v $(pwd):/home/whovian/host/ -v ~/.ssh:/home/whovian/.ssh:ro vivianmiranda/whovian-cocoa:thin

Warning

There is a weird bug on macOS that Matplotlib does not work unless you add the flags -e EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1 -e PYTHONUNBUFFERED=1 right after docker run --platform linux/amd64

This is a large image, approximately 13GB, as it already contains the conda cocoa environment. Users can now proceed to the Installation and Compilation of external modules section to continue the installation.

Tip

Once installation is complete, the user must learn how to start and exit the Docker container. Assuming the user maintained the container name cocoa2025 set on the flag --name cocoa2025, type:

 docker start -ai cocoa2025

to restart the container.

Tip

To run Jupyter Notebooks within the whovian-cocoa Docker container installed on a local machine, type the following command:

 jupyter notebook --no-browser --port=8888

The terminal will show a message similar to the following template:

 [... NotebookApp] Writing notebook server cookie secret to /home/whovian/.local/share/jupyter/runtime/notebook_cookie_secret
 [... NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
 [... NotebookApp] Serving notebooks from local directory: /home/whovian/host
 [... NotebookApp] Jupyter Notebook 6.1.1 is running at:
 [... NotebookApp] http://f0a13949f6b5:8888/?token=XXX
 [... NotebookApp] or http://127.0.0.1:8888/?token=XXX
 [... NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Now go to the local internet browser and type http://127.0.0.1:8888/?token=XXX, where XXX is the previously saved token displayed on the line

[... NotebookApp] or http://127.0.0.1:8888/?token=XXX  

Tip

To run the Jupyter Notebook on the whovian-cocoa Docker container installed on a remote server, adjust the command below

ssh your_username@your_sever.com -L 8888:localhost:8888

before typing http://127.0.0.1:8888/?token=XXX on the local desktop/laptop. This will bind the server port 8888 to the local port 8888.

[... NotebookApp] or http://127.0.0.1:8888/?token=XXX  

Tip

To delete a particular container, assuming the container name cocoa2025, type

 docker rm -f cocoa2025

Tip

The flag -v $(pwd):/home/whovian/host/ in the docker run command ensures that files in the host computer located within the folder where the Docker container was initialized are accessible inside the container.

Warning

Do not allow the Docker container to have system-wide access to your files. Accidents happen, especially when dealing with dangerous bash commands such as rm (deletion).

Note

The recipe of the whovian-cocoa image is maintained at cocoa_installation_libraries/docker/. To rebuild and publish the image (developers only), type

cd ./cocoa/cocoa_installation_libraries/docker

and

docker build . --platform linux/amd64 --build-arg COCOA_TAG=v5.09 -t vivianmiranda/whovian-cocoa # replace the tag with the Cocoa version whose yml seeds the conda env

and

docker push vivianmiranda/whovian-cocoa

To publish the :thin variant, run the container, install and compile Cocoa inside it, and type

docker commit <containerid> vivianmiranda/whovian-cocoa:thin # docker ps -a shows the container id

and

docker push vivianmiranda/whovian-cocoa:thin

⁉️ FAQ: How can users set the appropriate environment for ML?

By default, Cocoa installs several machine-learning-related libraries via pip; the keys below control this installation. To skip the GPU-based ML libraries, uncomment IGNORE_EMULATOR_GPU_PIP_PACKAGES. GPU users must also select the CUDA generation that matches their hardware.

[Adapted from Cocoa/set_installation_options.sh shell script] 
# ------------------------------------------------------------------------------
# If not set, pip_core_packages.sh will install several ML packages ------------
# ------------------------------------------------------------------------------
#export IGNORE_EMULATOR_GPU_PIP_PACKAGES=1
# New GPUs require cuda 13 / Old GPUs may still require cuda 11.8 --------------
#export ML_BLEEDING_EDGE_LIBS=1
#export ML_LEGACY_LIBS=1
(...)

If users have already run setup_cocoa.sh prior to commenting these flags, run the command.

source start_cocoa.sh # even if (.local) is already active, users must run start_cocoa.sh again to update bash environment values

and

source ./installation_scripts/setup_pip_core_packages.sh

Additional Appendices

⁉️ FAQ: Note on Planck-2018 low-ell SimAll EE and Gibbs TT likelihoods

Cocoa does not adopt Cobaya's Python reimplementation of SimAll EE and Gibbs TT likelihoods. Therefore, we patch the files.

cobaya/cobaya/likelihoods/planck_2018_lowl/TT.py
cobaya/cobaya/likelihoods/planck_2018_lowl/EE.py

to call the original Planck-2018 Clik likelihoods, as shown below.

[Adapted from Cocoa/cobaya/cobaya/likelihoods/planck_2018_lowl/TT.py]
from cobaya.likelihoods.base_classes import Planck2018Clik

#COCOA BEGINS
#class TT_clik(Planck2018Clik):
class TT(Planck2018Clik):
#COCOA ENDS
    r"""
    Low-$\ell$ temperature-only likelihood of Planck's 2018 data release
    """
    pass

and

[Adapted from Cocoa/cobaya/cobaya/likelihoods/planck_2018_lowl/EE.py]
from cobaya.likelihoods.base_classes import Planck2018Clik

#COCOA BEGINS
#class EE_clik(Planck2018Clik):
class EE(Planck2018Clik):
#COCOA ENDS
    r"""
    Low-$\ell$ EE likelihood of Planck's 2018 data release
    """

This ensures backward consistency in our code, as TT.py and EE.py used to point to Planck-2018 Clik before Cobaya's authors moved them to EE_clik.py and TT_clik.py.

⁉️ FAQ: How can users deal with pip getting stuck during installation (Linux): a possible cause

On some machines, the step Installing pip dependencies hangs (mamba shows repeated Failed to establish a new connection warnings). A possible cause: pip is configured with the retired NVIDIA index pypi.ngc.nvidia.com, planted by the nvidia-pyindex package (which Cocoa's machine-learning setup installed before v4.11.4). If that is the case, follow the steps below.

Step 1️⃣: Locate the pip configuration files

python3 -m pip config list -v

Step 2️⃣: Remove the extra-index-url and trusted-host entries that point to pypi.ngc.nvidia.com

sed -i '/extra-index-url/,+1d; /trusted-host/,+1d' ~/.pip/pip.conf ~/.config/pip/pip.conf

Step 3️⃣: Remove the pip.conf files located on older Cocoa installations (they take priority whenever that (.local) environment is active)

rm -f <old cocoa>/Cocoa/.local/pip.conf

Step 4️⃣: In case the interrupted installation already finished the conda stage, rerun only the pip stage

conda env update --name cocoa --file=cocoapy311.yml