Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Building All Domains with CstarSpecEngine

This notebook demonstrates how to use CstarSpecEngine to build all domain configurations from domains.yml in a single workflow.

CstarSpecEngine Overview

CstarSpecEngine is the high-level interface for managing ROMS model configurations and builds. It provides methods to:

  • Generate single domains: Build one domain at a time using generate_domain()

  • Generate multiple domains: Build all domains from domains.yml using generate_all()

  • Run simulations: Execute model runs using run_all()

The engine reads domain configurations from domains.yml and model specifications from models.yml, orchestrating the complete workflow from input generation through model compilation.

Batch Processing

The generate_all() method processes all domains defined in domains.yml sequentially. This is useful for:

  • Batch workflows: Generating multiple domain configurations in one operation

  • Consistent settings: Applying the same parameters (clobber flags, settings overrides) to all domains

  • Error handling: The method continues processing remaining domains even if one fails, reporting all errors at the end

Each domain goes through the complete workflow: PRECONFIG → Source Data → POSTCONFIG → BUILD → Pre-run.

Workflow Stages

For each domain, generate_all() executes the complete workflow:

  1. PRECONFIG: Initialize blueprint and grid object

  2. Source Data: Download and prepare required datasets (GLORYS, UNIFIED, SRTM15, etc.)

  3. POSTCONFIG: Generate all input files (grid, initial conditions, forcing)

  4. BUILD: Render configuration templates and compile the model executable

  5. Pre-run: Prepare for execution (partitioning, etc.)

The method returns a dictionary mapping domain names to CstarSpecBuilder instances, stored in engine.builder.

Setup

Enable autoreload for development and import the package.

%load_ext autoreload
%autoreload 2

Generate All Domains

Create a CstarSpecEngine instance and generate all domains from domains.yml.

Parameters:

  • clobber_inputs: If True, overwrite existing input files for all domains

  • clobber_source_data: If True, re-download source datasets

  • partition_files: If True, partition input files across tiles

  • test: If True, truncate generation loop after 2 iterations (for testing)

  • compile_time_settings: Optional dict of compile-time settings overrides

  • run_time_settings: Optional dict of run-time settings overrides

  • overrides: Optional dict of configuration overrides to apply to all domains

The method returns a dictionary mapping domain names to CstarSpecBuilder instances.

import cson_forge

Run All Simulations

Execute model simulations for all generated domains. The run_all() method handles execution for all builders and returns execution handlers for monitoring the runs.

Note: This cell is commented out by default. Uncomment to run all simulations.

engine = cson_forge.CstarSpecEngine(domains_file="domains.yml")
builders = engine.generate_all(clobber_inputs=True)

================================================================================
Starting generation for 4 domain(s)
================================================================================


--------------------------------------------------------------------------------
[1/4] Processing domain: ccs-12km
--------------------------------------------------------------------------------
✔️  Using existing GLORYS_REGIONAL file for 2024-01-01: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_ccs-12km_20240101.nc
✔️  Using existing GLORYS_REGIONAL file for 2024-01-02: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_ccs-12km_20240102.nc
✔️  TPXO dataset verified at: /Users/mclong/cson-forge-data/source-data/TPXO/TPXO10.v2
✔️  Using existing BGC dataset: /Users/mclong/cson-forge-data/source-data/UNIFIED_BGC/BGCdataset.nc

▶️  [1/8] Writing ROMS grid...

▶️  [2/8] Generating initial conditions...

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']

[########################################] | 100% Completed | 51.06 s

▶️  [3/8] Generating surface forcing...

Numba: Attempted to fork from a non-main thread, the TBB library may be in an invalid state in the child process.

[########################################] | 100% Completed | 8.85 sms

▶️  [4/8] Generating surface forcing...

[WARNING] Optional variables missing (but not critical): ['pco2_air_alt']

[########################################] | 100% Completed | 211.02 ms

▶️  [5/8] Generating boundary forcing...

[WARNING] The northern boundary is divided by land. It would be safer (but slower and more memory-intensive) to use `apply_2d_horizontal_fill = True`.

[########################################] | 100% Completed | 1.24 sms

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']


▶️  [6/8] Generating boundary forcing...
[########################################] | 100% Completed | 12.56 s

▶️  [7/8] Generating tidal forcing...
[########################################] | 100% Completed | 3.28 sms

▶️  [8/8] Generating river forcing...

[WARNING] No records found at or after the end_time: 2024-01-02 00:00:00.


✅ All input files generated.

[INFO] 🛠️ Configuring ROMSSimulation
[INFO] 🔧 Setting up ROMSExternalCodeBase...
[INFO] 🔧 Setting up MARBLExternalCodeBase...
[INFO] 📦 Fetching compile-time code...
[INFO] 📦 Fetching runtime code... 
[INFO] 📦 Fetching input datasets...
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_grid.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_initial_conditions.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_tidal.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_river.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_boundary-physics_202401.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_boundary-bgc_clim.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_surface-physics_202401.nc into (16,20)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_ccs-12km_320procs_surface-bgc_clim.nc into (16,20)

✓ Successfully completed domain: ccs-12km

--------------------------------------------------------------------------------
[2/4] Processing domain: gulf-guinea-toy
--------------------------------------------------------------------------------
✔️  Using existing GLORYS_REGIONAL file for 2012-01-01: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_gulf-guinea-toy_20120101.nc
✔️  Using existing GLORYS_REGIONAL file for 2012-01-02: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_gulf-guinea-toy_20120102.nc
✔️  TPXO dataset verified at: /Users/mclong/cson-forge-data/source-data/TPXO/TPXO10.v2
✔️  Using existing BGC dataset: /Users/mclong/cson-forge-data/source-data/UNIFIED_BGC/BGCdataset.nc

▶️  [1/8] Writing ROMS grid...

▶️  [2/8] Generating initial conditions...

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']

[########################################] | 100% Completed | 9.14 sms

▶️  [3/8] Generating surface forcing...
[########################################] | 100% Completed | 7.65 ss

▶️  [4/8] Generating surface forcing...

[WARNING] Optional variables missing (but not critical): ['pco2_air_alt']

[########################################] | 100% Completed | 103.71 ms

▶️  [5/8] Generating boundary forcing...
[########################################] | 100% Completed | 430.38 ms

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']


▶️  [6/8] Generating boundary forcing...
[########################################] | 100% Completed | 9.69 ss

▶️  [7/8] Generating tidal forcing...
[########################################] | 100% Completed | 2.00 sms

▶️  [8/8] Generating river forcing...

[WARNING] NaNs detected in 'river_volume' and set to zero. This may indicate missing river data and affect model accuracy. 


✅ All input files generated.

[INFO] 🛠️ Configuring ROMSSimulation
[INFO] 🔧 Setting up ROMSExternalCodeBase...
[INFO] 🔧 Setting up MARBLExternalCodeBase...
[INFO] 📦 Fetching compile-time code...
[INFO] 📦 Fetching runtime code... 
[INFO] 📦 Fetching input datasets...
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_gulf-guinea-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_gulf-guinea-toy_10procs_grid.nc into (2,5)

✗ Error processing domain gulf-guinea-toy: Dimension 'xi_coarse' of size 5 cannot be evenly divided into 2 partitions.

--------------------------------------------------------------------------------
[3/4] Processing domain: hvalfjörður-0
--------------------------------------------------------------------------------

/var/folders/x8/7n8hknbj717fxnf07pnk3pch0000gn/T/ipykernel_35535/3018024646.py:2: UserWarning: Error processing domain gulf-guinea-toy: Dimension 'xi_coarse' of size 5 cannot be evenly divided into 2 partitions.
  builders = engine.generate_all(clobber_inputs=True)

✔️  Using existing GLORYS_REGIONAL file for 2024-01-01: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_hvalfjörður-0_20240101.nc
✔️  Using existing GLORYS_REGIONAL file for 2024-01-02: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_hvalfjörður-0_20240102.nc
✔️  TPXO dataset verified at: /Users/mclong/cson-forge-data/source-data/TPXO/TPXO10.v2
✔️  Using existing BGC dataset: /Users/mclong/cson-forge-data/source-data/UNIFIED_BGC/BGCdataset.nc

▶️  [1/8] Writing ROMS grid...

▶️  [2/8] Generating initial conditions...

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']

[########################################] | 100% Completed | 131.24 s

▶️  [3/8] Generating surface forcing...
[########################################] | 100% Completed | 10.71 ss

[WARNING] Optional variables missing (but not critical): ['pco2_air_alt']


▶️  [4/8] Generating surface forcing...
[########################################] | 100% Completed | 209.59 ms

▶️  [5/8] Generating boundary forcing...
[########################################] | 100% Completed | 537.84 ms

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']


▶️  [6/8] Generating boundary forcing...
[########################################] | 100% Completed | 16.39 s

▶️  [7/8] Generating tidal forcing...
[########################################] | 100% Completed | 1.91 sms

▶️  [8/8] Generating river forcing...

[WARNING] No records found at or after the end_time: 2024-01-02 00:00:00.


✅ All input files generated.

[INFO] 🛠️ Configuring ROMSSimulation
[INFO] 🔧 Setting up ROMSExternalCodeBase...
[INFO] 🔧 Setting up MARBLExternalCodeBase...
[INFO] 📦 Fetching compile-time code...
[INFO] 📦 Fetching runtime code... 
[INFO] 📦 Fetching input datasets...
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_grid.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_initial_conditions.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_tidal.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_river.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_boundary-physics_202401.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_boundary-bgc_clim.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_surface-physics_202401.nc into (16,16)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_20240101-20240102/input_datasets/cson_roms-marbl_v0.1_hvalfjörður-0_256procs_surface-bgc_clim.nc into (16,16)

✓ Successfully completed domain: hvalfjörður-0

--------------------------------------------------------------------------------
[4/4] Processing domain: wio-toy
--------------------------------------------------------------------------------
✔️  Using existing GLORYS_REGIONAL file for 2012-01-01: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_wio-toy_20120101.nc
✔️  Using existing GLORYS_REGIONAL file for 2012-01-02: cmems_mod_glo_phy_my_0.083deg_P1D-m_REGIONAL_wio-toy_20120102.nc
✔️  TPXO dataset verified at: /Users/mclong/cson-forge-data/source-data/TPXO/TPXO10.v2
✔️  Using existing BGC dataset: /Users/mclong/cson-forge-data/source-data/UNIFIED_BGC/BGCdataset.nc

▶️  [1/8] Writing ROMS grid...

▶️  [2/8] Generating initial conditions...

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']

[########################################] | 100% Completed | 28.45 ss

▶️  [3/8] Generating surface forcing...
[########################################] | 100% Completed | 8.51 sms

[WARNING] Optional variables missing (but not critical): ['pco2_air_alt']


▶️  [4/8] Generating surface forcing...
[########################################] | 100% Completed | 106.69 ms

▶️  [5/8] Generating boundary forcing...
[########################################] | 100% Completed | 2.28 sms

[WARNING] Optional variables missing (but not critical): ['Lig', 'DIC_ALT_CO2', 'Alk_ALT_CO2']


▶️  [6/8] Generating boundary forcing...
[########################################] | 100% Completed | 9.96 ss

▶️  [7/8] Generating tidal forcing...
[########################################] | 100% Completed | 4.49 sms

▶️  [8/8] Generating river forcing...

[WARNING] NaNs detected in 'river_volume' and set to zero. This may indicate missing river data and affect model accuracy. 


✅ All input files generated.

[INFO] 🛠️ Configuring ROMSSimulation
[INFO] 🔧 Setting up ROMSExternalCodeBase...
[INFO] 🔧 Setting up MARBLExternalCodeBase...
[INFO] 📦 Fetching compile-time code...
[INFO] 📦 Fetching runtime code... 
[INFO] 📦 Fetching input datasets...
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_grid.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_initial_conditions.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_tidal.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_river.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_boundary-physics_201201.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_boundary-bgc_clim.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_surface-physics_201201.nc into (2,5)
[INFO] Partitioning /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_wio-toy_10procs_20120101-20120102/input_datasets/cson_roms-marbl_v0.1_wio-toy_10procs_surface-bgc_clim.nc into (2,5)

✓ Successfully completed domain: wio-toy

================================================================================
Completed generation for all 4 domain(s)
⚠ 1 domain(s) failed:
  - gulf-guinea-toy: Dimension 'xi_coarse' of size 5 cannot be evenly divided into 2 partitions.
================================================================================


/var/folders/x8/7n8hknbj717fxnf07pnk3pch0000gn/T/ipykernel_35535/3018024646.py:2: UserWarning: 1 domain(s) failed during generation: gulf-guinea-toy
  builders = engine.generate_all(clobber_inputs=True)

exec_handlers = engine.run_all()    
for key, exec_handler in exec_handlers.items():
    print("-" * 100)
    print(key)
    print(exec_handler)
    print()


================================================================================
Starting execution for 3 domain(s)
================================================================================


--------------------------------------------------------------------------------
[1/3] Running simulation: ccs-12km
--------------------------------------------------------------------------------
[INFO] Running mpirun -n 320 /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/compile_time_code/roms /Users/mclong/cson-forge-data/cson-forge-run/cson_roms-marbl_v0.1_ccs-12km_320procs_20240101-20240102/runtime_code/cson_roms-marbl_v0.1_ccs-12km_320procs.in
Monitoring execution status for ccs-12km...
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running
  Status: running

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[4], line 1
----> 1 exec_handlers = engine.run_all()    
      2 for key, exec_handler in exec_handlers.items():
      3     print("-" * 100)

File ~/codes/cson-forge/cson_forge/_core.py:2647, in CstarSpecEngine.run_all(self, poll_interval)
   2644             break
   2646         # Wait before next status check
-> 2647         time.sleep(poll_interval)
   2649     execution_results[grid_name] = execution_handler
   2651 except Exception as e:

KeyboardInterrupt: 
for key, exec_handler in exec_handlers.items():
    while not handler.ExecutionStatus.is_terminal(exec_handler.status):
        print("...", end="", flush=True)
        time.sleep(30)

    if exec_handler.status == handler.ExecutionStatus.COMPLETED:
        ocn.post_run()
    else:
        raise Exception("Model run failed")
CStarSpecEngine
Building a Single Domain with CstarSpecEngine
Catalog
Blueprint Catalog API