Example Radiation Pipeline

Example Radiation Pipeline#

Lets create a simple diagnostic pipeline where we do a few things:

  1. Open OLR, FSO, and FSR files from a CanESM run.

  2. Rename them to be CMOR compliant.

  3. Calculate monthly averages of the variables.

  4. Combine these variables into BALT

radiation.yaml#
# the `setup` section helps define I/O but does not directly compute anything.
setup:

    canesm_version: "6.0"

    # output directories define where results from each stage will go.
    output_directories:
        monthly: 'diags/monthly'

    # `stages` defines the order in which pipelines will run.
    stages:
        - transforms
        - monthly

# for this examples we'll create two stages: "transfroms" and "monthly"
# transforms are applied to variables after opening files but before any processing
transforms:
    variables:
        - OLR:
            rename: RLUT
        - FSO:
            rename: RSTD
        - FSR:
            rename: RSUT

# `monthly` is a special keyword whose `variables`` should resampled and averaged at monthly granularity.
# `daily` and `yearly` keywords are also available.
monthly:
    reuse: transforms
    # variables come from files generated by CanESM
    variables:
        - OLR
        - FSR
        - FSO
    # computed values are derived from `variables` in this stage.
        - BALT: "FSO-FSR-OLR"

Compose Pipelines#

To avoid unmanageably large yaml files, we can split our pipelines up into smaller pieces and compose them into larger pipelines using the pipelines option.

canesm_pipeline.yaml#
# the `setup` section helps define I/O but does not directly compute anything.
setup:

    output_directories:
        - daily: 'diags/daily'
        - monthly: 'diags/monthly'
        - rtd: 'diags/rtd'
        - variability: 'diags/rlr001'   # a custom pipeline to explore variability

pipelines:
    - cmip/core/core.yaml
    - rtd/core.yaml
    - custom/rlr001/variability.yaml

Running a Pipeline#

To run a pipeline you can use the command line utility:

canproc-pipeline canesm_pipeline.yaml path/to/input/data path/to/output/directory
Contents