.. _example:

Example Radiation Pipeline
--------------------------

Lets create a simple diagnostic pipeline where we do a few things:

#. Open :code:`OLR`, :code:`FSO`, and :code:`FSR` files from a CanESM run.
#. Rename them to be CMOR compliant.
#. Calculate monthly averages of the variables.
#. Combine these variables into :code:`BALT`


.. code-block:: YAML
    :caption: radiation.yaml

    # the `setup` section helps define I/O but does not directly compute anything.
    setup:

        canesm_version: "6.0"

        # output directories define where results from each stage will go.
        output_directories:
            monthly: 'diags/monthly'
  
        # `stages` defines the order in which pipelines will run. 
        stages:
            - transforms
            - monthly
  
    # for this examples we'll create two stages: "transfroms" and "monthly"
    # transforms are applied to variables after opening files but before any processing
    transforms:
        variables:
            - OLR:
                rename: RLUT
            - FSO:
                rename: RSTD
            - FSR:
                rename: RSUT

    # `monthly` is a special keyword whose `variables`` should resampled and averaged at monthly granularity.
    # `daily` and `yearly` keywords are also available.
    monthly:
        reuse: transforms
        # variables come from files generated by CanESM
        variables:
            - OLR
            - FSR
            - FSO
        # computed values are derived from `variables` in this stage.
            - BALT: "FSO-FSR-OLR"


Compose Pipelines
-----------------

To avoid unmanageably large yaml files, we can split our pipelines up into
smaller pieces and compose them into larger pipelines using the :code:`pipelines` option.

.. code-block:: YAML
    :caption: canesm_pipeline.yaml

    # the `setup` section helps define I/O but does not directly compute anything.
    setup:

        output_directories:
            - daily: 'diags/daily'
            - monthly: 'diags/monthly'
            - rtd: 'diags/rtd'
            - variability: 'diags/rlr001'   # a custom pipeline to explore variability
            
    pipelines:
        - cmip/core/core.yaml
        - rtd/core.yaml
        - custom/rlr001/variability.yaml

  
Running a Pipeline
------------------

To run a pipeline you can use the command line utility:

.. code-block:: bash

    canproc-pipeline canesm_pipeline.yaml path/to/input/data path/to/output/directory