Pipelines

Pipelines#

Pipelines#

class canproc.pipelines.Pipeline(config: str | Path, input_dir: str | Path, output_dir: str | Path | None = None)#

Convert a YAML configuration file to a DAG pipeline

Example

>>> from canproc.pipelines import Pipeline
>>> pipeline = Pipeline('config.yaml', '/space/hall6/...', '/space/hall6/...')
>>> dag = pipeline.render()

Variables#

class canproc.pipelines.variable.Variable(name: str, loader: FileLoader | None = None, writer: FileWriter | None = None, metadata: dict = {}, **kwargs)#

Class to keep track of operations that have been applied to dataset.

Example

>>> variable = Variable('temperature')
>>> variable.open('test/input_directory', engine='netcdf4')
>>> variable.scale(10.0)
>>> variable.write('test/output_directory')
>>> dag = variable.render()
Contents