The core of the workflow engine.
- class pydra.engine.AuditFlag(value)[source]
- class pydra.engine.DockerTask(container_info=None, *args, **kwargs)[source]
Extend shell command task for containerized execution with the Docker Engine.
- property container_args
Get container-specific CLI arguments, returns a list if the task has a state
- init = False
- class pydra.engine.ShellCommandTask(container_info=None, *args, **kwargs)[source]
Wrap a shell command as a task element.
- property cmdline
Get the actual command line that will be submitted Returns a list if the task has a state.
- property command_args
Get command line arguments
- input_spec = None
- output_spec = None
- class pydra.engine.Submitter(plugin='cf', **kwargs)[source]
Send a task to the execution backend.
- async expand_runnable(runnable, wait=False, rerun=False)[source]
This coroutine handles state expansion.
Removes any states from runnable. If wait is set to False (default), aggregates all worker execution coroutines and returns them. If wait is True, waits for all coroutines to complete / error and returns None.
- async expand_workflow(wf, rerun=False)[source]
Expand and execute a stateless
Workflow. This method is only reached by Workflow._run_task.
Workflow) – Workflow Task object
wf – The computed workflow
- Return type:
- async submit_from_call(runnable, rerun)[source]
This coroutine should only be called once per Submitter call, and serves as the bridge between sync/async lands.
There are 4 potential paths based on the type of runnable: 0) Workflow has a different plugin than a submitter 1) Workflow without State 2) Task without State 3) (Workflow or Task) with State
Once Python 3.10 is the minimum, this should probably be refactored into using structural pattern matching.
- class pydra.engine.Workflow(name, audit_flags: AuditFlag = AuditFlag.NONE, cache_dir=None, cache_locations=None, input_spec: List[str] | SpecInfo | None = None, cont_dim=None, messenger_args=None, messengers=None, output_spec: SpecInfo | BaseSpec | None = None, rerun=False, propagate_rerun=True, **kwargs)[source]
A composite task with structure of computational graph.
- property checksum
Calculates the unique checksum of the task. Used to create specific directory name for task that are run; and to create nodes checksums needed for graph checksums (before the tasks have inputs etc.)
- create_connections(task, detailed=False)[source]
Add and connect a particular task to existing nodes in the workflow.
TaskBase) – The task to be added.
bool) – If True, add_edges_description is run for self.graph to add a detailed descriptions of the connections (input/output fields names)
- create_dotfile(type='simple', export=None, name=None, output_dir=None)[source]
creating a graph - dotfile and optionally exporting to other formats
- property graph_sorted
Get a sorted graph representation of the workflow.
- property nodes
Get the list of node names.
- pydra.engine.audit module
- pydra.engine.boutiques module
- pydra.engine.core module
- pydra.engine.graph module
- pydra.engine.helpers module
- pydra.engine.helpers_file module
- pydra.engine.helpers_state module
- pydra.engine.specs module
- pydra.engine.state module
- pydra.engine.submitter module
- pydra.engine.task module
- pydra.engine.workers module