pydra.engine.specs module

Task I/O specifications.

class pydra.engine.specs.BaseSpec[source]

Bases: object

The base dataclass specs for all inputs and outputs.

check_fields_input_spec()[source]

Check fields from input spec based on the medatada.

e.g., if xor, requires are fulfilled, if value provided when mandatory.

check_metadata()[source]

Check contained metadata.

collect_additional_outputs(inputs, output_dir, outputs)[source]

Get additional outputs.

copyfile_input(output_dir)[source]

Copy the file pointed by a File input.

property hash

Compute a basic hash for any given set of fields.

retrieve_values(wf, state_index=None)[source]

Get values contained by this spec.

template_update()[source]

Update template.

class pydra.engine.specs.ContainerSpec(*, executable: Union[str, List[str]], args: Optional[Union[str, List[str]]] = None, image: Union[File, str], container: Optional[Union[File, str]], container_xargs: Optional[List[str]] = None)[source]

Bases: ShellSpec

Refine the generic command-line specification to container execution.

container: Optional[Union[File, str]]

The container.

container_xargs: Optional[List[str]]
image: Union[File, str]

The image to be containerized.

class pydra.engine.specs.Directory[source]

Bases: object

An os.pathlike object, designating a folder.

class pydra.engine.specs.DockerSpec(*, executable: Union[str, List[str]], args: Optional[Union[str, List[str]]] = None, image: Union[File, str], container_xargs: Optional[List[str]] = None, container: str = 'docker')[source]

Bases: ContainerSpec

Particularize container specifications to the Docker engine.

container: str

The container.

class pydra.engine.specs.File[source]

Bases: object

An os.pathlike object, designating a file.

class pydra.engine.specs.FunctionSpec[source]

Bases: BaseSpec

Specification for a process invoked from a shell.

check_metadata()[source]

Check the metadata for fields in input_spec and fields.

Also sets the default values when available and needed.

class pydra.engine.specs.LazyField(node, attr_type)[source]

Bases: object

Lazy fields implement promises.

get_value(wf, state_index=None)[source]

Return the value of a lazy field.

class pydra.engine.specs.MultiInputFile[source]

Bases: MultiInputObj

A ty.List[File] object, converter changes a single file path to a list

class pydra.engine.specs.MultiInputObj[source]

Bases: object

A ty.List[ty.Any] object, converter changes a single values to a list

classmethod converter(value)[source]
class pydra.engine.specs.MultiOutputFile[source]

Bases: MultiOutputObj

A ty.List[File] object, converter changes an 1-el list to the single value

class pydra.engine.specs.MultiOutputObj[source]

Bases: object

A ty.List[ty.Any] object, converter changes an 1-el list to the single value

classmethod converter(value)[source]
class pydra.engine.specs.Result(*, output: Optional[Any] = None, runtime: Optional[Runtime] = None, errored: bool = False)[source]

Bases: object

Metadata regarding the outputs of processing.

errored: bool
get_output_field(field_name)[source]

Used in get_values in Workflow

Parameters

field_name (str) – Name of field in LazyField object

output: Optional[Any]
runtime: Optional[Runtime]
class pydra.engine.specs.Runtime(*, rss_peak_gb: Optional[float] = None, vms_peak_gb: Optional[float] = None, cpu_peak_percent: Optional[float] = None)[source]

Bases: object

Represent run time metadata.

cpu_peak_percent: Optional[float]

Peak in cpu consumption.

rss_peak_gb: Optional[float]

Peak in consumption of physical RAM.

vms_peak_gb: Optional[float]

Peak in consumption of virtual memory.

class pydra.engine.specs.RuntimeSpec(*, outdir: Optional[str] = None, container: Optional[str] = 'shell', network: bool = False)[source]

Bases: object

Specification for a task.

From CWL:

InlineJavascriptRequirement
SchemaDefRequirement
DockerRequirement
SoftwareRequirement
InitialWorkDirRequirement
EnvVarRequirement
ShellCommandRequirement
ResourceRequirement

InlineScriptRequirement
container: Optional[str]
network: bool
outdir: Optional[str]
class pydra.engine.specs.ShellOutSpec(*, return_code: int, stdout: Union[File, str], stderr: Union[File, str])[source]

Bases: object

Output specification of a generic shell process.

collect_additional_outputs(inputs, output_dir, outputs)[source]

Collect additional outputs from shelltask output_spec.

generated_output_names(inputs, output_dir)[source]

Returns a list of all outputs that will be generated by the task. Takes into account the task input and the requires list for the output fields. TODO: should be in all Output specs?

return_code: int

The process’ exit code.

stderr: Union[File, str]

The process’ standard input.

stdout: Union[File, str]

The process’ standard output.

class pydra.engine.specs.ShellSpec(*, executable: Union[str, List[str]], args: Optional[Union[str, List[str]]] = None)[source]

Bases: BaseSpec

Specification for a process invoked from a shell.

args: Optional[Union[str, List[str]]]
check_metadata()[source]

Check the metadata for fields in input_spec and fields.

Also sets the default values when available and needed.

executable: Union[str, List[str]]
retrieve_values(wf, state_index=None)[source]

Parse output results.

class pydra.engine.specs.SingularitySpec(*, executable: Union[str, List[str]], args: Optional[Union[str, List[str]]] = None, image: Union[File, str], container_xargs: Optional[List[str]] = None, container: str = 'singularity')[source]

Bases: ContainerSpec

Particularize container specifications to Singularity.

container: str

The container.

class pydra.engine.specs.SpecInfo(*, name: str, fields: List[Tuple] = NOTHING, bases: Tuple[Type] = NOTHING)[source]

Bases: object

Base data structure for metadata of specifications.

bases: Tuple[Type]

Keeps track of specification inheritance. Should be a tuple containing at least one BaseSpec

fields: List[Tuple]

List of names of fields (can be inputs or outputs).

name: str

A name for the specification.

class pydra.engine.specs.TaskHook(*, pre_run_task: ~typing.Callable = <function donothing>, post_run_task: ~typing.Callable = <function donothing>, pre_run: ~typing.Callable = <function donothing>, post_run: ~typing.Callable = <function donothing>)[source]

Bases: object

Callable task hooks.

post_run: Callable
post_run_task: Callable
pre_run: Callable
pre_run_task: Callable
reset()[source]
pydra.engine.specs.attr_fields(spec, exclude_names=())[source]
pydra.engine.specs.attr_fields_dict(spec, exclude_names=())[source]
pydra.engine.specs.donothing(*args, **kwargs)[source]
pydra.engine.specs.path_to_string(value)[source]

Convert paths to strings.