pydra.engine.specs module

Task I/O specifications.

class pydra.engine.specs.BaseSpec[source]

Bases: object

The base dataclass specs for all inputs and outputs.

check_fields_input_spec()[source]

Check fields from input spec based on the medatada.

e.g., if xor, requires are fulfilled, if value provided when mandatory.

check_metadata()[source]

Check contained metadata.

collect_additional_outputs(inputs, output_dir, outputs)[source]

Get additional outputs.

copyfile_input(output_dir)[source]

Copy the file pointed by a File input.

property hash

Compute a basic hash for any given set of fields.

retrieve_values(wf, state_index=None)[source]

Get values contained by this spec.

template_update()[source]

Update template.

class pydra.engine.specs.ContainerSpec(*, executable: str | List[str], args: str | List[str] | None = None, image: File | str, container: File | str | None, container_xargs: List[str] | None = None)[source]

Bases: ShellSpec

Refine the generic command-line specification to container execution.

container: File | str | None

The container.

container_xargs: List[str] | None
image: File | str

The image to be containerized.

class pydra.engine.specs.Directory[source]

Bases: object

An os.pathlike object, designating a folder.

class pydra.engine.specs.DockerSpec(*, executable: str | List[str], args: str | List[str] | None = None, image: File | str, container_xargs: List[str] | None = None, container: str = 'docker')[source]

Bases: ContainerSpec

Particularize container specifications to the Docker engine.

container: str

The container.

class pydra.engine.specs.File[source]

Bases: object

An os.pathlike object, designating a file.

class pydra.engine.specs.FunctionSpec[source]

Bases: BaseSpec

Specification for a process invoked from a shell.

check_metadata()[source]

Check the metadata for fields in input_spec and fields.

Also sets the default values when available and needed.

class pydra.engine.specs.LazyField(node, attr_type)[source]

Bases: object

Lazy fields implement promises.

get_value(wf, state_index=None)[source]

Return the value of a lazy field.

class pydra.engine.specs.MultiInputFile[source]

Bases: MultiInputObj

A ty.List[File] object, converter changes a single file path to a list

class pydra.engine.specs.MultiInputObj[source]

Bases: object

A ty.List[ty.Any] object, converter changes a single values to a list

classmethod converter(value)[source]
class pydra.engine.specs.MultiOutputFile[source]

Bases: MultiOutputObj

A ty.List[File] object, converter changes an 1-el list to the single value

class pydra.engine.specs.MultiOutputObj[source]

Bases: object

A ty.List[ty.Any] object, converter changes an 1-el list to the single value

classmethod converter(value)[source]
class pydra.engine.specs.Result(*, output: Any | None = None, runtime: Runtime | None = None, errored: bool = False)[source]

Bases: object

Metadata regarding the outputs of processing.

errored: bool
get_output_field(field_name)[source]

Used in get_values in Workflow

Parameters:

field_name (str) – Name of field in LazyField object

output: Any | None
runtime: Runtime | None
class pydra.engine.specs.Runtime(*, rss_peak_gb: float | None = None, vms_peak_gb: float | None = None, cpu_peak_percent: float | None = None)[source]

Bases: object

Represent run time metadata.

cpu_peak_percent: float | None

Peak in cpu consumption.

rss_peak_gb: float | None

Peak in consumption of physical RAM.

vms_peak_gb: float | None

Peak in consumption of virtual memory.

class pydra.engine.specs.RuntimeSpec(*, outdir: str | None = None, container: str | None = 'shell', network: bool = False)[source]

Bases: object

Specification for a task.

From CWL:

InlineJavascriptRequirement
SchemaDefRequirement
DockerRequirement
SoftwareRequirement
InitialWorkDirRequirement
EnvVarRequirement
ShellCommandRequirement
ResourceRequirement

InlineScriptRequirement
container: str | None
network: bool
outdir: str | None
class pydra.engine.specs.ShellOutSpec(*, return_code: int, stdout: File | str, stderr: File | str)[source]

Bases: object

Output specification of a generic shell process.

collect_additional_outputs(inputs, output_dir, outputs)[source]

Collect additional outputs from shelltask output_spec.

generated_output_names(inputs, output_dir)[source]

Returns a list of all outputs that will be generated by the task. Takes into account the task input and the requires list for the output fields. TODO: should be in all Output specs?

return_code: int

The process’ exit code.

stderr: File | str

The process’ standard input.

stdout: File | str

The process’ standard output.

class pydra.engine.specs.ShellSpec(*, executable: str | List[str], args: str | List[str] | None = None)[source]

Bases: BaseSpec

Specification for a process invoked from a shell.

args: str | List[str] | None
check_metadata()[source]

Check the metadata for fields in input_spec and fields.

Also sets the default values when available and needed.

executable: str | List[str]
retrieve_values(wf, state_index=None)[source]

Parse output results.

class pydra.engine.specs.SingularitySpec(*, executable: str | List[str], args: str | List[str] | None = None, image: File | str, container_xargs: List[str] | None = None, container: str = 'singularity')[source]

Bases: ContainerSpec

Particularize container specifications to Singularity.

container: str

The container.

class pydra.engine.specs.SpecInfo(*, name: str, fields: List[Tuple] = _Nothing.NOTHING, bases: Tuple[Type] = _Nothing.NOTHING)[source]

Bases: object

Base data structure for metadata of specifications.

bases: Tuple[Type]

Keeps track of specification inheritance. Should be a tuple containing at least one BaseSpec

fields: List[Tuple]

List of names of fields (can be inputs or outputs).

name: str

A name for the specification.

class pydra.engine.specs.TaskHook(*, pre_run_task: ~typing.Callable = <function donothing>, post_run_task: ~typing.Callable = <function donothing>, pre_run: ~typing.Callable = <function donothing>, post_run: ~typing.Callable = <function donothing>)[source]

Bases: object

Callable task hooks.

post_run: Callable
post_run_task: Callable
pre_run: Callable
pre_run_task: Callable
reset()[source]
pydra.engine.specs.attr_fields(spec, exclude_names=())[source]
pydra.engine.specs.attr_fields_dict(spec, exclude_names=())[source]
pydra.engine.specs.donothing(*args, **kwargs)[source]
pydra.engine.specs.path_to_string(value)[source]

Convert paths to strings.