pydra.utils.profiler module

Utilities to keep track of performance and resource utilization.

class pydra.utils.profiler.ResourceMonitor(pid, interval=5, logdir=None, fname=None)[source]

Bases: Thread

A thread to monitor a specific PID with a certain frequency to a file.

property fname

Get/set the internal filename.

run()[source]

Core monitoring function, called by start().

stop()[source]

Stop monitoring.

pydra.utils.profiler.get_max_resources_used(pid, mem_mb, num_threads, pyfunc=False)[source]

Get the RAM and threads utilized by a given process.

Parameters:
  • pid (integer) – the process ID of process to profile

  • mem_mb (float) – the high memory watermark so far during process execution (in MB)

  • num_threads (int) – the high thread watermark so far during process execution

Returns:

  • mem_mb (float) – the new high memory watermark of process (MB)

  • num_threads (float) – the new high thread watermark of process

pydra.utils.profiler.get_system_total_memory_gb()[source]

Get the total RAM of the running system, in GB.

pydra.utils.profiler.log_nodes_cb(node, status)[source]

Record node run statistics to a log file as json dictionaries.

Parameters:
  • node (nipype.pipeline.engine.Node) – the node being logged

  • status (string) – acceptable values are ‘start’, ‘end’; otherwise it is considered and error

Returns:

this function does not return any values, it logs the node status info to the callback logger

Return type:

None