dqrtasks.workflow.utils

Functions

`config_override`(config, overrides)
`config_write`(config, output_loc)

Classes

Job(name, executable[, output_arg, ...])

Methods

class dqrtasks.workflow.utils.Job(name, executable, output_arg=None, output_file=None, accounting_group=None, extra_lines=[], arguments=None, retry=None, **kwargs)[source]

Bases: Job

Methods

`add_arg`(arg[, name, retry])	Add argument to Job
`add_args`(args)	Adds multiple arguments to Job
`add_child`(node)	Adds node to children list
`add_children`(nodes)	Adds nodes to the children list
`add_parent`(node)	Adds node to parents list
`add_parents`(nodes)	Adds nodes to the parents list
`build`([makedirs, fancyname])	Build and saves the submit file for Job
`build_submit`([makedirs, fancyname, ...])	Calls build and submit sequentially
`haschildren`()	Checks for any children nodes
`hasparents`()	Checks for any parent nodes
`submit_job`([submit_options])	Submits Job to condor

add_arg(arg, name=None, retry=None)[source]

Add argument to Job

Parameters:

argstr: Argument to append to Job args list.
namestr or None, optional: Option to specify a name related to this argument. If a name is specified, then a separate set of log, output, and error files will be generated for this particular argument (default is None).

Added in version 0.1.2.
retryint or None, optional: Option to specify the number of times to retry this node. Default number of retries is 0. Note: this feature is only available to Jobs that are submitted via a Dagman.

Added in version 0.1.2.

Returns:

selfobject: Returns self.

add_args(args)[source]

Adds multiple arguments to Job

Parameters:

argsiterable: Iterable of arguments to append to the arguments list

Returns:

selfobject: Returns self.

add_child(node)

Adds node to children list

Parameters:

nodeBaseNode: Job or Dagman to append to the children list.

Returns:

selfobject: Returns self.

add_children(nodes)

Adds nodes to the children list

Parameters:

nodeslist or tuple: List of nodes to append to the children list

Returns:

selfobject: Returns self.

add_parent(node)

Adds node to parents list

Parameters:

nodeBaseNode: Job or Dagman to append to the parents list.

Returns:

selfobject: Returns self.

add_parents(nodes)

Adds nodes to the parents list

Parameters:

nodeslist or tuple: List of nodes to append to the parents list

Returns:

selfobject: Returns self.

build(makedirs=True, fancyname=True)[source]

Build and saves the submit file for Job

Parameters:

makedirsbool, optional: If Job directories (e.g. error, output, log, submit) don’t exist, create them (default is True).
fancynamebool, optional: Appends the date and unique id number to error, log, output, and submit files. For example, instead of jobname.submit the submit file becomes jobname_YYYYMMD_id. This is useful when running several Jobs of the same name (default is True).

Returns:

selfobject: Returns self.

build_submit(makedirs=True, fancyname=True, submit_options=None)[source]

Calls build and submit sequentially

Parameters:

makedirsbool, optional: If Job directories (e.g. error, output, log, submit) don’t exist, create them (default is True).
fancynamebool, optional: Appends the date and unique id number to error, log, output, and submit files. For example, instead of jobname.submit the submit file becomes jobname_YYYYMMD_id. This is useful when running several Jobs of the same name (default is True).
submit_optionsstr, optional: Options to be passed to condor_submit for this Job (see the condor_submit documentation for possible options).

Returns:

selfobject: Returns self.

haschildren()

Checks for any children nodes

Returns:

bool: Returns whether or not this node has any child nodes.

hasparents()

Checks for any parent nodes

Returns:

bool: Returns whether or not this node has any parent nodes.

submit_job(submit_options=None)[source]

Submits Job to condor

Parameters:

submit_optionsstr, optional: Options to be passed to condor_submit for this Job (see the condor_submit documentation for possible options).

Returns:

selfobject: Returns self.

Examples

>>> import pycondor
>>> job = pycondor.Job('myjob', 'myscript.py')
>>> job.build()
>>> job.submit_job(submit_options='-maxjobs 1000 -interactive')