cwl_utils.cwl_v1_2_expression_refactor

CWL Expression refactoring tool for CWL v1.2 .

Module Contents

Functions

expand_stream_shortcuts(process)

Rewrite the "type: stdout" shortcut to use an explicit random filename.

escape_expression_field(contents)

Escape sequences similar to CWL expressions or param references.

clean_type_ids(cwltype)

Simplify type identifiers.

get_expression(string, inputs, self)

Find and return a normalized CWL expression, if any.

etool_to_cltool(etool[, expressionLib])

Convert a ExpressionTool to a CommandLineTool.

traverse(process, replace_etool, inside, ...)

Convert the given process and any subprocesses.

load_step(step, replace_etool, skip_command_line1, ...)

If the step's Process is not inline, load and process it.

generate_etool_from_expr(expr, target[, no_inputs, ...])

Convert a CWL Expression into an ExpressionTool.

get_input_for_id(name, tool)

Determine the CommandInputParameter for the given input name.

find_expressionLib(processes)

Return the expressionLib from the highest priority InlineJavascriptRequirement.

replace_expr_with_etool(expr, name, workflow, target, ...)

Modify the given workflow, replacing the expr with an standalone ExpressionTool.

replace_wf_input_ref_with_step_output(workflow, name, ...)

Refactor all reference to a workflow input to the specified step output.

empty_inputs(process_or_step[, parent])

Produce a mock input object for the given inputs.

example_input(some_type)

Produce a fake input for the given type.

process_workflow_inputs_and_outputs(workflow, ...)

Do any needed conversions on the given Workflow's inputs and outputs.

process_workflow_reqs_and_hints(workflow, replace_etool)

Convert any expressions in a workflow's reqs and hints.

process_level_reqs(process, step, parent, ...)

Convert expressions inside a process into new adjacent steps.

add_input_to_process(process, name, inptype, ...)

Add a new InputParameter to the given CommandLineTool.

traverse_CommandLineTool(clt, parent, step, ...)

Extract any CWL Expressions within the given CommandLineTool into sibling steps.

rename_step_source(workflow, old, new)

Update step source names to the new name.

remove_JSReq(process, skip_command_line1)

Since the InlineJavascriptRequirement is longer needed, remove it.

replace_step_clt_expr_with_etool(expr, name, workflow, ...)

Convert a step level CWL Expression to a sibling expression step.

replace_clt_hintreq_expr_with_etool(expr, name, ...[, ...])

Factor out an expression inside a CommandLineTool req or hint into a sibling step.

cltool_inputs_to_etool_inputs(tool)

Copy CommandLineTool input objects into the equivalent ExpressionTool input objects.

cltool_step_outputs_to_workflow_outputs(cltool_step, ...)

Copy CommandLineTool outputs into the equivalent Workflow output parameters.

generate_etool_from_expr2(expr, target, inputs[, ...])

Generate an ExpressionTool to achieve the same result as the given expression.

traverse_step(step, parent, replace_etool, ...)

Process the given WorkflowStep.

workflow_step_to_WorkflowInputParameters(step_ins, ...)

Create WorkflowInputParameters to match the given WorkflowStep inputs.

replace_step_valueFrom_expr_with_etool(expr, name, ...)

Replace a WorkflowStep level 'valueFrom' expression with a sibling ExpressionTool step.

replace_step_when_expr_with_etool(expr, workflow, ...)

Replace a WorkflowStep level 'when' expression with a sibling ExpressionTool step.

traverse_workflow(workflow, replace_etool, ...)

Traverse a workflow, processing each step.

Attributes

EMPTY_FILE

TOPLEVEL_SF_EXPR_ERROR

TOPLEVEL_FORMAT_EXPR_ERROR

PICKVALUE_FIRST_NON_NULL_EXPR

PICKVALUE_THE_ONLY_NON_NULL_EXPR

PICKVALUE_ALL_NON_NULL_EXPR

PICKVALUE_ERROR

cwl_utils.cwl_v1_2_expression_refactor.expand_stream_shortcuts(process)

Rewrite the “type: stdout” shortcut to use an explicit random filename.

Parameters:

process (cwl_utils.parser.cwl_v1_2.CommandLineTool) –

Return type:

cwl_utils.parser.cwl_v1_2.CommandLineTool

cwl_utils.cwl_v1_2_expression_refactor.escape_expression_field(contents)

Escape sequences similar to CWL expressions or param references.

Parameters:

contents (str) –

Return type:

str

cwl_utils.cwl_v1_2_expression_refactor.clean_type_ids(cwltype)

Simplify type identifiers.

Parameters:

cwltype (Union[cwl_utils.parser.cwl_v1_2.ArraySchema, cwl_utils.parser.cwl_v1_2.InputRecordSchema]) –

Return type:

Union[cwl_utils.parser.cwl_v1_2.ArraySchema, cwl_utils.parser.cwl_v1_2.InputRecordSchema]

cwl_utils.cwl_v1_2_expression_refactor.get_expression(string, inputs, self)

Find and return a normalized CWL expression, if any.

CWL expressions in the $() form are converted to the ${} form.

Parameters:

  • string (str) –

  • inputs (cwl_utils.types.CWLObjectType) –

  • self (Optional[cwl_utils.types.CWLOutputType]) –

Return type:

Optional[str]

cwl_utils.cwl_v1_2_expression_refactor.etool_to_cltool(etool, expressionLib=None)

Convert a ExpressionTool to a CommandLineTool.

Parameters:
Return type:

cwl_utils.parser.cwl_v1_2.CommandLineTool

cwl_utils.cwl_v1_2_expression_refactor.traverse(process, replace_etool, inside, skip_command_line1, skip_command_line2)

Convert the given process and any subprocesses.

Parameters:
Return type:

Tuple[Union[cwl_utils.parser.cwl_v1_2.CommandLineTool, cwl_utils.parser.cwl_v1_2.ExpressionTool, cwl_utils.parser.cwl_v1_2.Workflow], bool]

cwl_utils.cwl_v1_2_expression_refactor.load_step(step, replace_etool, skip_command_line1, skip_command_line2)

If the step’s Process is not inline, load and process it.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.generate_etool_from_expr(expr, target, no_inputs=False, self_type=None, extra_processes=None)

Convert a CWL Expression into an ExpressionTool.

Parameters:
Return type:

cwl_utils.parser.cwl_v1_2.ExpressionTool

cwl_utils.cwl_v1_2_expression_refactor.get_input_for_id(name, tool)

Determine the CommandInputParameter for the given input name.

Parameters:
Return type:

Optional[cwl_utils.parser.cwl_v1_2.CommandInputParameter]

cwl_utils.cwl_v1_2_expression_refactor.find_expressionLib(processes)

Return the expressionLib from the highest priority InlineJavascriptRequirement.

processes: should be in order of least important to most important (Workflow, WorkflowStep, … CommandLineTool/ExpressionTool)

Parameters:

processes (Sequence[Union[cwl_utils.parser.cwl_v1_2.CommandLineTool, cwl_utils.parser.cwl_v1_2.Workflow, cwl_utils.parser.cwl_v1_2.ExpressionTool, cwl_utils.parser.cwl_v1_2.WorkflowStep]]) –

Return type:

Optional[List[str]]

cwl_utils.cwl_v1_2_expression_refactor.replace_expr_with_etool(expr, name, workflow, target, source, replace_etool=False, extra_process=None, source_type=None)

Modify the given workflow, replacing the expr with an standalone ExpressionTool.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.replace_wf_input_ref_with_step_output(workflow, name, target)

Refactor all reference to a workflow input to the specified step output.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.empty_inputs(process_or_step, parent=None)

Produce a mock input object for the given inputs.

Parameters:
Return type:

Dict[str, Any]

cwl_utils.cwl_v1_2_expression_refactor.example_input(some_type)

Produce a fake input for the given type.

Parameters:

some_type (Any) –

Return type:

Any

cwl_utils.cwl_v1_2_expression_refactor.EMPTY_FILE: cwl_utils.types.CWLOutputType
cwl_utils.cwl_v1_2_expression_refactor.TOPLEVEL_SF_EXPR_ERROR = "Input '{}'. Sorry, CWL Expressions as part of a secondaryFiles specification in a Workflow level..."
cwl_utils.cwl_v1_2_expression_refactor.TOPLEVEL_FORMAT_EXPR_ERROR = "Input '{}'. Sorry, CWL Expressions as part of a format specification in a Workflow level input..."
cwl_utils.cwl_v1_2_expression_refactor.PICKVALUE_FIRST_NON_NULL_EXPR = Multiline-String
Show Value
"""${
for (let i = 0; i < self.length; i++) {
  if (self[i] !== null){
    return self[i];
  }
}
throw 'pickValue=first_non_null, but no non-null value found: ' + self;
}
"""
cwl_utils.cwl_v1_2_expression_refactor.PICKVALUE_THE_ONLY_NON_NULL_EXPR = Multiline-String
Show Value
"""${
var found = null;
self.forEach(function(item) {
  if (item !== null) {
    if (found !== null) {
      throw 'pickValue=the_only_non_null, but multiple non-null values found: ' + self;
    } else {
      found = item;
    }
  }
})
if (found !== null) {
  return found;
}
throw 'pickValue=the_only_non_null, but no non-null value found: ' + self;
}
"""
cwl_utils.cwl_v1_2_expression_refactor.PICKVALUE_ALL_NON_NULL_EXPR = Multiline-String
Show Value
"""${
var results = [];
self.forEach(function(item) {
  if (item !== null){
    results.push(item);
  }
})
return results;
}
"""
cwl_utils.cwl_v1_2_expression_refactor.PICKVALUE_ERROR = "pickValue '{}' is invalid. Should be one of 'first_non_null', 'the_only_non_null', 'all_non_null'"
cwl_utils.cwl_v1_2_expression_refactor.process_workflow_inputs_and_outputs(workflow, replace_etool)

Do any needed conversions on the given Workflow’s inputs and outputs.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.process_workflow_reqs_and_hints(workflow, replace_etool)

Convert any expressions in a workflow’s reqs and hints.

Each expression will be converted to an additional step. The converted requirement will be copied to all workflow steps that don’t have that requirement type. Those affected steps will gain an additional input from the relevant synthesized expression step.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.process_level_reqs(process, step, parent, replace_etool, skip_command_line1, skip_command_line2)

Convert expressions inside a process into new adjacent steps.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.add_input_to_process(process, name, inptype, loadingOptions)

Add a new InputParameter to the given CommandLineTool.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.traverse_CommandLineTool(clt, parent, step, replace_etool, skip_command_line1, skip_command_line2)

Extract any CWL Expressions within the given CommandLineTool into sibling steps.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.rename_step_source(workflow, old, new)

Update step source names to the new name.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.remove_JSReq(process, skip_command_line1)

Since the InlineJavascriptRequirement is longer needed, remove it.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.replace_step_clt_expr_with_etool(expr, name, workflow, target, step, replace_etool, self_name=None)

Convert a step level CWL Expression to a sibling expression step.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.replace_clt_hintreq_expr_with_etool(expr, name, workflow, target, step, replace_etool, self_name=None)

Factor out an expression inside a CommandLineTool req or hint into a sibling step.

Parameters:
Return type:

Union[cwl_utils.parser.cwl_v1_2.CommandLineTool, cwl_utils.parser.cwl_v1_2.ExpressionTool]

cwl_utils.cwl_v1_2_expression_refactor.cltool_inputs_to_etool_inputs(tool)

Copy CommandLineTool input objects into the equivalent ExpressionTool input objects.

Parameters:

tool (cwl_utils.parser.cwl_v1_2.CommandLineTool) –

Return type:

List[cwl_utils.parser.cwl_v1_2.WorkflowInputParameter]

cwl_utils.cwl_v1_2_expression_refactor.cltool_step_outputs_to_workflow_outputs(cltool_step, etool_step_id, etool_out_id)

Copy CommandLineTool outputs into the equivalent Workflow output parameters.

Connects the outputSources for each of the new output parameters to the step they came from.

Parameters:
Return type:

List[cwl_utils.parser.cwl_v1_2.OutputParameter]

cwl_utils.cwl_v1_2_expression_refactor.generate_etool_from_expr2(expr, target, inputs, self_name=None, process=None, extra_processes=None)

Generate an ExpressionTool to achieve the same result as the given expression.

Parameters:
Return type:

cwl_utils.parser.cwl_v1_2.ExpressionTool

cwl_utils.cwl_v1_2_expression_refactor.traverse_step(step, parent, replace_etool, skip_command_line1, skip_command_line2)

Process the given WorkflowStep.

Parameters:
Return type:

bool

cwl_utils.cwl_v1_2_expression_refactor.workflow_step_to_WorkflowInputParameters(step_ins, parent, except_in_id)

Create WorkflowInputParameters to match the given WorkflowStep inputs.

Parameters:
Return type:

List[cwl_utils.parser.cwl_v1_2.WorkflowInputParameter]

cwl_utils.cwl_v1_2_expression_refactor.replace_step_valueFrom_expr_with_etool(expr, name, workflow, target, step, step_inp, original_process, original_step_ins, source, replace_etool, source_type=None)

Replace a WorkflowStep level ‘valueFrom’ expression with a sibling ExpressionTool step.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.replace_step_when_expr_with_etool(expr, workflow, step, original_step_ins, replace_etool)

Replace a WorkflowStep level ‘when’ expression with a sibling ExpressionTool step.

Parameters:
Return type:

None

cwl_utils.cwl_v1_2_expression_refactor.traverse_workflow(workflow, replace_etool, skip_command_line1, skip_command_line2)

Traverse a workflow, processing each step.

Parameters:
Return type:

Tuple[cwl_utils.parser.cwl_v1_2.Workflow, bool]