cwl_utils.cwl_v1_1_expression_refactor
CWL Expression refactoring tool for CWL v1.1 .
Attributes
Functions
|
Rewrite the "type: stdout" shortcut to use an explicit random filename. |
|
Escape sequences similar to CWL expressions or param references. |
|
Simplify type identifiers. |
|
Find and return a normalized CWL expression, if any. |
|
Convert a ExpressionTool to a CommandLineTool. |
|
Convert the given process and any subprocesses. |
|
If the step's Process is not inline, load and process it. |
|
Convert a CWL Expression into an ExpressionTool. |
|
Determine the CommandInputParameter for the given input name. |
|
Return the expressionLib from the highest priority InlineJavascriptRequirement. |
|
Modify the given workflow, replacing the expr with an standalone ExpressionTool. |
|
Refactor all reference to a workflow input to the specified step output. |
|
Produce a mock input object for the given inputs. |
|
Produce a fake input for the given type. |
|
Do any needed conversions on the given Workflow's inputs and outputs. |
|
Convert any expressions in a workflow's reqs and hints. |
|
Convert expressions inside a process into new adjacent steps. |
|
Add a new InputParameter to the given CommandLineTool. |
|
Extract any CWL Expressions within the given CommandLineTool into sibling steps. |
|
Update step source names to the new name. |
|
Since the InlineJavascriptRequirement is longer needed, remove it. |
|
Convert a step level CWL Expression to a sibling expression step. |
|
Factor out an expression inside a CommandLineTool req or hint into a sibling step. |
Copy CommandLineTool input objects into the equivalent ExpressionTool input objects. |
|
|
Copy CommandLineTool outputs into the equivalent Workflow output parameters. |
|
Generate an ExpressionTool to achieve the same result as the given expression. |
|
Process the given WorkflowStep. |
|
Create WorkflowInputParameters to match the given WorkflowStep inputs. |
|
Replace a WorkflowStep level 'valueFrom' expression with a sibling ExpressionTool step. |
|
Traverse a workflow, processing each step. |
Module Contents
- cwl_utils.cwl_v1_1_expression_refactor.expand_stream_shortcuts(process)
Rewrite the “type: stdout” shortcut to use an explicit random filename.
- Parameters:
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.escape_expression_field(contents)
Escape sequences similar to CWL expressions or param references.
- cwl_utils.cwl_v1_1_expression_refactor.clean_type_ids(cwltype)
Simplify type identifiers.
- Parameters:
cwltype (Union[cwl_utils.parser.cwl_v1_1.ArraySchema, cwl_utils.parser.cwl_v1_1.InputRecordSchema])
- Return type:
Union[cwl_utils.parser.cwl_v1_1.ArraySchema, cwl_utils.parser.cwl_v1_1.InputRecordSchema]
- cwl_utils.cwl_v1_1_expression_refactor.get_expression(string, inputs, self)
Find and return a normalized CWL expression, if any.
CWL expressions in the $() form are converted to the ${} form.
- cwl_utils.cwl_v1_1_expression_refactor.etool_to_cltool(etool, expressionLib=None)
Convert a ExpressionTool to a CommandLineTool.
- Parameters:
expressionLib (Optional[List[str]])
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.traverse(process, replace_etool, inside, skip_command_line1, skip_command_line2)
Convert the given process and any subprocesses.
- Parameters:
process (Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.ExpressionTool, cwl_utils.parser.cwl_v1_1.Workflow])
replace_etool (bool)
inside (bool)
skip_command_line1 (bool)
skip_command_line2 (bool)
- Return type:
Tuple[Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.ExpressionTool, cwl_utils.parser.cwl_v1_1.Workflow], bool]
- cwl_utils.cwl_v1_1_expression_refactor.load_step(step, replace_etool, skip_command_line1, skip_command_line2)
If the step’s Process is not inline, load and process it.
- cwl_utils.cwl_v1_1_expression_refactor.generate_etool_from_expr(expr, target, no_inputs=False, self_type=None, extra_processes=None)
Convert a CWL Expression into an ExpressionTool.
- Parameters:
expr (str)
target (Union[cwl_utils.parser.cwl_v1_1.CommandInputParameter, cwl_utils.parser.cwl_v1_1.WorkflowInputParameter])
no_inputs (bool)
self_type (Optional[Union[cwl_utils.parser.cwl_v1_1.WorkflowInputParameter, cwl_utils.parser.cwl_v1_1.CommandInputParameter, List[Union[cwl_utils.parser.cwl_v1_1.WorkflowInputParameter, cwl_utils.parser.cwl_v1_1.CommandInputParameter]]]])
extra_processes (Optional[Sequence[Union[cwl_utils.parser.cwl_v1_1.Workflow, cwl_utils.parser.cwl_v1_1.WorkflowStep, cwl_utils.parser.cwl_v1_1.CommandLineTool]]])
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.get_input_for_id(name, tool)
Determine the CommandInputParameter for the given input name.
- Parameters:
name (str)
tool (Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.Workflow])
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.find_expressionLib(processes)
Return the expressionLib from the highest priority InlineJavascriptRequirement.
processes: should be in order of least important to most important (Workflow, WorkflowStep, … CommandLineTool/ExpressionTool)
- Parameters:
processes (Sequence[Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.Workflow, cwl_utils.parser.cwl_v1_1.ExpressionTool, cwl_utils.parser.cwl_v1_1.WorkflowStep]])
- Return type:
Optional[List[str]]
- cwl_utils.cwl_v1_1_expression_refactor.replace_expr_with_etool(expr, name, workflow, target, source, replace_etool=False, extra_process=None, source_type=None)
Modify the given workflow, replacing the expr with an standalone ExpressionTool.
- Parameters:
expr (str)
name (str)
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
target (Union[cwl_utils.parser.cwl_v1_1.CommandInputParameter, cwl_utils.parser.cwl_v1_1.WorkflowInputParameter])
source (Optional[Union[str, List[Any]]])
replace_etool (bool)
extra_process (Optional[Union[cwl_utils.parser.cwl_v1_1.Workflow, cwl_utils.parser.cwl_v1_1.WorkflowStep, cwl_utils.parser.cwl_v1_1.CommandLineTool]])
source_type (Optional[cwl_utils.parser.cwl_v1_1.CommandInputParameter])
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.replace_wf_input_ref_with_step_output(workflow, name, target)
Refactor all reference to a workflow input to the specified step output.
- Parameters:
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
name (str)
target (str)
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.empty_inputs(process_or_step, parent=None)
Produce a mock input object for the given inputs.
- Parameters:
process_or_step (Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.WorkflowStep, cwl_utils.parser.cwl_v1_1.ExpressionTool, cwl_utils.parser.cwl_v1_1.Workflow])
parent (Optional[cwl_utils.parser.cwl_v1_1.Workflow])
- Return type:
Dict[str, Any]
- cwl_utils.cwl_v1_1_expression_refactor.example_input(some_type)
Produce a fake input for the given type.
- Parameters:
some_type (Any)
- Return type:
Any
- cwl_utils.cwl_v1_1_expression_refactor.EMPTY_FILE: cwl_utils.types.CWLOutputType
- cwl_utils.cwl_v1_1_expression_refactor.TOPLEVEL_SF_EXPR_ERROR = "Input '{}'. Sorry, CWL Expressions as part of a secondaryFiles specification in a Workflow...
- cwl_utils.cwl_v1_1_expression_refactor.TOPLEVEL_FORMAT_EXPR_ERROR = "Input '{}'. Sorry, CWL Expressions as part of a format specification in a Workflow level input...
- cwl_utils.cwl_v1_1_expression_refactor.process_workflow_inputs_and_outputs(workflow, replace_etool)
Do any needed conversions on the given Workflow’s inputs and outputs.
- Parameters:
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.process_workflow_reqs_and_hints(workflow, replace_etool)
Convert any expressions in a workflow’s reqs and hints.
Each expression will be converted to an additional step. The converted requirement will be copied to all workflow steps that don’t have that requirement type. Those affected steps will gain an additional input from the relevant synthesized expression step.
- Parameters:
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.process_level_reqs(process, step, parent, replace_etool, skip_command_line1, skip_command_line2)
Convert expressions inside a process into new adjacent steps.
- Parameters:
parent (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
skip_command_line1 (bool)
skip_command_line2 (bool)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.add_input_to_process(process, name, inptype, loadingOptions)
Add a new InputParameter to the given CommandLineTool.
- Parameters:
process (cwl_utils.parser.cwl_v1_1.Process)
name (str)
inptype (Any)
loadingOptions (cwl_utils.parser.cwl_v1_1.LoadingOptions)
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.traverse_CommandLineTool(clt, parent, step, replace_etool, skip_command_line1, skip_command_line2)
Extract any CWL Expressions within the given CommandLineTool into sibling steps.
- Parameters:
parent (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
skip_command_line1 (bool)
skip_command_line2 (bool)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.rename_step_source(workflow, old, new)
Update step source names to the new name.
- Parameters:
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
old (str)
new (str)
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.remove_JSReq(process, skip_command_line1)
Since the InlineJavascriptRequirement is longer needed, remove it.
- Parameters:
process (Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.WorkflowStep, cwl_utils.parser.cwl_v1_1.Workflow])
skip_command_line1 (bool)
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.replace_step_clt_expr_with_etool(expr, name, workflow, target, step, replace_etool, self_name=None)
Convert a step level CWL Expression to a sibling expression step.
- Parameters:
expr (str)
name (str)
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
self_name (Optional[str])
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.replace_clt_hintreq_expr_with_etool(expr, name, workflow, target, step, replace_etool, self_name=None)
Factor out an expression inside a CommandLineTool req or hint into a sibling step.
- Parameters:
expr (str)
name (str)
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
self_name (Optional[str])
- Return type:
Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.ExpressionTool]
- cwl_utils.cwl_v1_1_expression_refactor.cltool_inputs_to_etool_inputs(tool)
Copy CommandLineTool input objects into the equivalent ExpressionTool input objects.
- Parameters:
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.cltool_step_outputs_to_workflow_outputs(cltool_step, etool_step_id, etool_out_id)
Copy CommandLineTool outputs into the equivalent Workflow output parameters.
Connects the outputSources for each of the new output parameters to the step they came from.
- Parameters:
cltool_step (cwl_utils.parser.cwl_v1_1.WorkflowStep)
etool_step_id (str)
etool_out_id (str)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.generate_etool_from_expr2(expr, target, inputs, self_name=None, process=None, extra_processes=None)
Generate an ExpressionTool to achieve the same result as the given expression.
- Parameters:
expr (str)
target (Union[cwl_utils.parser.cwl_v1_1.CommandInputParameter, cwl_utils.parser.cwl_v1_1.WorkflowInputParameter])
inputs (Sequence[Union[cwl_utils.parser.cwl_v1_1.WorkflowInputParameter, cwl_utils.parser.cwl_v1_1.CommandInputParameter]])
self_name (Optional[str])
process (Optional[Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.ExpressionTool]])
extra_processes (Optional[Sequence[Union[cwl_utils.parser.cwl_v1_1.Workflow, cwl_utils.parser.cwl_v1_1.WorkflowStep, cwl_utils.parser.cwl_v1_1.CommandLineTool]]])
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.traverse_step(step, parent, replace_etool, skip_command_line1, skip_command_line2)
Process the given WorkflowStep.
- Parameters:
parent (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
skip_command_line1 (bool)
skip_command_line2 (bool)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.workflow_step_to_WorkflowInputParameters(step_ins, parent, except_in_id)
Create WorkflowInputParameters to match the given WorkflowStep inputs.
- Parameters:
step_ins (List[cwl_utils.parser.cwl_v1_1.WorkflowStepInput])
parent (cwl_utils.parser.cwl_v1_1.Workflow)
except_in_id (str)
- Return type:
- cwl_utils.cwl_v1_1_expression_refactor.replace_step_valueFrom_expr_with_etool(expr, name, workflow, target, step, step_inp, original_process, original_step_ins, source, replace_etool, source_type=None)
Replace a WorkflowStep level ‘valueFrom’ expression with a sibling ExpressionTool step.
- Parameters:
expr (str)
name (str)
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
target (Union[cwl_utils.parser.cwl_v1_1.CommandInputParameter, cwl_utils.parser.cwl_v1_1.WorkflowInputParameter])
step_inp (cwl_utils.parser.cwl_v1_1.WorkflowStepInput)
original_process (Union[cwl_utils.parser.cwl_v1_1.CommandLineTool, cwl_utils.parser.cwl_v1_1.ExpressionTool])
original_step_ins (List[cwl_utils.parser.cwl_v1_1.WorkflowStepInput])
replace_etool (bool)
source_type (Optional[Union[cwl_utils.parser.cwl_v1_1.WorkflowInputParameter, List[cwl_utils.parser.cwl_v1_1.WorkflowInputParameter]]])
- Return type:
None
- cwl_utils.cwl_v1_1_expression_refactor.traverse_workflow(workflow, replace_etool, skip_command_line1, skip_command_line2)
Traverse a workflow, processing each step.
- Parameters:
workflow (cwl_utils.parser.cwl_v1_1.Workflow)
replace_etool (bool)
skip_command_line1 (bool)
skip_command_line2 (bool)
- Return type: