Decorator | Examples | |
---|---|---|
@follows
|
||
@files (see Manual)
|
Decorator | Examples | |
---|---|---|
@split (see Manual)
|
|
|
@transform (see Manual)
|
|
|
@merge (see Manual)
|
|
|
@posttask
|
|
Decorator Examples @collate (See Manual here and here)
- Groups multiple input files using regular expression matching
- Input resulting in the same output after substitution will be collated together.
@transform (Manual)
- Infers input as well as output from regular expression substitutions
- Useful for adding additional file dependencies
- @transform ( tasks_or_file_names, suffix(suffix_string), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] )
- @transform ( tasks_or_file_names, regex(regex_pattern), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] )
@split (see Manual)
- Splits multiple input each further into many more output
- Globs in output can specify an indeterminate number of files.
- @split ( tasks_or_file_names, regex(regex_pattern), [ inputs | add_inputs(input_pattern), ] output_pattern, [extra_parameters,...] )
@jobs_limit (see Manual)
- Limits the amount of multiprocessing for the specified task
- Ensures that fewer than N jobs for this task are run in parallel
- Overrides multiprocess parameter in pipeline_run(...)
- @jobs_limit ( NUMBER_OF_JOBS_RUNNING_CONCURRENTLY )
Decorator Examples @parallel
- By default, does not check if jobs are up to date
- Best used in conjuction with @check_if_uptodate
- @parallel ( parameter_list ) (see Manual)
- @parallel ( parameter_generating_function ) (see Manual)
@check_if_uptodate (see Manual)
- Custom function to determine if jobs need to be run
- @check_if_uptodate ( is_task_up_to_date_function )
Tip
- The use of this overly complicated function is discouraged.
@files_re
- I/O file names via regular expressions
- start from lists of file names or glob results
- skips up-to-date jobs
- @files_re ( tasks_or_file_names, matching_regex, [input_pattern,] output_pattern, ... )
input_pattern/output_pattern are regex patterns used to create input/output file names from the starting list of either glob_str or file names