Ruffus documentation

Start Here:

• Installation
• A simple tutorial: 8 steps to Ruffus
• Manual

Overview:

• Cheat Sheet
  • 1. Annotate functions with Ruffus decorators
  • 2. Print dependency graph if necessary
  • 3. Run the pipeline
• Pipeline functions
  • pipeline_run
  • pipeline_printout
  • pipeline_printout_graph
• Installation
  • The easy way
  • The most up-to-date code:
• Design & Architecture
  • GNU Make
  • Scons, Rake and other Make alternatives
  • Managing pipelines stage-by-stage using Ruffus
  • Alternatives to Ruffus
• Major Features added to Ruffus
  • version 1.0
  • version 1.0.7
  • version 1.1.4
  • version 2.0
  • version 2.0.2
  • version 2.0.8
  • version 2.0.9
  • version 2.0.10
  • version 2.1.0
  • version 2.1.1
  • version 2.2
• Fixed Bugs
• Future plans for Ruffus:
  • Update documentation
• Completed already:
• Left to do:
  • Exceptions
  • Clean up
  • (Plug-in) File Dependency Checking via MD5 or Databases
  • Remove intermediate files
  • Extra signalling before and after each task and job
  • SQL hooks
  • Return values
  • Run jobs on remote (clustered) processes via SGE/Hadoop
• FAQ
  • General
  • Windows
  • Sun Grid Engine
• Glossary
• Hall of Fame: User contributed flowcharts
  • RNASeq pipeline
  • non-coding evolutionary constraints
  • SNP annotation
  • Chip-Seq analysis
• Standard code for developing Ruffus pipelines
  • 1) Logging
  • 3) Tracing pipeline progress
  • 4) Printing a flowchart
  • 5) Running the pipeline on multiple processors

Examples

• Construction of a simple pipeline to run BLAST jobs
  • Overview
  • Prerequisites
  • Code
  • Step 1. Splitting up the query sequences
  • Step 2. Run BLAST jobs in parallel
  • Step 3. Combining BLAST results
  • Step 4. Running the pipeline
  • Step 5. Testing dependencies
  • What is next?
• Part 2: A slightly more practical pipeline to run blasts jobs
  • Overview
  • Step 1. Cleaning up any leftover junk from previous pipeline runs
  • Step 2. Adding a “flag” file to mark successful completion
  • Step 3. Allowing the script to be invoked on the command line
  • Step 4. Printing out a flowchart for the pipeline
  • Step 5. Errors
  • Step 6. Will it run?
• Ruffus code
• Ruffus code
• Flowchart colours
• Code for flowchart_colours
  • Code
• Code for flowchart_colours
  • Code
• Code for the manual tutorial
  • Table of Contents
• Code for the simple tutorial: 8 steps to Ruffus
  • Table of Contents

Reference:

Decorators

• Ruffus Decorators
• Indicator Objects

Basic

• @follows
• @files

Core

• @split
• @transform
• @merge
• @posttask

For advanced users

• @jobs_limit
• @split with regex(...), add_inputs and inputs
• @transform with add_inputs and inputs
• @collate
• @collate with add_inputs and inputs

Esoteric

• Generating parameters on the fly for @files
• @parallel
• @check_if_uptodate
• @files_re

Modules:

• ruffus.Task
  • ruffus.task – Overview
  • Decorators
  • Pipeline functions
  • Logging
  • Implementation:
  • Exceptions and Errors
• ruffus.proxy_logger
  • Create proxy for logging for use with multiprocessing
  • Proxies for a log:
  • Create a logging object

Indices and tables

• Index
• Module Index
• Search Page