Skip to content

nf-core tools for users

Objectives

  • Learn more about nf-core tooling for users.
  • Use nf-core list to view information about nf-core pipelines.
  • Use nf-core download to download a pipeline and it's singularity images.
  • Use nf-core launch to create a parameters file.

nf-core tools

nf-core tools has additional commands to help users execute pipelines. Although you do not need to use these commands to execute the nf-core pipelines, they can greatly improve and simplify your experience.

There are also nf-core tools for developers. However, these will not be covered as a part of this workshop. If you are curious to learn more about these tools you can find more information on the nf-core websites tools page. There are also lots of excellent ByteSize talks on the nf-core YouTube channel.

nf-core list

The nf-core list command can be used to print a list of remote nf-core pipelines along with your local pipeline information.

nf-core list

The output shows the latest pipeline version number and when it was released. You will also be shown if and when a pipeline was pulled locally and whether you have the latest version.

Keywords can also be supplied to help filter the pipelines based on matches in titles, descriptions, or topics:

nf-core list dna

Options can also be used to sort the pipelines by latest release (-s release, default), when you last pulled a pipeline locally (-s pulled), alphabetically (-s name), or number by the number of GitHub stars (-s stars).

Exercise

Filter the list of nf-core pipelines for those that are for dna and sort them by stars. Which dna pipeline has the most stars?

Solution

Execute the list command, filter it for dna, and sort by stars:

nf-core list dna -s stars

nf-core launch

A pipeline can have a large number of optional parameters. To help with this, the nf-core launch command is designed to help you write parameter files for when you launch your pipeline.

The nf-core launch command takes one argument - either the name of an nf-core pipeline which will be pulled automatically or the path to a directory containing a Nextflow pipeline:

nf-core launch nf-core/<pipeline>

When running this command, you will first be asked about which version of the pipeline you would like to execute. Next, you will be given the choice between a web-based graphical interface or an interactive command-line wizard tool to enter the pipeline parameters. Both interfaces show documentation alongside each parameter, will generate a run ID, and will validate your inputs.

The nf-core launch tool uses the nextflow_schema.json file from a pipeline to give parameter descriptions, defaults, and grouping. If no file for the pipeline is found, one will be automatically generated at runtime.

The launch tool will save your parameter variables as a .json file called nf-params.json. It will also suggest an execution command that includes the -params-file flag and your new nf-params.json file.

The command line wizard will finish by asking if you want to launch the pipeline. Any profiles or options that were set using the wizard will be included in your run command.

Exercise

Use nf-core launch to launch the the christopher-hakkaart/nf-core-demo pipeline. Use the test and singularity profiles and name your output folder my_test_output.

Solution

Use the nf-core launch command for the christopher-hakkaart/nf-core-demo pipeline. Your nf-params.json file should look like this:

{
    "outdir": "my_test_output"
}

Your final run command should look like this:

nextflow run christopher-hakkaart/nf-core-demo -r main -profile test,singularity -params-file nf-params.json

The launch website

You can also use the launch command directly from the nf-core launch website. In this case, you can configure your pipeline using the wizard and then copy the outputs to your terminal or use the run id generated by the wizard. You will need to be connected to the internet to use the run id.

nf-core launch --id <run_id>

nf-core download

Sometimes you may need to execute an nf-core pipeline on a server or HPC system that has no internet connection. In this case, you will need to fetch the pipeline files and manually transfer them to your offline system. To make this process easier and ensure accurate retrieval of correctly versioned code and software containers, nf-core has the download command.

The nf-core download command will download both the pipeline code and the institutional nf-core/configs files. It can also optionally download singularity image file.

nf-core download

If run without any arguments, the download tool will interactively prompt you for the required information. Each prompt option has a flag and if all flags are supplied then it will run without a request for any additional user input:

  • Pipeline name
    • Name of pipeline you would like to download.
  • Pipeline revision
    • The revision you would like to download.
  • Pull containers
    • If you would like to download Singularity images.
    • The path to a folder where you would like to store these images if you have not set your NXF_SINGULARITY_CACHEDIR.
  • Choose compression type
    • The compression type for Singularity images.

Alternatively, you could build your own execution command with the command line options.

Exercise

Use the nf-core download command to download the christopher-hakkaart/nf-core-demo pipeline with it's uncompressed Singularity images.

Solution

Use the nf-core download command for the christopher-hakkaart/nf-core-demo pipeline and follow the prompts.

Your output should look like this:

INFO Saving 'christopher-hakkaart/nf-core-demo'                                                                                                                               
  Pipeline revision: 'main'                                                                                                                                               
  Use containers: 'singularity'                                                                                                                                           
  Container library: 'quay.io'                                                                                                                                            
  Using $NXF_SINGULARITY_CACHEDIR': /Users/chrishakkaart/tools/singularity'                                                                                               
  Output directory: 'christopher-hakkaart-nf-core-demo_main'                                                                                                              
  Include default institutional configuration: 'False'


Key points

  • The nf-core list command can be used to view local and remote information about nf-core pipelines
  • The nf-core launch command can be a useful tool for writing parameter files
  • The nf-core download command is a powerful way to download a pipeline and its Singularity images