WEBINAR: Portable, reproducible and scalable bioinformatics workflows using Nextflow and Pawsey Nimbus Cloud

This record includes training materials associated with the Australian BioCommons webinar ‘Portable, reproducible and scalable bioinformatics workflows using Nextflow and Pawsey Nimbus Cloud’. This webinar took place on 20 September 2022.

Event description 

Bioinformatics workflows can support portable, reproducible and scalable analysis of omics datasets but using workflows can be challenging for both beginners and experienced bioinformaticians. Beginners face a steep learning curve to be able to build and deploy their own bioinformatics workflows while those with more experience face challenges productionising and scaling code for custom workflows and big data. 

Bioinformaticians across the world are using Nextflow to build and manage workflows. Many of these workflows are shared for others to use and supported by the community via nf-co.re. So far, 39 workflows for omics data are available with another 23 under development. These workflows cover common analyses such as RNAseq, mapping, variant calling, single cell transcriptomics and more and can be easily deployed by anyone, regardless of skill level.

In this webinar, Nandan Deshpande from the Sydney Informatics Hub, University of Sydney, will discuss how you can deploy freely available Nextflow (nf.co-re) bioinformatics workflows with a single command. We describe how you can quickly get started deploying these workflows using Pawsey Nimbus Cloud. For advanced users, we introduce you to Nextflow concepts to get you started with building your own workflows that will save you time and support reproducible, portable and scalable analysis.

In the latter half of the webinar, Sarah Beecroft from the Pawsey Supercomputing Research Centre will talk about their Nimbus Cloud systems. While Nextflow supports portability and can run on many computing infrastructures, we describe why we specifically love using Nimbus with Nextflow for many bioinformatics projects. We will describe some of the nf.co-re workflows that we have used on Nimbus and the research outcomes. We will also cover when not to use Nimbus and the alternatives we recommend.
 

Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.

Files and materials included in this record:

Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.


Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.


Nextflow_Nimbus_slides (PDF): A PDF copy of the slides presented during the webinar.

Materials shared elsewhere:

A recording of this webinar is available on the Australian BioCommons YouTube Channel:

https://youtu.be/VnLX63yXbJU

DOI: 10.5281/zenodo.7095271

Licence: Creative Commons Attribution 4.0

Contact: Melissa Burke (melissa@biocommons.org.au)

Keywords: Bioinformatics, Workflows, Nextflow, Containerisation


Additional information

Status: Active

Authors: Deshpande, Nandan (orcid: 0000-0002-0324-8728), Beecroft, Sarah (orcid: 0000-0002-3935-2279)

WEBINAR: Portable, reproducible and scalable bioinformatics workflows using Nextflow and Pawsey Nimbus Cloud https://dresa.org.au/materials/webinar-portable-reproducible-and-scalable-bioinformatics-workflows-using-nextflow-and-pawsey-nimbus-cloud This record includes training materials associated with the Australian BioCommons webinar ‘Portable, reproducible and scalable bioinformatics workflows using Nextflow and Pawsey Nimbus Cloud’. This webinar took place on 20 September 2022. Event description  Bioinformatics workflows can support portable, reproducible and scalable analysis of omics datasets but using workflows can be challenging for both beginners and experienced bioinformaticians. Beginners face a steep learning curve to be able to build and deploy their own bioinformatics workflows while those with more experience face challenges productionising and scaling code for custom workflows and big data.  Bioinformaticians across the world are using Nextflow to build and manage workflows. Many of these workflows are shared for others to use and supported by the community via nf-co.re. So far, 39 workflows for omics data are available with another 23 under development. These workflows cover common analyses such as RNAseq, mapping, variant calling, single cell transcriptomics and more and can be easily deployed by anyone, regardless of skill level. In this webinar, Nandan Deshpande from the Sydney Informatics Hub, University of Sydney, will discuss how you can deploy freely available Nextflow (nf.co-re) bioinformatics workflows with a single command. We describe how you can quickly get started deploying these workflows using Pawsey Nimbus Cloud. For advanced users, we introduce you to Nextflow concepts to get you started with building your own workflows that will save you time and support reproducible, portable and scalable analysis. In the latter half of the webinar, Sarah Beecroft from the Pawsey Supercomputing Research Centre will talk about their Nimbus Cloud systems. While Nextflow supports portability and can run on many computing infrastructures, we describe why we specifically love using Nimbus with Nextflow for many bioinformatics projects. We will describe some of the nf.co-re workflows that we have used on Nimbus and the research outcomes. We will also cover when not to use Nimbus and the alternatives we recommend.   Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. Files and materials included in this record: Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. Nextflow_Nimbus_slides (PDF): A PDF copy of the slides presented during the webinar. Materials shared elsewhere: A recording of this webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/VnLX63yXbJU Melissa Burke (melissa@biocommons.org.au) Bioinformatics, Workflows, Nextflow, Containerisation