WEBINAR: Where to go when your bioinformatics outgrows your compute
This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.
Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute...
Keywords: Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
WEBINAR: Where to go when your bioinformatics outgrows your compute
https://zenodo.org/records/5240578
https://dresa.org.au/materials/webinar-where-to-go-when-your-bioinformatics-outgrows-your-compute-7a5a0ff8-8f4f-4fd0-af20-a88d515a6554
This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.
Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute resources. “I don’t know what compute resources I will need”, “My analysis won’t run and I don’t know why” and "Just getting it to work" are common pain points for researchers. In this webinar, you will learn how to understand the compute requirements for your bioinformatics workflows. You will also hear about ways of accessing compute that suits your needs as an Australian researcher, including Galaxy Australia, cloud and high-performance computing services offered by the Australian Research Data Commons, the National Compute Infrastructure (NCI) and Pawsey. We also describe bioinformatics and computing support services available to Australian researchers.
This webinar was jointly organised with the Sydney Informatics Hub at the University of Sydney.
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
Where to go when your bioinformatics outgrows your compute - slides (PDF and PPTX): Slides presented during the webinar
Australian research computing resources cheat sheet (PDF): A list of resources and useful links mentioned during the webinar.
Materials shared elsewhere:
A recording of the webinar is available on the Australian BioCommons YouTube Channel:
https://youtu.be/hNTbngSc-W0
Melissa Burke (melissa@biocommons.org.au)
Samaha, Georgina (orcid: 0000-0003-0419-1476)
Chew, Tracy (orcid: 0000-0001-9529-7705)
Sadsad, Rosemarie (orcid: 0000-0003-2488-953X)
Coddington, Paul (orcid: 0000-0003-1336-9686)
Gladman, Simon (orcid: 0000-0002-6100-4385)
Edberg, Roger
Shaikh, Javed
Cytowski, Maciej (orcid: 0000-0002-0007-0979)
Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
From PC to Cloud or High Performance Computing
Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well.
We may refer to both types as “large scale computing” – but...
From PC to Cloud or High Performance Computing
https://intersect.org.au/training/course/compute001
https://dresa.org.au/materials/from-pc-to-cloud-or-high-performance-computing
Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well.
We may refer to both types as “large scale computing” – but what is the difference? Both systems target scalability of computing, but in different ways.
This webinar will give a good overview to the researchers thinking to make a move from their local computer to Cloud of High Performance Computing Cluster.
Introduction
HPC vs Cloud computing
When to use HPC
When to use the Cloud
The Cloud – Pros and Cons
HPC – Pros and Cons
The webinar has no prerequisites.
training@intersect.org.au
Intersect Australia
HPC
Getting started with HPC using PBS Pro
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...
Getting started with HPC using PBS Pro
https://intersect.org.au/training/course/hpc201
https://dresa.org.au/materials/getting-started-with-hpc-using-pbs-pro
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free?
High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using PBS Pro.
Connect to an HPC cluster
Use the Unix command line to operate a remote computer and create job scripts
Submit and manage jobs on a cluster using a scheduler
Transfer files to and from a remote computer
Use software through environment modules
Use parallelisation to speed up data analysis
Access the facilities available to you as a researcher
This is the PBS Pro version of the Getting Started with HPC course.
This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course.
training@intersect.org.au
Intersect Australia
HPC
Getting started with HPC using Slurm
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...
Getting started with HPC using Slurm
https://intersect.org.au/training/course/hpc202
https://dresa.org.au/materials/getting-started-with-hpc-using-slurm
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free?
High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using Slurm.
Connect to an HPC cluster
Use the Unix command line to operate a remote computer and create job scripts
Submit and manage jobs on a cluster using a scheduler
Transfer files to and from a remote computer
Use software through environment modules
Use parallelisation to speed up data analysis
Access the facilities available to you as a researcher
This is the Slurm version of the Getting Started with HPC course.
This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course.
training@intersect.org.au
Intersect Australia
HPC
Parallel Programming for HPC
You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs.
Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance...
Parallel Programming for HPC
https://intersect.org.au/training/course/hpc301
https://dresa.org.au/materials/parallel-programming-for-hpc
You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs.
Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance Computing.
OpenMP (Open Multi-Processing): a widespread method for shared memory programming
MPI (Message Passing Interface): a leading distributed memory programming model
To do this course you need to have:
A good working knowledge of HPC. Consider taking our
Getting Started with HPC using PBS Pro course to come up to speed beforehand.
Prior experience of writing programs in either C or Fortran.
training@intersect.org.au
Intersect Australia
HPC