Register training material
16 materials found

Keywords: fish  or HPC 


WEBINAR: Where to go when your bioinformatics outgrows your compute

This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.

Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute...

Keywords: Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing

WEBINAR: Where to go when your bioinformatics outgrows your compute https://dresa.org.au/materials/webinar-where-to-go-when-your-bioinformatics-outgrows-your-compute-7a5a0ff8-8f4f-4fd0-af20-a88d515a6554 This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021. Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute resources. “I don’t know what compute resources I will need”, “My analysis won’t run and I don’t know why” and "Just getting it to work" are common pain points for researchers. In this webinar, you will learn how to understand the compute requirements for your bioinformatics workflows. You will also hear about ways of accessing compute that suits your needs as an Australian researcher, including Galaxy Australia, cloud and high-performance computing services offered by the Australian Research Data Commons, the National Compute Infrastructure (NCI) and Pawsey.  We also describe bioinformatics and computing support services available to Australian researchers.  This webinar was jointly organised with the Sydney Informatics Hub at the University of Sydney. Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. Files and materials included in this record: Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. Where to go when your bioinformatics outgrows your compute - slides (PDF and PPTX): Slides presented during the webinar Australian research computing resources cheat sheet (PDF): A list of resources and useful links mentioned during the webinar. Materials shared elsewhere: A recording of the webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/hNTbngSc-W0 Melissa Burke (melissa@biocommons.org.au) Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
WEBINAR: High performance bioinformatics: submitting your best NCMAS application

This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021.

Bioinformaticians are increasingly turning to specialised compute infrastructure and...

Keywords: Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS

WEBINAR: High performance bioinformatics: submitting your best NCMAS application https://dresa.org.au/materials/webinar-high-performance-bioinformatics-submitting-your-best-ncmas-application-ee80822f-74ac-41af-a5a4-e162c10e6d78 This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021. Bioinformaticians are increasingly turning to specialised compute infrastructure and efficient, scalable workflows as their research becomes more data intensive. Australian researchers that require extensive compute resources to process large datasets can apply for access to national high performance computing facilities (e.g. Pawsey and NCI) to power their research through the National Computational Merit Allocation Scheme (NCMAS). NCMAS is a competitive, merit-based scheme and requires applicants to carefully consider how the compute infrastructure and workflows will be applied.  This webinar provides life science researchers with insights into what makes a strong NCMAS application, with a focus on the technical assessment, and how to design and present effective and efficient bioinformatic workflows for the various national compute facilities. It will be followed by a short Q&A session. Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. Files and materials included in this record: Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. High performance bioinformatics: submitting your best NCMAS application - slides (PDF and PPTX): Slides presented during the webinar   Materials shared elsewhere: A recording of the webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/HeFGjguwS0Y Melissa Burke (melissa@biocommons.org.au) Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS
WEBINAR: Pro tips for scaling bioinformatics workflows to HPC

This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023.

Event description 

High Performance Computing (HPC) infrastructures offer the computational scale and...

Keywords: Bioinformatics, Workflows, HPC, High Performance Computing

WEBINAR: Pro tips for scaling bioinformatics workflows to HPC https://dresa.org.au/materials/webinar-pro-tips-for-scaling-bioinformatics-workflows-to-hpc-9f2a8b90-88da-433b-83b2-b1ab262dd9df This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023. Event description  High Performance Computing (HPC) infrastructures offer the computational scale and efficiency that life scientists need to handle complex biological datasets and multi-step computational workflows. But scaling workflows to HPC from smaller, more familiar computational infrastructures brings with it new jargon, expectations, and processes to learn. To make the most of HPC resources, bioinformatics workflows need to be designed for distributed computing environments and carefully manage varying resource requirements, and data scale related to biology.   In this webinar, Dr Georgina Samaha from the Sydney Informatics Hub, Dr Matthew Downton from the National Computational Infrastructure (NCI) and Dr Sarah Beecroft from the Pawsey Supercomputing Research Centre help you navigate the world of HPC for running and developing bioinformatics workflows. They explain when you should take your workflows to HPC and highlight the architectural features you should make the most of to scale your analyses once you’re there. You’ll hear pro-tips for dealing with common pain points like software installation, optimising for parallel computing and resource management, and will find out how to get access to Australia’s National HPC infrastructures at NCI and Pawsey.  Materials Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. Files and materials included in this record: Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. Pro-tips_HPC_Slides: A PDF copy of the slides presented during the webinar. Materials shared elsewhere: A recording of this webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/YKJDRXCmGMo Melissa Burke (melissa@biocommons.org.au) Bioinformatics, Workflows, HPC, High Performance Computing
CheckEM User Guide

CheckEM is an open-source web based application which provides quality control assessments on metadata and image annotations of fish stereo-imagery. It is available at marine-ecology.shinyapps.io/CheckEM. The application can assess a range of sampling methods and annotation data formats for...

Keywords: stereo-video, fish, annotation

CheckEM User Guide https://dresa.org.au/materials/checkem-user-guide CheckEM is an open-source web based application which provides quality control assessments on metadata and image annotations of fish stereo-imagery. It is available at marine-ecology.shinyapps.io/CheckEM. The application can assess a range of sampling methods and annotation data formats for common inaccuracies made whilst annotating stereo imagery. CheckEM creates interactive plots and tables in a graphical interface, and provides summarised data and a report of potential errors to download. brooke.gibbons@uwa.edu.au stereo-video, fish, annotation
EventMeasure Annotation Guide

EventMeasure annotation guide for baited remote underwater stereo video systems (stereo-BRUVs) for count and length

Keywords: fish, stereo-video, annotation

EventMeasure Annotation Guide https://dresa.org.au/materials/eventmeasure-annotation-guide EventMeasure annotation guide for baited remote underwater stereo video systems (stereo-BRUVs) for count and length tim.langlois@uwa.edu.au fish, stereo-video, annotation
Stereo-video workflows for fish and benthic ecologists

Stereo imagery is widely used by research institutions and management bodies around the world as a cost-effective and non-destructive method to research and monitor fish and habitats (Whitmarsh, Fairweather and Huveneers, 2017). Stereo-video can provide accurate and precise size and range...

Keywords: stereo-video, fish, sharks, habitats

Resource type: tutorial

Stereo-video workflows for fish and benthic ecologists https://dresa.org.au/materials/stereo-video-workflows-for-fish-and-benthic-ecologists Stereo imagery is widely used by research institutions and management bodies around the world as a cost-effective and non-destructive method to research and monitor fish and habitats (Whitmarsh, Fairweather and Huveneers, 2017). Stereo-video can provide accurate and precise size and range measurements and can be used to study spatial and temporal patterns in fish assemblages (McLean et al., 2016), habitat composition and complexity (Collins et al., 2017), behaviour (Goetze et al., 2017), responses to anthropogenic pressures (Bosch et al., 2022) and the recovery and growth of benthic fauna (Langlois et al. 2020). It is important that users of stereo-video collect, annotate, quality control and store their data in a consistent manner, to ensure data produced is of the highest quality possible and to enable large scale collaborations. Here we collate existing best practices and propose new tools to equip ecologists to ensure that all aspects of the stereo-video workflow are performed in a consistent way. tim.langlois@uwa.edu.au stereo-video, fish, sharks, habitats
Pawsey: AWS Quantum 101 Using Amazon Braket

Join us as AWS Quantum Specialists introduce quantum simulators and gate-based quantum computers, before turning to more advanced topics.

Keywords: Pawsey Supercomputing Centre, AWS, quantum, HPC

Pawsey: AWS Quantum 101 Using Amazon Braket https://dresa.org.au/materials/pawsey-aws-quantum-101-using-amazon-braket Join us as AWS Quantum Specialists introduce quantum simulators and gate-based quantum computers, before turning to more advanced topics. training@pawsey.org.au Pawsey Supercomputing Centre, AWS, quantum, HPC
OpenCL

Supercomputers make use of accelerators from a variety of different hardware vendors, using devices such as multi-core CPU’s, GPU’s and even FPGA’s. OpenCL is a way for your HPC application to make effective use of heterogeneous computing devices, and to avoid code refactoring for new HPC...

Keywords: OpenCL, supercomputing, CPUs, GPUs, FPGAs, HPC

OpenCL https://dresa.org.au/materials/opencl-3eabb316-794d-4f46-959a-725be3ae1bde Supercomputers make use of accelerators from a variety of different hardware vendors, using devices such as multi-core CPU’s, GPU’s and even FPGA’s. OpenCL is a way for your HPC application to make effective use of heterogeneous computing devices, and to avoid code refactoring for new HPC infrastructure. Topics covered in this course are : - Introduction to OpenCL - How to build and run applications on Setonix with OpenCL and MPI - Matrix multiplication with OpenCL – fully explained line by line - How to debug OpenCL applications and kernels - Measure performance with OpenCL Events and open source tools - Memory management - Coarse and fine-grained shared memory - Strategies for building optimised OpenCL kernels - Optimise IO performance with asynchronous operations training@pawsey.org.au OpenCL, supercomputing, CPUs, GPUs, FPGAs, HPC
PCon Preparing applications for El Capitan and beyond

As Lawrence Livermore National Laboratories (LLNL) prepares to stand up its next supercomputer, El Capitan, application teams prepare to pivot to another GPU architecture.

This talk presents how the LLNL application teams made the transition from distributed-memory, CPU-only architectures to...

Keywords: GPUs, supercomputing, HPC, PaCER

PCon Preparing applications for El Capitan and beyond https://dresa.org.au/materials/pcon-preparing-applications-for-el-capitan-and-beyond As Lawrence Livermore National Laboratories (LLNL) prepares to stand up its next supercomputer, El Capitan, application teams prepare to pivot to another GPU architecture. This talk presents how the LLNL application teams made the transition from distributed-memory, CPU-only architectures to GPUs. They share institutional best practices. They discuss new open-source software products as tools for porting and profiling applications and as avenues for collaboration across the computational science community. Join LLNL's Erik Draeger and Jane Herriman, who presented this talk at Pawsey's PaCER Conference in September 2023. training@pawsey.org.au Pawsey Supercomputing Research Centre GPUs, supercomputing, HPC, PaCER masters phd researcher ecr support professional ugrad
WEBINAR: Pro tips for scaling bioinformatics workflows to HPC

This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023.

Event description 

High Performance Computing (HPC) infrastructures offer the computational scale and...

Keywords: Bioinformatics, Workflows, HPC, High Performance Computing

WEBINAR: Pro tips for scaling bioinformatics workflows to HPC https://dresa.org.au/materials/webinar-pro-tips-for-scaling-bioinformatics-workflows-to-hpc This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023. Event description  High Performance Computing (HPC) infrastructures offer the computational scale and efficiency that life scientists need to handle complex biological datasets and multi-step computational workflows. But scaling workflows to HPC from smaller, more familiar computational infrastructures brings with it new jargon, expectations, and processes to learn. To make the most of HPC resources, bioinformatics workflows need to be designed for distributed computing environments and carefully manage varying resource requirements, and data scale related to biology.   In this webinar, Dr Georgina Samaha from the Sydney Informatics Hub, Dr Matthew Downton from the National Computational Infrastructure (NCI) and Dr Sarah Beecroft from the Pawsey Supercomputing Research Centre help you navigate the world of HPC for running and developing bioinformatics workflows. They explain when you should take your workflows to HPC and highlight the architectural features you should make the most of to scale your analyses once you’re there. You’ll hear pro-tips for dealing with common pain points like software installation, optimising for parallel computing and resource management, and will find out how to get access to Australia’s National HPC infrastructures at NCI and Pawsey.  Materials Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. Files and materials included in this record: Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. Pro-tips_HPC_Slides: A PDF copy of the slides presented during the webinar. Materials shared elsewhere: A recording of this webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/YKJDRXCmGMo Melissa Burke (melissa@biocommons.org.au) Bioinformatics, Workflows, HPC, High Performance Computing
WEBINAR: Where to go when your bioinformatics outgrows your compute

This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.

Bioinformatics analyses are often complex, requiring multiple software tools and specialised...

Keywords: Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing

WEBINAR: Where to go when your bioinformatics outgrows your compute https://dresa.org.au/materials/webinar-where-to-go-when-your-bioinformatics-outgrows-your-compute This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021. Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute resources. “I don’t know what compute resources I will need”, “My analysis won’t run and I don’t know why” and "Just getting it to work" are common pain points for researchers. In this webinar, you will learn how to understand the compute requirements for your bioinformatics workflows. You will also hear about ways of accessing compute that suits your needs as an Australian researcher, including Galaxy Australia, cloud and high-performance computing services offered by the Australian Research Data Commons, the National Compute Infrastructure (NCI) and Pawsey.  We also describe bioinformatics and computing support services available to Australian researchers.  This webinar was jointly organised with the Sydney Informatics Hub at the University of Sydney. Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. **Files and materials included in this record:** - Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. - Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. - Where to go when your bioinformatics outgrows your compute - slides (PDF and PPTX): Slides presented during the webinar - Australian research computing resources cheat sheet (PDF): A list of resources and useful links mentioned during the webinar. **Materials shared elsewhere:** A recording of the webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/hNTbngSc-W0 Melissa Burke (melissa@biocommons.org.au) Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
WEBINAR: High performance bioinformatics: submitting your best NCMAS application

This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021.

Bioinformaticians are increasingly turning to specialised compute infrastructure and...

Keywords: Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS

WEBINAR: High performance bioinformatics: submitting your best NCMAS application https://dresa.org.au/materials/webinar-high-performance-bioinformatics-submitting-your-best-ncmas-application This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021. Bioinformaticians are increasingly turning to specialised compute infrastructure and efficient, scalable workflows as their research becomes more data intensive. Australian researchers that require extensive compute resources to process large datasets can apply for access to national high performance computing facilities (e.g. Pawsey and NCI) to power their research through the National Computational Merit Allocation Scheme (NCMAS). NCMAS is a competitive, merit-based scheme and requires applicants to carefully consider how the compute infrastructure and workflows will be applied.  This webinar provides life science researchers with insights into what makes a strong NCMAS application, with a focus on the technical assessment, and how to design and present effective and efficient bioinformatic workflows for the various national compute facilities. It will be followed by a short Q&A session. Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event. **Files and materials included in this record:** - Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc. - Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file. - High performance bioinformatics: submitting your best NCMAS application - slides (PDF and PPTX): Slides presented during the webinar **Materials shared elsewhere:** A recording of the webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/HeFGjguwS0Y Melissa Burke (melissa@biocommons.org.au) Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS
HPC file systems and what users need to consider for appropriate and efficient usage

Three videos on miscellaneous aspects of HPC usage - useful reference for new users of HPC systems.

1 – General overview of different file systems that might be available on HPC. The video goes through shared file systems such as /home and /scratch, local compute node file systems (local...

Keywords: HPC, high performance computer, File systems

Resource type: video, presentation

HPC file systems and what users need to consider for appropriate and efficient usage https://dresa.org.au/materials/hpc-file-systems-and-what-users-need-to-consider-for-appropriate-and-efficient-usage Three videos on miscellaneous aspects of HPC usage - useful reference for new users of HPC systems. 1 – General overview of different file systems that might be available on HPC. The video goes through shared file systems such as /home and /scratch, local compute node file systems (local scratch or $TMPDIR) and storage file system. It outlines what users need to consider if they wish to use any of these in their workflows. 2 – Overview of the different directories that might be present on HPC. These could include /home, /scratch, /opt, /lib and lib64, /sw and others. 3 – Overview of the Message-of-the-day file and the message that is displayed to users every time they log in. This displays info about general help and often current problems or upcoming outages. QCIF Training (training@qcif.edu.au) HPC, high performance computer, File systems
Basic Linux/Unix commands

A series of eight videos (each between 5 and 10 minutes long) following the content of the Software Carpentry workshop "The Unix Shell".

Sessions 1, 2 and 3 provide instructions on the minimal level of Linux/Unix commands recommended for new...

Keywords: HPC, high performance computer, Unix, Linux, Software Carpentry

Resource type: video, guide

Basic Linux/Unix commands https://dresa.org.au/materials/basic-linux-unix-commands A series of eight videos (each between 5 and 10 minutes long) following the content of the Software Carpentry workshop ["The Unix Shell"](https://swcarpentry.github.io/shell-novice/). Sessions 1, 2 and 3 provide instructions on the minimal level of Linux/Unix commands recommended for new users of HPC. 1 – An overview of how to find out where a user is in the filesystem, list the files there, and how to get help on Unix commands 2 – How to move around the file system and change into other directories 3 – Explains the difference between an absolute and relative path 4 – Overview of how to create new directories, and to create and edit new files with nano 5 – How to use the vi editor to edit files 6 – Overview of file viewers available 7 – How to copy and move files and directories 8 – How to remove files and directories Further details and exercises with solutions can be found on the Software Carpentry "The Unix Shell" page (https://swcarpentry.github.io/shell-novice/) QCIF Training (training@qcif.edu.au) HPC, high performance computer, Unix, Linux, Software Carpentry
Transferring files and data

A short video outlining the basics on how to use FileZilla to establish a secure file transfer protocol (sftp) connection to HPC to use a drag and drop interface to transfer files between the HPC and a desktop computer.

Keywords: sftp, file transfer, HPC, high performance computer

Resource type: video, guide

Transferring files and data https://dresa.org.au/materials/transferring-files-and-data A short video outlining the basics on how to use FileZilla to establish a secure file transfer protocol (sftp) connection to HPC to use a drag and drop interface to transfer files between the HPC and a desktop computer. QCIF Training (training@qcif.edu.au) sftp, file transfer, HPC, high performance computer
Connecting to HPC

A series of three short videos introducing how to use PuTTY to connect from a Windows PC to a secure HPC (high performance computing) cluster.

1 - The very basics on how to establish a connection to HPC.
2 - How to add more specific options for the connection to HPC.
3 - How to save the...

Keywords: HPC, high performance computer, ssh

Resource type: video, guide

Connecting to HPC https://dresa.org.au/materials/connecting-to-hpc A series of three short videos introducing how to use PuTTY to connect from a Windows PC to a secure HPC (high performance computing) cluster. 1 - The very basics on how to establish a connection to HPC. 2 - How to add more specific options for the connection to HPC. 3 - How to save the details and options for a connection for future use. QCIF Training (training@qcif.edu.au) HPC, high performance computer, ssh