WEBINAR: Where to go when your bioinformatics outgrows your compute
This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.
Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute...
Keywords: Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
WEBINAR: Where to go when your bioinformatics outgrows your compute
https://zenodo.org/records/5240578
https://dresa.org.au/materials/webinar-where-to-go-when-your-bioinformatics-outgrows-your-compute-7a5a0ff8-8f4f-4fd0-af20-a88d515a6554
This record includes training materials associated with the Australian BioCommons webinar ‘Where to go when your bioinformatics outgrows your compute’. This webinar took place on 19 August 2021.
Bioinformatics analyses are often complex, requiring multiple software tools and specialised compute resources. “I don’t know what compute resources I will need”, “My analysis won’t run and I don’t know why” and "Just getting it to work" are common pain points for researchers. In this webinar, you will learn how to understand the compute requirements for your bioinformatics workflows. You will also hear about ways of accessing compute that suits your needs as an Australian researcher, including Galaxy Australia, cloud and high-performance computing services offered by the Australian Research Data Commons, the National Compute Infrastructure (NCI) and Pawsey. We also describe bioinformatics and computing support services available to Australian researchers.
This webinar was jointly organised with the Sydney Informatics Hub at the University of Sydney.
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
Where to go when your bioinformatics outgrows your compute - slides (PDF and PPTX): Slides presented during the webinar
Australian research computing resources cheat sheet (PDF): A list of resources and useful links mentioned during the webinar.
Materials shared elsewhere:
A recording of the webinar is available on the Australian BioCommons YouTube Channel:
https://youtu.be/hNTbngSc-W0
Melissa Burke (melissa@biocommons.org.au)
Samaha, Georgina (orcid: 0000-0003-0419-1476)
Chew, Tracy (orcid: 0000-0001-9529-7705)
Sadsad, Rosemarie (orcid: 0000-0003-2488-953X)
Coddington, Paul (orcid: 0000-0003-1336-9686)
Gladman, Simon (orcid: 0000-0002-6100-4385)
Edberg, Roger
Shaikh, Javed
Cytowski, Maciej (orcid: 0000-0002-0007-0979)
Computational Biology, Bioinformatics, High performance computing, HPC, Galaxy Australia, Nectar Research Cloud, Pawsey Supercomputing Centre, NCI, NCMAS, Cloud computing
WEBINAR: High performance bioinformatics: submitting your best NCMAS application
This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021.
Bioinformaticians are increasingly turning to specialised compute infrastructure and...
Keywords: Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS
WEBINAR: High performance bioinformatics: submitting your best NCMAS application
https://zenodo.org/records/5239883
https://dresa.org.au/materials/webinar-high-performance-bioinformatics-submitting-your-best-ncmas-application-ee80822f-74ac-41af-a5a4-e162c10e6d78
This record includes training materials associated with the Australian BioCommons webinar ‘High performance bioinformatics: submitting your best NCMAS application’. This webinar took place on 20 August 2021.
Bioinformaticians are increasingly turning to specialised compute infrastructure and efficient, scalable workflows as their research becomes more data intensive. Australian researchers that require extensive compute resources to process large datasets can apply for access to national high performance computing facilities (e.g. Pawsey and NCI) to power their research through the National Computational Merit Allocation Scheme (NCMAS). NCMAS is a competitive, merit-based scheme and requires applicants to carefully consider how the compute infrastructure and workflows will be applied.
This webinar provides life science researchers with insights into what makes a strong NCMAS application, with a focus on the technical assessment, and how to design and present effective and efficient bioinformatic workflows for the various national compute facilities. It will be followed by a short Q&A session.
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
High performance bioinformatics: submitting your best NCMAS application - slides (PDF and PPTX): Slides presented during the webinar
Materials shared elsewhere:
A recording of the webinar is available on the Australian BioCommons YouTube Channel:
https://youtu.be/HeFGjguwS0Y
Melissa Burke (melissa@biocommons.org.au)
Samaha, Georgina (orcid: 0000-0003-0419-1476)
Chew, Tracy (orcid: 0000-0001-9529-7705)
Computational Biology, Bioinformatics, High Performance Computing, HPC, NCMAS
WEBINAR: Getting started with deep learning
This record includes training materials associated with the Australian BioCommons webinar ‘Getting started with deep learning’. This webinar took place on 21 July 2021.
Are you wondering what deep learning is and how it might be useful in your research? This high level overview introduces deep...
Keywords: Deep learning, Neural networks, Machine learning
WEBINAR: Getting started with deep learning
https://zenodo.org/records/5121004
https://dresa.org.au/materials/webinar-getting-started-with-deep-learning-986aa2d2-594a-4a7f-836c-44d6e9d5d017
This record includes training materials associated with the Australian BioCommons webinar ‘Getting started with deep learning’. This webinar took place on 21 July 2021.
Are you wondering what deep learning is and how it might be useful in your research? This high level overview introduces deep learning ‘in a nutshell’ and provides tips on which concepts and skills you will need to know to build a deep learning application. The presentation also provides pointers to various resources you can use to get started in deep learning.
The webinar is followed by a short Q&A session.
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
Getting Started with Deep Learning - Slides (PDF): Slides used in the presentation
Materials shared elsewhere:
A recording of the webinar is available on the Australian BioCommons YouTube Channel:
https://youtu.be/I1TmpnZUuiQ
Melissa Burke (melissa@biocommons.org.au)
Tang, Titus (orcid: 0000-0001-7496-1152)
Deep learning, Neural networks, Machine learning
WEBINAR: Pro tips for scaling bioinformatics workflows to HPC
This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023.
Event description
High Performance Computing (HPC) infrastructures offer the computational scale and...
Keywords: Bioinformatics, Workflows, HPC, High Performance Computing
WEBINAR: Pro tips for scaling bioinformatics workflows to HPC
https://zenodo.org/records/8008227
https://dresa.org.au/materials/webinar-pro-tips-for-scaling-bioinformatics-workflows-to-hpc-9f2a8b90-88da-433b-83b2-b1ab262dd9df
This record includes training materials associated with the Australian BioCommons webinar ‘Pro tips for scaling bioinformatics workflows to HPC’. This webinar took place on 31 May 2023.
Event description
High Performance Computing (HPC) infrastructures offer the computational scale and efficiency that life scientists need to handle complex biological datasets and multi-step computational workflows. But scaling workflows to HPC from smaller, more familiar computational infrastructures brings with it new jargon, expectations, and processes to learn. To make the most of HPC resources, bioinformatics workflows need to be designed for distributed computing environments and carefully manage varying resource requirements, and data scale related to biology.
In this webinar, Dr Georgina Samaha from the Sydney Informatics Hub, Dr Matthew Downton from the National Computational Infrastructure (NCI) and Dr Sarah Beecroft from the Pawsey Supercomputing Research Centre help you navigate the world of HPC for running and developing bioinformatics workflows. They explain when you should take your workflows to HPC and highlight the architectural features you should make the most of to scale your analyses once you’re there. You’ll hear pro-tips for dealing with common pain points like software installation, optimising for parallel computing and resource management, and will find out how to get access to Australia’s National HPC infrastructures at NCI and Pawsey.
Materials
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
Pro-tips_HPC_Slides: A PDF copy of the slides presented during the webinar.
Materials shared elsewhere:
A recording of this webinar is available on the Australian BioCommons YouTube Channel: https://youtu.be/YKJDRXCmGMo
Melissa Burke (melissa@biocommons.org.au)
Samaha, Georgina (orcid: 0000-0003-0419-1476)
Beecroft, Sarah (orcid: 0000-0002-3935-2279)
Downton, Matthew (orcid: 0000-0002-4693-1965)
Bioinformatics, Workflows, HPC, High Performance Computing
WEBINAR: AlphaFold: what's in it for me?
This record includes training materials associated with the Australian BioCommons webinar ‘WEBINAR: AlphaFold: what’s in it for me?’. This webinar took place on 18 April 2023.
Event description
AlphaFold has taken the scientific world by storm with the ability to accurately predict the...
Keywords: Bioinformatics, Machine Learning, Structural Biology, Proteins, Drug discovery, AlphaFold, AI, Artificial Intelligence, Deep learning
WEBINAR: AlphaFold: what's in it for me?
https://zenodo.org/records/7865494
https://dresa.org.au/materials/webinar-alphafold-what-s-in-it-for-me-4d1ea222-4240-4b68-b9ae-7769ac664ee0
This record includes training materials associated with the Australian BioCommons webinar ‘WEBINAR: AlphaFold: what’s in it for me?’. This webinar took place on 18 April 2023.
Event description
AlphaFold has taken the scientific world by storm with the ability to accurately predict the structure of any protein in minutes using artificial intelligence (AI). From drug discovery to enzymes that degrade plastics, this promises to speed up and fundamentally change the way that protein structures are used in biological research.
Beyond the hype, what does this mean for structural biology as a field (and as a career)?
Dr Craig Morton, Drug Discovery Lead at the CSIRO, is an early adopter of AlphaFold and has decades of expertise in protein structure / function, protein modelling, protein – ligand interactions and computational small molecule drug discovery, with particular interest in anti-infective agents for the treatment of bacterial and viral diseases.
Craig joins this webinar to share his perspective on the implications of AlphaFold for science and structural biology. He will give an overview of how AlphaFold works, ways to access AlphaFold, and some examples of how it can be used for protein structure/function analysis.
Materials are shared under a Creative Commons Attribution 4.0 International agreement unless otherwise specified and were current at the time of the event.
Files and materials included in this record:
Event metadata (PDF): Information about the event including, description, event URL, learning objectives, prerequisites, technical requirements etc.
Index of training materials (PDF): List and description of all materials associated with this event including the name, format, location and a brief description of each file.
Materials shared elsewhere:
A recording of this webinar is available on the Australian BioCommons YouTube Channel:
https://youtu.be/4ytn2_AiH8s
Melissa Burke (melissa@biocommons.org.au)
Morton, Craig (orcid: 0000-0001-5452-5193)
Bioinformatics, Machine Learning, Structural Biology, Proteins, Drug discovery, AlphaFold, AI, Artificial Intelligence, Deep learning
Accelerating skills development in Data science and AI at scale
At the Monash Data Science and AI platform, we believe that upskilling our research community and building a workforce with data science skills are key to accelerating the application of data science in research. To achieve this, we create and leverage new and existing training capabilities...
Keywords: AI, machine learning, eresearch skills, training, train the trainer, volunteer instructors, training partnerships, training material
Accelerating skills development in Data science and AI at scale
https://zenodo.org/records/4287746
https://dresa.org.au/materials/accelerating-skills-development-in-data-science-and-ai-at-scale-2d8a65fa-f96e-44ad-a026-cfae3f38d128
At the Monash Data Science and AI platform, we believe that upskilling our research community and building a workforce with data science skills are key to accelerating the application of data science in research. To achieve this, we create and leverage new and existing training capabilities within and outside Monash University. In this talk, we will discuss the principles and purpose of establishing collaborative models to accelerate skills development at scale. We will talk about our approach to identifying gaps in the existing skills and training available in data science, key areas of interest as identified by the research community and various sources of training available in the marketplace. We will provide insights into the collaborations we currently have and intend to develop in the future within the university and also nationally.
The talk will also cover our approach as outlined below
• Combined survey of gaps in skills and trainings for Data science and AI
• Provide seats to partners
• Share associate instructors/helpers/volunteers
• Develop combined training materials
• Publish a repository of open source trainings
• Train the trainer activities
• Establish a network of volunteers to deliver trainings at their local regions
Industry plays a significant role in making some invaluable training available to the research community either through self learning platforms like AWS Machine Learning University or Instructor led courses like NVIDIA Deep Learning Institute. We will discuss how we leverage our partnerships with Industry to bring these trainings to our research community.
Finally, we will discuss how we map our training to the ARDC skills roadmap and how the ARDC platforms project “Environments to accelerate Machine Learning based Discovery” has enabled collaboration between Monash University and University of Queensland to develop and deliver training together.
contact@ardc.edu.au
Tang, Titus
AI, machine learning, eresearch skills, training, train the trainer, volunteer instructors, training partnerships, training material
Monash University - University of Queensland training partnership in Data science and AI
We describe the peer network exchange for training that has been recently created via an ARDC funded partnership between Monash University and Universities of Queensland under the umbrella of the Queensland Cyber Infrastructure Foundation (QCIF). As part of a training program in machine learning,...
Keywords: data skills, training partnerships, data science, AI, training material
Monash University - University of Queensland training partnership in Data science and AI
https://zenodo.org/records/4287864
https://dresa.org.au/materials/monash-university-university-of-queensland-training-partnership-in-data-science-and-ai-8082bf73-d20f-4214-ad8c-95123e25a36c
We describe the peer network exchange for training that has been recently created via an ARDC funded partnership between Monash University and Universities of Queensland under the umbrella of the Queensland Cyber Infrastructure Foundation (QCIF). As part of a training program in machine learning, visualisation, and computing tools, we have established a series of over 20 workshops over the year where either Monash or QCIF hosts the event for some 20-40 of their researchers and students, while some 5 places are offered to participants from the other institution. In the longer term we aim to share material developed at one institution and have trainers present it at the other. In this talk we will describe the many benefits we have found to this approach including access to a wider range of expertise in several rapidly developing fields, upskilling of trainers, faster identification of emerging training needs, and peer learning for trainers.
contact@ardc.edu.au
Tang, Titus
data skills, training partnerships, data science, AI, training material
Pawsey: AWS Quantum 101 Using Amazon Braket
Join us as AWS Quantum Specialists introduce quantum simulators and gate-based quantum computers, before turning to more advanced topics.
Keywords: Pawsey Supercomputing Centre, AWS, quantum, HPC
Pawsey: AWS Quantum 101 Using Amazon Braket
https://www.youtube.com/playlist?list=PLmu61dgAX-abDLr86-bG8zqfBIffu6Eh2
https://dresa.org.au/materials/pawsey-aws-quantum-101-using-amazon-braket
Join us as AWS Quantum Specialists introduce quantum simulators and gate-based quantum computers, before turning to more advanced topics.
training@pawsey.org.au
Pawsey Supercomputing Research Centre
Pawsey Supercomputing Centre, AWS, quantum, HPC
PCon Preparing applications for El Capitan and beyond
As Lawrence Livermore National Laboratories (LLNL) prepares to stand up its next supercomputer, El Capitan, application teams prepare to pivot to another GPU architecture.
This talk presents how the LLNL application teams made the transition from distributed-memory, CPU-only architectures to...
Keywords: GPUs, supercomputing, HPC, PaCER
PCon Preparing applications for El Capitan and beyond
https://www.youtube.com/watch?v=cj7a7gWgt8o&list=PLmu61dgAX-aZ_aa6SmmExSJtXGS7L_BF9&index=4
https://dresa.org.au/materials/pcon-preparing-applications-for-el-capitan-and-beyond
As Lawrence Livermore National Laboratories (LLNL) prepares to stand up its next supercomputer, El Capitan, application teams prepare to pivot to another GPU architecture.
This talk presents how the LLNL application teams made the transition from distributed-memory, CPU-only architectures to GPUs. They share institutional best practices. They discuss new open-source software products as tools for porting and profiling applications and as avenues for collaboration across the computational science community.
Join LLNL's Erik Draeger and Jane Herriman, who presented this talk at Pawsey's PaCER Conference in September 2023.
training@pawsey.org.au
Erik Draeger
Jane Herriman
Pawsey Supercomputing Research Centre
GPUs, supercomputing, HPC, PaCER
masters
phd
researcher
ecr
support
professional
ugrad
From PC to Cloud or High Performance Computing
Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well.
We may refer to both types as “large scale computing” – but...
From PC to Cloud or High Performance Computing
https://intersect.org.au/training/course/compute001
https://dresa.org.au/materials/from-pc-to-cloud-or-high-performance-computing
Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well.
We may refer to both types as “large scale computing” – but what is the difference? Both systems target scalability of computing, but in different ways.
This webinar will give a good overview to the researchers thinking to make a move from their local computer to Cloud of High Performance Computing Cluster.
Introduction
HPC vs Cloud computing
When to use HPC
When to use the Cloud
The Cloud – Pros and Cons
HPC – Pros and Cons
The webinar has no prerequisites.
training@intersect.org.au
Intersect Australia
HPC
Getting started with HPC using PBS Pro
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...
Getting started with HPC using PBS Pro
https://intersect.org.au/training/course/hpc201
https://dresa.org.au/materials/getting-started-with-hpc-using-pbs-pro
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free?
High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using PBS Pro.
Connect to an HPC cluster
Use the Unix command line to operate a remote computer and create job scripts
Submit and manage jobs on a cluster using a scheduler
Transfer files to and from a remote computer
Use software through environment modules
Use parallelisation to speed up data analysis
Access the facilities available to you as a researcher
This is the PBS Pro version of the Getting Started with HPC course.
This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course.
training@intersect.org.au
Intersect Australia
HPC
Getting started with HPC using Slurm
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...
Getting started with HPC using Slurm
https://intersect.org.au/training/course/hpc202
https://dresa.org.au/materials/getting-started-with-hpc-using-slurm
Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free?
High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using Slurm.
Connect to an HPC cluster
Use the Unix command line to operate a remote computer and create job scripts
Submit and manage jobs on a cluster using a scheduler
Transfer files to and from a remote computer
Use software through environment modules
Use parallelisation to speed up data analysis
Access the facilities available to you as a researcher
This is the Slurm version of the Getting Started with HPC course.
This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course.
training@intersect.org.au
Intersect Australia
HPC
Parallel Programming for HPC
You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs.
Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance...
Parallel Programming for HPC
https://intersect.org.au/training/course/hpc301
https://dresa.org.au/materials/parallel-programming-for-hpc
You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs.
Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance Computing.
OpenMP (Open Multi-Processing): a widespread method for shared memory programming
MPI (Message Passing Interface): a leading distributed memory programming model
To do this course you need to have:
A good working knowledge of HPC. Consider taking our
Getting Started with HPC using PBS Pro course to come up to speed beforehand.
Prior experience of writing programs in either C or Fortran.
training@intersect.org.au
Intersect Australia
HPC
HPC file systems and what users need to consider for appropriate and efficient usage
Three videos on miscellaneous aspects of HPC usage - useful reference for new users of HPC systems.
1 – General overview of different file systems that might be available on HPC. The video goes through shared file systems such as /home and /scratch, local compute node file systems (local...
Keywords: HPC, high performance computer, File systems
Resource type: video, presentation
HPC file systems and what users need to consider for appropriate and efficient usage
https://www.youtube.com/watch?v=cNW7F9V1plA&list=PLjlLx279X4yO62jHF4rd7I9iEfbnz3Ts1
https://dresa.org.au/materials/hpc-file-systems-and-what-users-need-to-consider-for-appropriate-and-efficient-usage
Three videos on miscellaneous aspects of HPC usage - useful reference for new users of HPC systems.
1 – General overview of different file systems that might be available on HPC. The video goes through shared file systems such as /home and /scratch, local compute node file systems (local scratch or $TMPDIR) and storage file system. It outlines what users need to consider if they wish to use any of these in their workflows.
2 – Overview of the different directories that might be present on HPC. These could include /home, /scratch, /opt, /lib and lib64, /sw and others.
3 – Overview of the Message-of-the-day file and the message that is displayed to users every time they log in. This displays info about general help and often current problems or upcoming outages.
QCIF Training (training@qcif.edu.au)
Marlies Hankel
HPC, high performance computer, File systems
Basic Linux/Unix commands
A series of eight videos (each between 5 and 10 minutes long) following the content of the Software Carpentry workshop "The Unix Shell".
Sessions 1, 2 and 3 provide instructions on the minimal level of Linux/Unix commands recommended for new...
Keywords: HPC, high performance computer, Unix, Linux, Software Carpentry
Resource type: video, guide
Basic Linux/Unix commands
https://www.youtube.com/playlist?list=PLjlLx279X4yP5GodfbqQTJuJ1S9EJU3GM
https://dresa.org.au/materials/basic-linux-unix-commands
A series of eight videos (each between 5 and 10 minutes long) following the content of the Software Carpentry workshop ["The Unix Shell"](https://swcarpentry.github.io/shell-novice/).
Sessions 1, 2 and 3 provide instructions on the minimal level of Linux/Unix commands recommended for new users of HPC.
1 – An overview of how to find out where a user is in the filesystem, list the files there, and how to get help on Unix commands
2 – How to move around the file system and change into other directories
3 – Explains the difference between an absolute and relative path
4 – Overview of how to create new directories, and to create and edit new files with nano
5 – How to use the vi editor to edit files
6 – Overview of file viewers available
7 – How to copy and move files and directories
8 – How to remove files and directories
Further details and exercises with solutions can be found on the Software Carpentry "The Unix Shell" page (https://swcarpentry.github.io/shell-novice/)
QCIF Training (training@qcif.edu.au)
Marlies Hankel
HPC, high performance computer, Unix, Linux, Software Carpentry
Transferring files and data
A short video outlining the basics on how to use FileZilla to establish a secure file transfer protocol (sftp) connection to HPC to use a drag and drop interface to transfer files between the HPC and a desktop computer.
Keywords: sftp, file transfer, HPC, high performance computer
Resource type: video, guide
Transferring files and data
https://www.youtube.com/watch?v=9ABMxcKqfkQ&list=PLjlLx279X4yP3eTLu0S6nOt0HQ7XRf6WF
https://dresa.org.au/materials/transferring-files-and-data
A short video outlining the basics on how to use FileZilla to establish a secure file transfer protocol (sftp) connection to HPC to use a drag and drop interface to transfer files between the HPC and a desktop computer.
QCIF Training (training@qcif.edu.au)
Marlies Hankel
sftp, file transfer, HPC, high performance computer
Connecting to HPC
A series of three short videos introducing how to use PuTTY to connect from a Windows PC to a secure HPC (high performance computing) cluster.
1 - The very basics on how to establish a connection to HPC.
2 - How to add more specific options for the connection to HPC.
3 - How to save the...
Keywords: HPC, high performance computer, ssh
Resource type: video, guide
Connecting to HPC
https://www.youtube.com/playlist?list=PLjlLx279X4yPJBVQuIRhz1CVMfQpTuvZW
https://dresa.org.au/materials/connecting-to-hpc
A series of three short videos introducing how to use PuTTY to connect from a Windows PC to a secure HPC (high performance computing) cluster.
1 - The very basics on how to establish a connection to HPC.
2 - How to add more specific options for the connection to HPC.
3 - How to save the details and options for a connection for future use.
QCIF Training (training@qcif.edu.au)
Marlies Hankel
HPC, high performance computer, ssh
Deep Learning for Natural Language Processing
This workshop is designed to be instructor led and consists of two parts.
Part 1 consists of a lecture-demo about text processing and a hands-on session for attendees to learn how to clean a dataset.
Part 2 consists of a lecture introducing Recurrent Neural Networks and a hands-on session for...
Keywords: Deep learning, NLP, Machine learning
Resource type: presentation, tutorial
Deep Learning for Natural Language Processing
https://doi.org/10.26180/13100513
https://dresa.org.au/materials/deep-learning-for-natural-language-processing
This workshop is designed to be instructor led and consists of two parts.
Part 1 consists of a lecture-demo about text processing and a hands-on session for attendees to learn how to clean a dataset.
Part 2 consists of a lecture introducing Recurrent Neural Networks and a hands-on session for attendees to train their own RNN.
The Powerpoints contain the lecture slides, while the Jupyter notebooks (.ipynb) contain the hands-on coding exercises.
This workshop introduces natural language as data for deep learning. We discuss various techniques and software packages (e.g. python strings, RegEx, NLTK, Word2Vec) that help us convert, clean, and formalise text data “in the wild” for use in a deep learning model. We then explore the training and testing of a Recurrent Neural Network on the data to complete a real world task. We will be using TensorFlow v2 for this purpose.
datascienceplatform@monash.edu
Titus Tang
Deep learning, NLP, Machine learning
Introduction to Deep Learning and TensorFlow
This workshop is intended to run as an instructor guided live event and consists of two parts. Each part consists of a lecture and a hands-on coding exercise.
Part 1 - Introduction to Deep Learning and TensorFlow
Part 2 - Introduction to Convolutional Neural Networks
The Powerpoints contain...
Keywords: Deep learning, convolutional neural network, tensorflow, Machine learning
Resource type: presentation, tutorial
Introduction to Deep Learning and TensorFlow
https://doi.org/10.26180/13100519
https://dresa.org.au/materials/introduction-to-deep-learning-and-tensorflow
This workshop is intended to run as an instructor guided live event and consists of two parts. Each part consists of a lecture and a hands-on coding exercise.
Part 1 - Introduction to Deep Learning and TensorFlow
Part 2 - Introduction to Convolutional Neural Networks
The Powerpoints contain the lecture slides, while the Jupyter notebooks (.ipynb) contain the hands-on coding exercises.
This workshop is an introduction to how deep learning works and how you could create a neural network using TensorFlow v2. We start by learning the basics of deep learning including what a neural network is, how information passes through the network, and how the network learns from data through the automated process of gradient descent. Workshop attendees would build, train and evaluate a neural network using a cloud GPU (Google Colab).
In part 2, we look at image data and how we could train a convolution neural network to classify images. Workshop attendees will extend their knowledge from the first part to design, train and evaluate this convolutional neural network.
datascienceplatform@monash.edu
Titus Tang
Deep learning, convolutional neural network, tensorflow, Machine learning