1. Home
  2. Materials
Register training material
273 materials found
Collecting Web Data

Web scraping is a technique for extracting information from websites. This can be done manually but it is usually faster, more efficient and less error-prone if it can be automated.

Web scraping allows you to convert non-tabular or poorly structured data into a usable, structured format,...

Keywords: Python

Collecting Web Data https://dresa.org.au/materials/collecting-web-data Web scraping is a technique for extracting information from websites. This can be done manually but it is usually faster, more efficient and less error-prone if it can be automated. Web scraping allows you to convert non-tabular or poorly structured data into a usable, structured format, such as a .csv file or spreadsheet. But scraping is about more than just acquiring data: it can help you track changes to data online, and help you archive data. In short, it’s a skill worth learning. So join us for this web scraping workshop to learn web scraping, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. The concept of structured data The use of XPath queries on HTML document How to scrape data using browser extensions How to scrape using Python and Scrapy How to automate the scraping of multiple web pages A good knowledge of the basic concepts and techniques in Python. Consider taking our \Learn to Program: Python\ and \Python for Research\ courses to come up to speed beforehand. training@intersect.org.au Python
Mastering text with Regular Expressions

Have you ever wanted to extract phone numbers out of a block of unstructured text? Or email addresses. Or find all the words that start with “e” and end with “ed”, no matter their length? Or search through DNA sequences for a pattern? Or extract coordinates from GPS data?

Regular...

Keywords: Regular Expressions

Mastering text with Regular Expressions https://dresa.org.au/materials/mastering-text-with-regular-expressions Have you ever wanted to extract phone numbers out of a block of unstructured text? Or email addresses. Or find all the words that start with “e” and end with “ed”, no matter their length? Or search through DNA sequences for a pattern? Or extract coordinates from GPS data? Regular Expressions (regexes) are a powerful way to handle a multitude of different types of data. They can be used to find patterns in text and make sophisticated replacements. Think of them as find and replace on steroids. Come along to this workshop to learn what they can do and how to apply them to your research. Comprehend and apply the syntax of regular expressions Use the http://regexr.com tool to test a regular expression against some text Construct simple regular expressions to find capitalised words; all numbers; all words that start with a specific set of letters, etc. in a block of text Craft and test a progressively more complex regular expression Find helpful resources covering regular expressions on the web Comprehend and apply the syntax of regular expressions Use the http://regexr.com tool to test a regular expression against some text Construct simple regular expressions to find capitalised words; all numbers; all words that start with a specific set of letters, etc. in a block of text Craft and test a progressively more complex regular expression Find helpful resources covering regular expressions on the web training@intersect.org.au Regular Expressions
Unix Shell and Command Line Basics

The Unix environment is incredibly powerful but quite daunting to the newcomer. Command line confidence unlocks powerful computing resources beyond the desktop, including virtual machines and High Performance Computing. It enables repetitive tasks to be automated. And it comes with a swag of...

Keywords: Unix

Unix Shell and Command Line Basics https://dresa.org.au/materials/unix-shell-and-command-line-basics The Unix environment is incredibly powerful but quite daunting to the newcomer. Command line confidence unlocks powerful computing resources beyond the desktop, including virtual machines and High Performance Computing. It enables repetitive tasks to be automated. And it comes with a swag of handy tools that can be combined in powerful ways. Getting started is the hardest part, but our helpful instructors are there to demystify Unix as you get to work running programs and writing scripts on the command line. Every attendee is given a dedicated training environment for the duration of the workshop, with all software and data fully loaded and ready to run. We teach this course within a GNU/Linux environment. This is best characterised as a Unix-like environment. We teach how to run commands within the Bash Shell. The skills you’ll learn at this course are generally transferable to other Unix environments. Navigate and work with files and directories (folders) Use a selection of essential tools Combine data and tools to build a processing workflow Automate repetitive analysis using the command line The course has no prerequisites. training@intersect.org.au Unix
Data Manipulation and Visualisation in R

R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework.

In this workshop, you will learn how to manipulate, explore and get insights...

Keywords: R

Data Manipulation and Visualisation in R https://dresa.org.au/materials/data-manipulation-and-visualisation-in-r R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework. In this workshop, you will learn how to manipulate, explore and get insights from your data (Data Manipulation using the dplyr package), as well as how to convert your data from one format to another (Data Transformation using the tidyr package). You will also explore different types of graphs and learn how to customise them using one of the most popular plotting packages in R, ggplot2 (Data Visualisation). We teach using RStudio, which allows program code, results, visualisations and documentation to be blended seamlessly. Join us for a live coding workshop where we write programs that produce results, using the researcher-focused training modules from Intersect and the highly regarded Software Carpentry Foundation. DataFrame Manipulation using the dplyr package DataFrame Transformation using the tidyr package Using the Grammar of Graphics to convert data into figures using the ggplot2 package Configuring plot elements within ggplot2 Exploring different types of plots using ggplot2 Either \Learn to Program: R\ or \Learn to Program: R\ and \R for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: R\ and \R for Research\ courses to ensure that you are familiar with the knowledge needed for this course. training@intersect.org.au R
Traversing t tests in R

R has become a popular programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework.

The primary goal of this workshop is to familiarise you with basic statistical concepts in R...

Keywords: R

Traversing t tests in R https://dresa.org.au/materials/traversing-t-tests-in-r R has become a popular programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework. The primary goal of this workshop is to familiarise you with basic statistical concepts in R from reading in and manipulating data, checking assumptions, statistical tests and visualisations. This is not an advanced statistics course, but is instead designed to gently introduce you to statistical comparisons and hypothesis testing in R. Read in and manipulate data Check assumptions of t tests Perform one-sample t tests Perform two-sample t tests (Independent-samples, Paired-samples) Perform nonparametric t tests (One-sample Wilcoxon Signed Rank test, Independent-samples Mann-Whitney U test) This course assumes familiarity with R and RStudio. You should have a good understanding of R syntax and basic programming concepts. Please consider attending Intersect’s following courses to get up to speed: \Learn to Program: R\, \Data Manipulation and Visualisation in R\ training@intersect.org.au R
Introduction to Machine Learning using R: Classification

Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore...

Keywords: R

Introduction to Machine Learning using R: Classification https://dresa.org.au/materials/introduction-to-machine-learning-using-r-classification Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore the fundamentals of Machine Learning from a practical perspective with the help of the R programming language and its scientific computing packages. Comprehensive introduction to Machine Learning models and techniques such as Logistic Regression, Decision Trees and Ensemble Learning. Know the differences between various core Machine Learning models. Understand the Machine Learning modelling workflows. Use R and its relevant packages to process real datasets, train and apply Machine Learning models. \\Either \Learn to Program: R\ and \Data Manipulation in R\ or \Learn to Program: R\ and \Data Manipulation and Visualisation in R\needed to attend this course. If you already have experience with programming, please check the topics covered in courses above and \Introduction to ML using R: Introduction & Linear Regression\ to ensure that you are familiar with the knowledge needed for this course, such as good understanding of R syntax and basic programming concepts, familiarity with dplyr, tidyr and ggplot2 packages, and basic understanding of Machine Learning and Model Training.\\Maths knowledge is not required. There are only a few Math formula that you are going to see in this course, however references to Mathematics required for learning about Machine Learning will be provided. Having an understanding of the Mathematics behind each Machine Learning algorithms is going to make you appreciate the behaviour of the model and know its pros/cons when using them.\\ training@intersect.org.au R
Data Visualisation in R

R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework.

In this workshop, you will explore different types of graphs and learn how to...

Keywords: R

Data Visualisation in R https://dresa.org.au/materials/data-visualisation-in-r R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework. In this workshop, you will explore different types of graphs and learn how to customise them using one of the most popular plotting packages in R, ggplot2 (Data Visualisation). We teach using RStudio, which allows program code, results, visualisations and documentation to be blended seamlessly. Join us for a live coding workshop where we write programs that produce results, using the researcher-focused training modules from Intersect and the highly regarded Software Carpentry Foundation. Using the Grammar of Graphics to convert data into figures using the ggplot2 package Configuring plot elements within ggplot2 Exploring different types of plots using ggplot2 Either \Learn to Program: R\ or \Learn to Program: R\ and \R for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: R\ and \R for Research\ courses to ensure that you are familiar with the knowledge needed for this course. We also strongly recommend attending the \Data Manipulation in R\ course. training@intersect.org.au R
Exploring Chi-square and correlation in R

This hands-on training is designed to familiarise you with the data analysis environment of the R programming. In this session, we will traverse into the realm of inferential statistics, beginning with correlation and reliability. We will present a brief conceptual overview and the R procedures...

Keywords: R

Exploring Chi-square and correlation in R https://dresa.org.au/materials/exploring-chi-square-and-correlation-in-r This hands-on training is designed to familiarise you with the data analysis environment of the R programming. In this session, we will traverse into the realm of inferential statistics, beginning with correlation and reliability. We will present a brief conceptual overview and the R procedures for computing reliability and correlation (Pearson’s r, Spearman’s Rho and Kendall’s tau) in real world datasets. Obtain inferential statistics and assess data normality Manipulate data and create graphs Perform Chi-Square tests (Goodness of Fit test and Test of Independence) Perform correlations on continuous and categorical data (Pearson’s r, Spearman’s Rho and Kendall’s tau) This course assumes familiarity with R and RStudio. You should have a good understanding of R syntax and basic programming concepts, as well as familiarity with data manipulation (dplyr) and visualisation (ggplot2 package).  Please consider attending Intersect’s following courses to get up to speed: \Learn to Program: R\, \Data Manipulation and Visualisation in R\ training@intersect.org.au R
Regular Expressions on the Command Line

Would you like to use regular expressions with the classic command line utilities find, grep, sed and awk? These venerable Unix utilities allow you to search, filter and transform large amounts of text (including many common data formats) efficiently and repeatably.

find to locate files and...

Keywords: Regular Expressions

Regular Expressions on the Command Line https://dresa.org.au/materials/regular-expressions-on-the-command-line Would you like to use regular expressions with the classic command line utilities find, grep, sed and awk? These venerable Unix utilities allow you to search, filter and transform large amounts of text (including many common data formats) efficiently and repeatably. find to locate files and directories matching regexes. grep to filter lines in files based on pattern matches. sed to find and replace using regular expressions and captures. awk to work with row- and column-oriented data. This course assumes prior knowledge of the basic syntax of regular expressions. If you’re new to regular expressions or would like a refresher, take our Mastering text with Regular Expressions course first. This course also assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. Take our Unix Shell and Command Line Basics course to get up to speed quickly. training@intersect.org.au Regular Expressions
Learn to Program: Python

Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you'd expect of a modern programming language, and also a rich set of libraries for working with data.

We teach using Jupyter notebooks, which allow program code, results,...

Keywords: Programming, Python

Learn to Program: Python https://dresa.org.au/materials/learn-to-program-python Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you'd expect of a modern programming language, and also a rich set of libraries for working with data. We teach using Jupyter notebooks, which allow program code, results, visualisations and documentation to be blended seamlessly. Perfect for sharing insights with others while producing reproducible research. Join us for this live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. #### You'll learn: - Introduction to the JupyterLab interface for programming - Basic syntax and data types in Python - How to load external data into Python - Creating functions (FUNCTIONS) - Repeating actions and analysing multiple data sets (LOOPS) - Making choices (IF STATEMENTS - CONDITIONALS) - Ways to visualise data in Python #### Prerequisites: No prior experience with programming is needed to attend this course. We strongly recommend attending the Start Coding without Hesitation: Programming Languages Showdown and Thinking like a computer: The Fundamentals of Programming webinars. Recordings of previously delivered webinars can be found [here](https://intersect.org.au/training/webinars/). **For more information, please click [here](https://intersect.org.au/training/course/python101).** training@intersect.org.au Programming, Python
Data Manipulation in R

R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework.

In this workshop, you will learn how to manipulate, explore and get insights...

Keywords: R

Data Manipulation in R https://dresa.org.au/materials/data-manipulation-in-r R is quickly gaining popularity as a programming language for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio and the Shiny web application framework. In this workshop, you will learn how to manipulate, explore and get insights from your data (Data Manipulation using the dplyr package), as well as how to convert your data from one format to another (Data Transformation using the tidyr package). We teach using RStudio, which allows program code, results, visualisations and documentation to be blended seamlessly. Join us for a live coding workshop where we write programs that produce results, using the researcher-focused training modules from Intersect and the highly regarded Software Carpentry Foundation. DataFrame Manipulation using the dplyr package DataFrame Transformation using the tidyr package Either \Learn to Program: R\ or \Learn to Program: R\ and \R for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: R\ and \R for Research\ courses to ensure that you are familiar with the knowledge needed for this course. training@intersect.org.au R
Excel for Researchers

Data rarely comes in the form you require. Often it is messy. Sometimes it is incomplete. And sometimes there’s too much of it. Frequently, it has errors. We’ll use one of the most widespread data wrangling tools, Microsoft Excel, to import, sort, filter, copy, protect, transform, summarise,...

Keywords: Excel

Excel for Researchers https://dresa.org.au/materials/excel-for-researchers Data rarely comes in the form you require. Often it is messy. Sometimes it is incomplete. And sometimes there’s too much of it. Frequently, it has errors. We’ll use one of the most widespread data wrangling tools, Microsoft Excel, to import, sort, filter, copy, protect, transform, summarise, merge, and visualise research data. While aimed at novice Excel users, most attendees will walk away with new tricks to work more efficiently with their research data. ‘Clean up’ messy research data Organise, format and name your data Interpret your data (SORTING, FILTERING, CONDITIONAL FORMATTING) Perform calculations on your data using functions (MAX, MIN, AVERAGE) Extract significant findings from your data (PIVOT TABLE, VLOOKUP) Manipulate your data (convert data format, work with DATES and TIMES) Create graphs and charts to visualise your data (CHARTS) Handy tips to speed up your work In order to participate, attendees must have a licensed copy of Microsoft Excel installed on their computer. Speak to your local university IT or Research Office for assistance in obtaining a license and installing the software.   training@intersect.org.au Excel
From PC to Cloud or High Performance Computing

Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well.

We may refer to both types as “large scale computing” – but...

Keywords: HPC

From PC to Cloud or High Performance Computing https://dresa.org.au/materials/from-pc-to-cloud-or-high-performance-computing Most of you would have heard of Cloud and High Performance Computing (HPC), or you may already be using it. HPC is not the same as cloud computing. Both technologies differ in a number of ways, and have some similarities as well. We may refer to both types as “large scale computing” – but what is the difference? Both systems target scalability of computing, but in different ways. This webinar will give a good overview to the researchers thinking to make a move from their local computer to Cloud of High Performance Computing Cluster. Introduction HPC vs Cloud computing When to use HPC When to use the Cloud The Cloud – Pros and Cons HPC – Pros and Cons The webinar has no prerequisites. training@intersect.org.au HPC
Learn to Program: R

R is quickly gaining popularity as a programming language of choice for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio development environment and the Shiny web application framework.

But getting started with R can be...

Keywords: R

Learn to Program: R https://dresa.org.au/materials/learn-to-program-r R is quickly gaining popularity as a programming language of choice for statisticians, data scientists and researchers. It has an excellent ecosystem including the powerful RStudio development environment and the Shiny web application framework. But getting started with R can be challenging, particularly if you’ve never programmed before. That’s where this introductory course comes in. We teach using RStudio, which allows program code, results, visualisations and documentation to be blended seamlessly. Join us for a live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. Introduction to the RStudio interface for programming Basic syntax and data types in R How to load external data into R Creating functions (FUNCTIONS) Repeating actions and analysing multiple data sets (LOOPS) Making choices (IF STATEMENTS – CONDITIONALS) Ways to visualise data in R No prior experience with programming needed to attend this course. We strongly recommend attending the Start Coding without Hesitation: Programming Languages Showdown and Thinking like a computer: The Fundamentals of Programming webinars. Recordings of previously delivered webinars can be found \here\. training@intersect.org.au R
Data Manipulation and Visualisation in Python

Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data.

In this workshop, you will explore DataFrames in depth (using...

Keywords: Python

Data Manipulation and Visualisation in Python https://dresa.org.au/materials/data-manipulation-and-visualisation-in-python Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data. In this workshop, you will explore DataFrames in depth (using the pandas library), learn how to manipulate, explore and get insights from your data (Data Manipulation), as well as how to deal with missing values and how to combine multiple datasets. You will also explore different types of graphs and learn how to customise them using two of the most popular plotting libraries in Python, matplotlib and seaborn (Data Visualisation). We teach using Jupyter notebooks, which allow program code, results, visualisations and documentation to be blended seamlessly. Perfect for sharing insights with others while producing reproducible research. Join us for this live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. Working with pandas DataFrames Indexing, slicing and subsetting in pandas DataFrames Missing data values Combine multiple pandas DataFrames Using the Grammar of Graphics to convert data into figures using the seaborn and matplotlib libraries Configuring plot elements within seaborn and matplotlib Exploring different types of plots using seaborn Either \Learn to Program: Python\ or \Learn to Program: Python\ and \Python for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: Python\ and \Python for Research\ courses to ensure that you are familiar with the knowledge needed for this course. training@intersect.org.au Python
Introduction to Machine Learning using Python: Classification

Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore...

Keywords: Python

Introduction to Machine Learning using Python: Classification https://dresa.org.au/materials/introduction-to-machine-learning-using-python-classification Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore the fundamentals of Machine Learning from a practical perspective with the help of the Python programming language and its scientific computing libraries. Comprehensive introduction to Machine Learning models and techniques such as Logistic Regression, Decision Trees and Ensemble Learning. Know the differences between various core Machine Learning models. Understand the Machine Learning modelling workflows. Use Python and scikit-learn to process real datasets, train and apply Machine Learning models. Either \Learn to Program: Python\, \Data Manipulation in Python\ and \Introduction to ML using Python: Introduction & Linear Regression\ or \Learn to Program: Python\, \Data Manipulation and Visualisation in Python\ and \Introduction to ML using Python: Introduction & Linear Regression\ needed to attend this course.  If you already have experience with programming, please check the topics covered in courses above to ensure that you are familiar with the knowledge needed for this course, such as good understanding of Python syntax, basic programming concepts and familiarity with Pandas, Numpy and Seaborn libraries, and basic understanding of Machine Learning and Model Training. Maths knowledge is not required. However, there is a few Math formula covered in this course and the references will be provided. Having an understanding of the Mathematics behind each Machine Learning algorithms is going to make you appreciate the behaviour of the model and know its pros/cons when using them. training@intersect.org.au Python
Getting started with HPC using PBS Pro

Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...

Keywords: HPC

Getting started with HPC using PBS Pro https://dresa.org.au/materials/getting-started-with-hpc-using-pbs-pro Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free? High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using PBS Pro. Connect to an HPC cluster Use the Unix command line to operate a remote computer and create job scripts Submit and manage jobs on a cluster using a scheduler Transfer files to and from a remote computer Use software through environment modules Use parallelisation to speed up data analysis Access the facilities available to you as a researcher This is the PBS Pro version of the Getting Started with HPC course. This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course. training@intersect.org.au HPC
Introduction to Machine Learning using Python: SVM & Unsupervised Learning

Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore...

Keywords: Python

Introduction to Machine Learning using Python: SVM & Unsupervised Learning https://dresa.org.au/materials/introduction-to-machine-learning-using-python-svm-unsupervised-learning Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore the fundamentals of Machine Learning from a practical perspective with the help of the Python programming language and its scientific computing libraries. Comprehensive introduction to Machine Learning models and techniques such as Support Vector Machine, K-Nearest Neighbor and Dimensionality Reduction. Know the differences between various core Machine Learning models. Understand the Machine Learning modelling workflows. Use Python and scikit-learn to process real datasets, train and apply Machine Learning models. Either \Learn to Program: Python\, \Data Manipulation in Python\ and \Introduction to ML using Python: Introduction & Linear Regression\ or \Learn to Program: Python\, \Data Manipulation and Visualisation in Python\ and \Introduction to ML using Python: Introduction & Linear Regression\ needed to attend this course.  If you already have experience with programming, please check the topics covered in courses above to ensure that you are familiar with the knowledge needed for this course, such as good understanding of Python syntax, basic programming concepts and familiarity with Pandas, Numpy and Seaborn libraries, and basic understanding of Machine Learning and Model Training. Maths knowledge is not required. However, there is a few Math formula covered in this course and the references will be provided. Having an understanding of the Mathematics behind each Machine Learning algorithms is going to make you appreciate the behaviour of the model and know its pros/cons when using them. training@intersect.org.au Python
Getting started with HPC using Slurm

Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis...

Keywords: HPC

Getting started with HPC using Slurm https://dresa.org.au/materials/getting-started-with-hpc-using-slurm Is your computer’s limited power throttling your research ambitions? Are your analysis scripts pushing your laptop’s processor to its limits? Is your software crashing because you’ve run out of memory? Would you like to unleash to power of the Unix command line to automate and run your analysis on supercomputers that you can access for free? High-Performance Computing (HPC) allows you to accomplish your analysis faster by using many parallel CPUs and huge amounts of memory simultaneously. This course provides a hands on introduction to running software on HPC infrastructure using Slurm. Connect to an HPC cluster Use the Unix command line to operate a remote computer and create job scripts Submit and manage jobs on a cluster using a scheduler Transfer files to and from a remote computer Use software through environment modules Use parallelisation to speed up data analysis Access the facilities available to you as a researcher This is the Slurm version of the Getting Started with HPC course. This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. To come up to speed, consider taking our \Unix Shell and Command Line Basics\ course. training@intersect.org.au HPC
Data Visualisation in Python

Course Materials

Using the Grammar of Graphics to convert data into figures using the seaborn and matplotlib libraries

Configuring plot elements within seaborn and matplotlib

Exploring different types of...

Keywords: Python

Data Visualisation in Python https://dresa.org.au/materials/data-visualisation-in-python [Course Materials](https://intersectaustralia.github.io/training/PYTHON203/sources/Data-Adv_Python.zip) Using the Grammar of Graphics to convert data into figures using the seaborn and matplotlib libraries Configuring plot elements within seaborn and matplotlib Exploring different types of plots using seaborn Either \Learn to Program: Python\ or \Learn to Program: Python\ and \Python for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: Python\ and \Python for Research\ courses to ensure that you are familiar with the knowledge needed for this course. We also strongly recommend attending the \Data Manipulation in Python\. training@intersect.org.au Python
Python for Research

Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data.

This workshop is an introduction to data structures (DataFrames...

Keywords: Python

Python for Research https://dresa.org.au/materials/python-for-research Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data. This workshop is an introduction to data structures (DataFrames using the pandas library) and visualisation (using the matplotlib library) in Python. The targeted audience for this workshop is researchers who are already familiar with the basic concepts in programming such as loops, functions, and conditionals. We teach using Jupyter notebooks, which allow program code, results, visualisations and documentation to be blended seamlessly. Perfect for sharing insights with others while producing reproducible research. Join us for this live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. Introduction to Libraries and Built-in Functions in Python Introduction to DataFrames using the pandas library Reading and writing data in DataFrames Selecting values in DataFrames Quick introduction to Plotting using the matplotlib library \Learn to Program: Python\ or any of the \Learn to Program: R\, \Learn to Program: MATLAB\ or \Learn to Program: Julia\, needed to attend this course. If you already have some experience with programming, please check the topics covered in the \Learn to Program: Python\ course to ensure that you are familiar with the knowledge needed for this course. training@intersect.org.au Python
Data Manipulation in Python

Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data.

In this workshop, you will explore DataFrames in depth (using...

Keywords: Python

Data Manipulation in Python https://dresa.org.au/materials/data-manipulation-in-python Python has deservedly become a popular language for scientific computing. It has all the friendly features and conveniences you’d expect of a modern programming language, and also a rich set of libraries for working with data. In this workshop, you will explore DataFrames in depth (using the pandas library), learn how to manipulate, explore and get insights from your data (Data Manipulation), as well as how to deal with missing values and how to combine multiple datasets. We teach using Jupyter notebooks, which allow program code, results, visualisations and documentation to be blended seamlessly. Perfect for sharing insights with others while producing reproducible research. Join us for this live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. Working with pandas DataFrames Indexing, slicing and subsetting in pandas DataFrames Missing data values Combine multiple pandas DataFrames Either \Learn to Program: Python\ or \Learn to Program: Python\ and \Python for Research\ needed to attend this course. If you already have experience with programming, please check the topics covered in the \Learn to Program: Python\ and \Python for Research\ courses to ensure that you are familiar with the knowledge needed for this course. training@intersect.org.au Python
Introduction to Machine Learning using Python: Introduction & Linear Regression

Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore...

Keywords: Python

Introduction to Machine Learning using Python: Introduction & Linear Regression https://dresa.org.au/materials/introduction-to-machine-learning-using-python-introduction-linear-regression Machine Learning (ML) is a new way to program computers to solve real world problems. It has gained popularity over the last few years by achieving tremendous success in tasks that we believed only humans could solve, from recognising images to self-driving cars. In this course, we will explore the fundamentals of Machine Learning from a practical perspective with the help of the Python programming language and its scientific computing libraries. Understand the difference between supervised and unsupervised Machine Learning. Understand the fundamentals of Machine Learning. Comprehensive introduction to Machine Learning models and techniques such as Linear Regression and Model Training. Understand the Machine Learning modelling workflows. Use Python and scikit-learn to process real datasets, train and apply Machine Learning models Either \Learn to Program: Python\ and \Data Manipulation in Python\ or \Learn to Program: Python\ and \Data Manipulation and Visualisation in Python\ needed to attend this course.  If you already have experience with programming, please check the topics covered in courses above to ensure that you are familiar with the knowledge needed for this course, such as good understanding of Python syntax and basic programming concepts and familiarity with Pandas, Numpy and Seaborn libraries. Maths knowledge is not required. However, there is a few Math formula covered in this course and the references will be provided. Having an understanding of the Mathematics behind each Machine Learning algorithms is going to make you appreciate the behaviour of the model and know its pros/cons when using them. training@intersect.org.au Python
Version Control with Git

Have you mistakenly overwritten programs or data and want to learn techniques to avoid repeating the loss? Version control systems are one of the most powerful tools available for avoiding data loss and enabling reproducible research. While the learning curve can be steep, our trainers are there...

Keywords: Git

Version Control with Git https://dresa.org.au/materials/version-control-with-git Have you mistakenly overwritten programs or data and want to learn techniques to avoid repeating the loss? Version control systems are one of the most powerful tools available for avoiding data loss and enabling reproducible research. While the learning curve can be steep, our trainers are there to answer all your questions while you gain hands on experience in using Git, one of the most popular version control systems available. Join us for this workshop where we cover the fundamentals of version control using the researcher-focused training modules from the highly regarded Software Carpentry Foundation. keep versions of data, scripts, and other files examine commit logs to find which files were changed when restore earlier versions of files compare changes between versions of a file push your versioned files to a remote location, for backup and to facilitate collaboration The course has no prerequisites. training@intersect.org.au Git
Thinking like a computer: The Fundamentals of Programming

Human brains are extremely good at evaluating a small amount of information simultaneously, ignoring anomalies and coming up with an answer to a problem without much in the way of conscious thought. Computers on the other hand are extremely good at performing individual calculations, one at a...

Keywords: Python

Thinking like a computer: The Fundamentals of Programming https://dresa.org.au/materials/thinking-like-a-computer-the-fundamentals-of-programming Human brains are extremely good at evaluating a small amount of information simultaneously, ignoring anomalies and coming up with an answer to a problem without much in the way of conscious thought. Computers on the other hand are extremely good at performing individual calculations, one at a time, and can keep the results in a large bank of short-term memory for quick recall. These two approaches are fundamentally different. Humans can only reasonably retain seven plus or minus two pieces of information in short-term memory, and new items push older items out, whereas a computer is hopeless when given multiple pieces of information simultaneously. Understanding this fact is key to being able to write instructions for computers – also known as programs – in a way that takes advantage of their strengths, and overcomes their drawbacks. Suitable for the programming novice, this webinar is good preparation for researchers wanting to learn how to program. How a human solves tasks How a computer solves tasks Overview of programming concepts: Variables Loops Conditionals Functions Data types The webinar has no prerequisites. training@intersect.org.au Python
Parallel Programming for HPC

You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs.

Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance...

Keywords: HPC

Parallel Programming for HPC https://dresa.org.au/materials/parallel-programming-for-hpc You have written, compiled and run functioning programs in C and/or Fortran. You know how HPC works and you’ve submitted batch jobs. Now you want to move from writing single-threaded programs into the parallel programming paradigm, so you can truly harness the full power of High Performance Computing. OpenMP (Open Multi-Processing): a widespread method for shared memory programming MPI (Message Passing Interface): a leading distributed memory programming model To do this course you need to have: A good working knowledge of HPC. Consider taking our Getting Started with HPC using PBS Pro course to come up to speed beforehand. Prior experience of writing programs in either C or Fortran. training@intersect.org.au HPC
Start Coding without Hesitation: Programming Languages Showdown

Programming is becoming more and more popular, with many researchers using programming to perform data cleaning, data manipulation, data analytics, as well as creating publication quality plots. Programming can be really beneficial for automating processes and workflows. In this webinar, we are...

Keywords: Python, R, Matlab, Julia

Start Coding without Hesitation: Programming Languages Showdown https://dresa.org.au/materials/start-coding-without-hesitation-programming-languages-showdown Programming is becoming more and more popular, with many researchers using programming to perform data cleaning, data manipulation, data analytics, as well as creating publication quality plots. Programming can be really beneficial for automating processes and workflows. In this webinar, we are exploring four of the most popular programming languages that are widely used in academia, namely Python, R, MATLAB, and Julia. Why use Programming An overview of Python, R, MATLAB, and Julia Code comparison of the four programming languages Popularity and job opportunities Intersect’s comparison General guidelines on how to choose the best programming language for your research The webinar has no prerequisites. training@intersect.org.au Python, R, Matlab, Julia
R for Social Scientists

R is quickly gaining popularity as a programming language of choice for researchers. It has an excellent ecosystem including the powerful RStudio development environment.

But getting started with R can be challenging, particularly if you’ve never programmed before. That’s where this...

Keywords: R

R for Social Scientists https://dresa.org.au/materials/r-for-social-scientists R is quickly gaining popularity as a programming language of choice for researchers. It has an excellent ecosystem including the powerful RStudio development environment. But getting started with R can be challenging, particularly if you’ve never programmed before. That’s where this introductory course comes in. Join us for a live coding workshop where we write programs that produce results, using the researcher-focused training modules from the highly regarded Data Carpentry. Basic syntax and data types in R RStudio interface How to import CSV files into R The structure of data frames A brief introduction to data wrangling and data transformation How to calculate summary statistics A brief introduction to visualise data No prior experience with programming needed to attend this course. training@intersect.org.au R
Getting Started with Excel

We rarely receive the research data in an appropriate form. Often data is messy. Sometimes it is incomplete. And sometimes there’s too much of it. Frequently, it has errors. 

This webinar targets beginners and presents a quick demonstration of using the most widespread data wrangling tool,...

Keywords: Excel

Getting Started with Excel https://dresa.org.au/materials/getting-started-with-excel We rarely receive the research data in an appropriate form. Often data is messy. Sometimes it is incomplete. And sometimes there’s too much of it. Frequently, it has errors.  This webinar targets beginners and presents a quick demonstration of using the most widespread data wrangling tool, Microsoft Excel, to sort, filter, copy, protect, transform, aggregate, summarise, and visualise research data. Introduction to Microsoft Excel user interface Interpret data using sorting, filtering, and conditional formatting Summarise data using functions Analyse data using pivot tables Manipulate and visualise data Handy tips to speed up your work The webinar has no prerequisites. training@intersect.org.au Excel
Survey Tools in Research: REDCap and Qualtrics

Now more than ever researchers are needing to embrace electronic data capture methods to keep their research moving in the midst of social distancing restrictions and decreased access to survey participants. Using a research specific survey tool can not only solve this problem, but also set your...

Keywords: REDCap, Qualtrics

Survey Tools in Research: REDCap and Qualtrics https://dresa.org.au/materials/survey-tools-in-research-redcap-and-qualtrics Now more than ever researchers are needing to embrace electronic data capture methods to keep their research moving in the midst of social distancing restrictions and decreased access to survey participants. Using a research specific survey tool can not only solve this problem, but also set your research up for success through intuitive data collection and validation, scheduling and reporting. This webinar will introduce and compare two of the most popular research tools for the collection of survey data and patient records: REDCap and Qualtrics. Electronic Data Capture: Surveys vs Forms Confidential vs Anonymous data collection Strengths and weaknesses of Qualtrics and REDCap Real-life use cases for each tool Using survey tools for longitudinal studies The webinar has no prerequisites. training@intersect.org.au REDCap, Qualtrics