Introduction to High Performance Computing

The Jackson Laboratory

Online

Sep 29, 2020

9:00 am - 5:00 pm

Instructors: Jason Macklin, Aaron McDivitt, Matt Bradley, Sue McClatchy, Neil Kindlon, David McKenzie

Helpers: Ahmed Sadeque

Some adblockers block the registration window. If you do not see the registration box below, please check your adblocker settings.

General Information

This workshop introduces high performance computing using the Jackson Laboratory’s computing cluster. The goal is to teach novice programmers to use powerful tools and computing resources, and to engage in best practices for using these resources. The emphasis of this workshop is to give attendees a strong foundation in the fundamentals of the bash shell and to teach best practices for scientific computing in high performance computing environments. By the end of this workshop, participants will be able to:

Who: The course is aimed at graduate students and other researchers at the Jackson Laboratory. For IT security reasons, external participants will not be admitted. You don't need to have any previous knowledge of the tools that will be presented at the workshop.

Where: This training will take place online. The instructors will provide you with the information you will need to connect to this meeting.

When: Sep 29, 2020. Add to your Google Calendar.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.). They should have a few specific software packages installed (listed below).

Accessibility: We are dedicated to providing a positive and accessible learning environment for all. Please notify the instructors in advance of the workshop if you require any accommodations or if there is anything we can do to make this workshop more accessible to you.

Contact: Please email susan.mcclatchy@jax.org for more information.

Roles: To learn more about the roles at the workshop (who will be doing what), refer to our Workshop FAQ.


Collaborative Notes

We will use this collaborative document for chatting, taking notes, and sharing URLs and bits of code.



Schedule

Tuesday Sep 29

before Pre-workshop survey
09:00 Workshop overview and introductions
09:15 Why Use a Cluster?
09:45 Connecting to the remote HPC system
10:15 Moving around and looking at things
10:30 Morning break
10:45 Writing and reading files
11:30 Lunch break
12:30 Wildcards and pipes
13:30 Logging in and commands available to the user
14:00 SBATCH options/script
14:30 Batch jobs
15:00 Interactive jobs
15:30 Profiling
16:00 Basic job arrays
16:30 Moving data around (Globus)
16:45 Wrap-up
17:00 END

Syllabus

The Unix Shell

  • Files and Directories
  • History and Tab Completion
  • Pipes and Redirection
  • Looping Over Files
  • Creating and Running Shell Scripts
  • Finding Things
  • Reference...

High Performance Computing

  • Connect to a cluster
  • Use a scheduler to submit and manage jobs
  • Transfer files
  • Use software through environment modules
  • Reference...

Setup

To participate in a workshop, you will need access to the software described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

Install the videoconferencing client

If you haven't used Webex before, ask the JAX IT Help Desk to download and install the Webex client for your computer.

The Bash Shell

Bash is a commonly-used shell that gives you the power to do simple tasks more quickly.

  1. Download and install MobaXterm or PuTTY.
  2. If your "HOME" environment variable is not set (or you don't know what this is):
    1. Open command prompt (Open Start Menu then type cmd and press [Enter])
    2. Type the following line into the command prompt window exactly as shown:

      setx HOME "%USERPROFILE%"

    3. Press [Enter], you should see SUCCESS: Specified value was saved.
    4. Quit command prompt by typing exit then pressing [Enter]

The default shell in some versions of macOS is Bash, and Bash is available in all versions, so no need to install anything. You access Bash from the Terminal (found in /Applications/Utilities). See the Git installation video tutorial for an example on how to open the Terminal. You may want to keep Terminal in your dock for this workshop.

To see if your default shell is Bash type echo $SHELL in Terminal and press the enter/return key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash.

The default shell is usually Bash and there is usually no need to install anything.

To see if your default shell is Bash type echo $SHELL in a terminal and press the enter/return key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash.