The Unix shell has been around longer than most of its users have been alive. It has survived so long because it’s a power tool that allows people to do complex things with just a few keystrokes. More importantly, it helps them combine existing programs in new ways and automate repetitive tasks so they aren’t typing the same things over and over again. Use of the shell is fundamental to using a wide range of other powerful tools and computing resources (including “high-performance computing” supercomputers). These lessons will start you on a path towards using these resources effectively.
Prerequisites
This lesson guides you through the basics of file systems and the shell. If you have stored files on a computer at all and recognize the word “file” and either “directory” or “folder” (two common words for the same thing), you’re ready for this lesson.
If you’re already comfortable manipulating files and directories, searching for files with
grep
andfind
, and writing simple loops and scripts, you probably won’t learn much from this lesson.
00:00 | Introducing the Shell | What is a command shell and why would I use one? |
00:30 | Navigating Files and Directories |
How can I move around on my computer?
How can I see what files and directories I have? How can I specify the location of a file or directory on my computer? |
01:00 | Working With Files and Directories |
How can I create, copy, and delete files and directories?
How can I edit files? |
01:30 | Pipes and Filters | How can I combine existing commands to do new things? |
02:00 | Loops | How can I perform the same actions on many different files? |
02:30 | Shell Scripts | How can I save and re-use commands? |
03:00 | Finding Things |
How can I find files?
How can I find things in files? |
03:30 | Automating a workflow | How to run a workflow? |
04:30 | Connecting to clusters (ssh, scp) |
What is a cluster?
What HPC resources available at JAX? How do I connect to the cluster? How do I bring data in and out of the cluster? |
04:45 | Queues on HPC |
How do I know what is in the queue?
What are the types of queues available on JAX clusters? How do I find my jobs in the queue? |
04:50 | Using installed software (modules) |
Which tools are installed on the clusters?
How do I load a module installed on the cluster? |
05:00 | Running Jobs on HPC |
How do I submit jobs to the cluster?
How do I request an interactive terminal? |
05:15 | Job Arrays | How do I run similar job in parallel? |
05:30 | Writing scripts to submit jobs to the cluster | How do I write scripts to submit to the cluster? |
05:45 | Finish |
The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.