CS 450: Operating Systems

Staff

Instructor: Kyle C. Hale
Office Hours: Wed 3:05-5:05 PM (SB 229C)
E-mail: khale [at] cs [dot] iit [dot] edu

TAs:

Amal Rizvi
Office Hours: WF 10-11:30AM (SB 115)
E-mail: arizvi1 [at] hawk [dot] iit [dot] edu

Boyang Li
Office Hours: T 3:00-4:00PM, Th 1:30-3:00PM (SB 007B)
E-mail: bli70 [at] hawk [dot] iit [dot] edu

Course Info

Course number: CS 450

Semester: Spring 2019

Lecture Time: Mon/Wed 1:50PM - 3:05PM

Lecture Location: Pritzker 111

Overview

Operating systems run the world. Without an OS, you couldn't have hundreds of programs running at once, you'd have to have deep understanding of hardware, and you'd have to rewrite your code for each new machine on the market. You wouldn't have web servers, desktop machines, laptops, phones, or smart watches; no Siri, no Alexa, no TensorFlow or CUDA programs.

In this course, you will dive straight into the deepest and often most confounding component of a computer system's software stack. You will become familiar with both the high-level concepts underpinning modern operating systems and with low-level mechanisms, in addition to new and emerging techniques and programming models for computer systems. I am a strong believer in learning by doing, so the majority of your time for this course will be spent working on programming projects, both very low-level work and high-level (user-space) work. This course will be challenging! It will bring together a lot of concepts you've learned in your previous courses, from data structures and algorithms to assembly language, computer architecture, virtual memory, and advanced C programming.

For this course, I assume that you have basic familiarity with computer organization (as introduced in CS 350), data structures, C, and systems programming (as covered in CS 351). You will for sure need to be able to program in C and should be able to navigate around a UNIX environment.

Communication

We will be primarily using Piazza as a course communication mechanism. If you have an issue or question that is not strictly private (especially one that would benefit everyone were it answered), please use Piazza as your first resource. The instructor, the TA, and your fellow classmates will be there to help. Note that you can also post anonymously if you so choose.

Homeworks

While there will be no homework to turn it, it is a very good idea to do some of the homeworks associated with the readings as I post them. These are usually short coding tasks that use a simulator for the OS component in question. These homeworks will help solidify knowledge needed for the exams.

Readings

In addition to the projects (and homeworks if you're studious), you'll be needing to do the readings I post (almost) every lecture. These will primarily come from the OSTEP book, but I will also post other required and optional readings from seminal research papers, articles, and books.

We will, when time allows, also be spending some time reading code! This will likely include code snippets from real OSes including the Linux kernel.

The following is a rough schedule of the class. Note that it is subject to change.

Lecture Schedule

Week	Date	Item	Topic	Notes	Reading
1	Mon 1/14	Lec 1	Intro and Administrivia	slides; p1a out	Required Reading: OSTEP Preface, 1, 2
1	Wed 1/16	Lec 2	Resource Virtualization and Processes	slides	Required Reading: OSTEP 3, 4, 5, 6, System Calls
2	Mon 1/21	holiday	MLK Day (No class)	p1a in, p1b out
2	Wed 1/23	Lec 3	Scheduling	slides	Required Reading: OSTEP 7, 8, Summary, Linux processes, Interrupts (gentle) Optional Reading: A decade of wasted cores, OSTEP: Multi-processor Scheduling, Linux CFS
3	Mon 1/28	Lec 4	Virtual Memory	slides follow-up video	Required Reading: OSTEP 13, 15, 16, Anatomy of a program in memory Optional Reading: Kernel memory management, Closures, stacks, and the heap
3	Wed 1/30	--	Arctic Interrupt: University closed		Required Reading: OSTEP 18, 19
4	Mon 2/4	Lec 5	Intro to Paging	slides p1b in	Required Reading: OSTEP 20, Getting Physical with Memory Optional Reading: Memory Translation and Segmentation
4	Wed 2/6	Lec 6	Multi-level Paging and the TLB	Guest lecturer; slides	Required Reading: OSTEP 21, 22, Summary Optional Reading: OSTEP 23, The Page Cache
5	Mon 2/12	Lec 7	Virtual Memory and Paging Tricks	slides	Required Reading: OSTEP 25, 26
5	Wed 2/14	Lec 8	Concurrency I: Threads	slides	Required Reading: OSTEP 28, 29
6	Mon 2/18	Lec 9	Concurrency II: Mutual Exclusion (locks)	slides	Required Reading: OSTEP 30
6	Wed 2/20	Lec 10	Concurrency III: Signaling and Condition Variables	slides	Required Reading: OSTEP 31, 33, Signals, Linux kernel signals
7	Mon 2/25	Lec 11	Concurrency IV: Semaphores, Signals, Event-driven architecture	slides	Required Reading: OSTEP 32, Sys V. Shared Memory, Sys V. Message Queues
7	Wed 2/27	Lec 12	Concurrency V: Concurrency Bugs and IPC	slides	Required Reading: OSTEP d, 36
8	Mon 3/4	Lec 13	I/O I: Devices and Drivers		Required Reading: Overview of PCIe, Chipsets and the Memory Map Optional Reading: How PCI Express Works
8	Wed 3/6	Lec 14	I/O II: PCI and dynamic MMIO		Required Reading: OSTEP 37 Optional Reading: What your computer does while you wait
9	Mon 3/11	Lec 15	I/O III: Disks	slides Interested in research?	Required Reading: OSTEP 38 Optional Reading: Original RAID paper
9	Wed 3/13	Midterm (in class)	(See Exams)
10	Mon 3/18	holiday	Spring Break (No classes)
10	Wed 3/20	holiday	Spring Break (No classes)
11	Mon 3/25	Lec 16	I/O IV: RAID	slides	Required Reading: OSTEP 38
11	Wed 3/27	Lec 17	File Systems I: File Abstractions	slides	Required Reading: OSTEP 39
12	Mon 4/1	Lec 18	File Systems II: Implementation	slides	Required Reading: OSTEP 40
12	Wed 4/3	Lec 19	File Systems III: FFS	slides	Required Reading: OSTEP 41
13	Mon 4/8	Lec 20	File Systems IV: Journaling and LFS	slides	Required Reading: OSTEP 42, 43
13	Wed 4/10	Lec 21	SSD, Flash, and NVM	slides	Required Reading: OSTEP 44, NVDIMMs
14	Mon 4/15	Lec 22	Distributed Systems I	slides	Required Reading: OSTEP 48
14	Wed 4/17	Lec 23	Distributed Systems II: RPC and NFS	slides	Required Reading: OSTEP 49, 50
15	Mon 4/22	Lec 24	Programming Models I: MapReduce	slides	Required Reading:MapReduce
15	Wed 4/24	Lec 25	Case Study: GFS	slides	Required Reading:Google FS
16	Mon 4/29	Lec 26	Advanced kernel architecture
16	Wed 5/1	Lec 27	Future Directions: Unikernels, Containers, Serverless Computing
--	Mon 5/6, 2-4PM	Final Exam (same room)	(See Exams)

Exams

Week	Date	Item	Length	Covers	Notes
9	Wed 3/13	Midterm	50 min	Lec 1-15
17	Monday 5/6, 2-4pm (same room)	Final Exam	75 min	Apx 75% Lec 16 on, 25% midterm material	example 1, 2, 3

Projects

You'll notice that (almost) every project listed below for this class is split into two parts. In many cases these are really entirely separate projects, but are logically grouped together to stress some particular concept. The (a) parts are mostly concentrated on userspace components (part of the broader operating system) and the (b) parts are kernel components that will involve hacking on xv6. 3a and 4a are subject to change based on an in-class vote. Unless otherwise noted, grades for these projects are going to be based on whether or not your code works. Partial solutions will not be given credit! With that in mind, you should always start early.

Project	Topic	Due Date	Handout	Notes (TBP = To Be Posted)
1a	UNIX warmup	Monday, 1/21/2019 11:59PM CST	p1a description	Posted 1/14/2019 5:00PM
1b	xv6 warmup	~~Monday, 1/28/2019~~ Monday, 2/4/2019 11:59PM CST	p1b description	Posted 1/21/2019 9:00PM
2a	out of the depths of x86	Thursday, 2/14/2019 11:59PM CST	p2a description	Posted 2/4/2019 11:59PM
2b	scheduling in xv6	~~Thursday, 2/28/2019~~ Friday, 3/1/2019 11:59PM CST	p2b description	Posted 2/14/2019 11:00PM
3a	parallel zip	Friday, 3/15/2019 11:59PM CST	p3a description	Posted 3/1/2019 7:00PM
3b	null pointers and shared memory	~~Friday, 3/29/2019~~ Monday, 4/1/2019 11:59PM CST	p3b description	Posted 3/12/2019
4a	implementing kernel threads	~~Monday, 4/15/2019~~ Wednesday, 4/17/2019 11:59PM CST	p4a description	Posted 4/1/2019
4b	MapReduce	Wednesday, 5/1/2019, 11:59PM CST	p4b description	Posted 4/17/2019

Books

The following book is the only required textbook for this course. It is not only an excellent textbook, but is also freely availble online:

Operating Systems: Three Easy Pieces by Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau, 2015.

The below books are also great and often-used texts on Operating Systems. If you find a copy of one for a good price, snatch it. Keep in mind, however, that the best way to earn operating systems is by building!.

This is probably my favorite OS textbook after OSTEP. Full disclosure, I took undergrad OS with Dahlin many years ago.

Operating Systems: Principles and Practice (2nd Edition), by Thomas Anderson and Michael Dahlin, Recursive Books 2014.

Operating System Concepts (9th Edition), by Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, 2012 Wiley.

Modern Operating Systems (4th Edition), by Andrew S. Tanenbaum and Herbert Bos, Pearson 2014.

For general systems design, a good one to have on the shelf:

Principles of Computer System Design: An Introduction (1st Edition), by Jerome H. Saltzer and M. Frans Kaashoek, Morgan Kaufmann 2009.

The following book, while outdated (uses a 2.6.10 kernel as a reference), still provides one of the best references for practical kernel hacking:

Understanding the Linux Kernel (3rd Edition), by Daniel P. Bovet and Marco Cesati, O'Reilly Media 2005.

Good book on Linux kernel programming:

Linux Kernel Development (3rd Edition), by Robert Love, Addison-Wesley Professional, 2010.

If you're interested in the macOS kernel, this one is not out yet, but will be soon:

OSX Internals, Volume II, 2nd Edition, by Jonathan Levin.

Similarly for Windows:

Windows Internals, Part 1: System Architecture... (7th Edition), by Pavel Yosifovich, Mark E. Russinovich, Alex Ionescu, and David A. Solomon, Microsoft Press 2017.

Development Environment

We will be using the Fusion cluster for development and testing. This means you will be doing most of your development remotely. You should all have the ability to log in to the cluster when classes start. You might also want to get used to using tools like tmux or sshfs. All the tools you will need are on Fusion.