Staff
Instructor: Kyle C. Hale
Office Hours: Thurs 2-3 PM (SB 229C)
E-mail: khale [at] cs [dot] iit [dot] edu
Course Info
Course number: CS 595-02
Semester: Fall 2018
Lecture Time: Tues/Thurs 3:15PM - 4:30PM
Lecture Location: Engineering Center 027
Overview
Virtual machines have transformed the ways in which we build, manage, and interact with modern computer systems. A great deal of the network packets that you send as you sit at your computer are handled by virtual machines that may move to another side of the world in a matter of hours. If you have an Android phone, every application you launch executes in a virtual machine. Have you written a Java or Python program lately? It couldn't run without a virtual machine sitting underneath it. VMs have enabled new development practices for production applications, mobile applications, web applications, and operating system kernels. They allow datacenter operators to more efficiently provision hardware resources, saving money and energy. They allow service providers to quickly react to failures in an efficient and clean way. They enable the construction of portable and platform-agnostic programming languages by decoupling applications from the hardware on which they run. While VMs have been around since the early 1970s, modern developments, from cloudlets to containers, are only increasing the utility of virtualization technologies. One can expect that their importance and relevance will only increase.
This means that a basic understanding of the technological foundations underlying virtual machines should be part of any computer scientist's repertoire. This course will draw back the curtains and expose the magic that makes various types of VMs work. By the end of the course, you will have a deep understanding of hypervisors, system virtualization, machine emulation, language virtual machines, binary translators, virtual resource management, and more. You will gain exposure to a real-world hypervisor code-base. Furthermore, you will actually build a virtual machine and develop an intuition for using VMs to solve problems. The course will involve lectures, written assignments, involved programming projects, and discussions of foundational research papers.
Course Goals
The goals fo this course will be for you to develop a deep understanding of various types of virtualization techniques, their advantages and disadvantage, and to be able to apply them in a practical setting. You will be able to build basic VM constructs and understand how to evaluate them. You will therefore be expected to strengthen your system programming skills. You will also learn about new and upcoming technologies related to virtualization.
In general, you will learn about the following topics (and potentially others):
- General resource virtualization: virtual memory, virtualized CPUs, OS abstractions
- Basic Virtual Machines and historical context. Formal requirements for virtual machines
- Emulation: interpretation and binary translation
- Language VMs and Process VMs: JVM, Python, .NET CLR, Android's Dalvik JVM, VCODE, JIT compilation
- Full system virtualization: Type I & II VMMs, paravirtualization, device virtualization
- Virtualization in practice: virtual resource management, live migration, memory ballooning, fault tolerance
- Network virtualization: virtual overlays, bridged networking, NAT, software-defined networks
- Virtual storage: storage virtualization, virtual disk formats, copy-on-write formats
- Advanced topics in virtualization: hardware assisted virtualization, overheads of virtualization, performant VMs, interrupt virtualization, self-virtualizing devices, security, introspection, the semantic gap
- Current and future directions: VMMs as microkernels, Unikernels, chroot, jails, containers, cloudlets, virtual appliances
Communication
We will be primarily using Piazza as a course communication mechanism. If you have an issue or question that is not strictly private (especially one you think would benefit everyone were it answered), please use Piazza as your first resource. The instructor and your fellow classmates will be there to help. Note that you can also post anonymously if you so choose.Lecture Schedule
| Week | Date | Item | Topic | Notes | Reading |
|---|---|---|---|---|---|
| 1 | Tues 8/21 | No lecture | -- | ||
| 1 | Thurs 8/23 | Lec 1 | Class logistics; intro to VMs | Required reading: SN Ch. 1 | |
| 2 | Tues 8/28 | Lec 2 | VM overview | ||
| 2 | Thurs 8/30 | Lec 3 | Computer Architecture review | Lec 3 notes | Project 1 Preliminary Posted |
| 3 | Tues 9/4 | Lec 4 | Memory/DMA; MOS 6502 Introduction | Lec 4 notes | Required Reading: SN Ch. 2 |
| 3 | Thurs 9/6 | Lec 5 | Introduction to Hawknest system arch.; Project overview | Project 1 posted; due Thurs Oct 4 11:59PM | |
| 4 | Tues 9/11 | Lec 6 | Intro to emulation | ||
| 4 | Thurs 9/13 | No lecture | -- | ||
| 5 | Tues 9/18 | Lec 7 | QEMU Code walkthrough | ||
| 5 | Thurs 9/20 | Lec 8 | Binary Translation (Different source/target ISAs) | Required Reading: SN Ch. 3 | |
| 6 | Tues 9/25 | Lec 9 | Emulation Epilogue; Intro to HLL VMs | Required Reading: SN Ch. 5 | |
| 6 | Thurs 9/27 | Lec 10 | HLL Implementation Strategies | ||
| 7 | Tues 10/2 | Lec 11 | Garbage Collection | ||
| 7 | Thurs 10/4 | No lecture | 6502 Code walks/presentations (more GC?) | Project 2 posted; due Thurs Nov 1 11:59PM | |
| 8 | Tues 10/9 | Lec 12 | Hawkbeans JVM intro; Advanced issues in HLL implementation | ||
| 8 | Thurs 10/11 | Lec 13 | Theory of efficient virtualization | Required reading: Popek & Goldberg Recommended readfing: Survey of VM Research |
|
| 9 | Tues 10/16 | Lec 14 | Full system virtualization | Required reading: S&N 8.1 & 8.2 Required reading: AMD Systems Prog. Manual 15.1-15.8 |
|
| 9 | Thurs 10/18 | Lec 15 | Review of virtual memory & paging | Required reading:Using the KVM API Recommended reading (brush up on VM): OSTEP Chs. 18, 19, 20 | |
| 10 | Tues 10/23 | Lec 16 | Nested paging & shadow paging | ||
| 10 | Thurs 10/25 | Lec 17 | Interrupt & I/O virtualization | ||
| 11 | Tues 10/30 | Lec 18 | Palacios | Project 3 Preliminary posted; due Thurs 11/8 11:59PM | |
| 11 | Thurs 11/1 | Lec 19 | KVM codewalk | ||
| 12 | Tues 11/6 | Lec 20 | Network virtualization (virtual networking); VLANs; overlay networks | ||
| 12 | Thurs 11/8 | Lec 21 | Virtualization in the datacenter: consolidation, scheduling, live migration | ||
| 13 | Tues 11/13 | -- | Instructor at conference; No lecture | ||
| 13 | Thurs 11/15 | Lec 22 | Instructor at conference; Guest lecture (InfiniBand) | ||
| 14 | Tues 11/20 | -- | Thanksgiving Break: No class | ||
| 14 | Thurs 11/22 | holiday | Thanksgiving Break: No class | ||
| 15 | Tues 11/27 | Lec 24 | Virtualization security, potpurri | Recommended reading:
BluePill Cloud side channels Cache timing attacks Deterministic Replay Syscall Interposition (SubVirt) Guarded VMM modules |
|
| 15 | Thurs 11/29 | Lec 25 | Containers | ||
| 16 | Tues 12/4 | Finals week: Lec 26 | Future Directions: Specialization | ||
| 16 | Thurs 12/6 | Finals week: Lec 27 | Future Directions: Hybrid Virtualization, low-level platform virt |
Projects
| Project | Topic | Due Date | Handout | Notes (TBP = To Be Posted) |
|---|---|---|---|---|
| 1 | System Emulation (Hawknest) | Part 1 Posted 8/30, due next Thurs.
(9/6) Part 2 Posted 9/6, due Thurs. Oct 4, 11:59PM |
||
| 2 | High-Level Language VMs (Hawkbeans) | Posted 10/4, due Thurs. Nov 1, 11:59PM | ||
| 3 | System VMs and Virtual Devices | Part 1 Posted 10/30, due Thurs. Nov 8, 11:59PM | ||
| 4 | Containers | Posted 12/1, due Fri. Dec 7, 11:59PM |
Books
The following book is the only required textbook for this course. If you plan on pursuing computer systems seriously, it is a great book to have as a reference:
Development Environment
TBD
Other Useful Links and Resources
This is a list of other resources that you might find useful for this class and for doing work in the systems area in general. Feel free to peruse them at your own convenience.
