CS 562: Virtual Machines

Staff

Instructor: Kyle C. Hale
  Office Hours: M 12PM-1:30PM
  E-mail: khale [at] cs [dot] iit [dot] edu

Course Info

Course number: CS 562

Semester: Fall 2019

Lecture Time: Tues/Thurs 1:50PM - 3:05PM

Lecture Location: Stuart 111

Overview

Virtual machines have transformed the ways in which we build, manage, and interact with modern computer systems. A great deal of the network packets that you send as you sit at your computer are handled by virtual machines that may move to another side of the world in a matter of hours. If you have an Android phone, every application you launch executes in a virtual machine. Have you written a Java or Python program lately? It couldn't run without a virtual machine sitting underneath it. VMs have enabled new development practices for production applications, mobile applications, web applications, and operating system kernels. They allow datacenter operators to more efficiently provision hardware resources, saving money and energy. They allow service providers to quickly react to failures in an efficient and clean way. They enable the construction of portable and platform-agnostic programming languages by decoupling applications from the hardware on which they run. While VMs have been around since the early 1970s, modern developments, from cloudlets to containers, are only increasing the utility of virtualization technologies. One can expect that their importance and relevance will only increase.

This means that a basic understanding of the technological foundations underlying virtual machines should be part of any computer scientist's repertoire. This course will draw back the curtains and expose the magic that makes various types of VMs work. By the end of the course, you will have a deep understanding of hypervisors, system virtualization, machine emulation, language virtual machines, binary translators, virtual resource management, and more. You will gain exposure to a real-world hypervisor code-base. Furthermore, you will actually build a virtual machine and develop an intuition for using VMs to solve problems. The course will involve lectures, written assignments, involved programming projects, and discussions of foundational research papers.

Course Goals

The goals fo this course will be for you to develop a deep understanding of various types of virtualization techniques, their advantages and disadvantage, and to be able to apply them in a practical setting. You will be able to build basic VM constructs and understand how to evaluate them. You will therefore be expected to strengthen your system programming skills. You will also learn about new and upcoming technologies related to virtualization.

In general, you will learn about the following topics (and potentially others):

Communication

We will be primarily using Piazza as a course communication mechanism. If you have an issue or question that is not strictly private (especially one you think would benefit everyone were it answered), please use Piazza as your first resource. The instructor and your fellow classmates will be there to help. Note that you can also post anonymously if you so choose.

Lecture Schedule

Week Date Item Topic Notes Reading
1 Tues 8/20 Lec 1 Class logistics; intro to VMs Lec 1 notes Required reading: SN Ch. 1
1 Thurs 8/22 Lec 2 Computer architecture review Lec 2 notes
Lec 2 slides
2 Tues 8/27 Lec 3 Relative addressing, MMIO, DMA review Lec 3 notes
2 Thurs 8/29 Lec 4 Intro to Emulation Required reading: SN 2.1-2.3
3 Tues 9/3 Lec 5 6502 Overview
3 Thurs 9/5 --
4 Tues 9/10 Lec 6 Hawknest overview and codebase intro
4 Thurs 9/12 Lec 7 Binary Translation Required reading: SN 2.5-2.10
5 Tues 9/17 Lec 8 QEMU Code Walkthrough Required reading: SN Ch. 3
5 Thurs 9/19 Lec 9 Intro to HLL VMs Required reading: SN Ch. 5
6 Tues 9/24 Lec 10 HLL Implementation Strategies Project ideas posted
6 Thurs 9/26 Lec 11 Garbage Collection
7 Tues 10/1 Lec 12 Advanced issues in HLL implementation
7 Thurs 10/3 No Lecture Hawknest code walks/presentations
8 Tues 10/8 Lec 13 Hawkbeans JVM overview
8 Thurs 10/10 Lec 14 Intro to full system virtualization
9 Tues 10/15 Lec 15 Theory of efficient virtualization Required reading: Popek & Goldberg
9 Thurs 10/17 Lec 16 Virtual memory & paging review
10 Tues 10/22 Instructor at conference
10 Thurs 10/23 Instructor at conference
11 Tues 10/29 No lecture Project 2 presentations/code walks
11 Thurs 10/31 Lec 17 KVM overview/intro
12 Tues 11/5 Instructor at conference
12 Thurs 11/7 Lec 18 Nested & shadow paging
13 Tues 11/12 Lec 19 Interrupt & I/O virtualization
13 Thurs 11/14 Lec 20 Network virtualization
14 Tues 11/19 Lec 21 Containers intro
14 Thurs 11/21 Lec 22 Datacenter virtualization (or security)
15 Tues 11/26 holiday Thanksgiving break
15 Thurs 11/28 holiday Thanksgiving break
16 Tues 12/3 Finals week: Lec 23 Future directions and potpurri
16 Thurs 12/5 Finals week: final presentations Final project presentations

Projects

Project Topic Due Date Handout Notes (TBP = To Be Posted)
1 System Emulation (Hawknest) Part 1: Thursday, 8/29/2019, 11:59PM
Part 2: Friday, 10/4/2019, 11:59PM
Project 1 Preliminary posted 8/21/2019
Project 1 posted 9/10/2019
2 High-Level Language VMs (Hawkbeans) Friday, 11/1/2019, 11:59PM Project 2 posted 10/11/2019
3 System VMs/Open-ended project
4 Containers (Hawker)/Open-ended project

Books

The following book is the only required textbook for this course. If you plan on pursuing computer systems seriously, it is a great book to have as a reference:

Virtual Machines: Versatile Platforms for Systems and Processes (1st Edition), by Jim Smith and Ravi Nair, 2005 Morgan Kaufmann.

Development Environment

For all projects I will provide you with a Vagrant configuration which you can use to spawn a virtual machine to do your work on. To use Vagrant, you'll need a VMM installed on your machine. VirtualBox and libvirt (built on QEMU/kvm) are two free options. VMware Workstation or Fusion for Mac will work as well. If you're on a Windows box, HyperV is another option. See here for getting started with Vagrant VMs.

Other Useful Links and Resources

This is a list of other resources that you might find useful for this class and for doing work in the systems area in general. Feel free to peruse them at your own convenience.

Links