CSE 306: Lab 2: Copy-on-Write Fork

To become familiar with Unix-style forking and x86 memory management, you will convert the simple fork() implementation in xv6 to a copy-on-write fork(). This will involve writing a trap handler for page faults, augment the phyical memory management code, and, of course, manipulate page tables. This assignment does not involve writing that many lines of code (hundreds to a thousand), but the hard part is figuring out what to change and writing careful unit tests for each change. We strongly recommend starting early and writing many test cases.

Important note: Be sure to pull the latest code from the read-only (http) git repository. We have added a few essential helper functions and error code definitions you will need for this lab.

Hand-In Procedure

When you are ready to hand in your lab code and write-up, add an entry to the file slack.txt noting how many late hours you have used both for this assignment and in total. (This is to help us agree on the number that you have used.) This file should contain a single line formatted as follows (where n is the number of late hours):

Then run make handin-lab2 in the xv6 directory. If you submit multiple times, we will take the latest submission and count late hours accordingly.

In this and all other labs, you may complete challenge problems for extra credit. If you do this, please create a file called challenge.txt, which includes a short (e.g., one or two paragraph) description of what you did to solve your chosen challenge problem and how to test it. If you implement more than one challenge problem, you must describe each one.

This lab does not include any questions for you to answer, but you should document your design in the README file.

Core assignment

You will convert the xv6 fork() implemenation to use copy-on-write. The current version does a simple copy of each page in the address space. You will modify the xv6 kernel to do copy-on-write instead.

Tracking page reference counts

Currently, xv6 does not allow physical page frames to be shared. The first step in copy-on-write support will be adding a reference count to each physical page descriptor.

First, you will need to understand how physical pages are allocated. Begin by reading kalloc.c. Here, each 4KB page of free physical memory is represented as a struct run, and these structures are organized into a free list.

Exercise 1. (5 points) Add a reference count to this page descriptor structure. You should set the count to one when a page is allocated, write a helper function to increment and decrement the count (using appropriate locks or atomic instructions!), and assert that the count is one when a page is freed.

Checking that a page is not in use by more than one process at the time of freeing will help you find bugs later.

Catching page faults

The second building block you will need for copy-on-write fork is the ability to catch page faults. Read trap.c: you will see a number of fault handlers registered, such as for T_SYSCALL. You will register a handler for T_PGFLT in the trap() function. Currently, this handler can copy the default behavior, but you will later extend it to handle writes to a copy-on-write page.

Exercise 2. (5 points) Register a page fault handler for page faults, which prints a slightly different error message. Write a unit test that deliberately accesses an invalid address, and be sure that your handler is being called to kill the process.

Copy-on-write

The main part of the assignment is changing the fork implementation. We highly recommend that you keep the old version for easy comparison and debugging, such as with a #define#.

Begin by reading and understanding the default fork() implementation. The system call is defined in proc.c, although the main workhorse is the function copyuvm(), defined in vm.c. Before you write any code make sure you completely understand what this function (and its helpers) are doing. The following paragraphs will give some explanation, as will the relevant Intel manual entries below.

Before doing anything else, familiarize yourself with the x86's protected-mode memory management architecture: namely segmentation and page translation.

Exercise 3. Look at chapters 5 and 6 of the Intel 80386 Reference Manual, if you haven't done so already. Read the sections about page translation and page-based protection closely (5.2 and 6.4). We recommend that you also skim the sections about segmentation; while xv6 uses paging for virtual memory and protection, segment translation and segment-based protection cannot be disabled on the x86, so you will need a basic understanding of it.

Virtual, Linear, and Physical Addresses

In x86 terminology, a virtual address consists of a segment selector and an offset within the segment. A linear address is what you get after segment translation but before page translation. A physical address is what you finally get after both segment and page translation and what ultimately goes out on the hardware bus to your RAM.

A C pointer is the "offset" component of the virtual address. In bootasm.S, xv6 installed a Global Descriptor Table (GDT) that effectively disabled segment translation by setting all segment base addresses to 0 and limits to 0xffffffff. Hence the "selector" has no effect and the linear address always equals the offset of the virtual address. As for memory translation, we can ignore segmentation throughout the xv6 labs and focus solely on page translation.

Exercise 3. While GDB can only access QEMU's memory by virtual address, it's often useful to be able to inspect physical memory while setting up virtual memory. Review the QEMU monitor commands from the lab tools guide, especially the xp command, which lets you inspect physical memory. To access the QEMU monitor, press Ctrl-a c in the terminal (the same binding returns to the serial console).

Use the xp command in the QEMU monitor and the x command in GDB to inspect memory at corresponding physical and virtual addresses and make sure you see the same data.

Our patched version of QEMU provides an info pg command that may also prove useful: it shows a compact but detailed representation of the current page tables, including all mapped memory ranges, permissions, and flags. Stock QEMU also provides an info mem command that shows an overview of which ranges of virtual memory are mapped and with what permissions.

From code executing on the CPU, once we're in protected mode (which we entered first thing in boot/bootasm.S), there's no way to directly use a linear or physical address. All memory references are interpreted as virtual addresses and translated by the MMU, which means all pointers in C are virtual addresses.

Adding copy-on-write flags

You will need to define a page table flag for copy-on-write. The x86 architecture reserves a few flags in the page table for use by the OS. We recomment 0x800. Define this flag as PTE_COW in mmu.h. Note that this flag will not be interpreted by the CPU---it is only there for your code to use to identify COW pages.

Exercise 4. (40 points.) Implement a variant of copyuvm called cowuvm that does the following:

Converts each writeable page table entry in the parent and child to read-only and PTE_COW. You will need to also shoot down the TLB entries for any virtual addresses you change using the invlpg() function (note this was pushed recently to the read-only repository.
For each page that is shared copy-on-write, you will need to increment the refernce count.

Note that your cowuvm implementation will not need to allocate new page frames using kalloc() for the process. Rather, this will be done lazily in the page fault handler.

Finally, you will need to implement COW support in the page fault handler you registered above. This handler will need to:

Check if the fault is a write fault for a user address. Hint, look at the tf->err field, and the newly-added FEC_WR flags in mmu.h.
Check if the fault is for an address whose page table includes the PTE_COW flag. If not, kill the program as usual. If so, check the reference count on the page.
1. If the page has more than one reference, copy the page and replace it with a writeable copy in the local process. Be sure to invalidate the TLB!. Decrement the reference count on the original page.
2. If the page has only one reference, you can remove the PTE_COW flag and restore write permission to the page. No need to copy, as the other process has already copied the page. Be sure to invalidate the TLB!

At this point, you should be able to pass the forktest utility, as well as run all of the usertests provided. Of course, we want you to write addtional unit tests.

Exercise 4. (15 points.) Write at least 3 additional unit tests for fork. Extra credit is possible for particularly clever or tricky ways to detect edge cases in fork. This is very important going forward, as bugs in fork can cause other problems in future assignments.

Style and More

Aside from testing the proper functionality of your code, we will also evaluate the quality of your code. Be sure to use a consistent style, well documented, and break your code into separate functions and/or source files as it makes sense.

To be sure your code is very clean, it must compile with "gcc -Wall -Werror" without any errors or warnings!

If the various sources you use require common definitions, then do not duplicate the definitions. Make use of C's code-sharing facilities.

You must include a README file with this and any assignment. The README file should describe what you did, what approach you took, results of any measurements you made, which files are included in your submission and what they are for, etc. Feel free to include any other information you think is helpful to us in this README; it can only help your grade.