Automatic Memory Management

Outline

• Why automatic memory management?

• Garbage collection

• Three techniques:

  • Mark and sweep

  • Stop and copy

  • Reference counting

Why Automatic Memory Management?

• Storage management is still a hard problem in modern programming

• C and C++ programs have many storage bugs

  • forgetting to free unused memory

  • dereferencing a dangling pointer

  • overwriting parts of a data structure by accident

  • and so on \(\ldots\) (can be big security problems)

• Storage bugs are difficult to find; a bug can lead to a visible effect far away in time and program text from the source

Type Safety and Memory Management

• Some storage bugs can be prevented in a strongly typed language, for example, array bounds checking

• Can types prevent errors in programs with manual allocation and deallocation of memory?

  • Some fancy type systems (linear types) were designed for this purpose, but they complicate programming significantly

• If you want type safety, then you must use automatic memory management

Automatic Memory Management

• This is an old problem: studied since the 1950s for Lisp

• There are several well-known techniques for performing completely automatic memory management

• Until relatively recently (Java), they were unpopular outside the Lisp family of languages

The Basic Idea

• When an object that takes memory space is created, unused space is automatically allocated

• After a while there is no more unused space

• Some space is occupied by objects that will never be used again (dead objects)

• This space can be freed to be reused later

Dead Objects

• How can we tell whether an object will “never be used again”?

  • In general, it is impossible (undecidable) to determine

  • We will have to use a heuristic to find many, but not all, objects that will never be used again

• Observation: a program can use only the objects that it can find

• Java example:

  String s = new String("Hello");
  s = new String("Goodbye");
  // the original "Hello" string is unreachable

Garbage

• An object \(x\) is reachable if and only if:

  • A local variable (or register) contains a pointer to \(x\), or

  • Another reachable object \(y\) contains a pointer to \(x\)

• All reachable objects can be found by starting from local variables and following all the pointers (“transitive”)

• An unreachable object can never be referred to by the program; these objects are called garbage

Reachability is an Approximation

• Consider the program:

  x <- new A;
  y <- new B;
  x <- y;
  if true then x <- new C else x.m() fi;

• After x <- y (assuming y becomes dead there)

  • The object A is not reachable anymore
  • The object B is reachable (through x)
  • Thus B is not garbage and is not collected
  • But object B is never going to be used

Cool Garbage

• At run-time we have two mappings:

  • The environment \(E\) maps variable identifiers to locations

  • The store \(S\) maps locations to values

• Proposed garbage collector

  for each location l in domain(S)
      let can_reach = false
      for each (v, l2) in E
          if l = l2 then can_reach = true
          for each l3 in v  // v is X(..., ai = li, ...)
              if l = l3 then can_reach = true
      if not can_reach then reclaim_location(l)

Garbage Analysis

• Could we use the proposed Cool Garbage Collector in real life?

• How long would it take?

• How much space would it take?

• Are we forgetting anything?

  • Hint: yes

Tracing Reachable Values

• In Cool, local variables are easy to find

  • Use the environment mapping \(E\)
  • and one object may point to other objects, etc.

• The stack is more complex

  • each stack frame (activation record) contains method parameters (other objects)

• If we know the layout of a stack frame then we can find the pointers (objects) in it

Reachability

• Many things may look legitimate and reachable but will turn out not to be.

• How can we figure this out systematically?

A Simple Example

• Start tracing from local variables and the stack

• Note that B and D are not reachable from local vars or the stack

• Thus we can reuse their storage

Elements of Garbage Collection

• Every garbage collection scheme has the following steps
  • Allocate space as needed for new objects
  • When space runs out:
    • Compute which objects might be used again
    • Free space used by objects not found in the previous step
• Some strategies perform garbage collection before the space actually runs out

Mark and Sweep

• When memory runs out, the garbage collector executes two phases:
  • mark: traces reachable objects
  • sweep: collects garbage objects
• Every object has an extra bit: the mark bit
  • reserved for memory management
  • initially the mark bit is 0
  • set to 1 for the reachable objects in the mark phase

Mark and Sweep Example

The Mark Phase

let todo = { all roots }
while todo is not empty
    pick v in todo
    remove v from todo
    if mark(v) = 0 then
        mark(v) <- 1
        let v1, ..., vn be pointers contained in v
        add pointers to todo

The Sweep Phase

• The sweep phase scans the (entire) heap looking for objects with mark bit 0
  • these objects have not been visited in the mark phase
  • they are garbage
• Any such object is add to the free list
• The objects with a mark bit 1 have their mark bit reset to 0

The Sweep Phase

p <- bottom of the heap
while p < top of the heap
    if mark(p) = 1 then
        mark(0) <- 0
    else
        add block p...(p + sizeof(p)-1) to free list
    p <- p + sizeof(p)

Mark and Sweep Analysis

• While conceptually simple, this algorithm has a number of tricky details

• A serious problem with the mark phase
  • it is invoked when we are out of space
  • yet it needs space to construct the todo list
  • the size of the todo list is unbounded so we cannot reserve space for it ahead of time

Mark and Sweep Details

• The todo list is used as an auxiliary data structure to perform the reachability analysis

• There is a trick that allows the auxiliary data to be stored in the objects themselves
  • pointer reversal: when a pointer is followed it is reversed to point to its parent

• Similarly, the free list is stored in the free objects themselves

Mark and Sweep Evaluation

• Space for a new object is allocated from the new list
  • a block large enough is picked
  • an area of the necessary size is allocated from it
  • the left-over is put back into the free list
• Mark and sweep can fragment memory
• Advantage: objects are not moved during garbage collection

Stop and Copy

• Memory is organized into two areas:
  • old space: used for allocation
  • new space: used as a reserve for the garbage collector
• The heap pointer points to the next free word in the old space
  • Allocation just advances the heap pointer

Stop and Copy

• Starts when the old space is full
• Copies all reachable objects from old space into new space
  • garbage is left behind
  • after the copy phase the new space uses less space than the old one before the collection
• After the copy the roles of the old and new spaces are reversed and the program resumes

Stop and Copy Example

Implementing Stop and Copy

• We need to find all the reachable objects
  • just as in mark and sweep
• As we find a reachable object, we copy it into the new space
  • and we need to fix all pointers pointing to it
• As we copy an object we store in the old copy a forwarding pointer to the new copy
  • when we later reach an object with a forwarding pointer we know it was already copied
  • How can we identify forwarding pointers?

Implementing Stop and Copy

• We still have the issue of how to implement a traversal without using extra space

• The following trick solves the problem:
  • partition new space into three contiguous regions:
    • copied and scanned (copied objects whose pointer fields were followed and fixed)
    • copied (copied objects whose pointer fields were not followed)
    • empty

Stop and Copy Algorithm

while scan not equal to alloc
    let O be the object at scan pointer
    for each pointer p contained in O
        find O' that p points to
        if O' is without a forwarding pointer
            copy O' to new space (update alloc pointer)
            set first word of 0' to point to the new copy
            change p to point to the new copy of O'
        else
            set p in O equal to the forwarding pointer
    increment scan pointer to the next object

Stop and Copy Details

• As with mark and sweep, we must be able to tell how large and object is when we scan it
  • and we must also know where the pointers are inside the object
• We must also copy any objects pointed to by the stack and update pointers in the stack
  • this can be an expensive operation

Stop and Copy Evaluation

• Stop and copy is generally believed to be the fastest garbage collection technique
• Allocation is very cheap
  • just increment the heap pointer
• Collection is relatively cheap
  • especially if there is a lot of garbage
  • only touch reachable objects
• But some languages to not allow copying
  • Example: C, C++

Why Doesn’t C Allow Copying?

• Garbage collection relies on being able to find all reachable objects
  • and it needs to find all pointers in an object
• In C or C++ it is impossible to identify the contents of objects in memory
  • for example, how can you tell that a sequence of two memory words is a list cell (with data and next fields) or a binary tree node (with left and right fields)?
  • Thus we cannot tell where all the pointers are

Conservative Garbage Collection

• But, it is OK to be conservative:
  • If a memory word “looks like” a pointer then it is considered to be a pointer
    • it must be aligned
    • it must point to a valid address in the data segment
  • All such pointers are followed and we overestimate the reachable objects
• But, we still cannot move objects because we cannot update the pointers to them

Reference Counting

• Rather than wait for memory to be exhausted, try to collect an object when there are no more pointers to it

• Store in each object the number of pointers to that object
  • This is a reference count

• Each assignment operation has to manipulate the reference count

Implementing Reference Counts

• new returns an object with a reference count of 1

• If x points to an object then let rc(x) refer to the object’s reference count

• Every assignment x <- y must be changed:
  • `rc(y) <- rc(y) + 1
  • `rc(x) <- rc(x) - 1
  • if rc(x) equals 0 then mark x as free
  • x <- y

Reference Counting Evaluation

• Advantages:
  • Easy to implement
  • Collects garbage incrementally without large pauses in the execution
• Disadvantages:
  • Manipulating reference counts at each assignment is very slow
  • Cannot collect circular structures

Garbage Collector Evaluation

• Automatic memory management avoids some serious storage bugs

• But, it takes control away from the programmer
  • for example, layout of data in memory
  • for example, when memory is deallocated
• Most garbage collection implementations stop the execution during collection
  • not acceptable in real-time applications

Garbage Collector Evaluation

• Garbage collection is going to be around for a while

• Advanced garbage collection algorithms:
  • concurrent: allow the program to run while the collection is happening
  • generational: do not scan long-lived objects at every collection
  • parallel: several collectors working in parallel
  • real-time/incremental: no long pauses

Real Life Examples

• Python uses reference counting
• Ruby does mark and sweep
• OCaml does (generational) stop and copy
• Java does (generational) stop and copy

Summary

• An automatic memory management system deallocates objects when they are no longer used and reclaims their storage space
• We must be conservative and only free objects that will not be used later
• Garbage collection scans the heap from a set of roots to find reachable objects
• Reference counting stores the number of pointers to an object with that object and frees it when that count reaches zero