Automatic Memory Management
Outline
Why automatic memory management?
Garbage collection
Three techniques:
Mark and sweep
Stop and copy
Reference counting
Why Automatic Memory Management?
Storage management is still a hard problem in modern programming
C and C++ programs have many storage bugs
forgetting to free unused memory
dereferencing a dangling pointer
overwriting parts of a data structure by accident
and so on \(\ldots\) (can be big security problems)
Storage bugs are difficult to find; a bug can lead to a visible effect far away in time and program text from the source
Type Safety and Memory Management
Some storage bugs can be prevented in a strongly typed language, for example, array bounds checking
Can types prevent errors in programs with manual allocation and deallocation of memory?
- Some fancy type systems (linear types) were designed for this purpose, but they complicate programming significantly
If you want type safety, then you must use automatic memory management
Automatic Memory Management
This is an old problem: studied since the 1950s for Lisp
There are several well-known techniques for performing completely automatic memory management
Until relatively recently (Java), they were unpopular outside the Lisp family of languages
The Basic Idea
When an object that takes memory space is created, unused space is automatically allocated
After a while there is no more unused space
Some space is occupied by objects that will never be used again (dead objects)
This space can be freed to be reused later
Dead Objects
How can we tell whether an object will “never be used again”?
In general, it is impossible (undecidable) to determine
We will have to use a heuristic to find many, but not all, objects that will never be used again
Observation: a program can use only the objects that it can find
Java example:
String s = new String("Hello"); s = new String("Goodbye"); // the original "Hello" string is unreachable
Garbage
An object \(x\) is reachable if and only if:
A local variable (or register) contains a pointer to \(x\), or
Another reachable object \(y\) contains a pointer to \(x\)
All reachable objects can be found by starting from local variables and following all the pointers (“transitive”)
An unreachable object can never be referred to by the program; these objects are called garbage
Reachability is an Approximation
Consider the program:
x <- new A; y <- new B; x <- y; if true then x <- new C else x.m() fi;
After
x <- y
(assumingy
becomes dead there)- The object
A
is not reachable anymore - The object
B
is reachable (throughx
) - Thus
B
is not garbage and is not collected - But object
B
is never going to be used
- The object
Cool Garbage
At run-time we have two mappings:
The environment \(E\) maps variable identifiers to locations
The store \(S\) maps locations to values
Proposed garbage collector
for each location l in domain(S) let can_reach = false for each (v, l2) in E if l = l2 then can_reach = true for each l3 in v // v is X(..., ai = li, ...) if l = l3 then can_reach = true if not can_reach then reclaim_location(l)
Garbage Analysis
Could we use the proposed Cool Garbage Collector in real life?
How long would it take?
How much space would it take?
Are we forgetting anything?
- Hint: yes
Tracing Reachable Values
In Cool, local variables are easy to find
- Use the environment mapping \(E\)
- and one object may point to other objects, etc.
The stack is more complex
- each stack frame (activation record) contains method parameters (other objects)
If we know the layout of a stack frame then we can find the pointers (objects) in it
Reachability
Many things may look legitimate and reachable but will turn out not to be.
How can we figure this out systematically?
A Simple Example
Start tracing from local variables and the stack
Note that
B
andD
are not reachable from local vars or the stackThus we can reuse their storage
Elements of Garbage Collection
- Every garbage collection scheme has the following steps
- Allocate space as needed for new objects
- When space runs out:
- Compute which objects might be used again
- Free space used by objects not found in the previous step
- Some strategies perform garbage collection before the space actually runs out
Mark and Sweep
- When memory runs out, the garbage collector executes two phases:
- mark: traces reachable objects
- sweep: collects garbage objects
- Every object has an extra bit: the mark bit
- reserved for memory management
- initially the mark bit is 0
- set to 1 for the reachable objects in the mark phase
Mark and Sweep Example
The Mark Phase
let todo = { all roots }
while todo is not empty
pick v in todo
remove v from todo
if mark(v) = 0 then
mark(v) <- 1
let v1, ..., vn be pointers contained in v
add pointers to todo
The Sweep Phase
- The sweep phase scans the (entire) heap looking for objects with mark bit 0
- these objects have not been visited in the mark phase
- they are garbage
- Any such object is add to the free list
- The objects with a mark bit 1 have their mark bit reset to 0
The Sweep Phase
p <- bottom of the heap
while p < top of the heap
if mark(p) = 1 then
mark(0) <- 0
else
add block p...(p + sizeof(p)-1) to free list
p <- p + sizeof(p)
Mark and Sweep Analysis
While conceptually simple, this algorithm has a number of tricky details
- A serious problem with the mark phase
- it is invoked when we are out of space
- yet it needs space to construct the todo list
- the size of the todo list is unbounded so we cannot reserve space for it ahead of time
Mark and Sweep Details
The todo list is used as an auxiliary data structure to perform the reachability analysis
- There is a trick that allows the auxiliary data to be stored in the objects themselves
- pointer reversal: when a pointer is followed it is reversed to point to its parent
Similarly, the free list is stored in the free objects themselves
Mark and Sweep Evaluation
- Space for a new object is allocated from the new list
- a block large enough is picked
- an area of the necessary size is allocated from it
- the left-over is put back into the free list
- Mark and sweep can fragment memory
- Advantage: objects are not moved during garbage collection
Stop and Copy
- Memory is organized into two areas:
- old space: used for allocation
- new space: used as a reserve for the garbage collector
- The heap pointer points to the next free word in the old space
- Allocation just advances the heap pointer
Stop and Copy
- Starts when the old space is full
- Copies all reachable objects from old space into new space
- garbage is left behind
- after the copy phase the new space uses less space than the old one before the collection
- After the copy the roles of the old and new spaces are reversed and the program resumes
Stop and Copy Example
Implementing Stop and Copy
- We need to find all the reachable objects
- just as in mark and sweep
- As we find a reachable object, we copy it into the new space
- and we need to fix all pointers pointing to it
- As we copy an object we store in the old copy a forwarding pointer to the new copy
- when we later reach an object with a forwarding pointer we know it was already copied
- How can we identify forwarding pointers?
Implementing Stop and Copy
We still have the issue of how to implement a traversal without using extra space
- The following trick solves the problem:
- partition new space into three contiguous regions:
- copied and scanned (copied objects whose pointer fields were followed and fixed)
- copied (copied objects whose pointer fields were not followed)
- empty
- partition new space into three contiguous regions:
Stop and Copy Algorithm
while scan not equal to alloc
let O be the object at scan pointer
for each pointer p contained in O
find O' that p points to
if O' is without a forwarding pointer
copy O' to new space (update alloc pointer)
set first word of 0' to point to the new copy
change p to point to the new copy of O'
else
set p in O equal to the forwarding pointer
increment scan pointer to the next object
Stop and Copy Details
- As with mark and sweep, we must be able to tell how large and object is when we scan it
- and we must also know where the pointers are inside the object
- We must also copy any objects pointed to by the stack and update pointers in the stack
- this can be an expensive operation
Stop and Copy Evaluation
- Stop and copy is generally believed to be the fastest garbage collection technique
- Allocation is very cheap
- just increment the heap pointer
- Collection is relatively cheap
- especially if there is a lot of garbage
- only touch reachable objects
- But some languages to not allow copying
- Example: C, C++
Why Doesn’t C Allow Copying?
- Garbage collection relies on being able to find all reachable objects
- and it needs to find all pointers in an object
- In C or C++ it is impossible to identify the contents of objects in memory
- for example, how can you tell that a sequence of two memory words is a list cell (with data and next fields) or a binary tree node (with left and right fields)?
- Thus we cannot tell where all the pointers are
Conservative Garbage Collection
- But, it is OK to be conservative:
- If a memory word “looks like” a pointer then it is considered to be a pointer
- it must be aligned
- it must point to a valid address in the data segment
- All such pointers are followed and we overestimate the reachable objects
- If a memory word “looks like” a pointer then it is considered to be a pointer
- But, we still cannot move objects because we cannot update the pointers to them
Reference Counting
Rather than wait for memory to be exhausted, try to collect an object when there are no more pointers to it
- Store in each object the number of pointers to that object
- This is a reference count
Each assignment operation has to manipulate the reference count
Implementing Reference Counts
new
returns an object with a reference count of 1If
x
points to an object then letrc(x)
refer to the object’s reference count- Every assignment
x <- y
must be changed:- `rc(y) <- rc(y) + 1
- `rc(x) <- rc(x) - 1
- if
rc(x)
equals 0 then markx
as free x <- y
Reference Counting Evaluation
- Advantages:
- Easy to implement
- Collects garbage incrementally without large pauses in the execution
- Disadvantages:
- Manipulating reference counts at each assignment is very slow
- Cannot collect circular structures
Garbage Collector Evaluation
Automatic memory management avoids some serious storage bugs
- But, it takes control away from the programmer
- for example, layout of data in memory
- for example, when memory is deallocated
- Most garbage collection implementations stop the execution during collection
- not acceptable in real-time applications
Garbage Collector Evaluation
Garbage collection is going to be around for a while
- Advanced garbage collection algorithms:
- concurrent: allow the program to run while the collection is happening
- generational: do not scan long-lived objects at every collection
- parallel: several collectors working in parallel
- real-time/incremental: no long pauses
Real Life Examples
- Python uses reference counting
- Ruby does mark and sweep
- OCaml does (generational) stop and copy
- Java does (generational) stop and copy
Summary
- An automatic memory management system deallocates objects when they are no longer used and reclaims their storage space
- We must be conservative and only free objects that will not be used later
- Garbage collection scans the heap from a set of roots to find reachable objects
- Reference counting stores the number of pointers to an object with that object and frees it when that count reaches zero