Project 4 Description ---------------------- In this project you'll implement an explicit memory allocation/deallocation infrastructure. Getting rolecks --------------- Rolecks is available off of the course website (www.cs.umass.edu/~trekp/cs201). It is an executable Jar file, so any system running a reasonably recent version of Java should be able to run it. Make sure you get the latest version! Rolecks is regularly being extended and improved (read: bug fixes). Make sure you grab a recent copy before starting. Loading and running your code ----------------------------- You can write your code in your text editor of choice. To assemble it, click on the 'assemble file' button in rolecks. This will generate a simple binary version of the file (if the file was named foo.s the binary form will be named foo.bin). To load up the binary, click on the 'Open Executable' button in rolecks. The Problem ----------- In languages like C or C++, memory must not only be allocated, but explicitly deallocated. In C the memory allocation function is called malloc and the deallocation function is called free. Malloc takes an integer argument that is the number of bytes to allocate. It returns the address of the initial byte. Free takes as its argument the address of the memory block to free. It returns nothing. To use these methods, a program calls malloc() when it needs a block of memory, and when it is finished with it, it calls free(). Note that all malloc has to do is allocate the memory. There is no protection. Malloc doesn't check to make sure that the program isn't overwriting things or otherwise behaving badly. All malloc does is hand the program a block of contiguous bytes. All free does is return the block of bytes back to the pool of memory available. In this sense, malloc is similar to new in Java. Java has no equivalent to free (it is garbage collected) so this will be a new concept for many of you. The structure of an allocation ------------------------------- An allocation isn't merely a chunk of data however. Because free is handed the starting address, it has no direct knowledge of how large the block of bytes is. The way around this is to allocate (within malloc) a bit more data than is requested, and to put size information (metadata) in the extra space. So, if the user requests n bytes, you allocate n+4 bytes. The extra 4 are to store an integer which contains the size of the allocated block. Because malloc returns the address of the first allocated byte that the user can write to, usually the size is actually written to the first 4 bytes of the n+4 chunk. So, the structure of an allocated block is this: 0123 4 . . . n+3 +----+------------------------------------------------------------------+ |size| | +----+------------------------------------------------------------------+ ^ | address returned by malloc The user requests n bytes, you allocate n+4, you write n into the first 4 bytes, and you return the address of the next byte. Note that because you are writing an integer into the first four bytes, these allocations need to be on 4-byte boundaries (so that they are aligned). Now, the question is what to do when free is called. Because free will be handed (you hope) the address returned by malloc, the free code will have to look for the size 4 bytes back. So, basically LDR Rx, [Raddr, -#4]. It seems a little odd, but this is how many systems work. Information that malloc/free need to maintain ---------------------------------------------- If a program simply does allocations, then all you need to do is tack more pages onto the end of the heap and keep allocating into them. However, programs will allocate and free in an interleaved fashion, so you need to be able to keep track of the free gaps between currently active allocations. Consider the following call sequence: foo = malloc(16); bar = malloc(20); baz = malloc(24); free(bar); In this case, the 20 (actually 24 bytes) of space that were allocated to bar are now free. If these were the first three allocations in the program, that means that there are now 24 free bytes BETWEEN foo and baz. Your allocator MUST allocate into the first available free slot. So, if the code read thus: foo = malloc(16); bar1 = malloc(20); baz = malloc(24); free(bar1); bar2 = malloc(8); Then bar1 == bar2, because you can fit the bar2 allocation in the bar1 free space. What this means is that you must maintain a linked list of free records. This is known as a free list. Each node in a free list has the following structure: +-----------------------------+-----------------------------------------+ | Size (in bytes) | Address of next node | +-----------------------------+-----------------------------------------+ Now, you may well be asking yourself "where do I store the free list?" And the answer is, surprisingly enough, IN THE HEAP. This is where you need to be careful. You can easily construct functions to figure out if you have enough free space. To allocate new pages, when necessary, and to traverse and build a linked list. But where you actually store the free list nodes is perhaps the trickiest detail in this project. Allocating free nodes in the heap ---------------------------------- Here's how you should attack the free list problem. When a block is freed, write the free list node right there. In the free space. That way, when that space is allocated over it doesn't matter because that free node is no longer valid anyway. There are some issues to be aware of. Because this is a 32-bit addressable machine, all addresses are 32 bits wide (4 bytes). Size is an integer, so each free list node is 8 bytes large. What this means is that each allocation must take up AT LEAST 8 bytes. So even if the user requests 1 byte, you need to allocate 8. So, considering our hunk of code: foo = malloc(16); bar1 = malloc(20); baz = malloc(24); //line 3 free(bar1); //line 4 bar2 = malloc(8); //line 5 At line 3, the heap should look thus: |size| foo (16 bytes)|size| bar1 (20 bytes) |size| baz (24 bytes)| ^ ^ ^ | | | foo bar1 baz At line 4, the heap should look like: |size| foo (16 bytes)|size|next-node|<---16 bytes---->|size| baz (24 bytes)| ^ \____________/ ^ | | | foo free list node baz See how the free list node is written right over the spot where bar1 was? Now at line 5, the heap should look like: |size| foo (16 bytes)|size| 8 bytes |<-12 bytes->|size| baz (24 bytes)| ^ ^ ^ | | | foo bar2 baz See how the allocation happened right over the old list node? This is what you need to implement. There is a further issue, note that when bar2 was allocated, there are 12 bytes left over, but they're not in the free list. What this means is that an allocation that chops up a free area needs to create a new free list entry for the leftover space, so the heap should actually look like: |size|foo(16 bytes)|size| 8 bytes |size|next-node|4byte|size| baz(24bytes)| ^ ^ \____________/ ^ | | | | foo bar2 new free list node baz Linked List Management ---------------------- First you'll need a dummy node in static memory whose purpose is to point to the first node of the linked list. The last node in the list will have its next pointer set to 0. Therefore, if the free list is empty, the dummy node's next pointer will be 0. To insert a node into the list, simply tack it onto the front of the list. That is to say, after creating a free-list node, set its next pointer to the dummy's next pointer, then set the dummy's next pointer to the address of the created node. This is not the most efficient organization, but it is simple. You will still have to deal with traversing the linked list in order to locate appropriately sized nodes (which is the topic of the next section). Allocation ----------- When allocating, you need to first search the free list to find the first node whose size is larger than the allocation request + 4. If such a node exists, you can just allocate right there (remembering to make a new free-list-node entry if necessary). Otherwise, you need to get a new page from the OS. You may have to fetch multiple pages if the request is for more than 1020 bytes. When you allocate over a free-list node, you need to repair the list around it. What this means is that you should keep the address of the previous node on hand, so that you can update its next-address to be the next-address of the node that you're about to overwrite. Requirements ------------- Your allocator must: -allocate into the first available free slot (not always at the end of the heap) -be able to allocate objects less than 8 bytes in size -be able to allocate objects larger than 1020 bytes in size -deal with allocations that are smaller than the free block size You may assume: -malloc() will be handed reasonable values (e.g. I won't hand it 0xFFFFFFFF as the size), nothing so large as to exhaust the address space. -free() will be handed valid, aligned addresses -the program requesting memory won't overwrite your size metadata Additional Honors Requirements ------------------------------ The honors students will be required to maintain the linked list in sorted order. The nodes will be sorted by increasing size. In order to maintain the linked list, you need to have a dummy node whose only real data is a next pointer (address) to the first node. Now, you already know where in memory you need to allocate the free node, but the question remains: "where will it go in the free list?". In order to maintain sorted order, you will need to iterate through the list, until you encounter either the end ( indicated by having the next-address = 0) or a node whose size is larger than the node you're trying to insert. Then, you know you need to insert the node at that point. The following pseudo-code should illustrate this: prev_addr = address of dummy node pointing to first node; node_addr = address of node we're inserting; node_size = size of node we're inserting; next_addr = address of first node; size = size of first node; while((size < node_size) && (next_addr != 0)) { prev_addr = next_addr; next_addr = next addr in current node; size = size of current node; } //at this point we've encountered the insertion point our node next address = next_addr; next address of node pointed at by prev_addr = node_addr; Now, remember in order to get at the fields of the node, you need to load off of the base address (in this case, either prev_addr, next_addr or node_addr) with an offset so that it loads the appropriate field (0 for size, 4 for next address). Similarly to set these values you need to use STR with a base address and an offset. Also note that I've danced around issues such as what to do when your free list is empty. Hints ---------- -Start early. This project will take a while. -Get the linked list stuff working first. Make sure your insertion and searching code works before you use it to handle allocations. -Start with simple allocations. Make sure you can allocate 16 byte objects before you worry about allocating 1 byte and 4096 byte objects.