Process Memory Concepts

1.One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e., not all of these addresses actually can be used to store data.

2.The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a frame) or some secondary storage, usually disk space.

3.The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it – there’s just a flag saying it is all zeroes. The same frame of real memory or backing store can back multiple virtual pages be- longing to multiple processes. 

4.This is normally the case, for example, with virtual memory occupied by GNU C Library code. The same real memory frame containing the printf function backs a virtual memory page in each of the existing processes that has a printf call in its program.

5.In order for a program to access any part of a virtual page, the page must at that moment be backed by (“connected to”) a real frame.

-->But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called paging.

6.When a program attempts to access a page which is not at that moment backed by real memory, this is known as a page fault.

-->When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called “paging in” or “faulting in”), then resumes the process so that from the process’ point of view, the page was in real memory all along.

7. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions  can control it. Within each virtual address space, a process has tokeep track of what is at which addresses, and that process is called memory allocation. 

8.Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that’s not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn’t used tostore two different things.

9.Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it’s not very interesting. Exec is the operation of creating a virtual address space for a process, loading its basic program into it, and executing the program.

-->It is done by the “exec” family of functions (e.g. execl). The operation takes a program file (an executable), it allocates space to load all the data in the executable, loads it, and transfers control to it. That data is most notably the instructions of the program (the text), but also literals and constants in the program and even some variables

-->C variables with the static storage class  Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C Library, there are two kinds of programmatic allocation: automatic and dynamic.

10.Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process’ addresses shall be identical to the contents of a specified regular file.

-->The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory.

11.Just as it programmatically allocates memory, the program can programmatically deal- locate (free) it. You can’t free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot.

12.Program Termination A process’ virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are:
•The text segment contains a program’s instructions and literals and static constants.It is allocated by exec and stays the same size for the life of the virtual address space.
• The data segment is working storage for the program. It can be preallocated and preloaded by exec and the process can extend or shrink it by calling functions [Resizing the Data Segment].Its lower end is
fixed.
• The stack segment contains a program stack. It grows as the stack grows, but doesn’t
shrink when the stack shrinks.
------->Allocating Storage For Program Data
This section covers how ordinary programs manage storage for their data, including the famous malloc function and some fancier facilities special the GNU C Library and GNU
Compiler.

-->Memory Allocation in C Programs
The C language supports two kinds of memory allocation through the variables in C pro-
grams:
• Static allocation is what happens when you declare a static or global variable. Each
static or global variable defines one block of space, of a fixed size. The space is allocated once, when your program is started (part of the exec operation), and is never freed.
• Automatic allocation happens when you declare an automatic variable, such as a func-tion argument or a local variable. The space for an automatic variable is allocated
when the compound statement containing the declaration is entered, and is freed when that compound statement is exited.
In GNU C, the size of the automatic storage can be an expression that varies. In other C implementations, it must be a constant.
A third important kind of memory allocation, dynamic allocation, is not supported by
C variables but is available via GNU C Library functions.
-->Dynamic Memory Allocation
Dynamic memory allocation is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known beforethe program runs.

-->For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line.
-->May need a block for each record or each definition in the input data; since you can’t know in advance how many there will be, you must allocate a new block for each record or definition as you read it.


--->When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want.

--->Dynamic allocation is not supported by C variables; there is no storage class “dynamic”, and there can never be a C variable whose value is stored in dynamically allocated space.

--->The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C Library function call), and the only way to refer to dynamically allocated space is through a pointer.

-->Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamicallocation only when neither static nor automatic allocation will serve.
For example, if you want to allocate dynamically some space to hold a struct foobar,you cannot declare a variable of type struct foobar whose contents are the dynamicallyallocated space.

-->But you can declare a variable of pointer type struct foobar * and assignit the address of the space. Then you can use the operators ‘*’ and ‘->’ on this pointer
variable to refer to the contents of the space:
{
struct foobar *ptr
= (struct foobar *) malloc (sizeof (struct foobar));
ptr->name = x;
ptr->next = current_foobar;
current_foobar = ptr;
}
--->Unconstrained Allocation
The most general dynamic allocation facility is malloc. It allows you to allocate blocks ofmemory of any size at any time, make them bigger or smaller at any time, and free theblocks individually at any time (or never).
-->Basic Memory Allocation
To allocate a block of memory, call malloc. The prototype for this function is in ‘stdlib.h’.
void * malloc (size t size)
[Function]
This function returns a pointer to a newly allocated block size bytes long, or a nullpointer if the block could not be allocated.The contents of the block are undefined; you must initialize it yourself (or use calloc
the value as a pointer to the kind of object that you want to store in the block. Herewe show an example of doing so, and of initializing the space with zeros using the library
function memset
struct foo *ptr;
...
ptr = (struct foo *) malloc (sizeof (struct foo));
if (ptr == 0) abort ();
memset (ptr, 0, sizeof (struct foo));
You can store the result of malloc into any pointer variable without a cast, becauseISO C automatically converts the type void * to another type of pointer when necessary.But the cast is necessary in contexts other than assignment operators or if you might wantyour code to run in traditional C.
Remember that when allocating space for a string, the argument to malloc must be oneplus the length of the string. This is because a string is terminated with a null characterthat doesn’t count in the “length” of the string but does need space. For example:
char *ptr;
...
ptr = (char *) malloc (length + 1);

---> Examples of malloc
If no more space is available, malloc returns a null pointer. You should check the value ofevery call to malloc. It is useful to write a subroutine that calls malloc and reports anerror if the value is a null pointer, returning only if the value is nonzero. This function isconventionally called xmalloc. Here it is:
void *
xmalloc (size_t size)
{
register void *value = malloc (size);
if (value == 0)
fatal ("virtual memory exhausted");
return value;
}
Here is a real example of using malloc (by way of xmalloc). The function savestringwill copy a sequence of characters into a newly allocatednull terminated string:
char *
savestring (const char *ptr, size_t len)
{
register char *value = (char *) xmalloc (len + 1);
value[len] = ’\0’;
return (char *) memcpy (value, ptr, len);
}


--->The block that malloc gives you is guaranteed to be aligned so that it can hold anytype of data. On GNU systems, the address is always a multiple of eight on most systems,and a multiple of 16 on 64-bit systems.

-->Only rarely is any higher boundary (such as apage boundary) necessary; for those cases, use memalign, posix_memalign or valloc Note that the memory located after the end of the block is likely to be in use for somethingelse; perhaps a block already allocated by another call to malloc.

--> If you attempt to treat the block as longer than you asked for it to be, you are liable to destroy the data thatmalloc uses to keep track of its blocks, or you may destroy the contents of another block.If you have already allocated a block and discover you want it to be bigger, use realloc
--->Freeing Memory Allocated with malloc
When you no longer need a block that you got with malloc, use the function free to make the block available to be allocated again. The prototype for this function is in ‘stdlib.h’.
--->void free (void *ptr)
The free function deallocates the block of memory pointed at by ptr.
void cfree (void *ptr)
This function does the same thing as free. It’s provided for backward compatibilitywith SunOS; you should use free instead.

Freeing a block alters the contents of the block. Do not expect to find any data (such asa pointer to the next block in a chain of blocks) in the block after freeing it. Copy whateveryou need out of the block before freeing it! Here is an example of the proper way to free all the blocks in a chain, and the strings that they point to:
struct chain
{
struct chain *next;
char *name;
}
void
free_chain (struct chain *chain)
{
while (chain != 0)
{
struct chain *next = chain->next;
free (chain->name);
free (chain);
chain = next;
}
}
Occasionally, free can actually return memory to the operating system and make theprocess smaller. Usually, all it can do is allow a later call to malloc to reuse the space. Inthe meantime, the space remains in your program as part of a free-list used internally by malloc.

---->There is no point in freeing blocks at the end of a program, because all of the program’s space is given back to the system when the process terminates.

-->Changing the Size of a BlockOften you do not know for certain how big a block you will ultimately need at the time youmust begin to use the block. For example, the block might be a buffer that you use to holda line being read from a file; no matter how long you make the buffer initially, you mayencounter a line that is longer.
You can make the block longer by calling realloc. This function is declared in
‘stdlib.h’.
-->void * realloc (void *ptr, size t newsize)
The realloc function changes the size of the block whose address is ptr to be newsize.Since the space after the end of the block may be in use, realloc may find it necessary to copy the block to a new address where more free space is available.

--->The value of realloc is the new address of the block. If the block needs to be moved, realloc copies the old contents.If you pass a null pointer for ptr, realloc behaves just like ‘malloc (newsize)’.

-->This can be convenient, but beware that older implementations (before ISO C) may not support this behavior, and will probably crash when realloc is passed a null pointer.

--->Like malloc, realloc may return a null pointer if no memory space is available to make the block bigger. When this happens, the original block is untouched; it has not been modified or relocated.

---> In most cases it makes no difference what happens to the original block when realloc fails, because the application program cannot continue when it is out of memory, and the only thing to do is to give a fatal error message. Often it is convenient to write and use a subroutine, conventionally called xrealloc, that takes care of the error message as xmalloc
does for malloc:
void *
xrealloc (void *ptr, size_t size)
{
register void *value = realloc (ptr, size);
if (value == 0)
fatal ("Virtual memory exhausted");
return value;
}
You can also use realloc to make a block smaller. The reason you would do this is to avoid tying up a lot of memory space when only a little is needed.

-->In several allocation implementations, making a block smaller sometimes necessitates copying it, so it can fail if no other space is available. If the new size you specify is the same as the old size, realloc is guaranteed to change
nothing and return the same address that you gave.
-->Allocating Cleared Space
The function calloc allocates memory and clears it to zero. It is declared in ‘stdlib.h’.
void * calloc (size t count, size t eltsize)
This function allocates a block long enough to contain a vector of count elements, each of size eltsize. Its contents are cleared to zero before calloc returns.
You could define calloc as follows:
void *
calloc (size_t count, size_t eltsize)
{
size_t size = count * eltsize;
void *value = malloc (size);
if (value != 0)
memset (value, 0, size);
return value;
}
But in general, it is not guaranteed that calloc calls malloc internally. Therefore, if an application provides its own malloc/realloc/free outside the C library, it should always define calloc, too.
-->Efficiency Considerations for malloc
As opposed to other versions, the malloc in the GNU C Library does not round up block sizes to powers of two, neither for large nor for small sizes. Neighboring chunks can be coalesced on a free no matter what their size is.

--->This makes the implementation suitable for all kinds of allocation patterns without generally incurring high memory waste through fragmentation. Very large blocks (much larger than a page) are allocated with mmap (anonymous or via /dev/zero) by this implementation.

--->This has the great advantage that these chunks are returned to the system immediately when they are freed. Therefore, it cannot happen that a large chunk becomes “locked” in between smaller ones and even after calling free wastes memory. The size threshold for mmap to be used can be adjusted with mallopt. The use of mmap can also be disabled completely.

---> Allocating Aligned Memory Blocks
The address of a block returned by malloc or realloc in GNU systems is always a multiple of eight (or sixteen on 64-bit systems). If you need a block whose address is a multiple of a higher power of two than that, use memalign, posix_memalign, or valloc. memalign is declared in ‘malloc.h’ and posix_memalign is declared in ‘stdlib.h’.

----->With the GNU C Library, you can use free to free the blocks that memalign, posix_ memalign, and valloc return. That does not work in BSD, however—BSD does not provide any way to free such blocks.
void * memalign (size t boundary, size t size)

--->The memalign function allocates a block of size bytes whose address is a multiple of boundary. The boundary must be a power of two! The function memalign works by allocating a somewhat larger block, and then returning an address within the block that is on the specified boundary.
int posix_memalign (void **memptr, size t alignment, size t size)

---->The posix_memalign function is similar to the memalign function in that it returns a buffer of size bytes aligned to a multiple of alignment. But it adds one requirementto the parameter alignment: the value must be a power of two multiple of sizeof (void *).
----->If the function succeeds in allocation memory a pointer to the allocated memory is returned in *memptr and the return value is zero. Otherwise the function returns an error value indicating the problem.
void * valloc (size t size)
Using valloc is like using memalign and passing the page size as the value of the
second argument. It is implemented like this:
void *
valloc (size_t size)
{
return memalign (getpagesize (), size);
}
---> [How to get information about the memory subsystem?],
more information about the memory subsystem.
-->Malloc Tunable Parameters
You can adjust some parameters for dynamic memory allocation with the mallopt function. This function is the general SVID/XPG interface, defined in ‘malloc.h’. int mallopt (int param, int value)

---->When calling mallopt, the param argument specifies the parameter to be set, and
value the new value to be set. Possible choices for param, as defined in ‘malloc.h’,
are:
M_TRIM_THRESHOLD
This is the minimum size (in bytes) of the top-most, releasable chunk
that will cause sbrk to be called with a negative argument in order to
return memory to the system.
M_TOP_PAD
This parameter determines the amount of extra memory to obtain from
the system when a call to sbrk is required. It also specifies the number of
bytes to retain when shrinking the heap by calling sbrk with a negative
argument. This provides the necessary hysteresis in heap size such that
excessive amounts of system calls can be avoided.
Summary of malloc-Related Functions
------>Here is a summary of the functions that work with malloc:
void *malloc (size_t size)
Allocate a block of size bytes.
void free (void *addr)
Free a block previously allocated by malloc.
void *realloc (void *addr, size_t size)
Make a block previously allocated by malloc larger or smaller, possibly by
copying it to a new location.
void *calloc (size_t count, size_t eltsize)
Allocate a block of count * eltsize bytes using malloc, and set its contents to
zero.
void *valloc (size_t size)
Allocate a block of size bytes, starting on a page boundary.
void *memalign (size_t size, size_t boundary)
Allocate a block of size bytes, [Allocating Aligned Memory Blocks]
int mcheck (void (*abortfn) (void))
Tell malloc to perform occasional consistency checks on dynamically allocated
memory, and to call abortfn when an inconsistency is found.
-->Virtual Memory Allocation And Paging

void *(*__malloc_hook) (size_t size, const void *caller)
A pointer to a function that malloc uses whenever it is called.
void *(*__realloc_hook) (void *ptr, size_t size, const void *caller)
A pointer to a function that realloc uses whenever it is called.
void (*__free_hook) (void *ptr, const void *caller)
A pointer to a function that free uses whenever it is called.
void (*__memalign_hook) (size_t size, size_t alignment, const void *caller)
A pointer to a function that memalign uses whenever it is called.
struct mallinfo mallinfo (void)