VMALLOC(3)

NAME
vmalloc - virtual memory allocation

SYNOPSIS

  Regions

  Allocation functions

  Debugging

  Profiling

  Information and statistics

  Malloc-compatible functions

DESCRIPTION
These functions for dynamic storage allocation work in regions of memory. Each region has an allocation method for parceling out blocks of storage and a memory discipline for obtaining raw space. Automatic locking prevents interference by reentrant access to a region.

Pointers to space have type Void_t* where Void_t is #defined as void if possible, otherwise char. Space is counted in type size_t.

  Regions
Regions have type Vmalloc_t. Two predefined regions are pointed to by:

A general-purpose region, with best-fit
    allocation, and Unix memory discipline Vmdcsbrk.

These functions manipulate regions:

vmopen creates a region with memory discipline disc, allocation method meth, and a setting for control flags. It returns a pointer to the region on success and NULL on failure. The flags, represented by bit values or-ed together, are:

Disable locking and consistency checks, except under method Vmdebug.

Place tracing messages for each allocation event
    on the tracing file established by vmtrace. This turns off VM_TRUST.

VM_DBCHECK, VM_DBABORT
    See Debugging below.

vmclose closes a region and releases all associated memory according to the region's discipline. The first segment obtained from the discipline's memoryf function (see `Disciplines' below) will be the last released. vmclose returns -1 on failure and a non-negative value otherwise.

vmclear frees all allocated blocks in region regardless of methods. It returns -1 on failure and a non-negative value otherwise.

vmcompact releases as much of a region's free space to its discipline's memoryf function as possible. It returns a nonnegative value on success and -1 on failure.

vmset adjusts and queries a region's flags. The indicated flags are turned on if type is nonzero, off if zero. vmset returns the previous value of all flags. Thus, vmset(region,0,0) queries the flags without changing them. In addition to the settable flags, one of VM_MTBEST, VM_MTDEBUG, VM_MTPROFILE, VM_MTPOOL, or VM_MTLAST is returned to indicate the method used in creating the region.

vmdisc changes the discipline of region to the given new discipline disc if disc is not NULL and its memoryf function is the same as the current discipline. If the current discipline has an exceptf function, it will be called with event VM_DISC. This function always returns the current discipline.

vmmopen creates a region to allocate memory given by the system call mmap(2). The given file is the backing store for the map. If base is not NULL, it is the address to map the data to. The round argument asserts that the size of the region is always a multiple of this value. However, note that vmmopen has an internally defined minimum round value (typically 64K) which may be used if the given value is too small.

vmmset sets a data value associated with a key if set is non-zero. In this case, it returns the current data (before setting) if key is previously set; otherwise, it returns the new data value. If the argument set is zero, the call returns the current data value associated with key, if any.

  Allocation functions
vmalloc returns a pointer to a block of the requested size in a region, aligned to the strictest alignment that is suitable for the needs of any basic data type. It returns NULL on failure.

vmalign works like vmalloc, but returns a block aligned to a common multiple of align and the strictest alignment.

vmresize attempts to change the length of the block pointed to by addr to the specified size. If that is impossible and type has at least one of VM_RSMOVE and VM_RSCOPY, a new block is allocated and the old block is freed. The bit VM_RSCOPY also causes the new block to be initialized with as much of the old contents as will fit. When a resized block gets larger, the new space will be cleared if type has the bit VM_RSZERO. vmresize returns a pointer to the final block, or NULL on failure. If addr is NULL, vmresize behaves like vmalloc; otherwise, if size is 0, it behaves like vmfree.

vmfree makes the currently allocated block pointed to by addr available for future allocations in its region. If addr is NULL, vmfree does nothing. It returns -1 on error, and nonnegative otherwise.

vmnewof is a macro function that attempts to change the length of the block pointed to by addr to the size n*sizeof(type)+x. If the block is moved, new space will be initialized with as much of the old content as will fit. Additional space will be set to zero.

vmoldof is similar to vmnewof but it neither copies data nor clears space.

vmgetmem provides a handy function to creat/close regions and allocate/free memory based on chunks of memory obtained from the heap region Vmheap.

This call opens a new region.

This call closes the given region.

This call allocates a block of length n and clears it to zeros.

This call frees the block p.

This call resizes the block p to length n
    and clears the new memory to zeros if the block grows. The block may be moved as deemed necessary by the allocator.

  Memory disciplines
Memory disciplines have type Vmdisc_t, a structure with these members:

If this value is positive, all size arguments to the
    memoryf function will be multiples of it.

Points to a function to get or release segments of space for the
    region.

If this pointer is not NULL,
    the function it points to is called to announce events in a region.

There are two standard disciplines. In both, round is 0, and exceptf is NULL.

A discipline whose memoryf function gets space from sbrk(2).

A discipline whose memoryf function gets space from the region Vmheap.
    A region with Vmdcheap discipline and Vmlast allocation is good for building throwaway data structures.

A memoryf function returns a pointer to a memory segment on success, and NULL on failure. If csz is 0, the function returns a new segment of size nsz. Otherwise, the function attempts to change the length of the segment pointed to by addr from csz to nsz. If this is successful, memoryf should return addr (even if nsz is 0).

An exceptf function is called for events identified by type, which is coded thus:

A new region is being opened.
    If exceptf returns a zero value, the region opening proceeds normally. A negative return value causes vmopen to terminate with failure. A positive return value indicates that the new region is to manipulate memory already initialized by a previous vmopen call (perhaps by a different process on persistent or shared memory). In this case, the argument (Void_t**)obj should return the initial segment (which is of type (Void_t*)). vmopen will return failure if this segment is not returned or if it has not been properly initialized.

The region is being closed.
    The return value of exceptf is significant as follows. If negative, vmclose immediately returns with failure. If zero, vmclose proceeds normally by calling memoryf to free all allocated memory segments and also freeing the region itself. Finally, if positive, vmclose will only free the region without deallocating the allocated segments.

An attempt to extend the region by the amount
    (size_t)obj failed. The region is unlocked, so the exceptf function may free blocks. If the function returns a positive value the memory request will be repeated. If zero, the allocation method will again invoke memoryf to get space. If negative, the allocation request will fail.

Address obj, given to vmfree or vmresize,
    does not point to an allocated block from the region. The respective call will fail.

Announce that a memory allocation request is finished and returning.

Announce that a memory freeing request is finished and returning.

Announce that a memory resizing request is finished and returning.

  Allocation methods
There are five methods, of type Vmethod_t*:

An approximately best-fit allocation strategy.

A strategy for building structures that are only deleted in whole.
    Only the latest allocated block can be freed. This means that as soon as a block a is allocated, vmfree calls on blocks other than

A strategy for blocks of one size,
    set by the first vmalloc call after vmopen or vmclear.

An allocation strategy with extra-stringent checking and locking
    regardless of the VM_TRUST flag. It is useful for finding misuses of dynamically allocated memory, such as writing beyond the boundary of a block, or freeing a block twice.

An allocation method that records and prints summaries of memory usage.

  Debugging
The method Vmdebug is used to debug common memory violation problems. When a problem is found, a warning message is written to file descriptor 2 (standard error). In addition, if flag VM_DBABORT is on, the program is terminated by calling abort(2). Each message is a line of self-explanatory fields separated by colons. The optional flag -DVMFL, if used during compilation, enables recording of file names and line numbers. The following functions work with method Vmdebug.

vmdebug resets the file descriptor to write out warnings to the given argument. By default, this file descriptor is 2, the standard error. vmdebug returns the previous file descriptor.

vmdbcheck checks a region using Vmdebug or Vmbest for integrity. If Vmdebug, this also checks for block overwriting errors. On errors, vmdbwarn is called. If flag VM_DBCHECK is on, vmdbcheck is called at each invocation of vmalloc, vmfree, or vmresize.

vmdbwatch causes address addr to be watched, and reported whenever met in vmalloc, vmresize or vmfree. The watch list has finite size and if it becomes full, watches will be removed in a first-in-first-out fashion. If addr is NULL, all current watches are canceled. vmdbwatch returns the watch bumped out due to an insertion into a full list or NULL otherwise.

vmdbwarn is an internal function that processes warning messages for discovered errors. It can't be called from outside the vmalloc package, but is a good place to plant debugger traps because control goes there at every trouble.

  Profiling
The method Vmprofile is used to profile memory usage. Profiling data are maintained in private memory of a process so Vmprofile should be avoided from regions manipulating persistent or shared memory. The optional flag -DVMFL, if used during compilation, enables recording of file names and line numbers.

vmprofile prints memory usage summary. The summary is restricted to region vm if vm is not NULL; otherwise, it is for all regions created with Vmprofile. Summary records are written to file descriptor fd as lines with colon-separated fields. Here are some of the fields:

n_alloc,n_free:
    Number of allocation and free calls respectively. Note that a resize operation is coded as a free and an allocation.

s_alloc,s_free:
    Total amounts allocated and freed. The difference between these numbers is the amount of space not yet freed.

max_busy, extent:
    These fields are only with the summary record for region. They show the maximum busy space at any time and the extent of the region.

  Information and statistics
vmregion returns the region to which the block pointed to by addr belongs. This works only in regions that allocate with Vmbest, Vmdebug or Vmprofile. If multiple regions manipulate the same segment of memory, vmregion returns the region that causes the creation that memory segment.

vmsegment finds if some segment of memory in region contains the address addr. It returns the address of a found segment or NULL if none found.

vmwalk walks all segments in region or if region is NULL, all segments in all regions. At each segment, (*walkf)(vm,addr,size,disc) is called where vm is the region, addr is the segment, size is the size of the segment, and disc is the region's discipline. If walkf returns a negative value, the walk stops and returns the same value. On success, vmwalk returns 0; otherwise, it returns -1.

vmaddr checks whether addr points to an address within some allocated block of the given region. If not, it returns -1. If so, it returns the offset from the beginning of the block. The function does not work for a Vmlast region except on the latest allocated block.

vmsize returns the size of the allocated block pointed to by addr. It returns -1 if addr does not point to a valid block in the region. Sizes may be padded beyond that requested; in particular no block has size 0. The function does not work for a Vmlast region except on the latest allocated block.

vmstat gathers statistics on the given region and returns that information in the Vmstat_t structure pointed to by statb. A Vmstat_t structure has at least these members:

Bookeeping overhead is counted in extent, but not in s_busy or s_free.

vmtrace establishes file descriptor fd as the trace file and returns the previous value of the trace file descriptor. The trace descriptor is initially invalid. Output is sent to the trace file by successful allocation events when flag VM_TRACE is on.

Tools for analyzing traces are described in mtreplay(1). The trace record for an allocation event is a line with colon-separated fields, four numbers and one string.

old Zero for a fresh allocation; the address argument for freeing and resizing.

new Zero for freeing; the address returned by allocation or resizing.

size
    The size argument for allocation or resizing; the size freed by freeing. Sizes may differ due to padding for alignment.

region
    The address of the affected region.

method
    A string that tells the region's method: best, last, pool, profile, or debug.

vmtrbusy outputs a trace of all currently busy blocks in region vm. This only works with the Vmbest, Vmdebug and Vmprofile methods.

  Malloc-compatible functions
Functions in this set provide the behaviors of malloc(3). The functions memalign and valloc allocate aligned blocks; valloc further restricts alignment to page boundaries.

The malloc functions are instrumented for run-time debugging, profiling and tracing. When these modes are enable, time and space performance will be affected. For accurate reporting of files and line numbers, code should include vmalloc.h and compile with -DVMFL. The following environment variables should be set before any memory allocation (e.g., before a process starts). They drive different modes:

VMETHOD
    This defines the method to use for allocation. Its value should be one of the strings: Vmbest, Vmdebug, Vmprofile, Vmlast, Vmpool. The 'V' can be in lower case.

VMDEBUG
    This is ignored if a method other than Vmdebug has been selected with VMETHOD. VMDEBUG can be any combination of `a', a decimal number and a list of hexadecimal numbers. `a' causes the program to abort on any discovered allocation error. A hexadecimal number starts with either 0x or 0X and defines an address to watch (see vmdbwatch). Any other number is taken to be decimal and defines a period p to check the arena for integrity. The default period is 1, ie, the arena is checked on every call to a malloc function. Other letters not part of the defined set are ignored.

VMPROFILE
    This is ignored if a method other than Vmprofile has been selected by VMETHOD or VMDEBUG. VMPROFILE defines a file name to store profile data. Each instance of the pattern `%p' found in VMPROFILE is transformed to the process id of the running process. If the file cannot be created, file descriptor 2 (standard error) is used for output.

VMTRACE
    If this defines a valid writable file, trace messages of all allocation calls are written to the given file (see vmopen() and vmtrace()). Similar to VMPROFILE, each instance of the pattern `%p' found in VMTRACE is turned to the process id of the running process.

RECENT CHANGES
Vmlast: allocated blocks are now allowed to be resized (09/1998).

SEE ALSO
mtreplay(1), malloc(3).

AUTHOR
Kiem-Phong Vo, kpv@research.att.com