[project @ 2002-10-21 11:38:53 by simonmar]

[project @ 2002-10-21 11:38:53 by simonmar]
Bite the bullet and generalise the central memory allocation scheme.
Previously we tried to allocate memory starting from a fixed address,
which was set for each architecture (0x5000000 was a common one), and
to decide whether a particular address was in the heap or not we would
do a simple comparison against this address.

This doesn't work too well, because:

 - if we dynamically-load some objects above the boundary, the
   heap-allocated test becomes invalid

 - on windows we have less control, and the heap might be
   split into multiple sections

 - it turns out that on some Linux kernels we don't get memory where
   we asked for it.  This might be a bug in those kernels, but it
   exposes the fragility of our allocation scheme.

The solution is to bite the bullet and maintain a table mapping
addresses to a value indicating whether that address is in the heap or
not.  Since we normally allocate heap in chunks of 1Mb, the table is
quite small: 4k on a 32-bit machine, using one byte for each 1Mb
block.  Testing an address for heap residency now involves a memory
access, but the table is normally cache-resident.  I didn't manage to
measure any slowdown after making the change.

On a 64-bit machine, we'll need to use a 2-level table; I haven't
implemented that yet.

Now we can generalise the procedure used to grab memory from the OS.
In the general case, we allocate one megablock more than we need to,
and trim off the slop around the allocation to leave an aligned chunk.
The next time around, however, we try to allocate memory right after
the last chunk allocated, on the grounds that it is aligned and
probably free: if this doesn't work, we have to back off to the
general mechanism (it seems to work most of the time).

This cleans up the Windows story too: is_heap_alloced() has gone, and
we should be able to handle more than 256M of memory (or whatever the
arbitrary limit was before).

MERGE TO STABLE (after lots of testing)