Revision 2016-01-14
Background
Early Forth systems had little error-checking and it was very
easy to crash the system. In the late 1970's Fig-Forth added
compiler security and other checks. By the mid 1990's Forth-94
had standardized 58 error conditions. Today's Forth
compilers are capable of detecting all critical error conditions,
but one ... out-of-memory !
The problem with ALLOT
To understand why it is difficult for Forth to reliably detect
out-of-memory conditions (THROW
code -8 "dictionary
overflow") one must look at its memory allocation word ALLOT . Traditionally this
function was defined as follows:
: ALLOT ( n|u -- ) DP +! ;
where DP is a variable containing the address returned by HERE and n|u is the number of units to be allotted.
e.g. 10 ALLOT will increment the Dictionary Pointer by 10, thereby reserving 10 address units (bytes) of data space
But what if a program attempts to reserve more space than is
available? If ALLOT were defined
as above without any
memory check, the result is unpredictable. In all likelihood the
system would become unstable or crash. To avoid such a
scenario many of today's Forth systems include a memory check
within ALLOT. If insufficient
memory is available when
ALLOT is executed, it will be
detected and the function will abort with an error message or
number. At least this is the
hope.
The reason why memory checking in ALLOT
doesn't always work is due to another feature of ALLOT - the ability to
reclaim data space previously allotted. Just as 10 ALLOT reserves 10
bytes of data space, so -10
ALLOT will
return 10 bytes back to the system.
The problem with ALLOT is that
2's complement integer representation (most machines) cannot
distinguish between a
signed negative number and its corresponding unsigned positive
number. On a 16-bit system -9216
ALLOT can be
a valid operation; at the same time its alternate representation 55 1024 * ALLOT can
be an overflow condition.
Because ALLOT is required to
support the former, it may not exclude the latter. That makes
detecting out-of-memory
problematic, if not impossible.
Re-thinking ALLOT
As originally devised ALLOT
was very flexible. It worked with any number, signed or unsigned.
But this flexibility came
at the cost of compiler security. Today's Forth compilers
emphasize robustness, so with this in mind the following
re-working of ALLOT is presented.
To secure effective memory checking when ALLOTing data space, negative values
must be excluded. The easiest way to
achieve this is to split the existing functionality of ALLOT into two words:
: ALLOT ( u -- ) DUP UNUSED U> ABORT" out of memory" DP +! ;
: -ALLOT ( +n|u -- ) NEGATE DP +! ;
Under this scheme ALLOT is
used to reserve data space as before, while -ALLOT is used to reclaim data space
previously
allotted. In this implementation -ALLOT
has no memory checking on the grounds a programmer knows what he/she
is doing.
For non two's complement machines -ALLOT
is limited to positive signed numbers.
Compatibility
It may be argued such a change to ALLOT
would break existing code. In response it is suggested it is a
change that should
have occurred long ago. In practice relatively few programs will
break, and when they do, it is likely to be accompanied
by an out-of-memory error. From there it is a simple
matter of finding the offending ALLOT and
replacing it with
NEGATE -ALLOT .
History
2015-01-10 First release
2016-01-14 Revised text
Page updated: 2016-01-14