Re: EQObjects and garbage collection

[Michael McClary <michael>]

> From: hibbert (Chris Hibbert)
> 
> Mark,
> 
> Michael said (that he thought) that some things that were EQObjects
> were that way partly because they didn't want to be in the garbage
> collection system.  (e.g.  Category objects) They therefor didn't have
> SPTRs or CHKPTRs, so the effort wouldn't be spent on them.  I
> currently generate
> 
> DEFINE_CHKPTR($(CLASS));
> DEFINE_SPTR($(CLASS));
> 
> for ALL Heapers, and there isn't any attribute that tells me not to
> generate these definitions.  I think classes for which CHECKED_CLASS
> isn't supplied should be DONT_GC.  Does that seem right?  Would you
> rather that they didn't declare any attributes, and so had to do
> everything manually?

I started this letter indending to clarify that slightly, segued into
correcting my previous statements to Chris substantially, and in the
process I think I've just come up with something important - which is
why this is going to xtech.  ('Scuse me for quoting you without asking,
Chris, but I didn't think you'd want to be awakened just now.)

First the clarification and correction:

Certain objects (among them Categories and Recipies) are allocated once
at startup and never released.  Thus there is no need for the garbage
collector to be aware of them, or spend any of its time paying attention
to them.  Thus they had no CKPTRs or SPTRs defined.  (I had noticed this
when changing the WPTR() macro during the NOT_A_TYPE upgrade.)

Chris' modified script (at the time we were both working on it) attempted
to define CKPTRs and SPTRS for all Heapers.  This caused errors with
Categories and the like, because they did not have such pointers declared.

These objects happen to be EQObjects (for apparently unrelated reasons).
Chris mentioned this as were were chasing the bug in the stubble script
mods, and somehow I got the impression that:

 - EQObjects had been Tofus but not Heapers

 - They had been changed to be Heapers during the mod, and

 - This, in combination with the revised script generating *PTR references
   for all Heapers, is what caused the particular bug we were chasing.

In fact, EQObjects already were Heapers, and the bug was purely related to
the lack of the CHECKED_CLASS() macro.

As I read it now, to match what we had before:

 - All the EQObject stuff was a red herring.

 - Classes should be DONT_GC() if they had neither a CHECKED_CLASS() nor a
   CHECKED_CLASS_NOT_A_TYPE().

 - Classes should be NOT_A_TYPE() if they had a CHECKED_CLASS_NOT_A_TYPE()

 - SPTRs, CKPTRs, and WPTRs should be suppressed for anything that is
   >either< NOT_A_TYPE(), >or< DONT_GC().

Now the possibly important thing:

I note in passing (as others have noted before me) that having DONT_GC()
subclasses of a GCable class which is a TYPE creates the potential for
confusing the GC (and/or the programmer) with a *PTR(TheGCableTYPE)
pointing to a DONT_GC() object.  I think I see a way around this.
(I trust you'll forgive me if I have just reinvented something we
already have.)

Suppose DONT_GC() has the following side-effects:

 - DONT_GC() classes (and their superclasses) contain no "true" CKPTRs.

 - The DONT_GC() attribute is inherited (or required in a derived class
   if the base class has it.)

The first rule means a DONT_GC() class has a do-nothing markInstances()
method.  The second rule means a DONT_GC() class can never be subclassed
with a GCable class.  This prevents violation of the the first rule by
subclassing, so a pointer to a DONT_GC() class can never point to a class
with a non-trivial markInstances() routine.

Further, suppose we define another (non-inheritable) attribute,
"DONT_COLLECT()", which doesn't impose the CKPTR restriction.

Given this set of rules, stubble can generate for DONT_GC() classes
a set of "fake" SPTRs, CKPTRs, and WPTRs, and for both DONT_GC() and
DONT_COLLECT() an overriding definition of "operator new()".  The fake
PTRs have the same naming and operator replacement behavior as true PTRs,
but:

 - The class underlying a fake SPTR neglects the extra run-time
   behavior that links it into the bomb fuse and makes it visible
   to the garbage collector.

 - The fake CKPTR doesn't add markInstances() behavior in any class
   where it appears (and thus including a fake CKPTR in a DONT_GC()
   object doesn't violate the rules).
   
   (In its full form this will take a formic mod, to allow the formic
   script to discover whether a named class has, or has inherited, the
   DONT_GC() attribute.  We can limp along without it by prohibiting
   even fake CKPTRs in DONT_GC() classes.)

Thus the fake pointers are essentially just pointers, invisible to
garbage collection.

Finally, the replacement "operator new()" ignores the programmer's
instructions about what heap to use, and always allocates the object on
a GC-ignored heap (where objects, if freed at all, are freed manually).

This regime has the following interesting properties:

 - All rule violations can be detected by the compile-time tools.
   (We don't even need XLint for this one.  Formic/cfront/cc/ld is
    sufficient.)

 - The programmer can use SPTR/WPTR/CKPTRs to any object, regardless of
   its GCness.  Thus he can ignore the GC side-issues and concentrate
   on the data structure semantics.  ("OneOfOurTypes *" becomes an
   anacronism, which we can ferret out with Xlint or the like.)

 - The GCness of an object is controlled entirely by the attribute
   declarations in it and its superclasses.  Switching an object
   into or out of GC requires no changes to the client code.
   (This assumes any "Whatsis *"s have been fixed before the class
   is first switched into GC.)

 - We can start out with essentially everything GCed, and switch
   stuff to DONT_COLLECT() (or DONT_GC() if possible) when we are
   sure its manual deallocation is being handled correctly.

 - Alternatively, if a manually-GCed object springs a leak or starts
   evaporating prematurely, we can trivially switch it back to automatic
   GC, turn on GC enthusiasm, and track the bug down.
     - Perhaps we can fine-tune things further with an ENTHUSIASTIC()
       attribute which turns on GC enthusiasm for only the interesting
       objects.
     - Perhaps we could even jam the entire objectverse back into the
       GC by tweaking one line and recompiling.

 - DONT_GC() and DONT_COLLECT() objects can be constructed statically
   (as Recipes are).
   
 - DONT_GC() objects can even be allocated on the stack.

 - Yet another attribute, STACKABLE(), could cause the automatic
   generation of a (set of) subclass(es) of the implementation class
   that combined SPTR-like and DONT_COLLECT() behavior.  Such an object
   could be constructed on the stack, with no allocater hit and no more
   overhead than an SPTR, yet it can still use true CKPTRs to point to
   heap-allocated objects while serving as a GC root (thus participating
   fully in GC), and perform all the functions of a genuine Heaper.
   Except for the constructor distinction and the slightly increased size,
   it is indistinguishable from its base class.  (This can even be done
   without mods to the GC, just by adding the guts of the SPTR to the
   base class.)
    - (mayBecome() size tweaks might require a finnesse, such as a union
       or padding.)
    - (A STACKABLE() DONT_GC() class' subclass would automatically be the
       DONT_GC() class itself, with no GC overhead or programmer distraction.)

And the best part is that the changes needed are tiny, and can be
applied incrementally as they become useful.

Isn't that sweet?

	michael

PS:  A little algorithm for checking for improper deletion, either of
     normal GC objects or ones with the new DONT_COLLECT() attribute,
     under mark-and-sweep:

	- Hold the objects to be deleted by WPTR()s.
	- Delete them.
	- Force a garbage collection (or the 'mark' pass).
	- If any of 'em got marked, oopsie!