"become" in C++: shepherds and shepherd stubs

[Mark S. Miller <mark>]

Abstract: The shepherd/stub/flock system for transparently migrating
objects between core and disk must seem to replace an object of one
class with an object of a different class.  A cheap and completely
portable way to do this in C++ is explained.

When a shepherd is purged from memory, it must be "replaced" by a
shepherd stub which can fault the original back in.  All those which
point (or seem to point) at the shepherd must now effectively point at
the stub.  Later when one of the shepherd's client's sends the stub a
shepherd message, the stub must "fault" the shepherd (and its flock)
into core, and forward the message.  Once the shepherd is brought back
in, all those which (seemed to) point at the stub must now (seem to)
point at the shepherd.

Ideally, the shepherd and the stub would simply occupy the same block
of memory (one at a time, of course).  Originally, I thought there was
no way to do this portably in C++, so we've gone down another route
instead (which is why you see "seems to" above).  Oops, I was wrong.
It's no problem.  All you do is pre-allocate a chunk of memory which
is the max of the sizes of the shepherd & the stub.  When (for
example) the shepherd wants to turn into the stub, it explicitly
invokes its own destructor (not "delete") to vacate the memory, and
then uses a variation on the two argument "operator new" trick to
construct the stub in the same chunk of memory.  Both the class of the
shepherd object and the class of the stub should be private subclasses
of some more public common superclass.  All the client's pointers
should be declared as pointing to this (or some higher) common
superclass, and so everything remains properly type safe across the
replacement.

Some relevant code snippets:

CLASS(Heaper,Tofu) {
    ...
    static void *operator new (size_t actualSize, void* space, size_t maximumSize);
    ...
};

/*
This version of operator new lets an object be constructed in already
allocated space, after verifying that the space is indeed big enough.
The maxSize stuff can probably go away, as it would be quite a serious
bug in a simple mechanism for it to fail, and it is substantial
trouble to keep track of.
*/

void *Heaper::operator new (size_t actualSize, void* space, size_t maxSize)
{
    if (actualSize > maxSize) {
	BLAST(TOO_BIG_TO_OCCUPY_SPACE);
	/* purely for debugging */
    }
    return space;
}

CLASS(CommonType,...) {
    ...
    virtual int foo (int i) DEFERRED_FUNC;
	/* senders of foo only see this declaration */
    ...
};

CLASS(SomeShepherdClass,CommonType) {
    ...
    virtual void becomeStub ();
    ...
    LEAF int foo (int i);
	/* actually implements the foo request */
    ...
};

/* the pseudo-constructor */

STRP(SomeShepherdClass) someShepherdClass (...)
{
    size_t maxSize = max (sizeof (SomeShepherdClass), 
			  sizeof (CorrespondingStubClass));
    void * storage = Heaper::operator new (maxSize);
	/* actually allocate the storage */

    return new (storage, maxSize) SomeShepherdClass (...);
	/* construct the shepherd in it */
}

void SomeShepherdClass::becomeStub ()
{
    size_t maxSize = myMaxSize; /* I'm not seriously proposing an */
				/*  instance variable.  This is */
				/*  an example */
    this->SomeShepherdClass::~SomeShepherdClass();
	/* I vacate the memory without deallocating it */

    new (this, maxSize) CorrespondingStubClass (...);
	/* The stub is constructed in my memory */
}

CLASS(CorrespondingStubClass,CommonType) {
    ...
    LEAF int foo (int i);
	/* actually implemented only in the shepherd.  This */
	/* foo must fault the shepherd in and forward the */
	/* message */
    ...
};

int CorrespondingStubClass::foo (int i)
{
    size_t maxSize = myMaxSize;
    this->CorrespondingStubClass::~CorrespondingStubClass();
	/* I vacate the memory without deallocating it */

    WPTR(SomeShepherdClass) newSelf;
    newSelf = new (this, maxSize) SomeShepherdClass (...);
	/* The shepherd is reconstructed in my memory */
	/* The "..." above is actually a bit of a moose. */
	/* I suspect it will be mediated by the relevent */
	/* recipe & SnarfXcvr */
    
    return newSelf->foo (i);
	/* Forward the message.  We use "newSelf" instead of "this" */
	/* purely to be strictly safe accoring to the language */
	/* rules */
}


All clear?