Re: How to Restart an Object

[Michael McClary <michael>]

Abstract:

 - Reply to markm's letter.  Kudos, nits.
 - Something I've done to Xcvrs is closely related.
 - Semantics, modularity & containment confusions with SELF_COPY (data
   compression, protocol definition, restart/resonstitution mashed together)
   are now resolved.
 - An alternate interpretation of the design, phrased as pieces of a manual
   entry (with the emphasis changed from historical derivation to
   logic-of-the-design.)
 - Don't rock the boat: >I'M< already rocking it for you.  B-)



(I have reorganized the class hierarchy starting at Transceiver as part
 of the SnarfTransceiver revisions.  The top of the new hierarchy is the
 class Xcvr, which is essentially the subset of the former Transceiver
 class which applies to COPY classes.  Therefore, I will use Xcvr rather
 than Transceiver in this reply.)

The mechanism markm describes is, in my opinion, >precisely< the correct
one.  (Bravo!)  It's nice that implementing it is such a trivial change
(especially as compared to the consequences of not implementing it), and
that so much of it is already done (for instance, the changes I made a
few days ago to IntegerVar eliminated the original reason it was SELF_COPY).

The SELF_COPY mechanism had some semantic problems.  In the one place
the full form of it was used (IntegerVar), it confused the issues of
efficient transmission of data structures with efficient transmission
of each datum, stole protocol definition out of the Xcvr class
containment boundary, and then attempted to apply its data compression
only to the variables it sent, not to all values of the same type.
(All fixed now, though not yet merged.)

> Because not all instance variables of a "COPY(Foo):" class are
> necessarily saved and restored, some mechanism must be provided to
> cause the rest to be re-initialized on object creation.  We need a
> hook so that an optional user provided restart method will be called
> by the stubble-generated constructor.

Perhaps a better way to phrase this is that the COPY variables of a COPY
object may not encode all of the object's state.  (Hmmm... It's easier to
describe the implications of this interpretation by writing more "manual
entry" in this alternate language {and assuming Markm's approach is already
installed} than to compose it as a reply to Markm.  Here goes.
First - background:)
========================================================

Ideally, the contents of a COPY object's member variables would represent
the entire state of the COPY object, and vice-versa.  Such an object could
be saved or sent to another environment by saving or sending these values,
and the object could be reconstituted in its new environment by constructing
a new object from these values, translated as necessary for their new
environment.  (Pointers must be modified to point to the new environment's
equivalent target, number representations might change, and so on.)

In practice, many classes can work this way, but occasional classes
must violate one or both halves of the ideal.

First, some of the variables might not be significant to the object's
persistent state.  Some, for example, might encode transient state
during execution of member functions, and be insignificant when the
object is being stored or transmitted.

-----------
(Now to the meat:)
-----------

Second, some of the object's state might be something other than the
values of its variables.  Some might, for instance, be the object's
side-effects on its environment.  Examples:

  - The object might hold a lock, and a reconstituted version would
    have to acquire an equivalent lock.

  - An object might hold an open file descriptor, and a reconstituted
    object would have to open the equivalent file.

Third, the object may contain state which is in a convenient form for
processing in its current environment, but not for storage, transmission,
or processing in a new environment.  Perhaps extreme data compression
might be done by taking advantage of knowledge local to the object, or
an object may precompute something on construction to save processor
time later.  Examples:

 - A large, sparse hash table using a convenient hash function might
   be transmitted or stored as a tight table of key-value pairs, and
   reconstituted using a different hash function on a machine with
   a different byte-order.

 - An object may want to transmit or store its data as, say, PostScript
   code, but unfold it into a bitmap when in RAM.

The code generated by the Stubble tool supports all of these cases.

...

Stubble supports the transmission and storage of instances of COPY classes
by adding a member function and a constructor to the class definition.
The member function is named "sendSelfTo(Xcvr *)", and it packs up the
object's state for storage or shipment to some other environment.  The
constructor is "<ClassName>(Xcvr *, TCSJ)", and is responsible for
reconstituting the object when it is retrieved from storage or arrives
at its destination.

sendSelfTo(Xcvr *) and <ClassName>(Xcvr *, TCSJ) will automatically
send and receive any member variables in the "COPY(<ClassName>):"
PROTECTION_TYPE.  Any variables in "private:" "protected:", or
"public:" PROTECTION_TYPEs will not be automatically sent.

-----------------------------------------
(Hmmm... COPY(): variables are also private.
 Should there be an equivalent for protected?
 Yes, I know that's "not good OOP style"...)
-----------------------------------------

If you declare a member function with a single argument of type "Xcvr *"
in the "SENDER:" protection type, it will be called by sendSelfTo(Xcvr *)
after the COPY(): variables have been sent.  Such a member function may
call the public send(), sendData(), and sendString() member functions of
the Xcvr class to send any additional data it wishes.  Some caveats:

 - Your class will not be portable unless arguments to send() are, or
   are cast to, an everywhere-the-same-size type such as "UInt4".

 - You should never send anything that wouldn't make sense in the other
   environment (such as a file descriptor or a pointer-to-something-
   arbitrary.)  A pointer to a COPY class may be cast to "Heaper *"
   and sent, in which case the pointed-to object will automatically
   be sent as well.

If you declare a member function with a single argument of type "Xcvr *"
in the "RECEIVER:" protection type, <ClassName>(Xcvr *, TCSJ) will call it
during the construction of the reconstituted object.  When it is called:

 - The "private:" "protected:", and "public:" member variables will not
   have been initialized.

 - The "COPY(<ClassName>):" member variables will already be filled in, but

 - Pointers may contain invalid values or point to not-yet-constructed or
   partially-constructed objects.  (So DON'T deref or copy them!)  This
   usually happens because the partially-constructed object pointed to
   your object, and your object is being reconstructed so the other
   object's pointer can be filled in.  As viewed by your object, those
   pointers "become valid" when your object's constructor returns (though
   if some other object breaks the "Don't deref pointers during RECEIVER:
   execution" rule, your object's member functions could be called while
   your pointers are still invalid.)

The RECEIVER: function may do whatever (data-based) initialization it wishes,
but if a SENDER: function was used to send any additional data, the RECEIVER:
function must make EXACTLY the equivalent calls to the public receive(),
receiveData(), and/or receiveString() functions of the Xcvr class.

Some conventions and coding practices:

 - Name the RECEIVER: function receive<ClassName>(Xcvr *)

 - Name the SENDER: function send<ClassName>(Xcvr *)

 - If some member variables must be initialized by a RECEIVER: function,
   don't replicate the code that initializes them in ordinary constructors.
   Instead, where possible, have the ordinary constructors call the
   RECEIVER: function, or have both the ordinary constructors and the
   RECEIVER: call a common routine.

 - Don't use "public:" or "protected:" member variables.  Instead, use
   "COPY(<ClassName>):" or "private:" variables and provide "public:" or
   "protected:" member functions to return or twiddle them.


========================================================
Other comments:

> Passing the "trans" argument is also new.  Even though I've found no
> interesting use for it yet, it seems the right modularity.

It is.  See above for examples.

> It enables
> a restart method that wants to do something special given a particular
> transceiver that it was designed in concert with to check for that
> kind of transceiver and, in that case, invoke its special protocol:
> []

Well, yes, but I'd prefer, whenever possible, to keep protocol issues
completely inside Xcvrs, restricting copy objects to sending values
that all Xcvrs know how to send.  Copy objects can exercise plenty of
power by chosing what values to send and the order in which to send them,
They shouldn't be interested in how the values get where they're going,
and whether they've been translated to "g", Morse, Baudot, or Sanscrit
while on the way.  (The types of the arguments already provide lots of
hints for data compression, too.)

> Right now SELF_COPY is only used in IntegerVar, so this change should
> be no big deal (I've already changed my copy of IntegerVar).

Though IntegerVar is the only client of the SELF_COPY macro, the
PrimArray classes PtrArray, UInt4Array, and UInt1Array are effectively
SELF_COPY, defining their own sendSelfTo() functions and Recipes.
(That's all there are outside Tofu and the Comm package, however.)

> Anyone object to SELF_COPY becoming passe?

Yeah.  Don't make it passe.  KILL it!  B-)

> and to the end of the constructor (again before the '}'):
> 
> $(  FOR FUNCS pro: RECEIVER)
> 	this->$(FUNC) (trans);
> $(  ROF)

Actually, that should (just did...) replace the now-obsolete:

	$(  IF CLASS memf: restart)
		restart ();
	$(  FI)

> As these are not in Smalltalk, our normal merging procedure won't get
> these in.  Mr. Hill and Wjr:  can you guys add these to your files so
> they'll become part of the next release?  They should be completely
> upwards compatable.  Thanks.

Wjr: please don't.  I've put them into my version, and our merge will be
simpler if we keep your changes few and then merge them into my version.

(Oh, boy!  Just what I needed:  Another change at the root of the world,
 and another total recompile.  B-) )

	michael