zz: Proposed ZigZag Modules and Their Connection (zzmod D4

[Ted Nelson <ted@xxxxxxxxxx>]

zzmod D4
01.09.27
==========

PROPOSED ZIGZAG MODULES AND THEIR CONNECTION

CONTENTS
  Why mix and match parts
  Proposed module division
  Proposed convention: return of cell no.
  Proposed system of fixed cell IDs
  Notes on the gridwalker
  
  
==========

This is what I've been thinking about lately.

Note: special thanks to Andrew Pam for consultation and serving as a
reality check on some of these seemingly-retrograde ideas.


MIX'n'MATCH GENERAL

The idea is to be able to plug in sections from different sources and have
them work together.

WHEREAS
 - Sharing cell clusters and slices is highly desirable
 - Sharing some basic functionality is highly desirable 
 - We would like systems to be as compatible as possible
 - People will want to do things differently
 - Different people will do different optimization attempts, software and
hardware
 - Dimensions will be very different among users and applitudes
 - In arbitrary software, we want people to be able to replace lists and
tables with ZZstructures with as little difficulty as possible
 
VARIATIONS COULD INCLUDE
 - ROM for processor
 - ROM for certain sets of cells, e.g. system cells
 - co-processor board for speed in following structure
 - remote engines

THIS MEANS MODULARITY--
We want to be able to separate and combine, mix and match--
 - Structure engines
 - ZZ engines (performing ops)
 - Viewers
 - Slices and groups of cells

DIFFERENT PEOPLE CAN WORK IN PARALLEL
A good modular structure should allow parallel development by different
individuals and teams--
  - 'Good enough' modules can support each other as they are mutually improved
  - stable interfaces can make this possible

SPECIFICALLY, SOME STANDARD PARTS
 - e.g, structure engine (gridwalker) standard
  -- Divisible so that different dimensions or parts can be handled by
different methods
  -- Allowing some structures to be handled inside a processor, others
outside 


=========
PROPOSED MODULE STRUCTURE

The idea is to create as much referential stability as possible as we mix
and match.  We will use ancient and crufty techniques such as fixed addresses.

PRELIMINARY NAMES OF THE MAIN SECTIONS--

1A.  Gridwalk/ traversal  ( see last section* )
These functions return the cellno. but do not move the cursor.

 - Return arbitrary cell A
 - Return next cell in d.foo from A
 - Return headcell of cell A in d.foo
 
1B.  Gridwalk/traversal with cursor/stepper
Same as 1A but moving a cursor or stepper

 - Put stepper S on arbitrary cell A 
 - Put stepper S on next cell in d.foo from A
 - Put stepper S on headcell of cell A in d.foo
 
2A.  CORE OPERATIONS
  - new cell
  - delete cell
  - make connection
  - break connection
  - edit contents
  - clone (and clone constructs and rules)
  - alpha-shear
  - hop
  - shear
  - chug

    SELECTING FURTHER SUBDIVISION INTO PRIMITIVES IS UNNECESSARY.  I am not
defining quark-level operations within useful operations; I am only
defining USEFUL operations, which individual implementations may build ou
of their own quark-level operations as desired.
 
2B.  COMPOUND EXTENSIONS
  - marks-- AND DEFINING ALL OPS TO WORK WITH MARKED SETS
  - attributes
  - preflets
  - containment
  - slice
  - template
  - simple executions using d.calc
  - simple language using d.xeq [like Floating Clang]
 
3A.  COMPOUND VIEWS (cells and connections only)
  - rasters
    -- rows/columns (as in original)
    -- floating headcell
    -- 3D ball-and-stick
    -- others
  - cellviews
    -- brick
    -- ball-and-stick
    -- idiosyncratic
    -- etc.
  
 3B.  GLOBAL VIEWS, projecting unified visible constructs to a coordinate
space
  - desktop
  - neat new stuff
 
(OpenGL would be on both sides of the division into 3A and 3B.) 
 
==========
PROPOSED INTERCOMMUNICATION CONVENTION

Every module (and optionally every function in a module) RETURNS A CELL
NUMBER.  
 -  This will typically be a cell to go to, though certain numbers indicate
error messages and conditions.

 -  This is inspired by the convention from C and Java of a single returned
value (which is sometimes VOIDed).  In our case, though, the value is a cell.

 -  Returned values and additional conditions should be in specific
locations in the geography.

There are several main levels of granularity here--
 
1.  the vital lowest-level routine (NEXEL or STEP) which gets the next cell
in some direction
   -- needed for refresh (should be FAST)
   -- needed for every operation (should be FAST)
   -- needed for every cursor move

2.  Big operations
  - This allows operations done elsewhere to merge with local code

3.   Suboperations
This is less likely, but suboperations could be reusable by the same
mechanism.  
  - Functions inside main modules which do not return numbers are not
subusable.

DIVIDING IT MORE FINELY

A CASE-type selection mechanism could choose between alternative
mechanisms: for instance, cells 1-256 might be handled by some extremely
fast mechanism, known to the local processor.

HOW MIX AND MATCH THESE MODULES?

Probably there needs to be an interpreter-- an arbitrator kernel for each
platform-- that sorts the operations to the different modules.  However, it
would have to be very low overhead to make it worthwhile crossing language
lines.

 - Since Java knows about C better than C knows about Java, this might be
in Java.  
 - However, this might undo any speedup from efficiency mechanisms.


==========
FIXED ASSIGNMENT OF CELLNUMBERS

This is of course not necessarily seen by users, but could allow direct
addressability for all the basics.

It also allows
 - Predefined set of operations
 - predefined dimensions
 - predefined location names
   -- home cell
   -- headcells of key corner lists
   -- coathangers for returned values
   -- etc.

==========
*NOTES ON THE GRIDWALKER (continuing the above discussion)

Implementors should not get hung upon the gridwalker.

It will keep improving and should therefore be a replaceable subsystem.  No
"final version" is on the horizon.

ROBUST would be the best beginning, SCALING is the next issue.

We had a very good result in the prototype using Berkeley DB (the early
version under open license) as a hash database.  We observed no slowdown,
but probably did not get to cellsets higher than 500.

Andrew also suggests going to primitives and rolling our own hash database,
since we don't need a lot of general features.  That does not seem
appropriate for a first shot.

POSSIBLE GRIDWALK MECHANISM THAT LEVERAGES ON FILESYSTEMS

An extremely crufty and humorous mechanism would use the filesystem itself
for the connection mechanism.  Under a cell number would be found its
contents and connections, as follows.

Directories--
 - cells
   -- cellno
     --- contents (file containing contents)
     --- d.foo- (file containing cellno of cell in neg.direction on d.foo)
     --- d.foo-plus (file containing cellno of cell in pos.direction on
d.foo) 
     --- (AND ON AND ON FOR EVERY CONNECTION OUTWARD.)

This could be reasonably fast in RAMdisk, especially if each RAMdisk holds
only a certain number of cell references.

Andrew has suggested that EXT2, the standard Linux file system, should be
used for this because it can be mounted on Windows and that interface to
Windows should be reliable by now.  Presumably it will also mount on the
new BDS-based Macs.  If this could provide a robust method it would put us
up cross-platform fairly quickly.

?
_________________________________________
Theodor Holm Nelson              
Project Professor, Keio University SFC Campus, Fujisawa, Japan
 ?  e-mail: ted@xxxxxxxxxx   ?  world-wide fax 1/415/332-0136
 ?  http://www.sfc.keio.ac.jp/~ted/    ?  http://www.xanadu.net
 ? Coordinates in USA      Tel. 415/ 331-4422
  Project Xanadu, 3020 Bridgeway #295, Sausalito CA 94965
_________________________________________