Xanology (article draft)

[Andrew Pam <avatar@xxxxxxxxxxxxxx>]

OK, here's something to get some discussion started.  This is an article draft
I wrote in February, which I intend to update once I have completed my present
research into current work in distributed systems (meta-research!).

Comments are welcome, and I'd love to see anyone else evaluating and critiquing
any software which achieves or attempts any of the Xanadu goals.


Xanology: The State of the Art
==============================

  Copyright (c) 1994 Andrew Pam

  I see connections among much of the research into hypermedia,
  distributed systems and network information resources.  I believe
  that this area of overlap can be viewed as a field in its own
  right which has been referred to as "Xanology" (the study of
  "Xanalogical" systems) since the 1960's when Ted Nelson chose the
  name Xanadu and began to set forth many of the fundamental ideas
  of the field.

  I define Xanology as the study of information tools which aim to
  meet the following requirements:

1.  Every server is uniquely and securely identified.

2.  Every server can be operated independently or in a network.

3.  Every user is uniquely and securely identified.

4.  Every user can search, retrieve, create and store documents.

5.  Every document can consist of any number of parts each of which
    may be of any data type.

6.  Every document can contain links of any type including virtual
    copies ("transclusions") to any other document in the system
    accessible to its owner.  Permission to link to a document is
    explicitly granted by the act of publication.  Links are visible
    and can be followed from all endpoints.

7.  Every document can contain a royalty mechanism at any desired
    degree of granularity to ensure payment on any portion accessed,
    including virtual copies ("transclusions") of all or part of the
    document.

8.  Every document is uniquely and securely identified.

9.  Every document can have secure access controls.

10. Every document can be rapidly searched, stored and retrieved
    without user knowledge of where it is physically stored.

11. Every document is automatically moved to physical storage
    appropriate to its frequency of access from any given location.

12. Every document is automatically stored redundantly to maintain
    availability even in case of a disaster.

13. Every service provider can charge their users at any rate they
    choose for the storage, retrieval and publishing of documents.

14. Every transaction is secure and auditable only by the parties to
    that transaction.

15. The client-server communication protocol is an openly published
    standard.  Third-party software development and integration is
    encouraged.

  In this article, I intend to briefly summarise research activity
  and tools relevant to each of these requirements.

  Requirements 1, 3 and 8 discuss uniqueness of servers, users and
  documents respectively.  The Internet Engineering Task Force
  (IETF) Uniform Resource Identifier (URI) working group is drafting
  specifications for a Uniform Resource Name (URN) which would
  provide persistent global identifiers.  Xanadu designs have
  traditionally seen these as a hierarchy, where every user can be
  identified by a specific server and every document can be
  identified by a specific user (normally the "owner" of the
  document).  The Xanadu "tumbler" addressing scheme is described in
  Ted Nelson's works including the book "Literary Machines" and the
  video tape "A Technical Overview of the Xanadu System".

  The same three requirements also discuss security, as do
  requirements 9 and 14.  Secure authentication can be provided by
  hashing algorithms such as Message Digest 5 (MD5) described in
  RFC-1321 or digital signature systems such as Pretty Good Privacy
  (PGP) and the US national digital signature standard.

  Requirement 2 permits Xanadu software to be used for personal or
  corporate use without necessarily joining a national or
  international network, although I believe the benefits of doing so
  will inevitably lead most users to do so eventually.  Note that
  the uniqueness requirements described above should still apply to
  isolated Xanadu servers in order to permit them to join a network
  at a later date without naming conflicts.  The traditional
  approach to this problem is to require all naming authorities to
  register with a central name registry of some sort.  I am also
  investigating an alternative technique where all identifiers are
  expressed relative to the current server at all times, which would
  avoid the need for a central registry.

  Many network information retrieval (NIR) tools currently in use
  and under development maintain a distinction between consumers and
  producers of information.  Requirement 4 mandates that all users
  should be able to both produce and consume information, although
  naturally there may be limits or charges involved (as in
  requirement 13).  The Unix file transfer protocol (FTP) and
  network news transfer protocol (NNTP), for example, permit
  authorised users to create and store information as well as search
  and retrieve it, while Gopher and world wide web (WWW) currently
  only permit feedback using data entry forms.  A "PUT" method has
  been defined in the hypertext transfer protocol (HTTP) native to
  WWW, but is currently very sparsely implemented, if at all.
  Hyper-G apparently does allow users to store information.

  One attempt to standardise multi-part documents and data types as
  specified in requirement 5 is MIME (Multipurpose Internet Mail
  Extensions) described in RFC-1521.  It appears logical to provide
  support for multi-part documents and for transclusions (described
  in requirement 6) using the same mechanism; it may be possible to
  use the MIME "External-body" encoding for this purpose.  I also
  intend to investigate the OpenDoc standard being promoted by Apple
  and IBM.

  Requirement 6 defines hyperlinking, which was first proposed by
  Ted Nelson in his 1965 paper "A File Structure for the Complex,
  the Changing and the Indeterminate".  Since then it has become a
  standard paradigm for online help systems, and appears to be
  gaining increasing favour for NIR tools.  WWW is probably the most
  notable example, but does not support transclusions or
  bifollowable links.  HyperTed and Hyper-G appear to be the first
  tools to support bifollowable links, but I am not currently aware
  of any tools that provide full support for transclusions.

  There also do not currently appear to be any systems supporting
  charging mechanisms, although WWW is being enhanced to provide
  charging support in the near future.  I am certainly unaware of
  any other systems that aim to meet requirements 7 and 13, and this
  is therefore one of the focal points of Xanadu development.  We
  believe that other aspects of the Xanadu technical design (notably
  transclusions) and business model are essential to ultimately make
  charging workable for intellectual property.

  The Xanadu Operating Company (XOC) have done a great deal of
  research on access controls as specified in requirement 9, and the
  (as yet unreleased) XOC server embodies a very general and
  powerful access control mechanism as a result, described in their
  "Xanadu Hypermedia Server Developer Documentation".

  Requirement 10 defines a distributed file system with a global
  namespace.  This is currently the focus of a great deal of
  research, some notable examples including Prospero, AFS, Mungi,
  Sprite, Plan 9, DASH, GAFFES and DFS925.  Hyper-G also appears to
  include a distributed mode of operation.

  Local caching of documents helps to meet requirement 11, and cache
  consistency problems are avoided in the traditional Xanadu file
  system since documents can only be superceded, never altered.
  Another part of the solution is a hierarchical file system as
  implemented in a variety of near-line backup solutions.  I believe
  that implementation of file migration technology in future file
  systems will significantly aid storage management problems
  currently suffered by most computer users.

  Requirement 12 can be satisfied by RAID storage or "hot mirrors",
  but the Xanadu design explores the possibility of widely
  geographically dispersed servers being chosen randomly to mirror
  each document in the distributed file system.  This would require
  each server to allocate space to store these mirrored documents,
  but since they would be gaining an equal amount of redundant
  storage for their own documents it could be a reciprocal
  agreement.  In addition, mirroring should improve response times
  for infrequently requested (and thus not cached) documents.

  Finally, it is with requirement 15 in mind that I wrote this
  article.  Xanadu Australia intends to promote Xanadu as an open
  system, and therefore interoperate with and support the
  development and use of other open systems in the same field.  I
  welcome feedback from anyone interested in the field of Xanology
  as I have defined it in this article, most especially those
  involved with the tools and research projects mentioned above.

-- 
Andrew Pam                                      avatar@xxxxxxxxxxxxxx
Manager, Serious Cybernetics                    avatar@xxxxxxxxxxxxxxx
Coordinator, Xanadu Australia                   <http://www.aus.xanadu.com/>
P.O. Box 409, Canterbury VIC 3126 Australia     gopher gopher.aus.xanadu.com