Subject: Proximity application idea: intelligent email system

As you may know, I've been thinking of Proximity applications for
personal use (personal intelligence analysis, as David might put
it). The email I sent Tue, 13 Mar 2001 ("Proximity application idea:
personal information manager") described one possibility. Sadly, only
David commented on this - I was hoping it would get someone else
excited :-( However, in my boundless enthusiasm, and in the interest
of thinking big (for future proposals or products), I'd like to
briefly describe a second key idea - a Proximity-based intelligent
email system. As a motivation, I refer you to a recent article by John
Dvorak of PC Magazine called "The modernization of e-mail":

    http://www.zdnet.com/pcmag/stories/opinions/0,7802,2692066,00.html

The article is a bit vague, but he wants "a smart software subsystem
that can parse and organize my e-mail and abstract all my files." I
argue below that a Proximity-based system can do this very well.


You may have heard me mention that one "killer app" would be a
Proximity-based intelligent email system. This idea has both a short
term expression and a long term one, based on level of integration
with current email systems. In the short term, current text-based
email would be "objectified" by creating Proximity objects and links
based on the content. The bigger long-term idea is to *replace*
text-based email with Proximity object-based email. Descriptions of
these two ideas follow.


* Short term: Integration with text-based email: Every bit of email
you receive would be "objectified" into your Proximity database. (BTW,
what I describe here is the fourth "C" - Automated
Cross-Referencing. The other 'C's are Classification, Consolidation,
and Clustering.) The translation would perform these actions, ordered
roughly in increasing difficulty:

o break headers into separate objects (or attributes), and add links
  to them. ex: date, subject

o replace email addresses in headers and the body with links to person
  objects (again, may need to create new person place-holders)

o replace URLs in the body with links to URL objects (or URL
  attributes)

o break out forwarded email and MIME attachments into separate
  objects, and add links to them

o use "Message-Id", "In-reply-to", and similar header fields to create
  links to previous message objects. (may need to create blank
  place-holders if the messages aren't in the system)

o replace dates and times in the body with links to new objects (or
  attributes)

o replace text phrases in the body with links to existing objects


Of all of these, the last one is the most difficult. The others
basically require recognizing straightforward text patterns (what
David and I have called "data recognizers") that separate out
easily-identified data, including dates and times, email addresses,
URLs, proper names (getting harder), etc. I don't think this is too
hard. However, I think the last task of replacing text phrases is more
generally like information extraction, which I believe is made
difficult by its requirement for NLP. I need help working this bit
out. But we might be able to do some simpler but effective kinds of
linking if we're clever. One idea is to have a data structure that
maps English phrases to objects. (This could be an application of a
general-purpose Proximity dictionary feature.) Given such a mapping,
the system would simply replace every matching phrase with a link to
the mapped-to object. The system could help manage mappings by
providing nice tools, e.g., you might create a new mapping by
selecting a phrase in a newly-objectifed email and dragging it onto
the object to map to. Just one idea...

Now that we have the email in our semi-structured data representation,
we can use powerful Proximity analysis, organization, and
visualization tools on it, such as:

o train a classifier to recognize spam, urgent mail, etc. (note that
  it would be easy to organize email according to attributes - just
  write a program that categorizes the email according to settings,
  where "categorize" means add a link from the appropriate "category"
  object to the message!)

o find emails in more meaningful ways, such as by relationships to
  concepts

o show messages as graphs (in addition to standard lists and trees)

o ...


* Long term: Proximity-based object email: If we assume everyone is
using a Proximity-based system for personal information management,
it's obvious that one form of collaboration is to "send" an object to
someone else. There are two ways to do this: a) "flatten" the
object-to-email's graph structure (to some level) and send this in
encoded form, or b) just send a *reference* to the object on your
system (possibly stored in the "outbox" folder/object). Assuming that
Proximity is distributed and has the right security, recipients would
simply click the link to read the email. They could view it in
multiple ways, and navigate through related information as they would
with objects in their local database.