Part of unrevdb

-=-=-
References:

1. <http://www.burningchrome.com/~cdent/uviz/L594form.doc>
   SLIS L594 Proposal Form

2. <http://ella.slis.indiana.edu/~klabarre/unrev_firstpage.html>
   Unrev Archive Conceptual Access

3. <http://www.burningchrome.com/~cdent/index.cgi?word=88>
   Warp: uvizmessageeval. Notes from Kathryn about how to
   facillitate analysis

4. <http://ella.slis.indiana.edu/~cjdent/unrev/index.cgi>
   Existing interface to the database

-=-=-
Summary:

This project redesigns a database and retrieval interface
associated with a processed archive of the Bootstrap UNREV-II email
list to further work described in [2]. The purpose of the
redesign is to allow for the evaluation of individual messages and
semantic clusters of messages represented in the database.
Evaluations will help in the analysis required in the creation of
a faceted access structure. This project does not create the
access stucture, just the database structure and interfaces
required to evaluate messages.

-=-=-
List Legend:

- = required and incomplete
o = optional and incomplete
x = complete

-=-=-
Goals:

- Database redesign to provide:
  - Indications of cluster membership in database
  - Attributions of conceptual annotations and descriptors in
    database
  - Evaluations of
    - Messages
    - Clusters
    o Authors and evaluators
- Interface redesign to support:
  - Retrieval of messages from database by:
    - Message
    - Date
    - Subject
    - Author
    - Cluster membership
    - Freetext search
    o Evaluated score of the message
  - Annotations/Evaluations of messages to:
    - Describe messages with keywords
    - Describe messages with conceptual statements
    - Scores of relative importance of messages
    o Create and edit keywords
    o Create and edit conceptual statements
  o Evaluations of Authors
- Tool creation to support:
  - Import of cluster membership to database
    NOTE: Semantic Analysis tools can generate lists of documents
    that belong in a cluster from a set. The tool will identify
    messages and provide indicators of groupings to the database.

-=-=-
Milestones:

- Database redesign
  x Entity Relationship diagram
    See: <http://ella.slis.indiana.edu/~cjdent/unrev/htmlreport2/>
    This took longer than expected to create a design that should
    have some longevity.
    Status: behind schedule
  - Database creation
    - MySQL upgrade on hot.burningchrome.com
      NOTE: MySQL provides greatly enhanced freetext searching,
      both txidf and boolean.
    - Table creation
- Database testing
  - Data transfer
    - Modifying parser.pl, the mail archive to database tool
    - Processing archive
  - SQL queries to test
    - retrieval by various attributes
    - cluster membership
- Interface redesign
  - Reimplimenting existing interface with new schema (see [4])
  - Adding cluster view
  o Adding attributions
  o Adding evaluations
  o Retrieval by evaluations
- Tool creation
  - Standardizing a cluster transfer representation
    - Ask Katy
  - Creating a known dataset
    - Testing
  - Script for inputting to database
  - Test against interface
- Testing
  - Described tasks
  - With Kathryn
  - With Bootstrap participants
o Summarizing Paper
  o Short document describing the work done, placing it in the
    larger research context, with an emphasis on what next.

-=-=-
Tools:

- Embarcadero ER/Studio
- Perl, Perl DBI, Perl CGI
- MySQL, Oracle (in existing implementation)