Basic Concepts for Managing Semi-structured Information in Haystack
The Haystack platform (Huynh et al., 2002) is designed to store and manage personal information such as e-mail, documents, contacts, meetings, etc. These types of data are semi-structured in nature and thus are better expressed as semantic networks rather than as tables. We believe that any system whose primary data model is semi-structured has at least four intrinsic needs. First, as the semantic data networks inevitably grow, there is a need to organize them conveniently and flexibly and to navigate them systematically. Second, since there is an enormous amount of structured and unstructured data in existence, the system must also allow data exchange with the outside world to both extend its knowledgebase and to facilitate collaboration with users of other systems. Third, as a consequence of collaboration, managing the cooperation of services among various systems becomes important. Finally, cooperation requires trust management for different sources of information. In Haystack, we provide built-in solutions for all of these four needs for organization, external data integration, service cooperation, and trust management. This paper discusses the various techniques that we use to fulfill these needs.