ENQOIR

The ENQOIR 2009 workshop will be held 1st of April, 2009, in conjunction with the Joint International Conferences on Asia-Pacific Web Conference and Web-Age Information Management (APWeb-WAIM) in Suzhou, China. The joint APWeb-WAIM 2009 conferences will be held immediately after IEEE ICDE2009 to be held in Shanghai, China.

The ENQOIR workshop targets to deeper understanding and disseminate knowledge on advances in evaluation and application of ontology-based information retrieval (ObIR). The main areas of the workshop is an overlap between three evaluation aspects in ObIR, namely, evaluation of information retrieval, evaluation of ontology quality’s impact on ObIR results, and evaluation of user interaction complexity. The main objective is to contribute to optimization of ObIR by systematizing existing body of knowledge on ObIR and defining a set of metrics for evaluation of ontology-based search. The long-term goal of the workshop is to establish a forum to analyze and proceed towards a holistic evaluation method for evaluation of ontology-based information retrieval systems.

Topics
Evaluation of Ontology-based Information Retrieval Ontology quality aspects in Information Retrieval User acceptance of semantic technology
 * Information retrieval evaluation
 * Assessment of annotation quality/labour-load
 * Evaluation and benchmarking techniques and datasets
 * Quantitative / qualitative evaluation methods
 * Cost/ utility ratio
 * Ontology quality evaluation
 * Ontology utility
 * Ontology maintenance
 * Quantitative / qualitative evaluation methods
 * Usability evaluation
 * Quantitative / qualitative evaluation methods
 * Evaluation of human-computer interaction

Scope
In the recent years, a significant research effort has been devoted to ontology-based information retrieval (ObIR). The progress and results in this area offer a promising prospect to improve performance of current information retrieval (IR) systems. Furthermore, existing sparse evaluations of the ObIR tools report improvement compared to traditional IR systems. However, the results lack indications whether this improvement is optimal, causing difficulties to benchmark different ObIR systems. Yet, majority of IR evaluation methods is mainly based on relevance of retrieved information. While additional sophistication of the ObIR tools adds complexity on user interaction to reach improved results. Therefore, standard IR metrics as recall and precision do not suffice alone to measure user satisfaction because of complexity and efforts needed to use the ObIR systems. We need to investigate what ontology properties can even further enhance IR, to assess whether this improvement comes at a cost of interaction simplicity and user satisfaction, etc.

Furthermore, evaluation methods based on recall and precision do not indicate the causes for variation in different retrieval results. There are many other factors that influence the performance of ontology-based information retrieval, such as query quality, ontology quality, complexity of user interaction, difficulty of a searching topic with respect to retrieval, indexing, searching, and ranking methods. The detail analysis on how these factors and their interactions affect a retrieval process can help to dramatically improve retrieval methods or processes.

From other hand, ontology’s ability to capture the content of the universe of discourse at the appropriate level of granularity and precision and offer the application understandable correct information is important. An important body of work already exists in ontology quality assessment field. However, most of ontology evaluation methods are generic quality evaluation frameworks, which do not take into account application of ontology. Therefore there is a need for task- and scenario-based quality assessment methods that, in this particular case, would target and optimize ontology quality for use in information retrieval systems.

In order to promote more efficient and effective ontology usage in IR, there is a need to contemplate on analysis of ontology quality- and value-added aspects for this domain, summarize use cases and identify best practices. Several issues have been put forward by the current research, like the workload for annotation, the scalability, and the balance between the express power and reasoning capability. An approach to holistic evaluation should assess both technological and economical performance viewpoints. An aspect of value creation by semantics-based systems is important to demonstrate that the benefits of the new technology will overwhelm the payout.

The purpose of this workshop is to bring together researchers, developers, and practitioners to discuss experiences and lessons learned, identify problems solved and caused, synergize different views, analyse interplay between ontology quality and IR performance, and brainstorm future research/development directions. Particularly, we strongly encourage submissions dealing with ontology quality aspects and their impact on IR results, evaluation of usability of the ObIR systems, analysis of user behaviour, new evaluation methods enabling thorough and fine-grained analysis of ObIR technological and financial performance, etc.

Submissions
We invite submissions of two types: regular papers, and research in progress papers. Papers are restricted to a maximum length of 12 pages (including figures, references and appendices). Submissions must conform to Springer's LNCS format (see: http://www.springer.com/computer/lncs?SGWID=0-164-7-72376-0). All accepted papers will be published as post-proceedings in a combined APWeb-WAIM'09 workshops volume of Lecture Notes in Computer Science series by Springer. Please submit a paper in the PDF format.

Papers will be subject to a double-blind review process by the Program Committee, where both reviewers and authors remain anonymous throughout the review process. The text of the submitted paper should not reveal the identity of the authors.

Submissions for the workshop are handled by Easychair. Use the following link to access the submission system: http://www.easychair.org/conferences/?conf=enqoir2009

The extended best papers will be considered for publication in a standard issue of ACM JDIQ (ISSN: 1936-1955). Extended versions of other papers will be considered for publication in a special issue on Evaluation Aspects of Semantic Search Applications of the International Journal on Metadata, Semantics and Ontologies (by Inderscience, ISSN: 1744-2621, see: http://www.inderscience.com/browse/index.php?journalCODE=ijmso). Please note, that the extensions must be significant accounting for at least 30% difference from the papers published in the workshop proceedings. Preliminary deadline for submission of the extended papers is May, 2009. More details will follow later.

Important Dates

 * January 11, 2009: Extended Deadline for paper submission
 * February 2, 2009: Notification about decision
 * February 20, 2009: Camera-ready versions due
 * April 1, 2009: The workshop

Committees
Organising and Program Chairs
 * has OC member::Darijus Strasunskas, NTNU (Norway)
 * has OC member::Stein L. Tomassen, NTNU (Norway)
 * has OC member::Jinghai Rao, AOL (China)

Program Committee Members
 * has PC member::Xi Bai, University of Edinburgh (UK)
 * has PC member::Per Gunnar Auran, Yahoo! Technologies (Norway)
 * has PC member::Robert Engels, ESIS (Norway)
 * has PC member::Avigdor Gal, Technion (Israel)
 * has PC member::Jon Atle Gulla, NTNU (Norway)
 * has PC member::Sari Hakkarainen (Finland)
 * has PC member::Monika Lanzenberger, Vienna University of Technology (Austria)
 * has PC member::Federica Mandreoli, University of Modena e Reggio Emilia (Italy)
 * has PC member::Kin Fun Li, University of Victoria (Canada)
 * has PC member::James C. Mayfield, John Hopkins University (USA)
 * has PC member::Gabor Nagypal, disy Informationssysteme GmbH (Germany)
 * has PC member::Jaana Kekalainen, University of Tampere (Finland)
 * has PC member::Iadh Ounis, University of Glasgow (UK)
 * has PC member::Marta Sabou, The Open University (UK)
 * has PC member::Tetsuya Sakai, NewsWatch, Inc. (Japan)
 * has PC member::Amanda Spink, Queensland University of Technology (Australia)
 * has PC member::Peter Spyns, Vrije Universiteit Brussel (Belgium)
 * has PC member::Heiko Stoermer, University of Trento (Italy)
 * has PC member::Victoria Uren, The Open University (UK)