Coopetition in Information Retrieval Research

This talk explores an interesting approach for text retrieval: a community-based approach called “coopetitions.”

Coopetitions are activities in which competitors cooperate for a common good. Community evaluations such as the Text REtrieval Conference (TREC, trec.nist.gov) are prototypical examples of coopetitions in information retrieval, and they have now been part of the field for more than thirty years. This longevity and the proliferation of shared evaluation tasks suggest that, indeed, the net impact of community evaluations is positive.

Coopetitions can improve effectiveness for a retrieval task by setting up a collaborative structure: establishing a research cohort and constructing the infrastructure (including problem definition, test collections, scoring metrics, and research methodology) that the participants need to make progress on the task. They can also facilitate technology transfer and amortize the infrastructure costs. Yet these benefits only accrue when the infrastructure is a good abstraction of the real task. Information retrieval’s test collection paradigm is becoming increasingly untenable as corpus size grows and search engine effectiveness improves. This talk will review what we have learned about test-collection-based evaluation of search engines from TREC and examine the prospects of search evaluation in the future.

Speaker(s): Ellen Voorhees,

Room: Room 105, Bldg: Computer Science Building, Princeton University, Princeton, New Jersey, United States, 08544, Virtual: https://events.vtools.ieee.org/m/345494