SUBMISSIONS


Submission is closed now. Please see the final program.


CALL FOR PAPERS


Most research in data mining and knowledge discovery relies heavily on the availability of datasets. With the rapid growth of user generated content on the internet, there is now an abundance of sources from which data can be drawn. Compared to the amount of work in the field on techniques for pattern discovery and knowledge extraction, there has been relatively little effort directed at the study of effective methods for collecting and evaluating the quality of data.


Human computation is a new research area that studies the process of channeling the vast internet population to perform tasks or provide data towards solving difficult problems that no known efficient computer algorithms can yet solve. There are various genres of human computation applications available today. Games with a purpose (e.g., the ESP Game) specifically target online gamers who, in the process of playing an enjoyable game, generate useful data (e.g., image tags). Crowdsourcing marketplaces (e.g. Amazon Mechanical Turk) are human computation applications that coordinate workers to perform tasks in exchange for monetary rewards. In identity verification tasks, users need to perform some computation in order to access some online content; a recent example of such a human computation application is reCAPTCHA, which leverages millions of users who solve CAPTCHAs every day to correct words in books that optical character recognition (OCR) programs fail to recognize with certainty.


The goal of HCOMP 2009 is to bring together academic and industry researchers in a stimulating discussion of existing human computation applications and future directions of this new subject area. We solicit papers related to various aspects of both general human computation techniques and specific applications, e.g. general design principles; implementation; cost-benefit analysis; theoretical approaches; privacy and security concerns; and incorporation of machine learning / artificial intelligence techniques. An integral part of this workshop will be a demo session where participants can showcase their human computation applications. Specifically, topics of interests include, but are not limited to:


  1. Bullet Abstraction of human computation tasks into taxonomies of mechanisms

  2. Bullet Theories about what makes some human computation tasks fun and addictive

  3. Bullet Differences between collaborative vs. competitive tasks

  4. Bullet Tools and platforms to support human computation

  5. Bullet Domain-specific implementation challenges in human computation games

  6. Bullet Cost versus reliability of labelers

  7. Bullet Benefits of one-time versus repeated labeling

  8. Bullet Game-theoretic mechanism design of incentives for motivation and honest reporting

  9. Bullet Design of manipulation-resistance mechanisms in human computation

  10. Bullet Effectiveness of CAPTCHAs

  11. Bullet Concerns regarding the protection of labeler identities

  12. Bullet Active learning from imperfect human labelers

  13. Bullet Creation of intelligent bots in human computation games

  14. Bullet Limitations of human computation


Preliminary Workshop Format

  1. Bullet presentations by authors

  2. Bullet talks by invited speakers

  3. Bullet poster / demo session


Submission Information

  1. Bullet Papers and extended abstracts should be prepared as PDF files using the KDD conference-paper format and submitted via the CMT system.

  2. Bullet Long papers should be at most nine pages; short papers at most four pages; demo submissions should include either a previously published paper or a one page extended abstract about the demo.


Important Dates

April 18, 2009    Paper submission due

May 16, 2009    Acceptance announcement

May 22, 2009    Camera-ready paper submission due

June 28, 2009   Half-day Workshop at KDD


Program Committee


Serge Belongie (University of California at San Diego)

Laura Dabbish (Carnegie Mellon University)

Ralf Herbrich (Microsoft Research, Cambridge, UK)

John Langford (Yahoo! Research)

David Parkes (Harvard University)

Zoran Popovic (University of Washington)

Paul Resnick (University of Michigan)

Victor Sheng (New York University)

Alexander Sorokin (University of Illinois at Urbana-Champaign)


Organizing Committee

Paul Bennett (Microsoft Research)

Raman Chandrasekar (Microsoft Research)

Max Chickering (Microsoft Live Labs)

Panos Ipeirotis (New York University)

Edith Law (Carnegie Mellon University)

Anton Mityagin (Microsoft Live Labs)

Foster Provost (New York University)

Luis von Ahn (Carnegie Mellon University)


Contact email: hcomp2009@gmail.com