HCOMP 2014 Program

October 31st - November 1, 2014

November 2, 2014

November 3, 2014

November 4, 2014

October 31st - November 1, 2014

CrowdCamp at CMU (407 S. Craig)

November 2, 2014

Workshops & Doctoral Consortium

9:00am - 5:00pm

 Workshops

Crowdsourcing, Online Education, and Massive Open Online Courses

Markus Krause (Leibniz University), Praveen Paritosh (Google), Joseph Jay Williams (Stanford University)

Citizen + X: Workshop on Volunteer-based Crowdsourcing in Science, Public Health and Government

Edith Law (University of Waterloo) and Cliff Lampe (University of Michigan)

6:00pm - 8:00pm

Opening Reception at Omni Hotel 

   7:00pm:  reports from workshops

November 3, 2014

8:00am

Registration Opens

8:45am

Opening Remarks

David Parkes and Jeffrey Bigham

9:00am - 10:00am

Invited talk, Kristen Grauman, U. Texas at Austin

Chair: David Parkes

Humans teaching computers about visual categories

Visual recognition is critical to a variety of interesting applications, including image and video search, security, and robotics. The computer vision community has made important progress in recent years, leading to increased attention to the recognition task in terms of the complexity and scale of problems that are considered. However, while the difficulty of the recognition task has escalated dramatically, our means of "teaching" visual categories remains quite shallow. I will describe how teaching computers about visual categories can instead be an ongoing, interactive process, using communication with human supervisors that goes beyond category-labeled images. In particular, I will highlight our work on actively engaging human supervisors from the crowd and representing visual comparisons in interactive systems.

10:00am - 10:30am

Morning break

10:30am - 12:00pm

Technical Session I

Chair: Adam Marcus

Saving Money While Polling with InterPoll using Power Analysis

Benjamin Livshits (Microsoft Research); Todd Mytkowicz (Microsoft Research);

Attendee-Sourcing: Exploring The Design Space of Community-Informed Conference Scheduling

Anant Bhardwaj (MIT CSAIL); Juho Kim  (MIT CSAIL); Steven Dow (CMU HCII); David Karger  (MIT CSAIL); Sam Madden (MIT CSAIL); Robert Miller (MIT CSAIL); Haoqi Zhang (Northwestern University);

Cost-Effective HITs for Relative Similarity Comparisons

Michael Wilber (Cornell); Iljung Kwak (UCSD); Serge Belongie (Cornell);

Predicting Next Label Quality: A Time-Series Model of Crowdwork

Hyun Joon Jung (University of Texas at Austin); Yubin Park (University of Texas at Austin); Matt Lease (University of Texas at Austin);

Wish: Amplifying Creative Ability with Expert Crowds

Anand Kulkarni (Mobile Works); David Rolnitzky (Mobile Works); Prayag Narula (Mobile Works); Nathan Kontny (Mobile Works);

12:00pm - 12:05pm

Workshop Report

Crowdsourcing, Online Education, and Massive Open Online Courses

Markus Krause (Leibniz University), Praveen Paritosh (Google), Joseph Jay Williams (Stanford University)

Chair: David Parkes

12:05pm - 1:30pm

Lunch Break

1:30pm - 2:15pm

Invited Speaker, Kristy Milland “Spamgirl”

Spamgirl, a Turker from Toronto, moderates a large and extremely influential worker community called Turker Nation. We have invited her to HCOMP to share her perspectives on crowdsourcing research and the workers whose efforts fuel it. We will discuss researcher best practices as well as myths about the platform.

Chair: Michael Bernstein

2:15pm - 2:45pm

“Poster Madness”

Chair: Haoqi Zhang

Accepted WiP and Demos

2:45pm - 4:00pm

Poster session and Refreshments

4:00pm - 5:30pm

Technical Session II

Chair: Loren Terveen

Parallel Task Routing for Crowdsourcing

Jonathan Bragg (University of Washington); Andrey Kolobov (Microsoft Research); Mausam Mausam (IIT Delhi); Daniel Weld (University of Washington);

Incentives to Counter Bias in Human Computation

Boi Faltings (EPFL); Bao Duy Tran (EPFL); Radu Jurca  (EPFL); Pearl Pu (EPFL);

Community Poll: Externalizing Public Sentiments in Social Media in a Local Community Context

Patrick Shih (Penn State University); Kyungsik Han (Penn State University); John Carroll (Penn State University);

Mechanism Design for Crowdsourcing Markets with Heterogeneous Tasks

Gagan Goel (Google); Afshin Nikzad (Stanford University); Adish Singla (ETH Zurich);

Crowdsourced Explanations for Humorous Internet Memes Based on Linguistic Theories

Chi-Chin Lin (National Taiwan University); Yi-Ching Huang (National Taiwan University); Jane Yung-jen Hsu (National Taiwan University)

5:30-6:15pm

A Technical Discussion with Industry Representatives

Adam Marcus, Locu/GoDaddy

Burr Settles, DuoLingo

John Snowden, CloudFactory

Alexander Sorokin, CrowdFlower

Andy Schriner, LeadGenius

Chair: Jeffrey P. Bigham

6:45pm - 9:30pm

 Conference Banquet

Butcher and the Rye

212 6th St.

Pittsburgh, PA 15222

Google Maps Directions

November 4, 2014

8:45am - 9:40am

Technical Session III

Chair: Steven Dow

A Crowd of Your Own: Crowdsourcing for On-Demand Personalization

Peter Organisciak (University of Illinois); Jaime Teevan (Microsoft Research); Susan Dumais (Microsoft Research); Robert Miller (MIT CSAIL); Adam Kalai (Microsoft Research);

Output Agreement Mechanisms and Common Knowledge

Bo Waggoner (Harvard); Yiling Chen (Harvard);

Context Trees: Crowdsourcing Global Understanding from Local Views

Vasilis Verroios (Stanford University); Michael Bernstein (Stanford University);

9:40am - 10:00am

Pietro Michelucci: Report and Q&A in regard to CRA's Computing Community Consortium’s  “Human Computation Roadmap Summit”, June 2014

10:00 - 10:05am

Citizen + X: Workshop on Volunteer-based Crowdsourcing in Science, Public Health and Government

Edith Law (Harvard University) and Cliff Lampe (University of Michigan)

Chair: David Parkes

10:05am - 11:00am

Poster Session and Refreshments

11:00am - 12:10pm

Technical Session IV

Chair: Edith Law

Referral Incentives in Crowdfunding

Victor Naroditskiy (University of Southampton); Sebastian Stein (University of Southampton); Mirco Tonin (University of Southampton); Long Tran-Thanh  (University of Southampton); Michael Vlassopoulos (University of Southampton); Nick Jennings (University of Southampton);

To Re(label), or Not To Re(label)

Christopher Lin (University of Washington);  Mausam (IIT Delhi); Daniel Weld (University of Washington);

TRACCS: Trajectory-Aware Coordinated Urban Crowd-Sourcing

Cen Chen (Singapore Management University); Shih-Fen Cheng (Singapore Management University); Aldy Gunawan (Singapore Management University); Archan Misra (Singapore Management University); Deepthi Chander (Xerox); Koustuv Dasgupta (Xerox);

Robot Programming by Demonstration with Crowdsourced Action Fixes

Maxwell Forbes (University of Washington); Michael Chung  (University of Washington); Maya Cakmak  (University of Washington); Rajesh Rao (University of Washington);

12:10pm - 1:30pm

Lunch Break

1:30pm - 3:00pm

Technical Session V

Chair: Maxine Eskenazi

Groupsourcing: Distributed Problem Solving Using Social Networks

Jon Chamberlain (University of Essex);

Scaling-Up the Crowd: Micro-Task Pricing Schemes for Worker Retention and Latency Improvement

Djellel Eddine Difallah (Fribourgh University); Michele Catasta (EPFL); Gianluca Demartini (Fribourgh University); Philippe Cudré-Mauroux (Fribourgh University);

Monetary Interventions in Crowdsourcing Task Switching

Ming Yin (Harvard); Yiling Chen (Harvard); Yu-An Sun (Xerox);

Predicting Own Action: Self-fulfilling Prophecy Induced by Proper Scoring Rules

Yuko Sakurai (Kyushu University); masaaki oka (Kyushu University); taiki todo (Kyushu University); makoto yokoo (Kyushu University);

STEP: A Scalable Testing and Evaluation Platform

Maria Christoforaki (NYU); Panos Ipeirotis (NYU);

3:00pm - 3:30pm

Afternoon Break

3:30pm - 4:40pm

Technical session VI

Chair: Haoqi Zhang

Crowdsourcing for Participatory Democracies: Efficient Elicitation of Social Choice Functions

David Lee (Stanford University); Ashish Goel (Stanford University); Tanja Aitamurto (Stanford University); Helene Landemore (Yale University);

Detecting Non-adversarial Collusion in Crowdsourcing

Ashiqur KhudaBukhsh (CMU); Jaime Carbonell (CMU); Peter Jansen (CMU);

A Human Computation Framework for Boosting Combinatorial Solvers

Ronan Le Bras (Cornell); Yexiang Xue (Cornell); Richard Bernstein (Cornell); Carla Gomes (Cornell); Bart Selman (Cornell);

Instance-privacy Preserving Crowdsourcing

Hiroshi Kajino (University of Tokyo); Yukino Baba (National Institute of Informatics); Hisashi Kashima (Kyoto University);

4:45pm - 5:45pm

 Invited talk, Robert Kraut, CMU

Chair: Jeffrey P. Bigham

Managing volunteers in online production communities

Virtual organizations are becoming increasingly important in driving production and innovation in science, engineering and knowledge production. This talk will describe several longitudinal studies and several experiment to examine the conditions under which different coordination strategies online production communities use to coordinate the efforts of their contributors are successful. Previous research on leadership in online communities has focused on the behavior of a small set of elite users or people, like administrators, who have a formal leadership role. Our research introduced a model of shared leadership and demonstrated that most leadership behaviors in Wikipedia are produced by the non-elite. Using longitudinal, correlational methods and experiments, our research demonstrates that these leadership behaviors influence their recipients, although the influence depends both on the type of leadership behavior and the legitimacy of the person delivering it and the person receiving it. Directive and social communication boosted motivation the most, whereas negative feedback decreased motivation. Both the correlational and experimental research also showed that the effects of the communication were greatest when delivered by Wikipedia administrators and other elite members and had most impact when the recipients were relatively new to Wikipedia.

We also conducted experiments to show that feedback also has effects on those giving it. We investigated the effectiveness of three different strategies for organizing workers to accomplish management tasks: allowing individuals to provide evaluation and feedback to others, organizing interactive teams in which members of the teams had to reach consensus on the evaluation and feedback they would provide to others, and aggregating individual managers into nominal teams. In an experiment using Mechanical Turk workers, we showed that giving workers managerial responsibilities to assess, evaluate and approve other workers’ work increases their own motivation to work, enhances their understanding of the task domain, and helps them become better workers.

Although direct communication with both formal and peer leaders can influence volunteer behavior, too much explicit direction could backfire. If leaders try to exert too much control, volunteers are likely to leave, with fewer consequences to their economic welfare and their social capital than if they had quit a job in a conventional organization. To address the paradox of exerting control and yet retaining volunteers, we examined a control mechanism – group self-governance – which allows peer groups of individuals to self-regulate the behavior of their members on relevant tasks. Group self-governance works by promoting members’ identification with the group and the values it stands for. When people identify with a group, they incorporate important elements of the group into their own self-concepts and believe that events and actions that influence the welfare of the group also influence their own welfare. Therefore, if the group has mechanisms to draw attention to work that will benefit the group, members should voluntarily choose to work on these tasks because they believe it will improve their own welfare. Furthermore, instead of potentially harming volunteers’ motivation, control through group self-governance could increase volunteers’ general motivation.

5:45pm - 6:15pm

HCOMP Business Meeting