• Tracks

Accepted papers are grouped by track, with a link to each track group provided below.

Proceedings

View Conference Proceedings

Papers

A Checklist to Combat Cognitive Biases in Crowdsourcing (video)
  • Tim Draws, Alisa Rieger, Oana Inel, Ujwal Gadiraju and Nava Tintarev
Enhancing Image Classification Capabilities of Crowdsourcing-based Methods through Expanded Input Elicitation (video)
  • Romena Yasmin, Joshua Grassel, Md Mahmudulla Hassan, Adolfo Escobedo and Olac Fuentes
Explaining Autonomous Decisions in Swarms of Human-on-the-Loop Small Unmanned Aerial Systems (video)
  • Ankit Agrawal and Jane Cleland-Huang
Exploring the Music Perception Skills of Crowd Workers (video)
  • Ioannis Petros Samiotis, Sihang Qiu, Christoph Lofi, Jie Yang, Ujwal Gadiraju and Alessandro Bozzon
Forecast Aggregation via Peer Prediction (video)
  • Juntao Wang, Yang Liu and Yiling Chen
From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence (video)
  • David Alvarez Melis, Harmanpreet Kaur, Hal Daume, Hanna Wallach and Jennifer Wortman Vaughan
Human+AI Crowd Task Assignment Considering Result Quality Requirements (video)
  • Masaki Kobayashi, Kei Wakabayashi and Atsuyuki Morishima
Iterative Human-in-the-Loop Discovery of Unknown Unknowns in Image Datasets (video)
  • Lei Han, Xiao Dong and Gianluca Demartini
Iterative Quality Control Strategies for Expert Medical Image Labeling
  • Beverly Freeman, Naama Hammel, Sonia Phene, Abigail Huang, Rebecca Ackermann, Olga Kanzheleva, Miles Hutson, Caitlin Taggart, Quang Duong and Rory Sayres
Making Time Fly: Using Fillers to Improve Perceived Latency in Crowd-Powered Conversational Systems (video)
  • Tahir Abbas, Ujwal Gadiraju, Vassilis-Javed Khan and Panos Markopoulos
Modeling Simultaneous Preferences for Age, Gender, Ethnic, and Professional Profiles in Government-expense Spending: A Conjoint Analysis (video)
  • Lujain Ibrahim, Mohammad M. Ghassemi and Tuka Alhanai
On the Bayesian Rational Assumption in Information Design (video)
  • Wei Tang and Chien-Ju Ho
Pterodactyl: Two-step Redaction Of Images For Robust Face Deidentification (video)
  • Abdullah Alshaibani and Alexander Quinn
Rapid Instance-level Knowledge Acquisition from Class-level Common Sense (video)
  • Chris Welty, Lora Aroyo, Flip Korn, Sara Marie Mc Carthy and Shubin Zhao
StreamCollab: A Streaming Crowd-AI Collaborative System to Smart Urban Infrastructure Monitoring in Social Sensing (video)
  • Yang Zhang, Lanyu Shang, Ruohan Zong, Zeng Wang, Ziyi Kou and Dong Wang
Utility of Crowdsourced User Experiments for Measuring the Central Tendency of User Performance to Evaluate Error-rate Models on GUIs (video)
  • Shota Yamanaka

Graduate Consortium

Designing Human-AI Collaborative Systems for Historical Photo Identification (video)
  • Vikram Mohanty
Disentangling Inherent Ambiguity and Disagreement in Crowdsourced Annotation (video)
  • Quanze Chen
Empowering Workers with Artificial Intelligence Tools (video)
  • Carlos Toxtli-Hernandez
Human-UAV Collaborations in the Human-On-the-Loop Emergency Response Systems (video)
  • Ankit Agrawal
Incorporating Human In The Loop For Voice Assistants (video)
  • Shih-Hong Huang
Investigating Impacts of the Intra-Rural Digital Divide on Rural and Super-Rural Crowd Workers (video)
  • Rebecca Jonas
Supporting High-Stakes Investigations with Expert-Led Crowdsourcing
  • Sukrit Venkatagiri
Unlocking the Potential of the Crowd by Challenging its Assumptions (video)
  • Amy Rechkemmer
What is the Next Dataset Creation Pipeline? (video)
  • Yuan-Hong (Andrew) Liao

Works-in-Progress and Demonstrations

Group A

A General-Purpose Crowdsourcing Computational Quality Control Toolkit for Python (video)
  • Dmitry Ustalov, Nikita Pavlichenko, Vladimir Losev, Iulian Giliazev and Evgeny Tulin
A Human Computation Approach for Ontology Restrictions Verification (video)
  • Stefani Tsaneva and Marta Sabou
A Paradigm Shift towards Crowd-based Healthcare System (video)
  • Sujoy Chatterjee, Thipendra P Singh, Amitava Choudhury and Anirban Mukhopadhyay
Crowdsourcing Diverse Paraphrases for Training Task-oriented Bots (video)
  • Jorge Ramírez, Auday Berro, Marcos Baez, Boualem Benatallah and Fabio Casati
Crowdsourcing Exploratory Cues for Idea Browsing and Inspiration Discovery (video)
  • Xiaotong Tone Xu, Khuyen Le, Christopher Cha, Samuel Fleet and Steven Dow
Crowdsourcing Human Oversight on Image Tagging Algorithms: An initial study of image diversity (video)
  • Kyriakos Kyriakou, Pınar Barlas, Styliani Kleanthous, Evgenia Christoforou and Jahna Otterbacher
Double-Checking History: Designing Assessable Systems for Historical Photo Identification (video)
  • Vikram Mohanty and Kurt Luther
Exploring the Role of Domain Experts in Characterizing and Mitigating Machine Learning Errors (video)
  • Pavel Hoogland, Oana Inel, Jie Yang, Margje Schuur, Jasper van Vliet and Piet van Hoffen
FindItOut: A Multiplayer GWAP for Collecting Plural Knowledge (video)
  • Agathe Balayn, Gaole He, Andrea Hu, Jie Yang and Ujwal Gadiraju
Morphological Matrices as a Tool for Crowdsourced Ideation (video)
  • Jonas Oppenlaender

Group B

HAEM: Obtaining Higher-Quality Classification Task Results with AI Co-Workers (video)
  • Yu Yamashita, Masaki Kobayashi, Kei Wakabayashi, Hiroyoshi Ito and Atsuyuki Morishima
Hate2Explain: Crowdsourced Explanations as a Cultural Bridge in Understanding Hateful Memes (video)
  • Chao-Chun Han, Yi-Ching Huang, Nanyi Bi and Jane Hsu
How Much Do Platform Workers Value Reviews? An Experimental Method (video)
  • David Holtz, Liane Scult and Siddharth Suri
Rsourcer: Scaling Feedback on Research Drafts (video)
  • Yuchao Jiang, Boualem Benatallah and Marcos Báez
Supporting Dynamic Construction of Datasets with Annotator Suggestions (video)
  • Jeongeon Park, Eunyeong Ko, Donghoon Han, Jinyeong Yim and Juho Kim
The Role of Anthropomorphic Visual Cues in Human Interactions with Conversational Agents (video)
  • Emilija Zlatkutė, Sihang Qiu, Jie Yang and Ujwal Gadiraju
Towards a Requester-centered Study on the Use of ‘Bots’ for Completing Tasks (video)
  • Jonas Oppenlaender and Simo Hosio
When Content Decides Where You Belong: Investigating Micro Communities on VSCO (video)
  • Vikram Mohanty and Geneviève Patterson
iClarify - A Tool to Help Requesters Iteratively Improve Task Descriptions in Crowdsourcing (video)
  • Zahra Nouri, Nikhil Prakash, Ujwal Gadiraju and Henning Wachsmuth

Blue Sky Ideas Papers

Human Computation and Crowdsourcing for Earth (video)
  • Yasaman Rohanifar, Syed Ishtiaque Ahmed, Sharifa Sultana, Prateek Chanda and Malay Bhattacharyya
Human in the Loop for Machine Creativity (video)
  • Neo Christopher Chung
The Science of Rejection: A Research Area for Human Computation (video)
  • Burcu Sayin, Jie Yang, Andrea Passerini and Fabio Casati
  • Stay CONNECTED: HCOMP COMMUNITY

We welcome everyone who is interested in crowdsourcing and human computation to:

  • Join crowd-hcomp Google Group (mailing list) to post and receive crowdsourcing and human computation email announcements (e.g., calls-for-papers, job openings, etc.) including updates here about the conference. To subscribe send an email to crowd-hcomp+subscribe@googlegroups.com.
  • Check our Google Group webpage to view the archive of past communications on the HCOMP mailing list.
  • Visit the HCOMP blog where we post about new ideas for ideas related to crowd and social computing research.
  • Keep track of our twitter hashtag #HCOMP2021.
  • Join the HCOMP Slack Community to be in touch with researchers, industry players, practitioners, and crowd workers around Human Computation and relevant topics.