About IEEE HMData 2022
Overview
HMData workshop, which originally started as the "Human-Machine collaboration in BigData" workshop, will investigate the opportunities and challenges in human machine collaboration in work with bigdata, which are described by two terms: Human-in- the-Loop Methods and Future of Work. Human-in-the-Loop is a term focusing on the employer's viewpoint while Future of Work focuses more on worker's viewpoint, in both of which the division of labor among humans and machines is a key issue.
This area is likely to be heavily AI driven, and we intend to invite papers covering the following aspects, (a) Capturing human capabilities through intelligent models and how to adapt them through changing perceptions, needs, and skills. (2) High level tools that provide the ability for all stakeholders in the new ecosystem, including regulators for policies and AI workers, to specify their requirements. (3) system design and engineering of job platforms for collection, storage, retrieval, and analysis of data deluge about workers, jobs, and their activities. (4) Benchmarking and the development of appropriate metrics to measure system performance as well as human aspects, such as satisfaction, capital advancement, and equity.
We welcome any interesting ideas and results on any relevant topics, but this workshop encourages submitting papers the results of which have been or will be implemented as platforms, tools and libraries. This year, we plan to have a thematic session on improving the interoperability of tools on Human-in-the-loop Methods and Future-of-Work.
We also solicit practitioner papers as well as research papers, in order to facilitate discussion among researchers have solutions and practitioners who know problems.
All papers accepted for the workshop will be included in the Workshop Proceedings published by the IEEE Computer Society Press, made available at the Conference.
Journal publication of selected papers
After the conference, high quality papers will be selected and recommended for possible publication in a special issue of Information Systems Frontiers, Springer.
Topics
This workshop covers a wide range of topics of human-machine collaboration in work with bigdata. Keywords include: crowdsourcing, collaborative recommendation, crowdsensing, workflow model for humans and machines, incentives, human-assisted bigdata analysis, bigdata-human interaction, human-machine collaboration in real-world applications (such as natural disaster response, education, and citizen science), and ELSI in Human-in-the-loop systems and Future of Work.
We expect submissions to address some of the following issues:
- capturing human characteristics and capabilities,
- stakeholder requirement specification,
- social processes around the human-in-the-loop systems,
- platforms and ecosystems,
- computation capabilities, and
- benchmarks and metrics for human-in-the-loop systems and Future of Work
Keynotes
Mitigating Biases in Crowdsourcing Data Collection
Ming Yin (Purdue University)
Abstract: Data has become the secret sauce for the rapid progress of artificial intelligence (AI).
Over the past decade, crowdsourcing has become a prevalent paradigm for obtaining data from people to
enhance machine intelligence. However, there is a growing line of literature showing that data collected
from crowdsourcing efforts could have significant biases, which not only decreases the quality of the data,
but may even negatively impact the downstream algorithmic models built based on these data.
Many factors could contribute to the biases in crowdsourced data, including the composition of the dataset
on which annotations are solicited from the crowd (e.g., sampling bias of the dataset), and the cognitive
and behavioral limitations that the crowd is subject to when providing annotations (e.g., cognitive bias,
affective bias, social bias). In this talk, I'll present some of our recent efforts in mitigating biases
throughout the crowdsourcing data collection lifecycle.
Bio:
Ming Yin is an Assistant Professor in the Department of Computer Science, Purdue University. Her research broadly connects to the fields of human-computer interaction, applied artificial intelligence and machine learning, computational social science, and behavioral sciences. She uses both experimental and computational approaches to examine how to better utilize the wisdom of crowd to enhance machine intelligence (i.e., crowdsourcing and social computing), and how to better design intelligent systems that people can understand, trust and engage with effectively (i.e., human-AI interaction).
Prior to Purdue, She spent a year at Microsoft Research New York City as a postdoctoral researcher in the Computational Social Science group. She completed her Ph.D. in Computer Science at Harvard University, and received her bachelor degree from Tsinghua University, Beijing, China.
Program
The workshop starts at 8:50AM on Dec. 17 (Japan Standard Time)
Paris, France Sat, 17 Dec 2022 at 00:50 CET
New York, USA Fri, 16 Dec 2022 at 18:50 EST
Tokyo, Japan Sat, 17 Dec 2022 at 08:50 JST
Labels: [LR] - Research Paper (Long Presentation), [R] Research Paper (Short Presentation), [W] Project-in-Progress Paper (Short Presentation), [P] Practitioner Paper (Short Presentation)
8:50AM Opening (WS Chairs)
9:00-10:56 Session 1 (Chair: Senjuti Basu Roy)
9:00 Keynote by Ming Yin (Purdue University)
9:50-10:00 Short Break
10:00 [LR] Xiao Liu, Alan F. Smeaton, and Alessandra Mileo. An Adaptive Human-in-the-loop Approach to Emission Detection of Additive Manufacturing Processes and Active Learning with Computer Vision
10:20 [R] Seungun Kim, Masaki Matsubara, and Atsuyuki Morishima. Image Geolocation by Non-Expert Crowd Workers with an Expert Strategy
10:32 [R] Nailcan Kara, Yagiz Levent Gume, Umit Tigrak, Gokce Ezeroglu, Serdar Mola, Omer Burak Akgun, and Arzucan Ozgur. A SHAP-based Active Learning Approach for Creating High-Quality Training Data
10:44 [R] Hiroaki Tanaka, Yuuma Jitsunari, Wataru Yamada, Keiichi Ochiai, Shoko Wakamiya, and Eiji Aramaki. Graph Neural Network Tells Us Who is the Communication Enhancer
10:56-11:20 Break
11:20-12:36 Session 2 (Chair: Alex Quinn)
11:20 [LR] Adel Ardalan, Derek Paulsen, Amanpreet Singh Saini, Walter Cai, and AnHai Doan. Toward Data Cleaning with a Target Accuracy: A Case Study for Value Normalization
11:40 [LR] Tomoya Kanda, Hiroyoshi Ito, and Atsuyuki Morishima. Efficient Evaluation of AI Workers for the Human+AI Crowd Task Assignment
12:00 [R] Yunyi Xiao, Yu Yamashita, Hiroyoshi Ito, Masaki Matsubara, and Atsuyuki Morishima. Multi-Armed Bandit Approach to Qualification Task Assignment across Multi Crowdsourcing Platforms
12:12 [P] Kristina L. Kupferschmidt, Joshua G. Skorburg, and Graham W. Taylor. DelphAI: A human-centered approach to time-series forecasting
12:24 [W] Ryuya Itano, Tomoya Nohara, and Takahiro Koita. Crowd-Aided Anomaly Detection in Surveillance Videos
12:36-14:00 Lunch Break
14:00-15:16 Session 3 (Chair: Atsuyuki Morishima)
14:00 [LR] Kaname Muto, Satoshi Oyama, and Itsuki Noda. Explainable Recommendation Using Knowledge Graphs and Random Walks
14:20 [LR] Katia Bourahmoune, Karlos Ishac, Marc Carmichael, and Toshiyuki Amagasa. Owro: A Novel Robot for Sitting Posture Training Based on Adaptive Human Robot Interaction
14:40 [W] Bijoly Saha Bhattacharya, Biswajit Mandal, Arindam Biswas, and Malay Bhattacharyya. Improving Character Recognition by the Crowd Workers via Corrective Feedback
14:52 [R] Ryo Yanagisawa, Susumu Saito, Teppei Nakano, Tetsunori Kobayashi, and Tetsuji Ogawa. PostMe: Unsupervised Dynamic Microtask Posting For Efficient and Reliable Crowdsourcing
15:04 [W] Kazumu Nakahira, Shun Kuroiwa, and Takahiro Koita. Multi Speaker Identification System using Crowdsourcing-based Method
15:16 Closing Remarks (WS Chairs)
Important Dates
- Oct 14 (Fri), 2022: Due date for workshop papers submission Extended
(Authors have to submit the title and abstract by Oct. 7 (Fri))
- Nov 10 (Thu), 2022: Notification of paper acceptance to authors
- Nov 27 (Sun), 2022: Camera-ready of accepted papers Extended
- Dec 17-20 (Sat-Tue), 2022: Workshops
Submission
All submissions must be submitted electorically through the submission page (This will be open in August).
Please prefix your submission category such as [Research Paper] to the Title of Paper field in the submission page.
For example, if you would like to submit a project-in-progress paper "Crowd-centric Approach to Digital Archive Maintenance," you have to put "[project-in-progress paper] Crowd-centric Approach to Digital Archive Maintenance" into the Title of Paper field.
Reviews and Proceedings
All research papers are reviewed by at least two reviewers. All papers accepted for the workshop will be included in the Workshop Proceedings published by the IEEE Computer Society Press, made available at the Conference.
Submission Categories
- Research Papers (*) (long presentation): They report significant and original
results relevant to the scope of this workshop. We solicit innovative or
thought-provoking work but they do not necessarily have to reach the
level of completion. The expected length is between 4 and 6 pages. The maximum length is 10 pages, though the paper should be commensurate with the size of the contribution.
- Practitioner papers (*)(long presentation): They present interesting
problems that
require human-in-the-loop solutions in a variety of application domains, or
present the interesting results of applying existing human-in-the-loop
solutions to
their domains. The expected length is between 4 and 6 pages. The maximum length is 10 pages, though the paper should be commensurate with the size of the contribution.
-
Project-in-progress papers (short presentation): They present the goals,
challenges, and preliminary results of research or real-world projects
in progress.
The maximum length is 3 pages.
(*) Some of the papers submitted to the research or practitioner paper
categories may be accepted as project-in-progress papers and allotted to short
presentation slots.
Format
Papers should be formatted to IEEE Computer Society Proceedings Manuscript Formatting Guidelines in the IEEE Bigdata 2022 CFP page
Organization
Chairs
Senjuti Basu Roy (NJIT)
Alex Quinn (Purdue University)
Atsuyuki Morihsima (University of Tsukuba)
Program Committee
- Yukino Baba (The University of Tokyo)
- Wolf-Tilo Balke (Technische Universitaet Braunschweig)
- Ria Mae Borromeo (University of the Philippines Open University)
- Francois Charoy (University of Lorraine, Inria, CNRS)
- Marina Danilevsky (IBM Research - Almaden)
- Ashraf Dewan (Curtin University)
- Gianluca Demartini (University of Queensland)
- David Gross Amblard (Rennes 1 University / IRISA Lab)
- Itaru Kitahara (University of Tsukuba)
- Vana Kalogeraki (Athens University of Economics and Business)
- Masaki Matsubara (University of Tsukuba)
- Shigeo Matsubara (Osaka University)
- Satoshi Oyama (Hokkaido University)
- Raghav Rao (University of Texas at San Antonio)
- Yu Suzuki (Gifu University)
- Keishi Tajima (Kyoto University)
- Hisashi Toriya (Akita University)
- Demetris Zeinalipour (University of Cyprus)
Contact
hmdata.chairs@gmail.com
|
