The 3rd IEEE Workshop on
Human-in-the-loop Methods and
Human Machine Collaboration in BigData
(IEEE HMData 2019)

co-located with IEEE Bigdata 2019
Los Angeles, Dec. 9th (Fixed)

Photo courtesy of wikimedia commons
About Keynotes Dates and Submission Student Travel Support Organization HMData 2018 IEEE Bigdata 2019

About IEEE HMData 2019


Human power is a key factor to maximize the impact of bigdata technologies. This workshop addresses human-in-the-loop approaches in bigdata lifecycle - in collecting, processing, analyzing, utilizing, archiving and disposing them. The purpose of this workshop is to give excellent opportunities for students, researchers and practitioners to identify important research problems and exchange their ideas on human-in-the-loop in the bigdata context. To make the workshop an attractive place for those people, we solicit practitioner papers as well as research papers, in order to facilitate discussion among researchers who know solutions and practitioners who know problems. We also would like to make the place valuable for young researchers. All papers accepted for the workshop will be included in the Workshop Proceedings published by the IEEE Computer Society Press, made available at the Conference.


This workshop covers a wide range of human-related topics in the bigdata context, such as crowdsourcing, collaborative recommendation, crowdsensing, workflow model for humans and machines, incentives, human-assisted bigdata analysis, bigdata-human interaction, supporting tools for humans in human-in-the-loop systems, security and privacy in human-machine collaboration , human factors and ELSI (ethical, legal and social issues) in human-in-the-loop systems, and human-machine collaboration in real-world problems.


Computation and Organization
Michael Bernstein (Stanford University)
Abstract: Can computation amplify our ability to achieve complex collective goals? Today's techniques in areas such as crowdsourcing often fall short of this vision, in large part because their architecture is based on workflows, which are so inflexible that they can only be used for simple and modular goals. In this talk, I offer an alternative architecture based on computational organizational structures, and demonstrate that this approach enables groups to flexibly collaborate toward complex and open-ended goals such as product design, software engineering, and top-tier research. I will introduce techniques that fluidly assemble flash organizations and continuously adapt their efforts, evolve team structures and membership over time, and coordinate volunteers around the world in pursuing open-ended research. This research argues for a shift away from crowdsourcing as simple microtasks, wiki edits, or competitions, and toward computational systems that proactively aid groups in working together nimbly, reactively, and effectively toward complex goals.

Bio: Michael Bernstein is an Associate Professor of Computer Science at Stanford University, where he is a member of the Human-Computer Interaction group. His research focuses on the design of social computing and crowdsourcing systems. Michael's research has received awards at premier computing venues, and he has been recognized with an NSF CAREER award and an Alfred P. Sloan Fellowship. His Ph.D. students have gone on both to industry (e.g., Adobe Research, Facebook Data Science) and faculty careers (e.g., Carnegie Mellon, UC Berkeley). Michael holds a bachelor's degree in Symbolic Systems from Stanford University, as well as a master's degree and a Ph.D. in Computer Science from MIT.

Important Dates

  • Oct 12 (Sat), 2019: Due date for workshop papers submission Extended
    (Authors have to submit the title and abstract by Oct. 7 (Mon))
  • Nov 1 (Fri), 2019: Notification of paper acceptance to authors
  • Nov 15 (Fri), 2019: Camera-ready of accepted papers
  • Dec 9-12 (Mon-Thu), 2019: Workshops


All submissions must be submitted electorically through the submission page. Please prefix your submission category such as [Research Paper] to the Title of Paper field in the submission page. For example, if you would like to submit a project-in-progress paper "Crowd-centric Approach to Digital Archive Maintenance," you have to put "[project-in-progress paper] Crowd-centric Approach to Digital Archive Maintenance" into the Title of Paper field.
All papers accepted for the workshop will be included in the Workshop Proceedings published by the IEEE Computer Society Press, made available at the Conference.

Submission Categories

  • Research Papers (*) (long presentation): They report significant and original results relevant to the scope of this workshop. We solicit innovative or thought-provoking work but they do not necessarily have to reach the level of completion. The expected length is between 4 and 6 pages. The maximum length is 10 pages, though the paper should be commensurate with the size of the contribution.
  • Practitioner papers (*)(long presentation): They present interesting problems that require human-in-the-loop solutions in a variety of application domains, or present the interesting results of applying existing human-in-the-loop solutions to their domains. The expected length is between 4 and 6 pages. The maximum length is 10 pages, though the paper should be commensurate with the size of the contribution.
  • Project-in-progress papers (short presentation): They present the goals, challenges, and preliminary results of research or real-world projects in progress. The maximum length is 3 pages.
(*) Some of the papers submitted to the research or practitioner paper categories may be accepted as project-in-progress papers and allotted to short presentation slots.


Papers should be formatted to IEEE Computer Society Proceedings Manuscript Formatting Guidelines in the IEEE Bigdata 2019 CFP page

Student Travel Support

The IEEE BigData 2019 will offer student travel awards to student authors of not only main confernece papers but also workshop papers. Details will be given at IEEE Bigdata 2019 site later.



Senjuti Basu Roy (NJIT)
Alex Quinn (Purdue University)
Atsuyuki Morishima (Univesity of Tsukuba)

Program Committee (to be extended)

Mohammad Allahbakhsh (University of New South Wale)
Yukino Baba (University of Tsukuba)
Wolf-Tilo Balke (Technische Universitaet Braunschweig)
Adam Bradley (Amazon)
Daniel Barowy (Williams College)
Marco Brambilla (Politecnico di Milano)
Cinzia Cappiello (Politecnico di Milano)
Marina Danilevsky (IBM Research - Almaden)
Ujwal Gadiraju (L3S Research Center)
Munenari Inoguchi (University of Toyama)
Panos Ipeirotis (New York University)
Vana Kalogeraki (Athens University of Economics and Business)
Itaru Kitahara (University of Tsukuba)
Dongwon Lee (Penn State)
Masaki Matsubara (University of Tsukuba)
Satoshi Oyama (Hokkaido University/RIKEN AIP)
Nobuyuki Shimizu (Yahoo!Japan Research)
Elena Simperl (University of Southampton)
Yu Suzuki (Gifu University)
Keishi Tajima (Kyoto University)
Saravanan Thirumuruganathan (QCRI)
Ming Yin (Purdue University)
Demetris Zeinalipour (University of Cyprus)
Jing Zhang (Nanjing U. of Science & Technology)
Yudian Zheng (Twitter)