This year, I co-organized the workshop ‘Challenges of HRI in Real-World Contexts‘, together with Somaya Ben Allouch and Astrid Rosenthal – von der Putten, during the RO-MAN conference held in New York. The aim of our workshop was to bring together researchers from both industry and academia to discuss best practices as well as pitfalls of HRI research in real-world settings, and to provide the HRI community with guidelines to inform future developments of their robotic systems. The interactive character of our workshop gave room to many extended discussions on the challenges of doing HRI research outside the lab. What methodological issues arise when we try to replicate findings earlier established in lab settings? What role plays the research context? And what kind of technological challenges are we facing when performing research in the wild? Based on the presentations and the follow-up discussions after each presentation, we listed four topics for our main extend discussion session at the end of the morning.
Take what you can and give nothing back
The first discussion topic was guided by the following question: What can we learn from other fields about research in real-world contexts? One of the main points addressed in the discussion on this topic is that the HRI community may rely too much on theories and methods from (social) psychology. The assumption that human-robot interactions (should) follow guidelines of human-human interaction take a very prominent place in especially the way social behaviors of robots are developed and evaluated. However, there are many other fields that have relevant theories, methods and approaches. For example the fields of medicine or health research offer alternatives that can be useful when studying (long-term) effects on human behavior. The second main point addressed in this discussion was that ethical issues become more prominent when conducting research in the wild.
Everything inside out
The second discussion topic was guided by the following question: What does it mean to replicate results found in lab studies in real-world contexts? One of the main points addressed was that all researchers should acknowledge that there is a divide between lab studies and research in the wild. Not only with regard to the conclusions that can be drawn from the different types of studies, but also concerning the methods one could (or should) apply. But maybe an even more important divide in terms of its impact of the ability of the HRI field to move forward is that both types of research seem to be conducted by different sub-groups within the field. The HRI community should be more open to different types of methodologies and approaches to build a strong research field. This leads to the second main topic addressed in this discussion, which involves the necessity to include qualitative data when performing research in the wild. Quantitative data on its own is not able to capture all the complex phenomenon going on in real-world contexts. A third and final point addressed in this discussion, which is linked to the previous one, is that almost any type of HRI research is currently different from the ultimate real-world contexts since robots are not yet fully disseminated within our society. And participants in research are not the same as real end-users. Only when robots become mainstream and people start using robots on a regular basis, we can begin to unravel the sustained effects of our interactions with these artificial others.
Houston, we have a problem
A third discussion topic evolved around the technological challenges when conducting research in the wild. HRI researchers often encounter technical problems with the utility of the system, but also with regard to the collection of user data. The first main point addressed here is that researchers often choose the best solution for their technical constraints but not the perfect solutions. This is a result from either limitations in resources, the infeasibility of the perfect solutions, or other reasons. This does not have to be a problem, but its implications of the conclusion drawn from such studies should be properly addressed. Another main point addressed in this discussion is that innovation research often strive for patent registration, which has a negative effect on sharing progress with others in the field.
Long time no see
The fourth topic discussed was on long-term research. One of the main points addressed was that the definition of what makes a study long-term should not solely depend on the user’s perspective, but should be linked to the (cognitive) development of the robot as well. Not only users change their (use) behaviors over time, also the robot will develop over time when it learns to master its necessary skills. Another main point addressed in this discussion was that each users will have his or her own interpretation of the robot, resulting in different (social) roles they assign to the robot. This can even be the same robot; each user will establish its own use behaviors even though they are interacting with the same robot. One person may have daily social chit-chats with the robots, while another person may use the robot just as a tool. A final point addressed in the discussion on long-term research is that we may need to define classifications for both the technology as well as the user. For example, we could stereotype the technology based on its functionalities in a similar way we do with users and writing persona’s for each user group, and we could stereotype user groups on several aspects such as (not) wanting to interact socially with robots.