IROS2024 Workshop:
Workshop on
'The Grand Challenge of Cybernetic Avatars: Dreams and Facts'

2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2024), at ADNEC in Abu Dhabi ,UAE.

Duration and Date

Duration: Half-day
Date: AM Oct 14th, 2024

Organizers

Norihiro Hagita (ATR, Japan)
Khalifa Juma AlQama (DFL, UAE)
Miwako DOI (NICT, Japan)
Hiroshi Ishiguro (Osaka University, Japan)
Arianna Menciassi (SSSA, Italy)
Paolo Dario (SSSA, Italy / DFL, UAE)

Abstract

We propose a workshop on cybernetic avatar (CA) as a future vision of robots that can both improve productivity and enrich people's lives. Cybernetic Avatar (CA) is a generic term for human enhancement and augmentation systems. Using multiple avatars may free people from several physical, brain, space, and time constraints, and enable them to do what they want to do in good health and to live a new life based on socially accepted norms.

This workshop will introduce the vision, the benefits and some limitations posed by a future society with CAs in 2050, with particular reference to the Japanese Moonshot Goal 1 Program (https://www8.cao.go.jp/cstp/english/moonshot/sub1_en.html) and to the activities of the joint lab between JST and Dubai Future Foundation to be developed into an international R&D effort. Invited speakers will present the latest R&D activities and the scientific and technical challenges on CAs including androids, 3D-CG avatars, social robots, and in-body CAs within body and cells. The potential advantages of a well-balanced society of humans and CAs for a sustainable development – in terms of reduced needs for unnecessary traveling, efficiency, continuous and discrete assistance to citizens health, and more – is strong. Several CA systems will offer the audience several cases of in-depth analysis, inspiration and debate on opportunities and challenges for research and to explore creative solutions. In fact, a strong potential exists for several CAs to generate grand challenges for robotics and new frontiers for next generation of robotics, as well as to create new teleoperation markets with social acceptance. 

Content

The workshop aims at introducing the concept and the objectives of Cybernetic Avatars (CAs), as outlined in the Japanese Moonshot Goal 1 Program (https://www.jst.go.jp/moonshot/en/program/goal1/index.html), that is the realization of a society in which human beings can be free from limitations of body, brain, space, and time by 2050. 
CA is a generic term for human enhancement and augmentation systems. Using multiple avatars may free people from several physical, brain, space, and time constraints they have to deal with inside and outside their body, and enable them to do what they want to do in good health and to live a new life based on socially accepted norms.

This Workshop will address primarily the technical aspects of CAs, with reference to two different CA configurations (outside and inside the human body). A second Workshop is also proposed, addressing primarily the “philosophical” aspects of CAs and their acceptability for different classes of users, as evaluated through analysis and systematic tests conducted in different sites (including the Dubai Future Labs and the Dubai Mall) with different users. The two half-day workshops are proposed to be held both at IROS 2024, on the same dates.

After a general introduction to the objectives of the Moonshot Goal 1 Program two typical configurations of CAs will be illustrated in detail: the first involves the realization of an avatar-symbiotic society where everyone (including disabled, frail elderly, minorities) can perform active social roles without constraint; the second example relates to an in-body Cybernetic Avatar (in-body CA) that can visualize the state of health in the body.

The avatar-symbiotic society will include multiple teleoperated CAs that can fully transmit the user's actions, intentions, and reactions in scenarios which feature hospitality-rich dialogue and will allow users to take part in various social activities (work, education, medical care, daily life, etc.). A major scientific and technical challenge posed by this paradigm is the teleoperation of multiple CAs by a single user and the interaction of a single CA with multiple users.

During the workshop, the latest results of the next-generation Brain Machine Interface technology will also be presented, which allows CAs to express what they think in their heads by controlling their words and actions. The in-body CAs require to implement efficient spatio-temporal environmental information in the body. This can be achieved by distributing and coordinating multiple types of millimeter-, micro-, and nanoscale in-body CAs to realize health monitoring and ultraminimally invasive diagnostics. The ultimate goal is to develop a system useful for health maintenance, diagnosis, and disease prevention, that could be used by people in their daily lives, thereby contributing to societal health and longevity.

Invited speakers will present the latest R&D activities and the scientific and technical challenges on CAs including androids, 3D-CG avatars, social robots, and in-body CAs within body and cells. The potential advantages of a mixed society of humans and CAs for a sustainable development – in terms of reduced needs for unnecessary traveling, efficiency, optimal use of energy, continuous and discrete assistance to citizens health, especially for the elderly, increased safety of humans in dangerous tasks, reusability, and more – is strong and it will be addressed in detail.

In addition, short papers to be presented orally as teasers and discussed as posters in an interactive session will be solicited by means of a Call and by engaging the members of some endorsing RA TCs. Topics that are relevant to this Workshop include, but are not limited to: algorithms for Human-Robot-Avatar Interaction, predictive modeling of human intentions for avatar behavior, motion planning and coordination in collaborative tasks with avatars, user studies for evaluating human-avatar interaction, enhancing transparency and explainability in AI for avatars, benchmarking and use cases for understanding avatar explainability, social dynamics and empathy in avatar-mediated interactions, human-centered design approaches for avatar development, novel applications of cybernetic avatars in real-world scenarios, micro- and nano technologies for in-vivo avatars, and more.

Finally, a round table discussion with participation of the speakers and of the audience will offer all participants – along with the networking poster session - the possibility to interact with experts and peers, to ask questions and to express opinions on several hot issues, such as the intriguing scientific and technical problems that are posed by the CA scenarios, the opportunities for frontier research, the role of artificial intelligence, the importance of inter- and transdisciplinary research, the opportunity and need for international collaboration, for developing novel industrial products, and for addressing fundamental societal issues, with special attention to the respect of the human person, of diversity and of the planet.


Program

The Tentative Program of the Workshop is the following:

TimeTalkComments
09:00 - 09:05Opening Remarks (Paolo Dalio (SSSA))
09:05 - 09:15Keynote
Norihiro Hagita (ATR)
Introduction to the grand challenge of Cybernetic Avatars (CA)
09:15 - 09:30Session 1:
Invited Talk 1
Takahiro Miyashita (ATR)
Grand Challenge on hospitality - rich and moral dialogue Cybernetic Avatars
09:30 - 09:45Invited Talk 2
Kouta Minamizawa (Keio Univ.)
Grand Challenge on experience-sharing Cybernetic Avatars
09:45 - 10:00Invited Talk 3
Kan Akutsu (Araya inc.)
Grand Challenge on imagination-operated Cybernetic Avatars
10:00 - 10:10Invited Talk 4
Fumihito Arai (Univ. of Tokyo)
Grand Challenge on In-Body Cybernetic Avatars within body
10:10 - 10:20Invited Talk 5
Yoko Yamanishi (Kyushu Univ.)
Grand Challenge on In-Body Cybernetic Avatars within cells
10:20 - 11:00Session 2:
Poster session
Session Chair: Miwako Doi (NICT)
10:30 - 11:00Coffee break
11:00 - 12:00Session 3:
CA hands-on session
Session Chair: Yukiko Horikawa (ATR)
Teleoperation for CAs at Umekita2 in Osaka, Jetro Start-up exhibition, Dubai, and Orihime at Robot Cafe in Dubai
12:00 - 12:50Session 4:
Panel Discussion and Q&A
Moderators: Norihiro Hagita (ATR), Yukiko Horikawa (ATR)
Panelists: Paolo Dario (DFL/SSSA), Kouta Minamizawa (Keio Univ.), Fumihito Arai (Univ. of Tokyo), Yoko Yamanishi (Kyusyu Univ.), Tarek Taha (DFL)
12:50 - 13:00Closing Remarks (Paolo Dario (DFL/SSSA) and Norihiro Hagita (ATR))

At IROS 2024, we aim to introduce and discuss research and development focused on a new society with cybernetic avatars. We are planning to host another workshop (Ethical and Social Sustainability) in the afternoon at the same room #9 and one forum on 17th morning in conjunction. We encourage you to join all THREE events!


List of Invited Speakers and Panelists

Six males and two females will contribute our workshop as invited talks. They are the Program Director, the Sub-Program Director, Program Managers, and PI in the Moonshot Goal 1 Program  and/or  expertise researchers in the diverse range of research fields of the workshop.

Prof. Paolo Dario,
Male, Italy,
Professor and former-Director, Scuola Superiore Sant’Anna (SSSA) and
Chief Scientist, Dubai Future Labs (DFL) 

Paolo Dario is Emeritus Professor of Biomedical Robotics at the Scuola Superiore Sant’Anna, Pisa, Italy. He has been and is professor and fellow at various universities and scientific institutions worldwide. Currently, he serves as Chief Scientist at Dubai Future Labs (DFL). His current research interests are in the field of bio-robotics, bionics, and companion robotics. He has coordinated many large national and European projects, and served in many EU Committees. He is an IEEE Life Fellow and the Founding Editor-in-Chief of the IEEE Transactions on Medical Robotics and Bionics. He served as President of the IEEE Robotics and Automation Society, chaired several IEEE Conferences and received several prizes and Awards, including the 2024 IEEE Technical Field Award for establishing and advancing Bionics and Biorobotics. His research accomplishments have led to the creation of several start-up companies, and to several practical applications in medical robotics and medical bionics.

Prof. Norihiro Hagita,
Male, Japan,
Program Director, JST Moonshot Goal 1 Program, 
Director, ATR Norihiro Hagita Laboratory and 
Professor, Osaka University of Arts

Norihiro Hagita received B.S., M.S., and Ph.D. degrees in electrical engineering from Keio University, Japan in 1976, 1978, and 1986. From 1978 to 2001, he was with the Nippon Telegraph and Telephone Corporation (NTT). He joined the Advanced Telecommunications Research Institute International (ATR) and established the ATR Media Information Science Laboratories in 2001 and the ATR Intelligent Robotics and Communication Laboratories in 2002, and has been serving as the Director, ATR Fellow, ATR Norihiro Hagita Laboratory, respectively. He is also a chair and professor at Art Science departments in Osaka University of Arts. From 2014 to 2022, he served as the research supervisor in the JST CREST research programs on intelligent information processing systems creating co-experience knowledge and wisdom with human-machine harmonious collaboration. He was a member of Science Council of Japan in 2017-2022. Since 2020, he has been serving as the program director in the JST moonshot goal 1 program on realization of a society in which human beings can be free from limitations of body, brain, space, and time by 2050.

Dr. Takahiro Miyashita
Male, Japan,
Director, ATR Interaction Science Laboratories and
Director, ATR Interaction Technology Bank

Takahiro Miyashita received his Ph.D. degree in engineering for computer-controlled machinery from Osaka University, Japan, in 2002. Currently, he serves as the Director of Interaction Science Laboratories (ISL) and Interaction Technology Bank (ITB) at Advanced Telecommunications Research Institute International (ATR). Additionally, he serves as one of group leaders of Moonshot R&D Program Goal 1 Avatar-Symbiotic Society Project under the supervision of Prof. Hiroshi Ishiguro. His research focuses on Cybernetic Avatars (CA), communication infrastructure for CA called CA-PF, human-robot interaction, and cloud-networked robotics. Furthermore, he is dedicated to fostering entrepreneurship and facilitating industry collaboration for the societal implementation of cutting-edge technologies. In 2015, he established the i-RooBO Network Forum, a consortium of companies in the fields of robotics and IoT. In 2021, he also established the Corporate Consortium for Avatar-Symbiotic Society (C-CAS2), a consortium specifically dedicated to companies involved in cybernetic avatar technologies.

Prof. Kouta Minamizawa
Male, Japan,
Professor, Keio University and
Project Manager, JST Moonshot Goal 1 Program

After receiving his PhD. in Information Science and Technology from the University of Tokyo in 2010, he joined Keio University Graduate School of Media Design (KMD) and directs KMD Embodied Media Project, where conducts research and social deployment of embodied media that transfer, enhance, and create human experiences with digital technologies. His areas of research expertise include Haptics, Embodied Interaction, Virtual Reality and Telexistence. He also promotes activities on Haptic design, and Superhuman sports, also serves as a project manager of the Cybernetic being project under the Moonshot R&D program.

Mr. Kan Akutsu
Male, Japan,
Research Team Leader, Araya Inc.

Kan Akutsu received his M.S. degree in Information Science and Technology from the University of Tokyo in 2012. Following his graduation, he joined DeNA Co., Ltd., where he developed and planned several IP titles, one of which won the prestigious Famitsu Award for Excellence, a well-known accolade in the Japanese entertainment industry. He later worked at ZEALS, aiming to develop general-purpose interaction robots, where he served as a Product Owner. He is currently a team leader at Araya Inc. and serves as the Principal Investigator under PM Kanai for Moonshot Goal 1, a project led by the Cabinet Office. His research focuses on creating a common platform for operating avatars using multimodal biological signals, including brain information.

Prof. Fumihito Arai,
Male, Japan,
Professor, The University of Tokyo and
Project Manager, JST Moonshot Goal 1 Program

Fumihito Arai is a full Professor of the Department of Bioengineering, Department of Mechanical Engineering, at The University of Tokyo, Japan. He is mainly engaging in the research fields of bio-robotics, micro- and nano-robotics, micro- and nano-mechatoronics, MEMS, and Biomedical applications. He received a Doctor of Engineering degree from Nagoya University in 1993. Since 1998, he has been an Associate Professor at Nagoya University. Since 2005, he has been a Professor at Tohoku University. Since 2010, he has been a Professor at Nagoya University. Since 2020, he has been a Professor at the Department of Mechanical Engineering at The University of Tokyo. He was the Vice President of Technical Activities, at IEEE Robotics and Automation Society (RAS) (2014–2015) and (2016–2017). He is currently an Adcom member of RAS since 2022. He has been the Program Manager of the Moonshot R&D program on In-body Cybernetic Avatars since 2022.

Prof. Yoko Yamanishi
Female, Japan,
Professor, Kyushu University and
Project Manager, JST Moonshot Goal 1 Program

Yoko Yamanishi received the Ph.D. degree from Imperial College London in 2003. She became Assistant Professor of Dept. of Bioengineering and Robotics, Tohoku University in 2006-2009, mainly have engaged in the researches about Bio-medical Science and Engineering. She became Associate Professor of Dept. of Micro-Nano Systems Engineering of Nagoya University and also a member of PRESTO JST during 2010-2012, and have started her work of a needle-free bubble injector. She was Associate Professor of Dept. of Mechanical Engineering of Shibaura Institute of Technology during 2013-2015 and became a Professor in Kyushu University on 2016 (Dept. of Mechanical Engineering, Kyushu University) and is leading biomedical fluid engineering laboratory. She is also currently the Program Manager of moonshot Goal1(Creating A Society Whose Citizen’s Health is Monitored by Remote Control of Intracellular Cybernetic Avatars). Intracellular Cybernetic Avatar (hereinafter referred to as intracellular CA) technology will allow individuals to improve the state of their bodies by external remote control of intracellular CA constructed of nanometer-scale artificial and natural biomolecules. Remote control of the avatars is expected to keep people’s bodies in good condition at a cellular level using micrometer-scale cell-cell interactions.

Dr. Tarek Taha
Male, UAE
Robotics Lab Lead, Dubai Future Labs (DFL)

Tarek Taha has 20 years of experience in technology development in the fields of robotics, autonomous systems, and embodied artificial intelligence while working in industry and academia. He obtained his PhD in Robotics and Mechatronics from the Centre of Excellence for Autonomous Systems (CAS), University of Technology Sydney, Australia, followed by a degree in Research Commercialization from Australian Technology Network and Chief Technology Officer Degree from Massachusetts Institute of Technology (MIT).
He currently leads the Robotics Lab at the Dubai Future Labs with a focus on R&D initiative and applied research activities in the field of autonomous robotics and embodied A.I. Throughout his career, he has led productive teams to conduct advanced R&D activities focusing on developing cutting-edge engineering solutions and products for deployment in real-world applications. He has peer-reviewed papers on autonomous aerial systems, autonomous exploration, navigation and mapping, machine vision, human-robot interaction, assistive robotics and reinforcement learning.
He is passionate about technology development and the proper implementation of R&D strategies and programs to produce world-class products, services, and solutions that positively influence our lives.


Structure of the events

First, an overview of the future in 2050 and the need for research and development in CA to realize it and the international activities currently underway will be given in invited talks, followed by teasers and panel discussions intertwined with coffee breaks, and finally an open discussion where researchers new to this field, especially young people, can frankly talk with leading edge researchers and close the meeting.