A final blog from Open All Senses

Step into the world of “Open all Senses – Accessible Multimodal Telepresence Robots in Social Spaces”. Our year long research project investigated the responsible design and purposeful use of telepresence technologies in two ‘social’ spaces: healthcare and museums. Our empowering goal was to investigate additional requirements for the use of telepresence robots in various scenarios (in both healthcare and museum contexts) to provide a more complete experience for the tele-present and remote operator. We also explored the social, ethical and legal risks concerned with the deployment of these technologies and how these technologies can provide a more accessible platform for remote visits.

Telepresence robots are internet enabled mobile platforms which allow a person to attend and interact in a remote environment. Telepresence gives operators the appearance and feel of being present via a remote robotic system, usually equipped with a screen and video calling capabilities.  In a world where technology facilitates remote human connection, for example working from home using MS Teams or seeing a doctor online, telepresence robots offer a promising future to enable people to be co-present in social settings.

Our co-production approach facilitated stakeholders to be involved throughout the development lifecycle. We explored risks and accountabilities and co-developed scenarios and use cases.  New insights with respect to operator trust, acceptability and application areas emerged. Common and diverse themes have been identified across different use cases when comparing healthcare and museums examples.

The project was based in the Cobot Maker Space (CMS) and interleaved with research being led by the Trustworthy Autonomous Systems (TAS) Hub, Horizon and academics from the School of Computer Science, such as Professor Caleb-Solly.

Figure 1: The medical ward room

We highlight the development of two new agile research environments in the CMS:

1. Medical Ward Room (Figure 1): We designed and set up a controlled medical ward room explicitly tailored for testing telepresence and robotic equipment in a controlled environment. This involved identifying and installing essential infrastructure, including network connectivity, power supply, and optimized furniture arrangements. Integrating telepresence and robotic technologies into the medical ward room allowed us to assess human-robot and human-computer interactions in medical settings. Our stakeholder interviews and test scenarios in this space yielded valuable insights into the responsible design of telepresence technologies within healthcare contexts. This activity was led by Professor Praminda Caleb-Solly and Dominic Price.
2. Smart Museum Room (ongoing): Our work involves equipping a dedicated smart museum room, fostering testing and evaluation of telepresence and other robotic technologies in museum and gallery settings. By developing various deployment scenarios within the smart museum room, we showcase the immense potential of telepresence and robotic technologies for remote visits, guided tours, and catering to specific groups, such as vulnerable individuals. Carefully configuring the room with appropriate hardware, software, and interactive displays enables seamless user experiences and functionalities. The room is still under development and work was co-funded by a School of Computer Science Research Grant.

The smart museum and medical ward room will provide a testbed for community events to engage with stakeholders, including museum professionals, healthcare practitioners, researchers, technology developers and others. These spaces and the investigated telepresence technologies are fantastic opportunities to inspire young people in the community towards STEM, Responsible Research and Innovation, Technology, Multi-disciplinary studies and many others. Throughout the project, we connected with local organisations and got involved in outreach activity with the community, including Inspire Foundation Nottingham, encouraging participants to provide feedback and share their experiences.  We are now exploring potential collaborations to further advance the field of accessible multimodal telepresence robots.

At the core of the Open all Senses project was Responsible Research and Innovation (RRI) and engagement with the hoRRIzon project team to address and embed (RRI) throughout our work was key.

Involving stakeholders (museum professionals, healthcare practitioners, policymakers, industry practitioners, and end-users) throughout the research and development process provided valuable insights, perspectives, and feedback. This engagement ensured that the project’s outcomes and technologies were aligned with the real needs and expectations of the target audience, increasing the chances of successful and responsible adoption and impact.

The project facilitated an initial RRI workshop to investigate ethical, technological, social and legal implications of deploying telepresence robots in spaces such as healthcare and museum galleries. The need for data privacy, security measures, and responsible use of personal information emerged as crucial considerations. The project emphasised the importance of staying informed about relevant regulations, engaging with legal experts, and advocating for ethical guidelines and standards to ensure the responsible and accountable deployment of accessible multimodal telepresence robots.

The project demonstrated the power of interdisciplinary collaboration in addressing complex challenges. By bringing together experts from computer science, robotics, psychology, human-computer interaction, social science, law, and healthcare, we were able to leverage diverse perspectives, knowledge, and methodologies. This collaboration facilitated the development of innovative telepresence solutions that considered both technical feasibility and practical applicability of telepresence robots in social spaces.

Harriet ‘Alfie’ Cameron joined the project at the start of 2023 after submitting their PhD thesis as part of the Horizon CDT. Harriet has been building on insights from a stakeholder workshop held late in 2022. The workshop findings supported researchers from Horizon to create a series of questions and prompts to explore and document ethical, social, and physical concerns relating to the use of telepresence robots in museum and health care environments. Harriet has since used these questions and prompts to guide semi-structured discussions with professionals that consider potential applications for telepresence robots in the participants’ workplaces, as well as the barriers to implementation and theoretical risks that might arise. Thirteen participants were interviewed – the goal being to interview 10 people from each profession.  Participant organisations included the NHS, private healthcare providers, the National Trust, and independent museums. The interviews are being analysed using using a PACT framework specifically modified to explore how People, Activities, Context, and Technologies co-exist and shape healthcare and museum environments. Our preliminary findings provide some deeply insightful and novel applications for the use of telepresence robots that recognise the concerns and risks unique to healthcare and museum domains.

Feng Zhou joined the project towards the end of 2022 and has been exploring the integration of Soma Bits [a growing library of shapes and actuators] into human-robot interaction (HRI) to enrich interactive strategies.

Feng expanded the application of Soma Bits to HRI by developing “Somatobotics” [1], a system that combines controlling mechanisms and modularized hardware.

Somaboticsgraphical user interface

Somatobotics features include:

Applicability to various applications through the analysis of commonalities among diverse actuators, controllable via button values (BV) and mapping values (MV)

• facile implementation of complex controlling logic by generating BV and MV
• seamless extensibility to a range of devices through the utilization of Robot Operating System (ROS) and modularized hardware design(4) user-friendliness achieved by concealing programming complexities using the visual programming tool Grasshopper and a customised interface.

Somatobotics is now being used by researchers at the University of Nottingham to integrate haptic and other modalities to provide feedback and enhance the experience of being co-present using telepresence technologies.

Open All Senses was a great avenue to build new research collaborations. Some of the closest partners that we worked with included The Lincoln Museum (The Collection), The Contemporary, Lakeside Arts museum, Hastings (the first to deploy telepresence robots in their settings).

The project has come to an end, however, new collaborations and research projects have emerged and its impact and efforts are extending far beyond its initial expectations.

[1] Zhou, F., Price, D., Pacchierotti, C. and Kucukyilmaz, A., 2023, July. Somabotics Toolkit for Rapid Prototyping Human-Robot Interaction Experiences using Wearable Haptics. In IEEE World Haptics Conference (Work-in-Progress paper).

Tags: cobot maker space, horizon cdt, hoRRIzon 2.0, human robot interaction, mobile robotic telepresence, Praminda Caleb-Solly, RRI, soma design, Somatobotics, STEM, TAS hub, telepresence