Student Showcase: Stimulating Psychological Behaviour in MiRo

Our Student Showcase blog series highlights some of the amazing student projects utilising MiRo. These articles are written by the students about their projects and this piece is written by Aung Htet from the University of Sheffield.


Introduction

My name is Aung Htet, and I am currently working on the MiRo to produce a simulation of a psychological experiment called the Strange Situation. It is a continuation of a group project I worked on while at the University of Sheffield. Being allowed to continue on this project along with my supervisor, Dr. Alejandro Jimenez-Rodriguez, I have been working on this project at Sheffield Robotics to be able to embody the simulation and write a paper related to it. The idea of the project is to be of use in affective and social robotics, where it will allow the MiRo to simulate behaviours rather than following a set of instructions that have been tasked for completion.

 

What the project is about

As an introduction to the project, it is based on a psychological experiment called the Strange Situation Procedure, which is based on the relationship between the caretaker and the child. The purpose of this experiment is to observe different attachment behaviours between the mother and child, which can be exhibited in different scenarios. The possible attachment styles can either be secure, ambivalent or avoidant.

Thus, in our experiment for simulating behaviour, we will be simulating these three types of behaviour by having the MiRo decide whether to explore or care for each other. The decision process will be made through a coupled oscillator dependent on the ambivalent level, avoidant level and the information inferred from one another. There are two parts to the project: first, to allow the MiRos to directly infer from each other, followed by the MiRos having the ability to take cues for inferring information.

 

Progress on what is being done or what will be done

The first step of the project involves using a QR code to determine where the MiRos would meet, and sound recognition will be involved in determining whether care is being given or received. In this step, the parent and child share a common state that would be communicated to all the nodes. The step has been successful, with a demonstration being done at the Sheffield Robotics lab.

First stage of the project

Second stage of the project

The second step of the project would involve the MiRo to determine each other's state through inference on cues like sounds and gestures.

Thus, further improvements will be made concerning the detection of audio through speech recognition and detecting the “coo” of another MiRo. This will be used on communications between the MiRos with current progress being done on speech recognition. Currently, the MiRo is focused on being able to detect a few commands, which set the MiRo to do some actions.

The MiRo will also be able to detect each other through image recognition rather than make use of a QR code to allow proper simulation of detecting the other MiRo and care being given. This will allow the MiRo to meet each other rather than setting a rendezvous point through the use of a QR code.

 

POST AUTHOR

Aung Htet

Computer Science Student at The University of Sheffield

 

 

 

Student Showcase: Interacting with MiRo via Auditory-Mediated Sensations

The start of our Student Showcase blog series which highlights some of the amazing student projects utilising MiRo. These articles are written by the students about their projects and this piece is written by Logan Miller from the University of Sheffield.


During my 3rd year at the University of Sheffield my team and I were tasked with adding human-robot interactions to the MiRo robot. The MiRo has a soft tail and ears which produce a specific crunchy noise when pinched or stroked. We planned to use the four microphones present in the MiRo’s body to detect the noise from stroking or pinching these body parts. This could then be used to train the robot, such as stroking to reward it and pinching to punish it – a common machine learning technique called reinforcement learning.

To detect if MiRo has been stroked or pinched, and where it happened, we used a classifier. A classifier is a form of supervised learning which uses labelled training data to map the various sounds to the actions that produced them.

We used the MiRo’s four microphones to record five minutes of us stroking the tail and ears, followed by five minutes of us pinching them and then five minutes of background noise in our lab. After recording our training data, we used Google’s VGGish library in Python to convert our audio files into semantically meaningful high-level 128-dimension embed files which produced better results when fed into a classification model compared to the audio files.

We then mapped each 128D embed file from the training data to a corresponding action and trained a classification model using a linear support-vector machine SVM classifier from the Ski-Kit Learn Python library. As our project focused on detecting the pinch or stroke the MiRo responded with either a blue, green or red LED for if it detected background noise, stroking, or pinching respectively.

Images of MiRo demonstrating the three LED colour results

Background noise can cause a lot of errors in audio classification, so removing the background noise from the pinching and stroking recording would improve the classifier’s results. One way of achieving this is a noise gate, which defines a specific sound level that audio can pass. If this level is not met, no audio signal is recorded (Hodgson, 2010).

We opted to use a slightly more advanced technique where the noise gate is adapted into a spectral noise gate. This is where a signal is split into individual frequencies, with custom thresholds set for each frequency. A noise gate is then applied to each frequency, resulting in a more tailored approach depending on the situation (Audacity, 2021).

We used Tiny Machine Learning (tinyML) techniques to allow our detection to run on the low-powered Raspberry Pi 3B+ inside the MiRo. This was to improve the speed MiRo could respond to pinches and strokes compared to if our detection was running of an external machine communicating with MiRo over a WiFi network. The main objective in tinyML is to reduce the pre-trained models, which is achieved through quantization, pruning and fusing.

Quantization its often the most effective technique and reduces the accuracy of values by using fewer bits. PyTorch’s quantization functionality can reduce the model up to four times its original size by using 8-bit integers over floating point numbers (Quantization, n.d.).

Pruning is a technique which eradicates useless parameters. First a pruning criterion has to be chosen, which assigns a rank to a set of elements to be pruned. The main pruning methods are random pruning, where elements are ranked in a random order, and magnitude-based pruning, where weights below a certain threshold are set to zero.

Finally, fusing combines multiple operators together, however we decided not to implement it as there is very little performance difference in fused and non-fused models with small models, such as this one. (PyTorch Fusing, n.d.).

Overall, this was a really fun project which allowed my team and I to get hands on experience with robots. It also gave us a greater understanding of the theoretical biomimetics we learnt alongside this project – which is the combination of analysis (investigating and understanding complex biological systems), and synthesis (building the system to test our scientific understanding).

 
 

POST AUTHOR

Logan Miller

Computer Science Graduate BSc from The University of Sheffield

 

Hodgson, J. (2010). Understanding Records: A Field Guide To Recording Practice. Bloomsbury Academic.

Alternative Noise Reduction Techniques. (2021, November 16). Audacity Manual. Retrieved May 12, 2022, from https://manual.audacityteam.org/man/alternative_noise_reduction_techniques.html

Quantization — PyTorch 1.11.0 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/docs/stable/quantization.html

Fuse Modules Recipe — PyTorch Tutorials 1.11.0+cu102 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/tutorials/recipes/fuse.html

Education Secretary Prioritizes Coding and Cloud Accessible Digital Tools to Improve Computer Science Provision in Schools

Education Secretary Prioritizes Coding and Cloud Accessible Digital Tools to Improve Computer Science Provision in Schools

We at Consequential Robotics have been extremely encouraged to learn about the UK Government’s own assessment and priorities for the needs of schools and families. Read on to learn about why coding in schools is top of mind.

Are Social Robots Effective in Robot-Assisted Interventions for Social Anxiety?

Are Social Robots Effective in Robot-Assisted Interventions for Social Anxiety?

Last month, the University of Waterloo (Samira Rasouli, Garima Gupta, Elizabeth Nilsen & Kerstin Dautenhahn) published a paper on the effectiveness of social robots for clinicians and patients in Robot-Assisted Interventions for Social Anxiety. They used MiRo and a range of social robots in their study.

Can Robots Care?

Can Robots Care?

Looking for a fun exhibition for the New Year? Come and visit MiRo at the prestigious Thackray Museum of Medicine! The playful ‘Can Robots care?’ exhibition offers visitors the chance to explore a range of robots designed to care for humans.

How can universities stimulate interest and learning in large classes with both in-person and remote participation?

How can universities stimulate interest and learning in large classes with both in-person and remote participation?

On-campus learning is key to student experience but how can this be balanced with the benefits of teaching online or to utilise the best that technology has to offer? I will answer this question here.

Why Universities are Moving to Student-Centred Blended Learning

Why Universities are Moving to Student-Centred Blended Learning

The popular demand for university education is having two observable effects.

Read on to learn more from Professor David lane as he speaks about the challenge that universities face due to the changing landscape of teaching and learning.

Supporting creativity in young people

Supporting creativity in young people

Creative skills are invaluable in every industry and in everyday life and we all share a measure of responsibility for helping young people get started. However, when the pandemic hit, the cultural sector and the creative industries suffered more than most. It decimated the National Youth Theatre’s (NYT) income, and they were unable to host in-person fundraising events to raise vital funds required to support their work. Read on to learn how MiRo helped.

How can you stimulate interest in coding?

How can you stimulate interest in coding?

In the future, there will be a demand for more and more Artificial Intelligence [AI] coding to run our cities and the devices we use in our home for housework, security, homeworking and entertainment. Staring at a screen looking at line after line of code appeals to some more than others so how can you stimulate yours or your pupils' interest in coding?

Can Robots Support Social Distancing within the Home?

Can Robots Support Social Distancing within the Home?

Hear from Professor Lynne Baillie from the School of Mathematical and Computer Sciences at Heriot-Watt University, who, along with Computer Science graduate Neve Curnyn, investigated whether MiRo could assist with social distancing within the home.