STEM

Student Showcase: Interacting with MiRo via Auditory-Mediated Sensations

The start of our Student Showcase blog series which highlights some of the amazing student projects utilising MiRo. These articles are written by the students about their projects and this piece is written by Logan Miller from the University of Sheffield.


During my 3rd year at the University of Sheffield my team and I were tasked with adding human-robot interactions to the MiRo robot. The MiRo has a soft tail and ears which produce a specific crunchy noise when pinched or stroked. We planned to use the four microphones present in the MiRo’s body to detect the noise from stroking or pinching these body parts. This could then be used to train the robot, such as stroking to reward it and pinching to punish it – a common machine learning technique called reinforcement learning.

To detect if MiRo has been stroked or pinched, and where it happened, we used a classifier. A classifier is a form of supervised learning which uses labelled training data to map the various sounds to the actions that produced them.

We used the MiRo’s four microphones to record five minutes of us stroking the tail and ears, followed by five minutes of us pinching them and then five minutes of background noise in our lab. After recording our training data, we used Google’s VGGish library in Python to convert our audio files into semantically meaningful high-level 128-dimension embed files which produced better results when fed into a classification model compared to the audio files.

We then mapped each 128D embed file from the training data to a corresponding action and trained a classification model using a linear support-vector machine SVM classifier from the Ski-Kit Learn Python library. As our project focused on detecting the pinch or stroke the MiRo responded with either a blue, green or red LED for if it detected background noise, stroking, or pinching respectively.

Images of MiRo demonstrating the three LED colour results

Background noise can cause a lot of errors in audio classification, so removing the background noise from the pinching and stroking recording would improve the classifier’s results. One way of achieving this is a noise gate, which defines a specific sound level that audio can pass. If this level is not met, no audio signal is recorded (Hodgson, 2010).

We opted to use a slightly more advanced technique where the noise gate is adapted into a spectral noise gate. This is where a signal is split into individual frequencies, with custom thresholds set for each frequency. A noise gate is then applied to each frequency, resulting in a more tailored approach depending on the situation (Audacity, 2021).

We used Tiny Machine Learning (tinyML) techniques to allow our detection to run on the low-powered Raspberry Pi 3B+ inside the MiRo. This was to improve the speed MiRo could respond to pinches and strokes compared to if our detection was running of an external machine communicating with MiRo over a WiFi network. The main objective in tinyML is to reduce the pre-trained models, which is achieved through quantization, pruning and fusing.

Quantization its often the most effective technique and reduces the accuracy of values by using fewer bits. PyTorch’s quantization functionality can reduce the model up to four times its original size by using 8-bit integers over floating point numbers (Quantization, n.d.).

Pruning is a technique which eradicates useless parameters. First a pruning criterion has to be chosen, which assigns a rank to a set of elements to be pruned. The main pruning methods are random pruning, where elements are ranked in a random order, and magnitude-based pruning, where weights below a certain threshold are set to zero.

Finally, fusing combines multiple operators together, however we decided not to implement it as there is very little performance difference in fused and non-fused models with small models, such as this one. (PyTorch Fusing, n.d.).

Overall, this was a really fun project which allowed my team and I to get hands on experience with robots. It also gave us a greater understanding of the theoretical biomimetics we learnt alongside this project – which is the combination of analysis (investigating and understanding complex biological systems), and synthesis (building the system to test our scientific understanding).

 
 

POST AUTHOR

Logan Miller

Computer Science Graduate BSc from The University of Sheffield

 

Hodgson, J. (2010). Understanding Records: A Field Guide To Recording Practice. Bloomsbury Academic.

Alternative Noise Reduction Techniques. (2021, November 16). Audacity Manual. Retrieved May 12, 2022, from https://manual.audacityteam.org/man/alternative_noise_reduction_techniques.html

Quantization — PyTorch 1.11.0 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/docs/stable/quantization.html

Fuse Modules Recipe — PyTorch Tutorials 1.11.0+cu102 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/tutorials/recipes/fuse.html

Education Secretary Prioritizes Coding and Cloud Accessible Digital Tools to Improve Computer Science Provision in Schools

Education Secretary Prioritizes Coding and Cloud Accessible Digital Tools to Improve Computer Science Provision in Schools

We at Consequential Robotics have been extremely encouraged to learn about the UK Government’s own assessment and priorities for the needs of schools and families. Read on to learn about why coding in schools is top of mind.

How can you stimulate interest in coding?

How can you stimulate interest in coding?

In the future, there will be a demand for more and more Artificial Intelligence [AI] coding to run our cities and the devices we use in our home for housework, security, homeworking and entertainment. Staring at a screen looking at line after line of code appeals to some more than others so how can you stimulate yours or your pupils' interest in coding?

Can Robots Support Social Distancing within the Home?

Can Robots Support Social Distancing within the Home?

Hear from Professor Lynne Baillie from the School of Mathematical and Computer Sciences at Heriot-Watt University, who, along with Computer Science graduate Neve Curnyn, investigated whether MiRo could assist with social distancing within the home.

Engineering a brain for MiRo - Part 2

What does a model of a brain actually look like? The answer is that they come in many, many forms. For example, some scientific groups work at a very detailed level, trying to understand how individual neurons, or groups of neurons, work to perform the functions that form the building blocks of behaviour.

Engineering a brain for MiRo - Part 1

MiRo was created to sense the world and then act on it. In this two part blog post, we are asked Consequential Robotics CTO Dr Ben Mitchinson to talk us through the origins of MIRO’s biomimetic brain.

The winners of the 2020 school robot competition!

The winners of the 2020 school robot competition!

We’re excited to share that the first-place winners have bagged themselves a MiRo-E robot for their winning space robot design!

Press Release: MiRoCLOUD is a simple to use AI robotics creative coding home learning tool for all

Press Release: MiRoCLOUD is a simple to use AI robotics creative coding home learning tool for all

Consequential Robotics, developers of the popular MiRo robot, today announced the release of MiRoCLOUD, a new browser-based platform for people of all ages and abilities to understand and develop essential coding skills. The unique combination of MiRoCLOUD with the charming MiRo-E biomimetic robot makes coding engaging, easier to learn and more enjoyable.

Are friends electric? The benefits and risks of human-robot relationships

Are friends electric? The benefits and risks of human-robot relationships

Social robots that can interact and communicate with people are growing in popularity for use at home and in customer-service, education, and healthcare settings. Although growing evidence suggests that co-operative and emotionally aligned social robots could benefit users across the lifespan, controversy continues about the ethical implications and their potential harms. In this condensed post, Professor Prescott and Professor Robillard highlight the benefits and risks of human-robot relationships.

Designing my own world – interview with Aarna (10 years old)

Designing my own world – interview with Aarna (10 years old)

In her first blog post with us, our global GIFT challenge winner shares her coding motivations and goals. Read on to learn more about Aarna and what she has learnt through coding.