Bounding box annotation from segmentation maps/id maps for computer vision.

Sometimes you want to generate synthetic data for training computer vision models. It is often easy to generate segmentation maps or class ID maps where objects in the image become uniform colours, simply using AOVs within 3D modelling programs, such as Maya, or by using substitute shaders in Unity or Unreal (or swapping out the material for one which just presents a uniform colour).

For many standard computer vision models, we require bounding box annotations, often given in the form of x1, y1, x2, y2, where these represent the coordinates of the bottom left and top right corners of the bounding box.

Here is a python script for converting from a segmentation map to bounding box annotations in a csv file. It also shows the bounding box annotation and you can save the bounding boxes also as images.

Please head over to my GitHub to check it out.

Example segmentation map.

Example segmentation map.

Bounding box output.

Bounding box output.

"Liquid Crystals" by Benjamin Outram published on IoP


My book, “Liquid Crystals” has been published through Institute of Physics, and is available to buy on Amazon here! Chapter 1 is reproduced below.


Chapter 1

What is it about nature that we find so beautiful? Maybe it is its colours and compositions, or the emotions and thoughts it provokes. This book will explore not only the vibrant visual aesthetic of liquid crystals, but also the deeper beauty that emerges as we come to understand their science and involvement in life.

The often untold story of liquid crystals lies somewhere between chemistry and biology. Most people are familiar with phase transitions like those between ice and water, and water and steam. In some materials, which are common in biological systems, there exist extra phases of matter called liquid crystals. Unlike crystals or liquids, they have a unique combination of fluidity and structure that enables them to spontaneously self-assemble into complex forms while remaining sensitive to environmental changes—key features of biological machinery—with mesmerising visual effect when viewed under a polarising optical microscope. Their fluid clockwork has been critical to the functioning of every biological cell since life began. The lipids, proteins, DNA and chromosomes in your body silently play out their dramas hundreds of billions of times every day.

Considering the shape-shifting, dynamic forms of liquid crystals when viewed under a microscope, it is not surprising that early researchers thought they may be some form of life. Over the last handful of generations, scientists have uncovered many different types of liquid crystals. We have come to understand why they exist, the laws that govern how they interact with themselves, with other materials, with light, and with fields. We have found them in the silk dope of spiders and worms, in the membranes of biological cells and many other biological systems. Chemists have engineered liquid crystals to exhibit new and unusual physical properties. And we have exploited liquid crystals in technology, mainly, thus far, for liquid crystal displays (LCDs) such as those in television and smartphone screens. It turns out that evolution has also worked out how to use liquid crystals for display purposes, as in the iridescence of beetles. We, however, have only just begun to understand the wider implications of imitating how biology exploits liquid crystals for our own technology.

Most people first hear the term 'liquid crystal' in relation to LCDs. Under normal circumstances, the kinds of liquid crystal structures useful for technology tend to be uniform, controlled, stationary, and boring. Some, out of curiosity, may have poked the screens of computers and calculators and been delighted by the strange liquid-like behaviour they saw. By making them flow, a fascinating aspect of their behaviour is revealed. The pretty structures you see throughout this book have come out of the same impetus of curious exploration, taking liquid crystals into conditions that are outside of their use in technology. Messing around with floating pools of liquid crystal near their melting temperature, inducing flow, adding additional chemicals like detergent, and playing with unusual liquid crystal phases—misspent hours of mucking around doing things you're not supposed to just for the thrill of seeing nature doing something intricate and amazing.

In chapter 2 we will build a basic understanding of matter, from the sub-atomic world of quarks to the solids, liquids and gases we experience in everyday life, and we will come to understand what a liquid crystal is. Chapter 3 will introduce light, and how it interacts with matter. These first chapters will form the basis upon which we will understand how matter can produce the images in this book. The remainder of the book chapters will explore the unique features and visual qualities of a series of different liquid crystals. Chapter 4 will explore nematic liquid crystals. The molecular properties that lead to the special physical properties of nematics are discussed. We will introduce key concepts related to the topology and symmetry of liquid crystals, and describe how they are used in LCD technology. Chapters 5 and 6 will introduce the concept of chirality in liquid crystals and explore a range of phenomena related to the twisted structures they form at different length scales. Chapter 7 follows, describing a range of less common and more complex structures, including the cubic-structured blue phases, the lyotropic liquid crystals most commonly seen in biological systems, and the layered smectic and twist-grain-boundary phases, and we will describe how spiders exploit liquid crystals to produce silk with extraordinary physical properties. Finally, chapter 8 will explore the discotic and columnar liquid crystal phases. On our journey, we will learn how liquid crystals interact with light and electric fields, the wonderful structures that liquid crystals can form, and some ways in which biology exploits the unique properties of liquid crystals.

Crystal Vibes finally Released on Steam! Download now.

Download 3D version now on SteamVR!

Experience body vibrations of candy colored psychedelic sound rippling through an endless crystal universe. Crystal Vibes utilizes the cutting edge of spatial 3D audio, full-body 'Synesthesia Suit' vibro-tactile stimulation, and sound visualization that maps sound and light based on the science of the human senses, to push the frontiers of technology-mediated sensory experience in virtual reality. With the project's predecessor described as “transcendent” and “like traveling through a psychedelic kaleidoscope” (Forbes 2016), this piece ups the ante with music from producer Ott. and is all-new for a trippier Sundance Film Festival and New Frontier 2017. 

Procedural Audio in Unity - Noise and Tone

Procedural audio is audio that is created using code at run time, rather than using a wav or mp3 sound file.  It can be very useful for creating audio that reacts to the situation and does not sound repetitive, which is difficult with prerecorded sound files.

To use the following script, create a GameObject in Unity and add an AudioSource.  Add the following C# script to the GameObject.  The script will automatically add a AudioLowPassFilter to the GameObject.

Press play and listen to the procedural audio!  You can mess around with the public variables on this script.  Frequency is set to 330Hz (an A tone) but change this to change the pitch.  The amount of noise compared to the tone volume is set by noiseRatio.  You can change the sound of the noise using the settings on the AudioLowPassFilter

//By Benjamin Outram
//C# script for use in Unity 

// create a new file and copy paste this.  Name the file:
// ProceduralAudio.cs
// attach script to a GameObject that has an AudioSource on it.
// adjust the public variables to experiment.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class ProceduralAudio : MonoBehaviour
    private float sampling_frequency = 48000;

    [Range(0f, 1f)]
    public float noiseRatio = 0.5f;

    //for noise part
    [Range(-1f, 1f)]
    public float offset;

    public float cutoffOn = 800;
    public float cutoffOff = 100;

    public bool cutOff;

    //for tonal part

    public float frequency = 440f;
    public float gain = 0.05f;

    private float increment;
    private float phase;

    System.Random rand = new System.Random();
    AudioLowPassFilter lowPassFilter;

    void Awake()
        sampling_frequency = AudioSettings.outputSampleRate;

        lowPassFilter = GetComponent<AudioLowPassFilter>();

    void OnAudioFilterRead(float[] data, int channels)
        float tonalPart = 0;
        float noisePart = 0;

        // update increment in case frequency has changed
        increment = frequency * 2f * Mathf.PI / sampling_frequency;

        for (int i = 0; i < data.Length; i++)
            noisePart = noiseRatio * (float)(rand.NextDouble() * 2.0 - 1.0 + offset);

            phase = phase + increment;
            if (phase > 2 * Mathf.PI) phase = 0;

            tonalPart = (1f - noiseRatio) * (float)(gain * Mathf.Sin(phase));

            data[i] = noisePart + tonalPart;

            // if we have stereo, we copy the mono data to each channel
            if (channels == 2)
                data[i + 1] = data[i];



    void Update()
        lowPassFilter.cutoffFrequency = cutOff ? cutoffOn : cutoffOff;



Virtual Reality Reading UIs

Virtual Reality (VR) devices have increasingly sparked both commercial and academic interest. While applications range from immersive games to real-world simulations, little attention has been given to the display of text in virtual environments. Since reading remains to be a crucial activity to consume information in the real and digital world, we set out to investigate user interfaces for reading in VR. To explore comfortable reading settings, we conducted a user study with 18 participants focusing on parameters, such as text size, convergence, as well as view box dimensions and positioning. This paper presents the first step in our work towards guidelines for effectively displaying text in VR.

Read full text here!

Liquid Crystal Music Video With Max Cooper

Max Cooper is one of my favourite artists in the world.  He takes electronic music to fine-art status and brings in many ideas from physics and science.

I was fortunate enough to collaborate with him on a music video for his track – “Music of the Tides” (Out now on Balance 030 -

The footage shows many liquid crystal phase transitions, including isotropic, nematic, cholesteric, columnar, smectic A, smectic C, twist-grain-boundary, and crystal phases of matter.

You can purchase special edition prints from Max Cooper's online store here, or a wider selection from my shop here.

Recently featured on Vice here

Evolutionary Algorithm with Neural Networks for Modelling Behaviour

Below are the result of some of my tinkering with machine learning and evolutionary algorithms with neural networks.  The first shows the network learning to get better at better at moving around in the space while not hitting itself, while the second is more advanced and shows a predator-prey scenario with co-evolving behaviours.  More information is in the video descriptions.

You can download a unity project and run the first one for yourself from my GitHub repository.

Neutrino at Dubai International Film Festival

Neutrino went down well at DIFF 2017 with over 100 people trying the demo. In addition, I was with Tanner Person demoing Flow Zone (Tanner Person, Benjamin Outram, Youssef Bouzarte), a poi-based virtual reality flow toy designed to induce states of flow, creativity and freedom.

Neutrino is a completely new kind of virtual reality experience. Dance dance revolution meets juggling in virtual reality for a rhythm action experience like never before As you pass through the levels you learn more and more complicated juggling tricks. This demo is being exhibited at Dubai International Film Festival this week.

Time control in virtual reality

This demo shows moving the controller through space to control time in the virtual environment. This kind of instant replay could be very useful for a variety of applications, and gives the user a feeling of having extra-ordinary control over their environment.

You can download the Unity project (Unity 5.6.1f1) on my GitHub here.