blockplay.ai prototype app

Blockplay.ai is a free prototype educational app, which uses AI-based computer vision software to recognise standard Cuisenaire rods or multilink cubes placed on a tabletop under a webcam, and responds with audio and animations. For example, it can speak the number of blocks and their colours, or play musical pitches proportional to the length of the rods, or display the molecules that would be made if the black, white and red cubes were atoms of Carbon, Hydrogen and Oxygen, to model basic climate chemistry.

You can download the app from the link below, and there is a ‘How to’ step-by-step guide which tells you what kind of Windows laptop, webcam and blocks you ideally need to run it. Please note the app is an early prototype and may not run perfectly every time. We would welcome any feedback and ideas for future development and partnerships – email us on contact@blockplay.ai

Download the app – How to install and run the app

*** We are looking for volunteers to trial the app – please click here for more details ***

  • An animated gif of a laptop connected to a webcam showing the app recognising and displaying blocks in different ways.
    Users place blocks on a tabletop under a webcam and take their hands away. The AI-based app recognises the blocks and says how many there are – or plays tones proportional to their length, or shows the molecules they would make if they were atoms joined together.
  • An image of the main menu of the blockplay.ai app, showing the four modes, for maths, music, molecule and animation.
    On the main menu users can choose whether to use multilink cubes or Cuisenaire rods. Then choose from four modes: maths, music, molecule and animation. Each mode has several variations which users can select from the dropdown menus, before hitting ‘Play’.
  • A set of the playthings including building blocks designed by Friedrich Froebel
    The app was a spinoff from an unsuccessful attempt to develop an AI-based research tool to recognise children’s 3D constructions made with wooden building blocks like these Froebelian Kindergarten cubes. It proved easier for the AI algorithm to recognise coloured cubes, placed flat, like Cuisenaire rods or multilink cubes.
  • A spinning model of a molecule of lactic acid, in anaglyph 3D. in
    In 3D mode the app can display stereoscopic 3D images of spinning molecules, like this one of lactic acid C3H6O3 visible with red/cyan glasses.
  • An image of a spinning 3D C60 buckyball molecule
    Tip: to unlock the 3D buckyball, place two carbon atoms together!
Credits

The blockplay.ai app was developed by a team at the University of Bristol, at the Research Centre for Teaching, Learning and Curriculum in the School of Education, in partnership with computer vision software developer Pysource, and a local primary school.

The app’s development was funded by the Bristol Digital Futures Institute, the Jean Golding Institute, and the Runway programme.

We would like to thank the university’s Research Software Engineering team for their technical advice, all the children and staff at the participating primary school for trialling the app and their invaluable feedback, and Martha for the voiceover recordings.

App menu graphics from https://www.pngwing.com and https://www.flaticon.com.

For more info or to send feedback please email: contact@blockplay.ai