Spot It! Mathematics through computer vision

: The Electrical Engineering and Computer Science Department of the institute of Technology of Valenci-ennes has been participating to Festival of Science for many years. Workshops about scientific subjects are organised in sessions of one and a half hour for groups of twelve participants. Participants come from different types of schools: from primary schools up to high schools. Workshops are designed, prepared and then animated by the students of the institute. The workshop that is presented in this article has been designed by 6 second year students as a project linked with a Computer Vision and Robotics module. Knowledge from the first-year module about Logic and Numeration (especially binary calculation) and more generally from the modules of Mathematics (especially homogenous coordinates for geometric transformations used in Robotics). The workshop is built around a popular matching cards game called “ Spot It!” or “Dobble” depending on the countries . In the first part of the article the context of the experiences is described. In the second part, the game is described, a mathematical model is explained and allows the design of a simplified Spot It! game. The third part explains the method chosen to write the screenplay of the workshop, while part four gives the screenplay of the workshop as it results from the work of students.


EXPERIENCE CONTEXT 1.Hardware
Most experiments described in the article have been made on a Raspberry Pi 3. The Raspberry Pi Foundation is a UK based charity founded in 2009.This organization intends to help people all around the planet to access to digital world.To accomplish this goal, the Raspberry Pi Foundation has worked with academic, scientific and research community, and in particular with Cambridge University, for the development of a low cost and flexible hardware for experimenting with programming and electronics.This collaboration is at the origin of the Raspberry Pi family, and in particular the Raspberry Pi 3, which we have been working with.The Raspberry Pi is equipped with General Purpose Input Output pins which allow the use of a wide range of actuators and sensors.A huge number of low-cost projects based on Raspberry Pi has been imagined by makers and gives a source of inspiration for students: retrogaming, drone designing for example.As for the camera, a Raspberry Picamera is used.

Software
Python is an interpreted language created in 1991 by Guido von Rossom.Python interpreters are available for most operating systems, and in particular available for the Rasbian Strech version of Linux that is installed on Raspberry Pi3.Python supports oriented-object programming.
Free distributions of Python and of useful libraries for Python can have been found on the Internet.Among the libraries that are used for the work presented here are Numpy (for numerical work), Matplotlib (for graphical work), and OpenCV (for computer vision).

Reference books
This teaching experience of Computer Vision in the context of Mathematics teaching was first made by Mathematics teachers: they found great help in two books, one by Jain [1], the other by Gonzalez [2].Giving students the habit of searching answers in these books was one of the challenges of the experience.

Students
The theater set of the experiences described here is a group of twelve students with no experience in computer vision.These students are given an initiation to computer vision in the context of the Festival of Science or University open days, and within courses on Robotics and Vision.A more precise description of the context can be found in [3].One goal of the initiation is to show them that Mathematics are a useful tool and are used in domains they appreciate like image construction.The paper used for the cards is a glossy paper which makes it difficult to take pictures of quality from the cards without an elaborate lighting.Playing with fewer symbols and with less complicated symbols would allow to focus on main ideas about Computer Vision: students are supposed to be working with Computer Vision for the first time.For these reasons, an adapted game of Spot It! has to be designed, which implies to think about the underlying mathematical model of the game.

Questions
The main questions that arise when trying to design a Spot It! game find their answer from combinatorics.What is the minimal number of symbols required to design a Spot It! game?For a suitable number of symbols, what is the greatest number of cards that can be designed?A Spot It! game with such a number of cards will be said to be an optimal game.In particular, it appears that the official game is not optimal and could contain 57 cards, satisfying then a more precise property: two different cards have exactly one common symbol.

Answers
Answers to these questions are exposed in [4] and [5].The main answer is: "An optimal Spot It! game can be modelled by a finite projective space".A projective space is a pair of two sets ( , ) with an incidence relation between elements (called points) and elements of (called lines).× is completed to obtain ( ) with 8 ideal points (points at infinity) corresponding to the 8 possible directions for lines (slopes from 0 to 6 and vertical direction).Therefore ( ) has 57 elements.The correspondence between ( ) and the game Spot It! is obtained taking Cards as Points and Symbols as Lines.For example, axiom (A1) gives "two different cards have a unique common symbol".The simplified game is built from the smallest projective plane, that is ( ), the finite projective plane built from the set of integers modulo 2: = {0,1} The affine plan × has 4 points:

The choice of the symbols
The choice of symbols is made to allow both a work on colour representation and a work on shapes.Choosing 7 different colours would imply that the work on shapes is useless, and would also imply a very performant lighting.On the other hand, one colour would imply a very poor work on colours (only colour for background and colour for symbols).A choice of three different colours has been made.The final selection is: -Yellow (with a maximal intensity in Red and in Green, and an intensity of 0 in Blue in RGB description) -Violet (with a maximal intensity in Red and in Blue, and an intensity of 0 in Green in RGB description) -Cyan (with a maximal intensity in Green and in Blue, and an intensity of 0 in Blue in RGB description) -Dark Grey for the background (with an intensity of a third of maximal intensity in Red, Green, and Blue in RGB description) Three basic shapes have been chosen for symbols: circle, square and triangle.The calculation of moments with OpenCV makes it easy to separate those shapes.In a purpose of simplification, only the moments of order 0, i.e. the areas of the symbols are used in the project.It is made possible by choosing constant sizes for the different symbols: a big circle, a medium square and a smaller triangle.This condition is a very restrictive condition, but is necessary for a computer vision workshop that is supposed to last one hour and a half.For example, the point p3 is the card "Cyan Disk, Cyan Triangle, Yellow Square" as this point is the intersection of these three lines.

The DARE method
Students have to organize the workshop using the pedagogical DARE method developed by the authors [6].DARE is the acronym for Discover, Apply and Resolve.This method can be perceived as a rewriting, for computer languages, of the classical three-steps tests developed for language teaching: first translating from foreign language to native language, then translating from native language to foreign language and last writing an essay in foreign language.The Discover step is devoted to translating computer language (here Python and OpenCV instruction) into native language.Workshop participants are given scripts they have to run and comment.In the Apply step, workshop participants have to answer very simple questions using the instructions seen in the first step: they simply translate basic works into computer language.The last step Resolve is the most complicated but also most interesting step: workshop participants must use their judgement, creativity and skills to solve a specific problem, in link with previous steps.

Writing the screenplay of the workshop.
First thing to do when writing the screenplay of the workshop is to begin by the end, that is here, playing Spot It! with the Raspberry, i.e. "Matching symbols between two cards".Students have to resolve the problem, compare their solutions and draw up a list of the different instructions used.This is the" Resolve" part.Those instructions correspond to the different steps of the resolution of the problem: these steps will give the subjects of the different exercises of the "Apply" part.Once again students have to propose a list of exercises for each subject, solve them and then write scripts corresponding to solutions of similar exercises.Those solutions give the "Discover" part.Three main steps can be established in the resolution of the problem.The first mathematical object encountered is the colour BGR image.This image is converted in the HSV format.In both formats a colour image can be thought as three greyscale (or intensity) images  The OpenCV function findContours has then to be used to find blobs, that is white connected parts, in the image.Other functions allow the calculation of the moments of those blobs, and in particular the calculation of their area.
Three main steps have been obtained for our pedagogical sequence: first Black and White images, then Greyscale images, and finally Colour images.

Resolve
The following challenge is proposed at the end of the workshop: "Two cards are placed under the Picamera.Write a program that indicates the name of their common symbol"

CONCLUSION
This workshop has been proposed to pupils from highschool visiting the University in June 2019.The work of our students has been appreciated by pupils and their teachers who specially enjoyed making Mathematics in an applied context.Students have appreciated the modelling activity of the project, and some of them have extended their work discovering the links between error correcting codes and finite projective plans.Maybe the subject for another article!

2 SPOT IT! 2 . 1
The official gameSpot It! is a matching game by Asmodee Company.This card game is sold under the name Dobble in Europe.

fig 4 :
fig 4: Spot It! card game.The game consists in 55 cards.There are 8 different symbols on each card.Two different cards have at the most one common symbol.The total amount of symbols used is 57.

2. 3
Design of a simplified game 2.3.1 The geometry of the simplified game.

fig 9 :
fig 9: Fano plane.( ) In this diagram, the ideal line is represented by a circle.Applying these results to the problem of the construction of a simplified Spot It! game indicates that 7 symbols (lines) are needed to design 7 cards (points) with 3 different symbols on each card (three lines through each point).
fig 10: Simplified cards fig 12: BGR image of a card.
fig 13: Greyscale image (Hue component)These greyscale images are usually transformed in black and white images using a threshold method.

fig 14 :
fig 14: Black and White after Thresholding