Emotions Handpose with LEDs

Youtube Video Demonstration

P5 Sketch Code: https://editor.p5js.org/mai8189/sketches/PwBwWnAuQ

Introduction to Project:

This project combines interactively choosing your emotion on screen using p5 machine learning handpose, with LEDs that light up based on the corresponding emotion.

Step 1: Applying the Thumbs Up and Down Example

The first thing I did was to get my thumbs up and down example working from class. The following is the circuit and code I used to get this to run:

I used this circuit, but actually only two of the LED’s. I added another 2 for my emotions project

int ledPin1 = 2;
int ledPin2 = 3;

void setup() {
  pinMode(ledPin1, OUTPUT);  // sets the pin as output
  pinMode(ledPin2, OUTPUT);
  Serial.begin(9600);        // initialize serial communications
}
 
void loop() {
 if (Serial.available() > 0) { // if there's serial data available
   int inByte = Serial.read();   // read it
   if (inByte == 1) {
     digitalWrite(ledPin1, HIGH);  // use it to turn on the LED 1
     digitalWrite(ledPin2, LOW);
   } else if (inByte == 2) {
     digitalWrite(ledPin2, HIGH);  // use it to turn on the LED 2
     digitalWrite(ledPin1, LOW);
   } else {
     digitalWrite(ledPin1, LOW);  // sets the LED off
     digitalWrite(ledPin2, LOW);  // sets the LED off
   }
   delay(200);                // waits
 }
}

P5 sketch can be found here: https://editor.p5js.org/mai8189/sketches/PwBwWnAuQ

Step 2: Making my Handpose

To make my handpose, I firstly changed the keypoints I was working. Through some trial and error, I deduced which keypoint coincides with the tip of my pointer finger. I isolated this point, made it bigger, and changed it’s color based on where on the video screen I was pointing. The following is my code, and a video demonstration of how I completed this process:

emotions handpose.mp4

int ledPin1 = 2;
int ledPin2 = 3;
int ledPin3 = 4;
int ledPin4 = 5;

void setup() {
  pinMode(ledPin1, OUTPUT);  // sets the pin as output
  pinMode(ledPin2, OUTPUT);
  Serial.begin(9600);        // initialize serial communications
}
 
void loop() {
 if (Serial.available() > 0) { // if there's serial data available
   int inByte = Serial.read();   // read it
   if (inByte == 1) {
     digitalWrite(ledPin1, LOW);  // use it to turn on the LED 1
     digitalWrite(ledPin2, LOW);
     digitalWrite(ledPin3, LOW);  // sets the LED off
     digitalWrite(ledPin4, HIGH);  // sets the LED off
   } else if (inByte == 2) {
     digitalWrite(ledPin2, LOW);  // use it to turn on the LED 2
     digitalWrite(ledPin1, LOW);
     digitalWrite(ledPin3, HIGH);  // sets the LED off
     digitalWrite(ledPin4, LOW);  // sets the LED off
   } else if (inByte == 4) {
     digitalWrite(ledPin2, HIGH);  // use it to turn on the LED 2
     digitalWrite(ledPin1, LOW);
     digitalWrite(ledPin3, LOW);  // sets the LED off
     digitalWrite(ledPin4, LOW);  // sets the LED off
   } else if (inByte == 3) {
     digitalWrite(ledPin2, LOW);  // use it to turn on the LED 2
     digitalWrite(ledPin1, HIGH);
     digitalWrite(ledPin3, LOW);  // sets the LED off
     digitalWrite(ledPin4, LOW);  // sets the LED off
   } else if (inByte == 5) {
     digitalWrite(ledPin2, HIGH);  // use it to turn on the LED 2
     digitalWrite(ledPin1, HIGH);
     digitalWrite(ledPin3, HIGH);  // sets the LED off
     digitalWrite(ledPin4, HIGH);  // sets the LED off
   } else {
    //no gesture
     digitalWrite(ledPin1, LOW);  // sets the LED off
     digitalWrite(ledPin2, LOW);  // sets the LED off
     digitalWrite(ledPin3, LOW);  // sets the LED off
     digitalWrite(ledPin4, LOW);  // sets the LED off
   }
   delay(200);                // waits
 }
}

P5 Code: https://editor.p5js.org/mai8189/sketches/szoEa4xLs

P5 Sketch full-screen: https://editor.p5js.org/mai8189/full/szoEa4xLs

Step 3: Demonstration with LEDs

Emotions Handpose with LEDs

Reflections on where to take Project Next:

I think for this particular concept, I may want to expand on it in certain ways. An example of how I might expand on it is by perhaps including a wider combination of possible outputs, or maybe exploring other types of communication like sound input. It is still a work in progress.