Did you know you can use Scratch to create fun virtual costumes like hats and glasses? Or that you can make games and projects that use your facial features? You can try all this in Scratch Lab, where we test new coding blocks and ask for your feedback.
Face Sensing is one of the cool experiments in Scratch Lab. It uses AI (Artificial Intelligence) to detect parts of your face. Let’s explore how it works! Eric, the Director of Scratch Lab, is here to help us experiment.
To try Face Sensing, go to the Scratch Lab website and click the “Try it out” button. This will load the Face Sensing blocks. You might need to allow camera access so your video appears on the screen. Then, you can use the “go to nose” block. When you click it, the cat will jump to your nose!
If you want the cat to stay on your face, use a “forever” block. Now, when you click it, the cat follows you around. You can also make it stick to your ear or even spin while it’s there!
Let’s try another experiment by making a hat or other object stick to your head, like a face filter. Choose a sprite, like a hat or glasses, from the sprite library. Or, use the Paint Editor to draw a funny hat with the brush or shape tools.
Using what you learned, can you write a script to make the sprite stick to the top of your head? I adjusted a script Eric showed us to make the hat stay on my head. If the hat doesn’t fit right, you can move it around in the costume editor until it’s perfect.
If the hat is too big or small, resize it using the tools or some code. How does your costume react if you’re close to the camera or far away? Does it follow your head tilt or sit awkwardly? Use code blocks to fix this.
For example, add the “set size to face size” block. This makes the hat change size based on your face’s size in the webcam. Add the “point in direction of face tilt” block, so when you turn your head, the hat moves with you, just like a real hat!
What else can you try? Visit the Face Sensing homepage to see other starter projects. At Scratch, we focus on helping learners express their creative ideas. That’s why we’re experimenting with AI, to let you explore new possibilities like face sensing.
It’s important to know that Face Sensing blocks can detect if a face exists, but they can’t tell who it belongs to or even if it’s a human face. This means the AI sometimes makes funny mistakes. Spotting these mistakes helps us understand the difference between human intelligence and AI.
For example, the AI might detect a toy’s face if you hold it close to the camera. But what about a whale? It has a face, but it’s not very human-like, so the AI might make the costume jump around a lot.
What about a drawing of a face? It can detect a simple drawing like a smiley face, especially if it’s close up. I’ve learned that even a filled circle can be detected as a face if the screen is filled with white paper. This shows how AI sees things differently from people.
If your face is partly covered, the AI might not see you. But if it can see just the top of your head and eyes, it works well, which is surprising!
The AI for Face Sensing was trained on millions of photos of people’s faces. It detects faces based on patterns it has seen, like the arrangement of eyes, nose, and mouth. That’s why it sometimes makes mistakes.
Thanks, Eric! I can’t wait to experiment with objects around my house to see what the AI recognizes as a face and create games and other interactive projects with these blocks.
Use the Scratch Lab to design a unique face filter. Choose a sprite, like a hat or glasses, and write a script to make it stick to your head. Experiment with resizing and positioning to ensure it fits perfectly. Share your creation with the class and explain how you made it work!
Challenge yourself to make a sprite follow different parts of your face, like your nose or ear. Use the “go to nose” and “forever” blocks to make the sprite move with you. Test how well it works and adjust your code to improve its accuracy.
Experiment with different objects and drawings to see what the AI recognizes as a face. Try using toys, drawings, or even your hand. Document any funny mistakes the AI makes and discuss why you think it happened. Share your findings with the class!
Use the “set size to face size” and “point in direction of face tilt” blocks to make your sprite react to changes in your face’s size and tilt. Test how well your sprite adjusts when you move closer or farther from the camera. Present your results and any improvements you made.
Create a short story or game using Face Sensing blocks. Incorporate different sprites and scripts to make your story interactive. Use your face to control the characters or elements in your story. Share your project with the class and explain how you used Face Sensing to enhance your story.
Here’s a sanitized version of the provided YouTube transcript:
—
Did you know that you can use Scratch to create virtual costumes, like a hat and glasses? Or that you can experiment with games and interactive projects that utilize facial features? You can try it out in Scratch Lab, where we create new experimental coding blocks and ask for your feedback.
Face Sensing is one of our Scratch Lab experiments. It uses AI to detect parts of your face. Let’s try it out! I’ve got Eric, the Director of Scratch Lab, here to help us experiment.
To try out the Face Sensing blocks, navigate to our Scratch Lab site and click the “Try it out” button at the top of the page. This loads up the Face Sensing blocks. You may need to allow camera permissions to get your video to appear on the stage. Then, you can use the “go to nose” block. If you click on it, the cat will jump to your nose.
If I want the cat to stick to my face, I can use a “forever” block. Now, when I click on it, the cat follows me around. I can also stick it to my ear or even make it spin while it’s there.
Let’s try another experiment and make a hat (or another object) stick to our head, like a face filter. You can choose a sprite that is a fashion item (like a hat or glasses) from the sprite library. Or, in the costume Paint Editor, try drawing a funny hat using the brush or shape tools.
Using what you learned in our first experiment, can you write a script to make the sprite stick to the top of your head? Here, I’ve altered a script that Eric showed us to make this hat continually go to the top of my head. Notice that this hat doesn’t quite fit on my head the way I expected. It’s positioned low on my face. I can change the position of this hat by going back to my costume, selecting the entire costume, and then moving it around until it’s just in the right place on the top of my head.
I also noticed that this hat may be too big or too small for my head. I could resize it using the tools or use a bit of code. How does your costume respond if you’re close to the camera versus further away? When you tilt your head, does it follow the tilt, or does it just sit awkwardly atop your forehead? We can use some code blocks to change this.
For instance, I can add the “set size to face size” block. When I get close to the camera or further away, the hat changes size based on the size of my face in the webcam. I can also add the “point in direction of face tilt” block. Now, when I turn my head, my hat goes with me, just like a normal hat would.
What else could you try? Navigate to the Face Sensing homepage and check out our other starter projects. At Scratch, we’re focused on making tools to help learners express their own creative ideas. That’s why we’re experimenting with AI, to let you explore new possibilities like face sensing.
It’s important to note that our Face Sensing blocks try to detect if a face exists, but they can’t identify who the face belongs to or even if it is a human face. This means the AI we use for Face Sensing sometimes makes interesting mistakes. Identifying these mistakes can help us see the difference between human intelligence and AI.
For example, the AI can detect the face of a toy if I hold it up close to the camera. But what about a whale? It definitely has a face, but it’s not very human-like. So you can see the AI makes the costume jump around a lot.
Now, what about a drawing of a face? It will detect even a very simple drawing like a smiley face, especially if we get it close up. One thing I’ve learned is that the drawing can be just a filled circle. However, I need to fill the screen with white paper, and then it thinks that’s a face. This shows how the AI sees things differently from a person.
What if my face is partly covered up? If it’s covered all the way, it can’t see me. But even if it can just see the top of my head and my eyes, it works really well. That’s kind of surprising.
The AI we’re using for Face Sensing was created by training it on millions of photos of people’s faces. It detects faces based on the patterns it has seen in those photos, like the arrangement of the eyes, nose, and mouth relative to each other. That’s also why it sometimes makes mistakes.
Thanks, Eric! I can’t wait to experiment with objects around my house to see what the AI recognizes as a face, and then create games and other interactive projects with these blocks.
—
This version maintains the original content while ensuring clarity and appropriateness.
Scratch – A programming language and online community where you can create your own interactive stories, games, and animations. – In Scratch, I made a fun game where a cat chases a mouse.
AI – Short for Artificial Intelligence, it refers to computer systems that can perform tasks that usually require human intelligence, like understanding language or recognizing pictures. – The AI in my game can recognize when I say “start” and begins the game automatically.
Face – The front part of a person’s head, or in technology, it can refer to recognizing or detecting a person’s face using a computer. – The new app uses face recognition to unlock the phone.
Sensing – Detecting or feeling something, often used in technology to describe how devices can detect changes in their environment. – The robot uses sensing technology to avoid obstacles in its path.
Camera – A device used to take pictures or videos, often used in computers and phones to capture images. – I used the camera on my tablet to take a picture of my project.
Sprite – A character or object in a computer game or animation, often used in Scratch to represent the characters you can control. – I created a sprite of a dog that can jump over obstacles in my game.
Filter – A tool that changes the appearance of an image or video, often used to enhance or modify pictures. – I applied a filter to my photo to make it look like a cartoon.
Code – A set of instructions written in a programming language that tells a computer what to do. – I wrote code to make my robot move forward and turn left.
Project – A task or assignment that involves creating something, often used in technology to describe a program or application you are working on. – My science project this year is to build a simple weather app.
Costume – A different appearance or outfit for a sprite in a game or animation, often used in Scratch to change how a character looks. – I gave my sprite a superhero costume for the new level of my game.
