Image Model Guide

Before you start

Open Blocks in a supported browser. To make Marty respond, select Marty in the Devices panel and connect over Bluetooth. You can collect examples and train a model before connecting a robot.

This guide uses your computer or tablet's camera. Allow camera access when prompted. Select the target you want to use before creating the model: saved models belong to that target.

Create and name an image model

Open the Machine Learning tab, select Add New Model (the + button at the bottom left), then choose New Image Model (Device). “Device” means the camera on the computer or tablet running Blocks. If an unsaved-changes warning appears, continue only after saving any model you want to keep.

Enter a clear name in Model name, such as “Image example”. The editor guides you through Collect examples, Train, and Test & save.

Add classes and record examples

A class is something you want the model to recognise, such as a red object or a blue object. Type its name in Add class, then select + Add class. Create at least two classes. Use descriptive names in your own project; “Class 1” and “Class 2” in the screenshots are example names.

Select a class and show its object to the camera. Press Record examples for [class name], or Record Data on its card. Each recording captures several images and stops automatically. Repeat for every class. You can delete unwanted examples before saving the model.

Aim for at least 10 varied images per class. Change the object's angle, distance and position, and include the lighting and backgrounds you expect to use. Keep the number of examples reasonably balanced. Consider a separate class for an empty scene or “none of these” so the model has an alternative when none of your objects is visible.

Train the model

When every class has examples and the editor says Ready to train, select Train model. Wait for training to finish. The Training loss graph shows how the model fits the training examples; a falling line alone does not prove that it will recognise new images correctly.

Test with new images

Select Test model. Show the camera each object again and watch Live predictions, which shows the best match and the confidence for every class. Test positions, angles and backgrounds that were not used during collection.

Select Stop testing when finished. If the predictions are unreliable, add or replace examples and train again before saving. Use Test again to repeat the test.

Save the model and back up the project

Select Save model when you are happy with the results. The model appears in the Models list and becomes available in Code.

A saved model cannot be given more training examples or retrained. Create a new model if you need to improve it later.

To keep a portable backup of your code and saved model, open Save/Load and select Save to File under Portable File Storage. Keep the downloaded .sb3 file. Use Choose File in the same section to reopen it later. Saving the model and saving the whole project are separate steps.

Load the model in Code

Select Code, then the Machine Learning blocks category. Add Load Machine Learning model [Image example] to your program and choose your saved image model from its menu. If the model is missing, select the target on which you created it and check that you saved it.

Run this block to load the model. Image classification starts automatically, using your device's camera. Use turn classification [off] to pause predictions, and turn classification [on] to resume them. classify image requests an image classification when needed in a script.

Use predictions in your program

The screenshot shows separate example blocks. To try the response without moving Marty, create a variable named prediction using Variables → Make a Variable. The example event and condition set this variable to “Class 1”.

After loading the model, its class names are available in the image-label menus.

• Event: put your response beneath when received image label: [class].
• Condition: use image [class] detected in an if block, inside a loop if you want to check repeatedly.
• Reporter: use image label for the recognised class name, or confidence of image [class] for its confidence. You can tick the checkbox beside image label in the Machine Learning category to display it on the stage.

Choose your own class in place of “Class 1” in the example. Begin with a response that does not move the robot, then add a Marty action once you have checked the predictions. Put Marty on a clear, stable surface before running movement commands.

Show the camera on the stage (optional)

Use turn video [on] to display your device's camera feed on the stage. Select the large-stage control beside the green flag and stop button if you need more room. Use turn video [off] to hide the feed again.

The stage preview is optional; you do not need to show it to use the model's predictions.

Adjust the classification interval and confidence

Label once every [1] seconds changes how often the model checks an image. The default interval is one second.

set confidence threshold [0.5] changes the minimum confidence needed for a recognised label. The default is 0.5 (50%). A higher threshold can reduce unwanted responses but may miss some objects. If no prediction meets the threshold, image label can be empty.

Use turn classification [off] when your program no longer needs predictions. Save the project again after changing your code.