Before you start

Open Blocks in a supported browser. To make Marty respond, select Marty in the Devices panel and connect over Bluetooth. You can collect examples and train a model before connecting a robot.

The model uses your computer or tablet's microphone. Allow microphone access when prompted. Select the target you want to use before creating the model: saved models belong to that target.

Create and name an audio model

Open the Machine Learning tab, select Add New Model (the + button at the bottom left), then choose New Audio Model. If an unsaved-changes warning appears, continue only after saving any model you want to keep.

Enter a clear name in Model name, such as “Audio example”. The editor guides you through Collect examples, Train, and Test & save.

Add classes and record examples

A class is a sound you want the model to recognise, such as a clap or a whistle. Type its name in Add class, then select + Add class. Use descriptive names in your own project; “Class 1” in the screenshots is an example name.

Select a class, then press Record examples for [class name], or Record Data on its card. Each recording captures a short sound sample and stops automatically. Repeat to collect several examples of that sound. You can play back or delete individual examples before saving the model.

The editor creates a background noise class, displayed as _background_noise_. Record at least 10 examples for this class with your usual room noise and without deliberately making the sounds you want to recognise. This class cannot be renamed or removed.

You need at least two classes, including background noise, and examples in every class before training. Aim for at least 10 varied examples per sound class too. Record the same sound at different distances and volumes, and keep the number of examples reasonably balanced.

Train the model

When the editor says Ready to train, select Train model. Wait for training to finish. The Training loss graph shows how the model fits the training examples; a falling line alone does not prove that it will recognise new sounds correctly.

Test with new sounds

Select Test model. Make each sound again and watch Live predictions, which shows the best match and the confidence for every class. Also try ordinary background noise. Test with sounds you did not use during collection.

Select Stop testing when finished. If the predictions are unreliable, add or replace examples and train again before saving. Use Test again to repeat the test.

Save the model and back up the project

Select Save model when you are happy with the results. The model appears in the Models list and becomes available in Code.

A saved model cannot be given more training examples or retrained. Create a new model if you need to improve it later.

To keep a portable backup of your code and saved model, open Save/Load and select Save to File under Portable File Storage. Keep the downloaded .sb3 file. Use Choose File in the same section to reopen it later. Saving the model and saving the whole project are separate steps.

Load the model in Code

Select Code, then the Machine Learning blocks category. Add Load Machine Learning model [Audio example] to your program and choose your saved audio model from its menu. If the model is missing, select the target on which you created it and check that you saved it.

Run this block to load the model. Sound classification starts automatically, using your device's microphone. Use turn classification [off] to pause predictions, and turn classification [on] to resume them.

Use predictions in your program

The screenshot shows separate example blocks. To try the response without moving Marty, create a variable named prediction using Variables → Make a Variable. The example event and condition set this variable to “Class 1”.

After loading the model, your sound class names are available in the sound-label menus. The background-noise class is excluded from those menus.

• Event: put your response beneath when received sound label: [class].
• Condition: use sound [class] detected in an if block, inside a loop if you want to check repeatedly.
• Reporter: use sound label for the recognised class name, or confidence of sound [class] for its confidence. You can tick the checkbox beside sound label in the Machine Learning category to display it on the stage.

Choose your own class in place of “Class 1” in the example. Begin with a response that does not move the robot, then add a Marty action once you have checked the predictions. Put Marty on a clear, stable surface before running movement commands.

Adjust the classification interval and confidence

Label once every [1] seconds changes how often the model checks for a sound. The default interval is one second.

set confidence threshold [0.5] changes the minimum confidence needed for a recognised label. The default is 0.5 (50%). A higher threshold can reduce unwanted responses but may miss some sounds. If no prediction meets the threshold, sound label can be empty.

Use turn classification [off] when your program no longer needs predictions. Save the project again after changing your code.