On system wake phrase on ESP32-S3 is right here – Voice: Chapter 6

TL;DR: We’ve got added on-device wake phrase detection (microWakeWord)! It’s sooner and extra scalable than processing the wake phrase in House Assistant. We’ll hold supporting wake phrase processing in House Assistant. Additionally new is extra customization for sentence triggers, extra intents for controlling extra gadgets, and higher error messages and debugging instruments.

Watch the complete Voice chapter 6 livestream

2023’s Yr of the Voice constructed a stable basis for letting customers management House Assistant by talking in their very own language.

We proceed with enhancements to Help, together with:

Oh, and “yet another factor”: on-device, open supply wake phrase detection in ESPHome! 🥳🥳🥳

Try this video of the brand new microWakeWord system working on an ESP32-S3-BOX-3 alongside one doing wake phrase detection inside House Assistant:

On-device vs. streaming wake phrase


Thanks to the incredible microWakeWord created by Kevin Ahrendt, ESPHome can now carry out wake phrase detection on gadgets just like the ESP32-S3-BOX-3.
You’ll be able to set up it in your S3-BOX-3 at this time to attempt it out.

Again in Chapter 4, we added wake phrase detection utilizing openWakeWord. Sadly, openWakeWord was too giant to run on low energy gadgets like S3-BOX-3.
So we selected to run wake phrase detection inside House Assistant as an alternative.

Doing wake phrase detection in HA permits tiny gadgets just like the M5 ATOM Echo Improvement Package to easily stream audio and let all the processing occur elsewhere. That is nice, because it permits low-powered gadgets utilizing a easy ESP32 chip to be remodeled right into a voice assistant even when they don’t pack the mandatory energy to detect wake phrases.
The draw back is that including extra voice assistants requires extra CPU utilization in HA in addition to extra community site visitors.

Enter microWakeWord. After listening to an interview with Paulus Schoutsen (founding father of House Assistant) on the Self Hosted podcast, Kevin Ahrendt created a mannequin primarily based on Google’s Inception neural network. As an present contributor to ESPHome, Kevin was capable of get this new mannequin working on the ESP32-S3 chip contained in the S3-BOX-3! (It additionally works on the, now discontinued, S3-BOX and S3-BOX-Lite)

Kevin has educated three models for the launch of microWakeWord:

  • “okay nabu”
  • “hey jarvis”
  • “alexa”

You’ll be able to attempt these out your self now by following the ESP32-S3-BOX tutorial. Altering the default “okay nabu” wake phrase would require adjusting your ESPHome configuration and recompiling the firmware, which can take a very long time and requires a machine with greater than 2GB of RAM.

We’re grateful to Kevin for growing microWakeWord, and making it part of the open residence!

Sentence set off responses

Including customized sentences to Help is as simple as including a sentence trigger to an automation. This allows you to trigger any action in Home Assistant with whatever sentences you want.

Now with the new conversation response action in HA 2024.2, you can also customize the response spoken or printed back to you. Using templating, your response can refer to the current state of your home.

You can also refer to wildcards in your sentence trigger. For example, the sentence trigger:

could have the response:

Playing  trigger.slots.album  by  trigger.slots.artist 

in addition to calling a media service.

You can experiment now with sentence triggers, and custom conversation responses in our automation editor by clicking here:
Open your Home Assistant instance and show your automations.

Improved errors and debugging

Assist users know the phrase “Sorry, I couldn’t understand that” all too well. This generic error message was given for a variety of reasons, such as:

Beginning in HA 2024.2, Help supplies totally different error messages for every of those instances.

Screenshot showing the new errors Assist will return in case the intention is understood, but something else is missing.

Now when you encounter errors, you’ll know the place to begin wanting! The very first thing to verify is that your system is uncovered to Help. Some kinds of gadgets, equivalent to lights, are uncovered by default. Different, like locks, usually are not and should be manually uncovered.

As soon as your gadgets are uncovered, be sure to’ve added an applicable alias so Help will know precisely the way you’ll be referring to them. Units and areas can have a number of aliases, even in a number of languages, so everybody’s desire will be accommodated.

In case you are nonetheless having issues, the Help debug device has additionally been improved. Utilizing the device, you see how Help is deciphering a sentence, together with any lacking items.

Open your Home Assistant instance and show your Assist developer tools.

Our neighborhood language leaders are exhausting at work translating sentences for Help. If in case you have recommendations for brand new sentences to be added, please create a problem on the intents repository or drop us a line at [email protected]

Thank you

Thank you to the Home Assistant community for subscribing to Home Assistant Cloud to assist voice and growth of House Assistant, ESPHome and different initiatives generally.

Because of our language leaders for extending the sentence assist to all the assorted languages.

Thank you for supporting the Home Assistant project