Yr of the Voice – Chapter 1: Help

This 12 months is House Assistant’s 12 months of the voice. It’s our objective for 2023 to let customers management House Assistant in their very own language. Right now, one month into 2023, we begin our first chapter.

At House Assistant we imagine that expertise is supposed to be performed with, and tasks needs to be usable as quickly as attainable. Along with the group we are able to then iterate and refine. That’s why immediately, we’re delivering a fundamental expertise for 22 languages to work together with House Assistant. Oh, and we’re additionally releasing some enjoyable stuff that we cooked up alongside the best way.

To observe the video presentation of this weblog submit, together with dwell demos, test the recording of our live stream.

Intentions

The core of a voice assistant is to be able to understand the intention of a spoken sentence. What is it the user wants to do? To extract these intentions we created our own template sentence matching format and intent recognizer named Hassil.

This new format is utilized by our new Home Assistant Intents undertaking. The objective of this undertaking is to gather house automation sentences in each attainable language. Because it’s begin a month in the past, we have now had 112 individuals contribute. The undertaking now helps 22 languages and 14 more are in progress.

Assist

We have added a new feature to Home Assistant: Assist icon Assist. It allow users to use natural language to control Home Assistant. It is powered by Hassil and the sentences from the Home Assistant Intent project.

We want Assist to be as accessible to as many people as possible. To do this, we made it work without requiring extra hardware – just update to Home Assistant 2023.2 and you can start! Through a combination of smart algorithms combined with sheer brute force (we are collecting a lot of sentences), we have been able to make a system that works for most common sentences. Support for more powerful, AI-powered, intent recognizers might come in the future as an opt-in feature.

Assist is enabled by default in the Home Assistant 2023.2 release. Tap the new Assist icon Assist icon at the top right of the dashboard to use it.

Assist documentation.

Screenshot of the Assist dialog

Assist on Android Wear

We want to make it as easy as possible to use Assist. To enable this for Android users, we have added a new tile to the Android Wear app. A simple swipe from the clock face will show the assist button and allows you to send voice commands.

Assist on Android Wear documentation.

The brand new tile is at the moment accessible within the Android beta and can be a part of the subsequent Android launch.

Assist via Siri and Apple Shortcuts

For Apple devices we have been able to create a fully hands-free experience by integrating with Siri. This is powered by a new Apple Shortcut action called Assist, which is part of the Home Assistant app. This shortcut action can also be manually triggered from your Mac taskbar, iPhone home screen or Apple Watch complication. We have two ready-made shortcuts that users can import from the documentation with a single tap to unlock these features.

Assist via Siri and Apple Shortcuts documentation.

The Help shortcut can be accessible within the Mac and iOS beta channel immediately and can be a part of the subsequent launch for iOS and Mac.

Customized Sentences

With House Assistant we imagine that each house is uniquely yours and that technology should adapt to you, not the other way around. That’s why we have architected Home Assistant to allow users to extensively customize their experience. Our Assist feature is no different.

  • Are you into Game of Thrones and want every response to be “hodor”?
  • Want to turn on lights in rooms by saying “Hocus pocus living room”?
  • Want to trigger your party mode script using a custom sentence?

Assist includes support for custom sentences, responses and intents, allowing you to achieve all of the above, and more. We’ve designed the custom sentence format in a way that it can be easily shared with the community.

Read the documentation on how you can get began.

In a future launch we’re planning on including a person interface to customise and import sentences.

Custom Assist engines

By default Assist is powered by our own intent recognizer. It is local but it’s limited to controlling devices. Maybe you want to be able to ask more wide-range queries or you are looking for a conversational AI that will make up responses and present it as the truth. For such cases the Assist feature supports swapping out its engine that handles all Help interactions.

The House Assistant 2023.2 launch contains two various Help engines that you would be able to allow: Google Assistant and OpenAI GPT-3.

The Google Assistant Help engine is ready to management your units when you’ve got linked up your House Assistant occasion to Google Assistant.

All methods to intereact with Help will work, as they don’t seem to be sure to the the Help engine that’s getting used. So in case you ever needed to make use of Google Assistant in your HomePod, now you’ll be able to 🤭

The OpenAI GPT-3 Help engine will course of all of your interactions utilizing GPT-3, a sibling of the notorious ChatGPT. It isn’t capable of management your own home or show you how to automate your own home. Something you ask it could or might not be factually appropriate. However it may be enjoyable!

In a future launch we’re planning to make it attainable to configure a number of Help engines to deal with interactions.

What’s next

For Year of the Voice – Chapter 1 we focused on building intent recognition into Home Assistant while relying on Google and Apple to do the hard parts (speech recognition). This allowed us the fastest path to get something to the community to play with.

We will continue collecting home automation sentences for all languages (anyone can help!). Updates can be included with each main launch of House Assistant.

Our subsequent step is integrating Speech-to-Textual content and Textual content-to-Speech with Help. We don’t have a timeline but when that can be prepared. Keep tuned!

Credits

A lot of people have worked very hard to make all of the above possible.

Technology:
Mike Hansen, Paulus Schoutsen, Daniel Shokouhi, Zac West, Rosemary Orchard, Tronikos

Language Leaders:
@AalianKhan, @Ahmed-farag36, @alpdmrel, @arunshekher, @auanasgheps, @benjaminlecouteux, @bluefoxlee, @cibernox, @cvladan, @davefx, @dinhchinh82, @dsimop, @duhow, @easterapps, @ErnestStaug, @fadamsen, @flexy2dd, @gabimarchidan, @haim-b, @halecivo, @HepoH3, @hertzg, @hristo-atanasov, @huusissa, @joaorgoncalves, @larsdunemark, @leranp, @LubosKadasi, @makstech, @mojikosu, @MTrab, @nagyrobi, @schizza, @Scorpoon, @skynetua, @spuljko, @tetele, @TheFes, @Uriziel01, @xraver, @zubir2k

Voice Neighborhood:
@Alexivia, @Atalonica, @AwesomeGuy000, @BossNeo, @CedricFinance, @Davidsoff, @EmilZackrisson, @FragMenthor, @InfiniteBed, @Kalma-House, @Licmeth, @Marlo461, @N3rdix, @Nismonx, @Robin-St, @TaQuangTien, @ThomDietrich, @TomaszPilch, @Wojciechgc, @alessandroias, @bemble, @berendhaan, @dejan2101, @dependabot[@bot], @dobromir-hristov, @frenck, @hugovsky, @iddiek, @jfisbein, @jharrvis, @jorclaret, @kamildoleglo, @kblin, @khymmera, @kroimon, @lellky, @ludeeus, @lukahra, @lunmay, @mardito, @martindybal, @mib1185, @michaelmior, @orrc, @pckahrs, @piitaya, @pmentis, @poltalashka, @rPonuganti, @rechin304, @relust, @rickydg, @rpochot, @rrakso, @rumbu13, @sanyatuning, @tasmin, @thecode, @waltlillyman, @witold-gren, @x15pa3ck15x, @yuvalabou