2mo ago

Voice control is dangerous

This morning my kid asked the voice assistant to "Turn off the computers in this house".

I heard it, thought well that's a strange request but seems harmless because how is home assistant gonna turn off computers.

Me a little while later, "why is shit broken? What's happening!"

Turns out dumb me had adguard exposed to the voice assistant, it switched off all the adguard settings including the DNS rewriting that is the cornerstone of many of my self-hosted services.

I've since revoked that access.

32 comments

A while back, I saw a story in the Home Assistant Facebook group about someone's child saying "Hey Google, turn on everything" and it messing things up. I was telling the story to my wife and forgot to replace "Hey Google" with something Google wouldn't pick up on. Oops. It heard my "turn on everything" and chaos ensued. I have some Zigbee alarms that all started sounding. It enabled several different scenes and ran several scripts. All TVs turned on. My Xbox and Nvidia Shield were fighting for control of the TV (there's some issue with HDMI-CEC that I haven't figured out where if both are on, they get stuck in a loop changing the TV input between HDMI2 and HDMI3 about once per second).
Don't do that. "Turn off everything" is bad too. I ~~have~~ used to have my server rack plugged into a smart plug to measure power usage, and "turn off everything" turns that off. I want to figure out how to disable these two voice commands.
- Instead, use a UPS that has telemetry that you can read back. Bonus: it's a ups
  
  I'm using a UPS now so I'll get rid of the smart plug. I've been using it for three years and haven't had issues with it cycling though.
  
  Or syncing with ohmhour, forgetting, removing physical devices and smart strip, later reusing smart strip for aforementioned server, losing your mind every few days when you notice it went down for no reason but no idea why bc nothing in the software you're running is instructing it to do so.
- Haha it's a fine balance between preventing this sort of craziness and having a voice assistant that is actually useful because it can do things.
  I want to figure out how to disable these two voice commands, and how to completely disable turning off the smart plug my server rack uses.
  In the settings->voice assistants page you can see exposed entities. Review them and remove anything you don't want it to be able to control.
  You can also set up custom commands as an automation. You could probably set up an automation trigger to pick up on the phrases you want to block and respond with "I'm sorry Dave, I'm afraid I can't do that" (or something boring).
- In home assistant every entity has "assist" config, you can remove any entity from assist which in turn will prevent it from being included in "all" assist/voice target.
  So you can have a plug for server without worry but need to remove it from assist.
- Open the Google Home app, go to " Automations", and make one for the household for when someone says "turn everything on" and any other variations you want, then just make it respond with something instead of actually doing the thing.
  
  This sounds great until you realize the number of invocations that mean the same thing:
  everything on
  turn on everything
  turn everything on
  switch on everything
  switch everything on
  turn on all devices
  There’s more I’m sure, but you get the point.
  
  Thanks for the explanation!
- When the voice assistant came out I remember one of my coworkers showing it off to another.
  To which the other coworker just said "hey Google, text Mom saying 'you're fat'".
  Because of that I've never had any voice tools connected to anything I own.
I half expected this post ending with HA turning itself off
- I'm surprised it didn't turn off the voice assistant. It does have the ability to mute itself (for when the kids are using it when they are supposed to be doing something else or are just generally annoying).
It was DNS.
- Is it DNS?
  
  I spent a week trying to work out why my traefik instance wasn't forwarding http & Https traffic to any of my docker containers. By the fifth day I remembered that the week before I'd decided it would be a good idea to test locally before uploading to my vps, by adding my site URLs to my hosts file. (All except traefik: which meant I was able to access the dashboard and see all my routers online... I just couldn't reach any of them, with no logs for access etc... drove me insane.).
  It was only until I tried accessing from my mobile that I realised what I'd done.
  Should've known better, but that's the price we pay for "good ideas" at 2am.
  
  I heard a legend that there was a time where it wasn't true.
  But then humans come up with all kinds of weird shit.
  
  Apt
I’ve since revoked that access.
A few releases ago they made it so you can DEFAULT new entities NOT to be controlled by the VA.. I cleared ALL entities and changed the setting. I only add things one at a time to voice control now.
SO many integrations ad switches it is a high risk to default the VA to have access to all new ones.
- I think that setting has been there as long as I've had a voice assistant (since I got my HA Voice Preview in January). I knew it existed but I let it expose them. I probably shouldn't, because I always have to review it after adding anything new since it normally doesn't get what I wanted exposed/not exposed.
That seems like a major bug if it thought DNS settings were a computer.
- It's an LLM that has access to run commands. It's a major bug by design 😅. But it does do a decent job if I keep tweaking after thing kind of thing happens.
  Without the LLM you have to phrase things very specifically, or it will say it doesn't understand. With the LLM the kids can do things like ask for "the song that goes [lyrics here]" and it can play it. It's a very cool thing to play with, e.g. "can you tell me what the weather will be like today, phrased as a haiku", but it's full of traps as well. I have a "Home Assistant Voice Preview", the "Preview" bit is to make it clear this is not ready for the general public yet 🙂
  P.S. if you're wondering, the weather today:
  Showers grace the sky,
  Rain will fall, then clear away,
  Gentle winds will sigh.
just be thankfull that you don't host lemmy.nz at home @Dave@lemmy.nz
- Haha I guess I should be grateful for that, though I do host another dozen or more services at home so a lot of things broke!
Might be a good idea to expose things through a reverse proxy if this is web interfaces. I have *.local.domain.nexus all pointed to my reverse proxy's internal ip address and the reverse proxy makes everything available through https.
- Everything is exposed through a reverse proxy. E.g. homeassistant.mydomain.nz
  However, I have DNS rewriting set in Adguard that does *.mydomain.nz -> 192.168.1.XX
  This means a) things don't need to go external if I'm at home, and b) I have many things only accessible internally, which rely on this otherwise they won't work at all.
  It's all HTTPS, I just use a cloudflare integration in Traefik to do the Let's Encrypt validation for domains not accessible externally.

32 comments