2y ago

When do you think there will be an Android app that could accurately perform real time object recognition?

In Star Wars Rebels, there was an E-XD-series infiltrator droid that could quickly take inventory of everything in a Rebel warehouse. With the advanced object recognition capabilities of modern AI, it’s only a matter of time before an app for Android can accurately and rapidly identify and store objects in real-time from video capture. This could be similar to a home inventory app where users only need to capture video and move around the house instead of taking pictures and labeling items. When do you think such an app will become available? Also, what is the closest app available right now?

edit: I didn't say offline or on-device, I don't know why everyone assumes that. I mean a service offered through an Android app.

23 comments

Modern AI, as you're seeing it today, is processed by massive data centers online with thousands of processing units running in parallel, not by your local device. Your device would be way too slow to expect any sort of realtime object recognition, at least with the current state of technology.
TL;DR - I don't think it'll happen anytime soon, at least not on your local device. It would take a super fast and steady connection to the AI service.
- Don’t underestimate the potential for optimization when you can constrain the problem to a narrow range of uses. Model pruning and custom silicon go far. Voice assistants used to be purely cloud compute, but a lot of common use cases are done on device now.
  
  Yes, I've been testing FUTO Voice Recognition lately. It's awesome as hell, but it is far from realtime. And this ain't even object recognition, it's only voice recognition.
  https://voiceinput.futo.org/
  https://play.google.com/store/apps/details?id=org.futo.voiceinput
- Honestly, I expect some form of it in the next five years. Tech can move fast when it wants to and there’s 💵 involved.
  
  Is Moore's Law Finally Dead?
- We're not talking about image rendering, we're talking about image recognition. Although they may seem related, they are not.
  It's one thing to sling a 3D model and textures to a GPU, but it's totally a different thing to take a photo and sling it against a humongous AI model being run at a datacenter with billions of images to compare it to.
There is an app called Object Detector which does this. It's not particularly accurate and can't recognize a lot of objects though. It does run on phones in realtime though.
An isolated phone, not for a while. A phone with a dedicated 5g connection would be pretty close.
Frigate does this on a raspberry pi or Intel NUC already. Would be power hungry in a phone but if you are not training ML models and just looking for objects they already know, the tech would be ready today.
EDIT: Here’s Google’s article on how to create your own TensorFlow app for Android https://developers.google.com/ml-kit/vision/object-detection/custom-models/android
Don’t know about an Android app but YOLOv8 Detect and similar models can detect objects in videos and classify them.
Google Goggles used to be able to do that
- Google Lens still can.
GPT 4 with image uploads gets pretty damn close, though it's not real-time and processed server side
Think about what Apple currently has: dedicated ML processing chip with multiple cores, and yet the on-device object recognition is still an “overnight while plugged in” process for a single image, and only detects a limited number of object types.
Real-time mobile offline OR is still the mythical “at least ten years out.” It needs improvements in processors, sample sets, training data and algorithms to get to real-time.
Doesn’t google lens basically do this already?
- Yes, but OP is referring to realtime object recognition. Although we don't have to wait very long for object recognition right now, we still have to wait a bit. That's not quite realtime.
Not likely in the near future, probably not feasible in the long term either. It's not just about recognising an object. You could write a program that recognises a screw but you'd need far more complicated sensors and algorithms to identify the dimensions, specific characteristics, material composition, design specifications, etc, then apply that to every screw, bolt, washer, small component and assembly, tubes, threaded rods, tyres, pistons, brake pads, resistors, capacitors, diodes, seals, consumables, etc.
For a long time, I think that kind of thing would be wildly inaccurate, hugely expensive, massively complicated, and much less efficient than asking a human to kindly go over there and check all those things manually.

23 comments