61% of customers report no ROI or disappointing ROI on their AI investments.
61% of customers report no ROI or disappointing ROI on their AI investments.
garymarcus.substack.com
Poor ROI for GenAI
15 comments
-
-
Good.
-
I guess I’m in the minority then, and that’s fine by me.
- AI generated meeting notes make it easy to produce summaries and action items for all parties, including those who couldn’t make the meeting
- I can give Copilot a few sentences recounting a story and what key points I want to convey in an essay and it writes most of it for me, often in a more professional sounding tone than I would have written. I spend 5 minutes in the front and 5 minutes on the back and it cuts down the remaining 30 minutes to hour it would have taken.
- I’ve used it to find and replace portions of code and XML. I’ve used it to redact PII in user stories. I’ve used it to assist me in making SOWs and help content more clear and robust.
- Education is using it to help identify college students who need more support prior to dropping out.
- Biologists recently used it to help determine the folding patterns of nearly every protein that does or could exist, which will likely lead to very precise medications that combat cancers and illnesses.
- Per a recent Malcolm Gladwell podcast I heard, fire chiefs can now use AI to listen to multiple radio channels at the same time in a large fire fighting scenario and have it identify firefighters who are in the most distress, therefore mitigating risk and saving lives.
AI is buzzy now, and buzz deserves a healthy bit of scrutiny. As we move down the hype curve, I think we’ll achieve a certain threshold that’s going to vastly improve the human experience and perhaps improve how we live with rest of this planet too.
-
I’ve used it to redact PII in user stories.
This seems like an inappropriate use case that could have legal repercussions :/
-
Not a public facing AI :)
-
-
Maybe ask AI what ROI means
-
AI generated meeting notes make it easy to produce summaries and action items for all parties, including those who couldn’t make the meeting
I'm only familiar with the Zoom version of this, but every time I've seen it used, it made so many mistakes that I would never trust it.
Maybe it's okay in certain scenarios, but it's like having someone taking notes that has no understanding of the context (our project, industry, etc).
Edit: I should emphasise the worst part. A human with no context would write "(something technical about GPUs that I don't understand)", whereas the AI confidently makes up some bullshit.
-
I've not used zoom but the copilot (teams premium) version of this isn't much better.
I've had it imagine extra points of discussion that never happened. I've had it add names of people to the summary that not only weren't in the meeting but aren't known to any of the participants.
At first look it is very impressive and we were able to convince the business to purchase the licences. But now that we try to rely on it, we find we have to fully scrutinize the output. Sometimes having to relisten to the recorded meeting to check accuracy.
I've resorted to making my own notes and then comparing with the AI summary. But then I ask myself what's the point?
-
-
There's an AI meeting summary I get every week, I've never clicked on it but I did yesterday. The customer owes me $89. I'm not sure why, but the AI was very clear I am owed this debt, so I'm considering sending an invoice
On the other hand, integrating AI with conventional systems is a specialty of mine. Or can be done, it can be done clearly, and it's incredibly powerful
-
Reminds me of the 90s dot com bust. Yes, it's the next big thing, really big thing. Every venture capitalist is throwing money at the wall to see what sticks, they would be stupid not to. Guarantee some good shit is going to stick, but the vast majority is going to fall.
You nailed some great bullets.
- Used Zoom AI a year ago to summarize meetings and was astonished at how well it did.
- Yep, used it to get a clue coding PowerShell. It's not a replacement for programmers, but it's a great way to get over a hump or get inspiration for other pathways. Same goes for writing anything. I wrote a horror short, be interesting to see what AI does with the concept, might give me ideas.
- Biology seems to been booming with uses, seeing a new story every day.
The lemmy hate is silly, like everyone is trying to be so much smarter than the suckers falling for AI. Again, reminds me of all the stupid shit that came out in the late 90s, but what stuck is the internet today.
-
I think the biggest problem with AI is that people expect it to fully do the work for you rather than be a tool. Imagine we live in a world without cameras and someone introduces that as something that will make paintings for you. Then users dislike it, expecting cameras to frame, aim, zoom and shoot for them.
I use AI for coding and it's amazing... at giving you an 80% correct boilerplate code that you then finish up editing yourself. There's real time savings there. I don't ask it to make the whole code because then I'm going to have to find the mistakes.
I use it also to summarise 3000-commit changelogs, which after some refinement it does way better than I could do in any reasonable timespan.
A colleague with dyslexia now writes without worrying that his grammar isn't making much sense, then an LLM fixes it for him.
The problem is when you use the result of the AI as a final product, because the reality of the technology is that then you get slop. There are so many people that just can't see past this and either use AI directly as an unattended slop generator, or don't use AI because they don't think it can be anything else. But I'm convinced you can use it as a tool with an input in less than 20% of a creative process (by this I don't mean "art" but any type of creation) and still save a real and significant chunk of time.
-
It's just going to make the average person lazier than they already are.
-
Excellent.
15 comments
Because it's all a bubble.
The fun part is that all this market disruption is before the AI bubble pops. Take a look at what’s been driving up indices the last 2 years (overweight in tech stocks as they are). A fair bit of real estate is hoping to become AI datacenters. A lot of promised investment in building out electrical grid for the AI.