What do you mean by "it"? The chatGPT interface? Could be, but then you are also missing the point I am making.

After all, chatGPT is just one of the possible implementations of LLMs and indeed not perfect in how they implemented some things like search. In fact, I do think that they shot themselves in the foot by implementing search through bing and implementing it poorly. It basically is nothing more than a proof of concept tech demo.

That doesn't mean that LLM's are useless for tasks like searching, it just means that you need to properly implement the functionality to make it possible. It certainly is possible to implement search functionality around LLMs that is both capable and can be reviewed by a human user to make sure it is not fucking up.

Let me demonstrate. I am doing some steps that you would normally automate with conventional code:

I started about by asking chatGPT a simple question.

*"I want to provide a list of all the children bach had. What would be a suitable google query"

It then responded with.

"To compile a comprehensive list of Johann Sebastian Bach's children, a suitable Google query would be: "List of Johann Sebastian Bach's children names". This query is specific enough to indicate that you're looking for a detailed list, including the names of all the children, which should yield targeted results."

The following step I did manually, but is something you would normally have automated. I put the suggested query in google, I quickly grabbed the first 5 links and then put the following in chatGPT.

The top 5 results in google are the pages I provided below. Browse to each page, and answer the query "List of Johann Sebastian Bach's children names". Just answer the query, but let me know if there is conflicting information on the page.

https://en.wikipedia.org/wiki/Bach_family
https://www.classicfm.com/composers/bach/guides/children/
https://www.bachonbach.com/100-bach-faq-and-soon-there-are-500-bach-faq/faq-26-what-were-the-names-of-all-of-johann-sebastian-bach-s-kids/
https://www.spiritsound.com/music/bachchildren.html
https://www.grahamsmusic.net/post/bach-s-wives-and-children

It then proceeded to give me the following answer

Johann Sebastian Bach had 20 children in total. With his first wife, Maria Barbara Bach, he had seven children, four of whom survived into adulthood: Catharina Dorothea, Wilhelm Friedemann, Carl Philipp Emanuel, and Johann Gottfried Bernhard. With his second wife, Anna Magdalena Wilcke, he had 13 children, of whom six survived into adulthood: Gottfried Heinrich, Elisabeth Juliane Friederica, Johanna Carolina, Regina Susanna, Johann Christoph Friedrich, and Johann Christian

Going over the search results myself seems to confirm this list. Most importantly, except for the initial input, all of this can be automated. And of course, a lot of it can be done better, as I didn't want to spend too much time.

I live in Spain

From what I know of Spain's healthcare system is that it is generally quite good for traditional healthcare. Reading up on it, it does seem that mental healthcare is not as good in the sense there are very few therapists and such. But does that also mean that it isn't covered by your insurance at all?

For real, it almost felt like an LLM written article the way it basically said nothing. Also, the way it puts everything in bullet points is just jarring to read.

True, though that isn't all that different from people doing knee jerk responses on the internet...

I am not claiming they are perfect, but for the steps I described a human aware of the limitations is perfectly able to validate the outcome. While still having saved a bunch of time and effort on doing an initial search pass.

All I am saying is that it is fine to be critical of LLM and AI claims in general as there is a lot of hype going on. But some people seem to lean towards the "they just suck, period" extreme end of the spectrum. Which is no longer being critical but just being a reverse fanboy/girl/person.

I don't know how to say this in a less direct way. If this is your take then you probably should look to get slightly more informed about what LLMs can do. Specifically, what they can do if you combine them with with some code to fill the gaps.

Things LLMs can do quite well:

• Generate useful search queries.
• Dig through provided text to determine what it contains.
• Summarize text.

These are all the building blocks for searching on the internet. If you are talking about local documents and such retrieval augmented generation (RAG) can be pretty damn useful.

You are glossing over a lot of infrastructure and development, when boiled down to the basics you are right. So it is basically a question of getting enough users to have that app installed. Which is not impossible given that we do have initiatives like OpenStreetMap.

It depends on the platform you are using. But, for platforms like github and gitlab there are extensions for popular IDEs and editors available that allow you to review all changes in the editor itself.

This at the very least allows you to simply do the diffing in your own editor without having to squash or anything like that.

At least for the instance this was posted on: the February 2024 Beehaw Financial Update

Well, the problem is you don’t know what you don’t know.

This is true, even recognized in the paper. People that spend more time on writing prompts (probably knowing that this is important) actually did manage to do reasonably well. Which is exactly what I in the previous reply was hinting at.

Because, let's be honest, this statement is true for everything where someone starts out new. In the past (and probably still) you had people blindly copying code blocks from stackoverflow not understanding what the code or realizing how outdated the answer might be.

So the key is still education of people and making them aware of their limitations. You can try to block the usage of tools like this, some companies actively do so. But people will be people and as long as the tools are available they will try to use them. So the more realistic approach, in my opinion, is to educate them in the usage of these tools.

For LLM training I do wonder if they assigned a weight, but I doubt it.

Given my experience with models I think they might actually do assign a weight. Otherwise, I would get a lot more bogus results. It also isn't as if it is that difficult to implement some basic, naive, weighing based on the amount of stars/forks/etc.

Of course it might differ per model and how they are trained.

Having said that, I wouldn't trust the output from an LLM to write secure code either. For me it is a very valuable tool on the end of helping me debug issues on the scale of being a slightly more intelligent rubber ducky. But when you ask most models to create anything more than basic functions/methods you damn well make sure it actually does what it needs it to do.

I suppose there is some role there for seniors to train juniors in how to properly use this new set of tooling. In the end it is very similar to having to deal with people who copy paste answers directly from stack overflow expecting it to magically fix their problem as well.

The fact that you not only need your code/tool to work but also understand why and how it works is also something I am constantly trying to teach to juniors at my place. What I often end up asking them is something along the lines of "Do you want to have learned a trick that might be obsolete in a few years? Or do you want to have mastered a set of skills and understanding which allows you to tackle new challenges when they arrive?".

If everything you have read is saying that it is fine, then why does it not feel right for you? Looking around I do get the same impression, it is non-combustible so there is not really a concern there. Basically from what I gather as long as you use the proper wire for use in walls/isolation, leave enough space and generally take good practices in account like using conduit where needed you should be good to go.

I am not an electrician though and certainly not aware of your local code and regulations.

Talking about electricians, if you are worried about doing it not right, why not hire one to do it for you?

Most code on GitHub either is unsecure, or it was written without needing to be secure.

That is a bit of a stretch imho. There are myriads of open source projects hosted on github that do need to be secure in the context where they are used. I am curious how you came to that conclusion.

I’m already getting pull requests from juniors trying to sneak in AI generated code without actually reading it.

That is worrysome though. I assume these people have had some background/education in the field before they were hired?

Long term wearing of vr headsets might indeed be not all that good. Though, the article is light on actual information and is mostly speculation. Which for the Apple Vision Pro can only be the case as it hasn't been out long enough to conduct anything more than a short term experiment. So that leaves very little data in the way of long term data points.

As far as the experiment they did, there was some information provided (although not much). From what was provided this bit did stand out to me.

The team wore Vision Pros and Quests around college campuses for a couple of weeks, trying to do all the things they would have done without them (with a minder nearby in case they tripped or walked into a wall).

I wonder why the Meta Oculus Quests were not included in the title. If it is the meta Quest 3, it is fairly capable as far as pass through goes. But, not nearly as good as I understand the Apple Vision Pro's passthrough is. I am not saying the Apple Vision Pro is perfect, in fact it isn't perfect if the reviews I have seen are any indicator. It is still very good, but there is still distortion around edges of vision, etc.

But given the price difference between the two I am wondering if the majority of the particpants actually used Quests as then I'd say that the next bit is basically a given:

They experienced "simulator sickness" — nausea, headaches, dizziness. That was weird, given how experienced they all were with headsets of all kinds.

VR Nausea is a known thing even experienced people will get. Truly walking around with these devices with the distorted views you get is bound to trigger that. Certainly with the distortion in pass through I have seen of Quests 3 videos. I'd assume there are no Quests 2 in play as the passthrough there is just grainy black and white video. :D

Even Apple with all their fancy promo videos mostly shows people using the Vision pro sitting down or in doors walking short distances.

So yeah, certainly with the current state of technology I am not surprised there are all sorts of weird side effects and distorted views of reality.

What I'd be more interested in, but what is not really possible to test yet, is what the effects will be when these devices become even better. To the point where there is barely a perceivable difference in having them on or off. That would be, I feel, the point where some speculated downsides from the article might actually come into play.

Would you like me to quote every single one of your lines, line by line, and respond to them?

No, that's not really what I'm asking for. I'm also not looking for responses that isolate a single sentence from my longer messages and ignore the context. I'm not sure how to make my point any clearer than in my first reply to you, where I started with two bullet points. You seemed to focus on the second, but my main point was about the first. If we do want to talk about standard behavior in human conversation, generally speaking, people do acknowledge that they have heard/read something someone said even if they don't respond to it in detail.

Again, I've been agreeing that AI is causing significant problems. But in the case of this specific tweet, the real issue is with a pay to publish journal where the peer review process is failing, not AI. This key point has mostly been ignored. Even if that was not the case, if you want to have any change of trying to combat the emergence of AI I think it is pretty reasonable to question if the basic processes in place are even functioning in the first place. Where my thesis (again, if this wasn't a pay to publish journal) would be that this is likely not the case as in that entire process clearly nobody looked closely at these images. And just to be extra clear, I am not saying that AI never will be an issue, etc. But if reviewing already isn't happening at a basic level how are you ever hoping to combat AI in the first place?

When did anyone say

But by just shouting, “AI is at it again with its antics!” at every turn instead of looking further and at other core issues we will only make things worse”

The context of this tweet, saying "It’s finally happened. A peer-reviewed journal article with what appear to be nonsensical AI-generated images. This is dangerous.", does imply that. I've been responding with this in mind, which should be clear. It is this sort of thing I mean when I say selective reading when you seemingly take it as me saying that you personally said exactly that. Which is a take, but not one I'd say is reasonable if you take the whole context into account.

And in that context, I've said:

that doesn’t mean all bullshit out there is caused by AI

Which I stand by. In this particular instances, in this particular context AI isn't the issue and somewhat clickbait. Which makes most of what you argued about valid concerns. Youtube struggling, SEO + AI blog spam, etc are all very valid and concerning of AI causing havoc. But in this context of me calling a particular tweet clickbait they are also very much less relevant. If you just wanted to discuss the impact of AI in general and step away from the context of this tweet, then you should have said so.

Now, about misrepresenting arguments:

If you are reaffirming somebody else’s comment, you are generally standing behind most if not all of what they said. But nobody here is saying or doing the things you are claiming. You are tilting at windmills.

Have you looked back at your own previous comments when you wrote that? Because while have this, slightly bizarre, conversation I have gone back to mine a few times. Just to check if I actually did mess up somewhere or said things differently that I thought I did. The reason I am asking is that I have been thrown a few of these remarks from you where I could have responded with the above quote myelf. Things like "It’s passing the buck and saying that AI in no way, shape, or form, bears any responsibility for the problem."

The fact that you specifically respond to this one highly specific thing. While I clearly have written more is exactly what I mean.

shrugs

I feel like this is the third time people are selective reading into what I have said.

I specifically acknowledge that AI is already causing all sorts of issues. I am also saying that there is also another issue at play. One that might be exacerbated by the use of AI but at its root isn't caused by AI.

In fact, in this very thread people have pointed out that *in this case" the journal in question is simply the issue. https://beehaw.org/comment/2416937

In fact. The only people likely noticed is, ironically, the fact that AI was being used.

And again I fully agree, AI is causing massive issues already and disturbing a lot of things in destructive ways. But, that doesn't mean all bullshit out there is caused by AI. Even if AI is tangible involved.

If that still, in your view, somehow makes me sound like an defensive AI evangelist then I don't know what to tell you...

I said clickbait about the AI specific thing. Which I do stand by. To be more direct, if peer reviewers don't review and editors don't edit you can have all the theoretical safeguards in place, but those will do jack shit. Procedures are meaningless if they are not being followed properly.

Attributions can be faked, just like these images are now already being faked. If the peer review process is already under tremendous pressure to keep up for various reasons then adding more things to it might actually just make things worse.

I feel like two different problems are conflated into one though.

1. The academic review process is broken.
2. AI generated bullshit is going to cause all sorts of issues.

Point two can contribute to point 1 but for that a bunch of stuff needs to happen. Correct my if I am wrong but as far as my understanding of peer-review processes are supposed to go it is something along the lines of:

1. A researcher submits their manuscript to a journal.
2. An editor of that journal validates the paper fits within the scope and aims of the journal. It might get rejected here or it gets send out for review.
3. When it does get send out for review to several experts in the field, the actual peer reviewers. These are supposed to be knowledgeable about the specific topic the paper is about. These then read the paper closely and evaluate things like methodology, results, (lack of) data, and conclusions.
4. Feedback goes to the editor, who then makes a call about the paper. It either gets accepted, revisions are required or it gets rejected.

If at point 3 people don't do the things I highlighted in bold then to me it seems like it is a bit silly to make this about AI. If at point 4 the editor ignores most feedback for the peer reviewers, then it again has very little to do with AI and everything the a base process being broken.

To summarize, yes AI is going to fuck up a lot of information, it already has. But by just shouting, “AI is at it again with its antics!” at every turn instead of looking further and at other core issues we will only make things worse.

Edit:

To be clear, I am not even saying that peer reviewers or editors should "just do their job already". But fake papers have been increasingly an issue for well over a decade as far as I am aware. The way the current peer review process works simply doesn't seem to scale to where we are today. And yes, AI is not going to help with that, but it is still building upon something that already was broken before AI was used to abuse it.

Oh huh, you are right. I threw that exact prompt in Dall-e and got indeed legible letters.

I totally see why you are worried about all the aspects AI introduces, especially regarding bias and the authenticity of generated content. My main gripe, though, is with the oversight (or lack thereof) in the peer review process. If a journal can't even spot AI-generated images, it raises red flags about the entire paper's credibility, regardless of the content's origin. It's not about AI per se. It is about ensuring the integrity of scholarly work. Because realistically speaking, how much of the paper itself is actually good or valid? Even more interesting, and this would bring AI back in the picture. Is the entire paper even written by a human or is the entire thing fake? Or maybe that is also not interesting at all as there are already tons of papers published with other fake data in it. People that actually don't give a shit about the academic process and just care about their names published somewhere likely already have employed other methods as well. I wouldn't be surprised if there is a paper out there with equally bogus images created by an actual human for pennies on Fiverr.

The crux of the matter is the robustness of the review process, which should safeguard against any form of dubious content, AI-generated or otherwise. Which is what I also said in my initial reply, I am most certainly not waving hands and saying that review is enough. I am saying that it is much more likely the review process has already failed miserably and most likely has been for a while.

Which, again to me, seems like the bigger issue.