piggy [they/them] @ piggy @hexbear.net

Posts

7

Comments

131

Joined

2 wk. ago

I'm so excited for this

Yes and?

1. They're getting paid.
2. It's a job.
3. They're humans who can choose to be better.
4. They're humans who can choose to fight their bosses out of some idiotic love of the game to the detriment of their own mental health because they're crazy. (I'm describing myself).
5. They're humans who can stall or break awful things from coming to pass by refusing to work on something or sabotaging it.

This is about a door to those possibilities closing, not about how many software developers are forced through it. I'm not going to cheer on an awful totalizing future dark age of technology simply because the current odds are bad.

And yeah this won't actually kill higher end devs in my understanding of the world, I'll be able to find a job. But, it will kill the social reproduction of people like me. In the same way that the iPad killed broad user-focused technological literacy from zoomers to millenials, LLMs will ultimately destroy the current level of developer-focused technological literacy. There won't even be guys who can't code their way out of a paper bag using StackOverflow or guys who memorize LeetCode solutions. It will just be old-heads powerful enough to avoid the cull and nobody else, until we die.

Every large coporation uses this method because they want to have fungible devs. Since developers with actual skill don't want to be treated as fungible cogs, the selection pressures ensure that people who can't get jobs with better conditions end up working in these places. They're just doing it to get a paycheck, and they basically bang their heads against the keyboard till something resembling working code falls out. I'll also remind you of the whole outsourcing craze which was basically exact same goal corps want to accomplish with AI now.

Damn that's crazy, imagine working a coding job for a paycheck! Soon you won't even be able to!

Also, for devices like airconditioners or televisions that use IR remotes and presumably some subset of a standard code is there a structured way to build an interface for them?

You can DIY with https://tasmota.github.io/docs/Tasmota-IR/

Some people recommend Broadlink RM3/RM4 and just walling it off from the web.

If your PC/Raspberry PI is in range you can get a cheap linux compatible IR Emitter and use LIRC https://www.lirc.org/

StackOverflow copypasta wasn't a productive processes that was seeking to remove the developer from the equation though.

This isn't about a tech scaling strategy of training high quality high productivity engineers vs "just throwing bodies at it" anymore. This is about the next level of "just throwing bodies at it", "just throwing compute at it".

This is something technically feasible within the next decade unless, inshallah, these models collapse from ingesting their own awful data, rather than improving.

For everyone of you there's 1000 junior engineers running copilot.

The way that you're applying the tool "properly" is ultimately the same way that middle managers want to apply the tool, the only difference is that you know what you're doing as a quality filter, where the code goes and how to run it. AI can't solve the former (quality) but there are people working on a wholesale solution for the latter two. And they're getting their data from people like you!

In terms a productive process there's not as much daylight between the two use cases as you seem to think there is.

Nobody is arguing for using the AI for problems you keep mentioning, and you keep ignoring that.

This is absolutely not true. Almost every programmer I know has had their company try to "AI" their documentation or "AI" some process only to fail spectacularly because the basis of what the AI does to data is either missing or doesn't have enough quality. I have several friends at the Lead/EM level take too much time out of their schedules to talk down a middle manager from sapping resources into AI boondoggles.

I've had to talk people off of this ledge, and lead that works under me (I'm technically a platform architect across 5 platform teams) actually decided to try it anyway and burn a couple days on a test run and guess what the results were garbage.

Beyond that the problem is that AI is a useful tool in IGNORING the problems.

I've given you concrete examples of how this tool is useful for me, you've just ignored that and continued arguing about the straw man you want to argue about.

I started this entire comment thread with an actual critique, a point, that you have in very debate bro fashion have consistently called a strawman. If I were a feeling less charitable I could call the majority of your arguments non-sequitors to mine. I have never argued that AI isn't useful to somebody. In fact I'm arguing that it's dangerously useful for decision makers in the software industry based on how they WANT to make software.

If a piece of software is a car, and a middle manager wants that car to have a wonderful proprietary light bar on it and wants to use AI to build such a light bar on his wonderful car. The AI might actually build the light bar in a narrow sense to the basic specs the decision maker feels might sell well on the market. However the light bar adds 500lbs of weight so when the driver gets in the car the front suspension is on the floor, and the wiring loom is also now a ball of yarn. But the car ends up being just shitty enough to sell, and that's the important thing.

And remember the AI doesn't complain about resources or order of operations when you ask it do make a light bar at the same time as a cool roof rack, a kick ass sound system and a more powerful engine, and hey if the car doesn't work after one of these we can just ask it to regenerate the car design and then just have another AI test it! And you know what it might even be fine to have 1 or 2 nerds around just in case we have to painfully take the car apart only to discover we're overloading the alternator from both ends.

I've never said that AI is the cause of those problems that's words you're putting in my mouth. I've said that AI is being used as a solution to those problems in the industry when in reality the use of AI to solve those problems exacerbates them while allowing companies to reap "productive" output.

For some reason programmers can understand "AI Slop" but if the AI is generating code instead of stories, images, audio and video it's no longer "AI Slop" because we're exalted in our communion with the machine spirits! Our holy logical languages could never encode the heresy of slop!

This is a quantization function. It's a fairly "math brained" name I agree, but the function is called qX_K_q8_K because it quantizes a value with a quantization index of X (unknown) to one with a quantization index of 8 (bits) which correlates to the memory usage. The 0 vs K portions are how it does rounding, 0 means it does rounding by equal distribution (without offset), and K means it creates a distribution that is more fine grained around more common values and is more rough around least common values. e.g. I have a data set that has a lot of values between 4 and 5 but not a lot of 10s. I have lets say 10 brackets between 4 and 5 but only 3 between 5 and 10.

Basically it's a lossy compression for a data set into a specific enumeration (roughly correlates with size), so it's a way to given 1,000,000 numbers from 1-1000000, of putting their values into a range of numbers based on the q level How using different functions affects the output of models is more voodoo than anything else. You get better "quality" output from higher memory space, but quality is a complex metric and doesn't necessarily map to factual accuracy in the output, just statistical correlation with the model's data set.

An example of a common quantizer is an analog to digital converter. It must take continuous values from a wave that goes 0 to 1 and transform them into digital values of 0 and 1 with a specific sample rate.

Taking a 32 bit float and copying the value into 32 bit float is an identity quantizer.

You're making up a giant straw man of how you pretend software development works which is utterly divorced from what we see happening in the real world. The AI doesn't change this one bit.

Commenting this under a post where an AI has spit out a dot product function optimization for an existing dot product function that's already ~150-250 lines long depending on architectural implementation of which there are about 6. The PR for which has an interaction that is two devs finger pointing about who is responsible for writing tests. The PR for which notes that the original and new function often don't give the correct answer. Just an amazing response. Chefs kiss.

What a wonderful way to engage with my post. You win bud. You're the smartest. This industry would never mystify a basic concept that's about 250 years old with a 716 line PR through its inability to communicate, organize and follow an academic discipline.

I know what I want to do conceptually, and I have plenty of experience designing applications.

How does AI help you actually traverse the concepts of React that you admit you don't have nitty gritty knowledge of how they work in terms of designing your application? React is a batteries included framework that has specific ways of doing things that impact the design and concepts that are technically feasible within React itself.

For example React isn't really optimized to crunch a ton of data performantly so if you're getting constant data updates over a web socket from multiple points and you want some or all the changes to be reflected you're gonna have a bad time vs something that has finer grained change controls out of the box such as Angular.

How does AI help you choose between functional and class based React components? How much of your application is doing typical developer copy-pasta instead of creating HOCs for similar functionalities? How did AI help you with that? How is AI helping apply concepts like SOLID into the design of your component tree? How does AI help you decide how to architect components and their children that need to have a lifecycle outside of the typical change-binding flow?

This in my opinion is the crux of the issue, AI cannot solve this problem for you nor can it reasonably explain it in a technical way beyond parroting the vagaries of what I said above. It cannot confer understanding of complex abstract concepts that are fuzzy and have grey areas. It can tell you something may not work explicitly but it cannot educate you realistically on the tradeoffs.

It seems to me that your answer boils down to "code monkey stuff". AI might help you swing a pickaxe, but it's not good at explaining where the mine is going to collapse based on the type of rock you're digging in. Another way of thinking about it is that you could build a building to the "building code" but it will still collapse. AI can explain the building code and loosely verify that you built something to it, but it cannot validate that your building is going to stay standing nor can it practically tell you what you need to change.

My problem with AI tools boils down to this. Software is a medium of communication. It communicates the base of a problem and the technical process of solving it. Software Engineering is a field that attempts to create strong patterns of communication and practices in order to efficiently organize the production of Software. The software industry at large (where most programmers get exposed to the process of building software) often eschews this discipline because of scientific management (the idea you can simply manage a process through fiduciary/managerial knowledge rather than domain knowledge) and the need for instant development to maintain fictional competitive advantage and fictional YoY growth. The industry welcomes AI for 2 reasons:

1. It can code monkey...eventually. Why pay programmers when you can ask CahpGBT to do it?
2. It can fix the problem of needing to deliver without knowing what you're doing... eventually. It fixes the problem of communication without relying on building up the knowledge and practice of Software Engineering. In essence why have people know this discipline and its practical application when you can continue to have the blind leading the blind because ChadGTP can see for us?

This is a disservice to programmers everywhere especially younger ones because it destroys the social reproduction of the capacity to build scalable software and replaces it with you guessed it machine rites. In practice it's the apotheosis of Conway's Law in the software industry. We build needlessly complex software that works coincidentally, and soon that software will be analyzed, modified, and ultimately created by a tool that is an overly complex statistical model that also works through the coincidence of statistical approximations.

Okay let me ask this question:

Who is this useful for? Who is the target audience for this?

That's just a straw man, because there's no reason why you wouldn't be looking through your code. What LLM does is help you find areas of the code that are worth looking at.

It's not a strawman because classifying unperformant code is a different task than generating performant replacement code. LLM can only generate code via it's internal weights + input it doesn't guarantee that that code is compilable, performant, readable, understandable, self documenting or much of anything.

The performance gain here is coincidental simply because the generated code uses functions that call processor features directly rather than get optimized into processor features by a compiler. LLM classifiers are also statistically analyzing the AST for performance they aren't actually performing real static analysis of the AST or it's compiled version. It doesn't calculate a BigO or really know how to reason through this problem, it's just primed that when you write the for loop to sum, that's "slower" than using _mm_add_ps. It doesn't even know which cases of the for loop compile down to a _mm_add_ps instruction on which compilers and which optimization levels.

Lastly you injected this line of reasoning when you basically said "why would I do this boring stuff as a programmer when I can get the LLM to do it". It's nice that there's a tool that you can politely ask to parse your garbage and replace with other garbage that happens to use a function that's more performant. But not only is this not Software Engineering, but a performant dot product is a solved problem at EVERY level of abstraction. This programming equivalent of tech bros reinventing the train every 5 years.

The fact that this is needed is a problem in and of itself with how people are building this software. This is machine spirit communion with technojargon. Instead of learning how to vectorize algorithms you're feeding your garbage code through a LLM to produce garbage code with SIMD instructions in it. That is quite literally stunting your growth as a Software Engineer. You are choosing to ignore learning how things actually work because it's too hard to parse through the existing garbage. A SIMD dot product algo is literally a 2 week college junior homework assignment.

Understanding what good uses for it are and the limitations of the tech is far more productive than simply rejecting it entirely.

I quite literally pointed several limitations in the post you replied to and in this post from a Software Engineering perspective.

This type of tooling isn't new and doesn't require AI models. Performance linters exist in many languages. Rubocop perf, perlint in python, eslint perf rules etc. For C++, clang-tidy and cpp-perf exist.

The only reason LLMs are in this space is because there is a lack of good modern tooling in many languages. Jumping straight to LLMs is boiling the ocean (literally and figuratively).

Not only that but if we're really gonna argue that "most code is very boring". That already negates your premise, most boring code isn't really highly perf sensitive and unique enough to be treated individually through LLMs. Directly needing to point out SIMD instructions in your C++ code basically shows that your compiler tool chain sucks or you're writing your code in such a "clever" way that it isn't getting picked up. This is an optimization scenario from 1999.

Likewise if you're not looking through the code you're not actually understanding what the performance gaps are or if the LLM is making new ones by generating sub-optimal code. Sometimes the machine spirits react to the prayer protocols and sometimes they don't. That's the level of technology you're arguing at. These aren't functional performance translations being applied. Once your system is full of this kind of junk, you won't actually understand what's going on or how things practically work. Standard perf linters are already not side effects free in some cases but they publish their side effects. LLMs cannot do this by comparison. That's software engineering, it's mostly risk management and organization. Yes it's boring.

You can distill your own models based on your own models, the fact that OpenAI isn't doing this is more evidence that they are "competing" via investment capital and not tech.

I'm going to say 2 things that are going to be very unpopular but kinda need to be heard.

1. DeepSeek is turning this place into /r/OpenAI but red, which is incredibly lame
2. If LLMs are significantly helping your development workflow, you are doing grunt work, you're not improving your skills, and you're not working on problems that have any significant difficulty beyond memorizing multiplication tables type recall but for tech.

This optimization is actually grunt work, it's not a new discovery, it's simply using SIMD instructions on matrices something that should have been done in the first place either by hand or by a compiler.

This is actually not new in the US. It's a well entrenched part of history that is creatively and politely skirted by. For example Castillo De San Marcos in St.Augustine Florida was a fort built by the Spanish in 1672 it's quite literally a mound of rock and was never really modernized or updated in the last 350 years except for tourism (okay fine the Dept of War built some industrial era canons and lets be honest for what?). The US Dept of War controlled the fort until 1942, and what do you think that the Americans used it for? That's right, mostly ethnic cleansing of indigenous people!

I fucking hate that place. The US National Park service is so effective at white washing the history of it. The signage is egregious, it's like going to Aushwitz and seeing signage that says "Jews were forcibly put into train cars and delivered to the camps. How have long journeys affected your life?". Half the signage is like that, it's like a large amount of Native Americans died here we don't really know their names or their cultures (maybe they were seminoles or some shit but some weren't and who cares?) or ultimately have a full count of how many there were it would be pointless bureaucracy. How has your life been affected by pointless bureaucracy?

As a sidenote "putting things in boxes" is the very thing that itself upholds bourgeois democracies and national boundaries as well.

I mean at this raw of an argument you might as well argue for Lysenkoism because unlike Darwinism/Mendelian selection it doesn't "put things in boxes". In practice things are put in boxes all the time, it's how most systems work. The reality is that as communists we need to mediate the negative effects of the fact that things are in boxes, not ignore the reality that things are in boxes.

The failure of capitalism is the fact that it's systems of meaning making converge into the arbitrage of things in boxes. At the end of the day this is actually the most difficult part of building communism, the Soviet Union throughout it's history still fell ill with the "things in boxes" disease. It's how you get addicted to slave labor, it's how you make political missteps because it's so easy to put people in a "kulak" in a box that doesn't even mean anything anymore, it's how you start disagreements with other communist nations because you really insist that they should put certain things into a certain box.

Geopolitical power comes mainly from 3 things, resources, technology, and controlling your "excess" (i.e. the people that do the "worst" work) population. Historically borders have been an effective means to more-or-less control all 3.

Controlling your own borders is really childs play, controlling other people's borders is where the fun really starts. Sykes-Picot for example ensured that the Middle East would fight over resources (water, arable land) and who the "excess" population should be by drawing borders in creative ways preventing the reformation of the Ottoman Empire after its defeat.