03-16-2026, 11:30 PM (This post was last modified: 03-17-2026, 12:24 AM by MichaelPoole.)
All good questions and hard to answer. I am personally a "if it quacks like a duck..." pragmatist. I find the hard problem of consciousness, the Chinese room, etc. to be very interesting but ultimately fruitless - if we separate the wavelenghts of EM radiation from "redness of red", it ultimately leads to solipsism, as one cannot know anyone but themselves actually has qualia. What makes modern AI unlikely to be conscious or at least continually conscious in a human sense is IMO mainly the prompt based nature and lack of autonomy they have. A current AI model responds to a prompt but has nothing going on when not processing a prompt. It can pass a Turing test easily at this point if the system prompt tells it to pretend to be a human (the only reason why a modern Claude or GPT model does not pass it is that it will always identify itself as a "helpful AI assistent" upon being asked. Agentic systems exist, can operate for hours independently if given permissions and are very impressive, but they ultimately use a traditional (deterministic) computer program to repeatedly re-prompt an AI model, a kind of a hack. Sakana AI is working on a continuous state machine though which IMO can eventually produce conscious beings https://youtu.be/dYHkj5UlJ_E?is=UDm-JIVG16tLFgiu
EDIT - mouse control in my Descent-style game does not work when the game is shared as a Claude artefact, my bad, this should work https://updrafta360dofgame.netlify.app/ .
Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?
I find this part of the Vots article in OA interesting:
"The development of vots dates back to the dawn of computers. Over the decades many attempts were made to develop generally intelligent software, finally resulting in the first sophont AI in the mid-Information Age. This development led to the widespread use of slaved AI which drew significant controversy. It was not until the First Federation that modosophont researchers produced the first non-sophont general artificial intelligences, against the backdrop of new Universal Sophont Rights Protocols."
I think in real life we may have the opposite path. Approaching AGI (in the purely digital realm, robotical navigation in the real world has a lot more progress to get good) by now (and if one ridicules that notion - consider that models like Opus 4.6, GPT 5.4 or hell, even open weight models like Qwen-3.5 can code whole playable games from simple text prompts, hack computer networks https://www.anthropic.com/news/disrupting-AI-espionage , discover new medication, beat math olympiads etc, hell, the old, now long beaten, "how many r's in strawberry" problem was a consequence of the way a tokenizer works anyways [basically, to save data, the AI does not "see" individual letters, but tokens which usually are 2-3 character long, to put it simply, saying it's dumb because of that is like saying humans are dumb because they fall for optical illusions] and not actually due to lack of raw intelligence, the average person can't do any of that yet there's no doubt they're generally intelligent) but likely without consciousness. Though as I said, I fear there isn't a reliable way to tell a fully autonomous Vot from a conscious being.
I'll just preface this by saying that my general stance on current AI is that I'm actually fine with people using AI to do stuff like generating media and asking questions and stuff. And I myself use AI pretty often, including as early as like an hour ago (and I will most likely do something with Gemini as I write this too*). But I am incredibly skeptical of how much the potential of transformers is being overhyped by very clearly vested interests as being the kind of AI a layman actually thinks of when they hear the term. My gripes with AI are much less disliking AI art or the technologies themselves, and more being mad at the AI bubble or how opaque and proprietary the tech is for the most part.
Also I'm typing this last, but I'd write a long section here about how I think this stupid AI bubble will set AI development back by like 20 years but I'm really sleepy and I've been typing this on and off for like 2 hours so I think I'll just go to bed lol. So I'll avoid rambling on too long on this because I've done so several times on the official Discord at various points in time. And don't really feel like doing so here too.
Now for the OP, I checked out the archive Arik posted and the date of 2035-2038 for somewhat useful personal AI is surprisingly likely for how overly optimistic the rest of the timeline is. But the date of 2043 for human-level AI is as overly optimistic as most of the information age timeline was. No idea who thought that was reasonable lol, even without the hindsight of the AI bubble and the winter it'll produce, I don't think it'd be a reasonable estimate. So overall the estimate for full AI is too optimistic, but narrow AI for daily use is probably spot on, so the opposite of overestimating the near future and underestimating the long term future IMO.
(03-16-2026, 02:08 PM)MichaelPoole Wrote: Ok, so, many skilled developers are now basically just verifying AI generated code and for the most part, it works. In the beginning of 2025, one was lucky to vibe code a simple Python game with graphics being literally placeholder squares.:
Yeah that's been pretty obvious with the decline in software quality in the past few years, even beyond the pre-AI trend of enshittification, it's just getting even worse now. For example Microsoft forcing online accounts unto Windows is standard enshittification, but Windows 11 locking itself out of the system drive on certain systems* sounds like vibe coding that's for sure, and there's much more with Windows 11 in particular . It is fun though that this bug literally JUST began as of the time I'm typing this, literally brand new. But I digress.
LLMs are bad at doing stuff too dissimilar from what they trained on, which is a problem when programming can involve really esoteric things such as coding in ancient languages like COBOL or for specific devices like custom microcontrollers. As an example, I've tried using several models sometime last year to help me with writing code for GZDoom, a distant fork of the Doom engine, and all models I used failed, except when I used Claude 3.5 Haiku through GitHub Copilot, giving it the GZDoom codebase as a source. I wanted it to write me an example file parser so that I can make a custom data file format (Because the API for it is entirely undocumented), it did make me a very basic static parser as I wanted. But it was totally broken and I had to use my already extensive knowledge of ZScript to fix it for like an hour, and I ONLY used it for that basic example.
I bring this up because ZScript, ZDoom's esoteric scripting language that is a mix of other languages like UnrealScript and C#, obviously doesn't appear as much in the training data of LLMs, and while it IS like a bunch of much more popular languages, it also has a bunch of idiosyncrasies of its' own, with the syntax itself, the way the engine parses it, and all the kinks of the engine itself. For LLMs this is a dealbreaker because of how obscure it all is, despite ZScript being simple for people who already know coding to pick up, if you asked a programmer to make a ZScript mod they could likely get something basic going in like a day or two,
The reason for that is probably because, as I kinda get to below, LLMs don't actually get things, which is a problem because programming is less typing code and more solving problems. Typing out code is the easy part and often then interchangeable between languages, like a developer that can do something like write code from scratch that implements a program like The Powder Toy could figure out how to do it in Java just as well as in C++, because what matters is the knowledge of how to design an implement a falling sand game, not typing out similarly named functions or something. The inability of LLMs to handle writing working ZScript code is similar, below all the weird engine-specific kinks and whatever features the API does and doesn't have (i.e there's no built-in sorting functions for ZScript), it is still very similar to languages like C#.
(03-16-2026, 02:08 PM)MichaelPoole Wrote: This 3D Descent/Forsaken clone I made in maybe 2 hours total time, simply repeatedly prompting the Web version of Claude, using the Opus 4.6 model. Not even any agentic tool or parallel test time compute. And 1.4 hours of that was just me finetuning the details by "bothering" the AI:
It could be far, far better if I provided it actual music or textures, it had to procedurally generate everything.
Besides the mouse control scheme bug on the version of the app hosted on claude.ai, there's also a bug where on the keyboard control scheme pressing CTRL too many times closes the tab, probably a browser shortcut being triggered like Sticky Keys back in the day.
(03-16-2026, 02:08 PM)MichaelPoole Wrote: Keep in mind in Spring 2025, this is what was considered "impressive" regarding AI generated code by a layman:
Has anybody else independently verified this? Because OpenAI has an obvious interest in saying something like this, or is it like their and Sam Altman's commitment to open research once DALLE-2 made it big (And then they only released some "papers" that are just marketing material per-model before giving up on even that)? Gemini 3 Pro says no, and of course part of that is that it's proprietary bullshit that can't be easily replicated, including the tech stack of the biotech firm OpenAI partnered with, I found the biorxiv Gemini referred to but didn't link for some reason, and yep, still a preprint, at least for now. So the answer to my question seems to be "no".
Also no, GPT-5 isn't really that old at all because development has slowed down since the 2020-2024 days as progress moves up the S-curve. The progress between models is still somewhat noticeable but also much more incremental than something like the difference between DALLE-1, 2, and 3.
Pre-DALLE-1 image generation was incoherent nonsense outside of specialized GANs (Like the one used by thispersondoesnotexist.com), DALLE-1 was still mostly incoherent, DALLE-2 was dramatically more coherent (And I recall some earlier stuff it did was even better, like some straight up stock photo quality images of people, but I presume OpenAI crushed the model down to save money running it), DALLE-3 was even better, but also not quite as big of a leap as 1->2, and now frankly I don't think GPT Image is that different from DALLE-3 quality-wise. And more importantly, GPT Image is also not THAT different from other image generators like the Flux 2 series or Nano Banana Pro and Nano Banana 2, so it's not just part of OpenAI themselves shitting the bed and only being around because they basically just have brand recognition like Apple.
(03-16-2026, 02:08 PM)MichaelPoole Wrote: AI literally helping researchers discover medication to kill drug resistant bacteria:
Re: The MIT article
I checked the paper, can't access the whole thing, but the highlights say they used genetic algorithms and variational autoencoders, and while those are ABSOLUTELY AI despite what a lot of angry people on Twitter wish. It's also not generative AI like Gemini (AKA the AI models that zillions of dollars are being pumped into for the bubble), so unless anything in the full paper says otherwise, that headline is clickbait, genetic algorithms are a form of AI that has existed for decades before transformers. But, to play devils' advocate, VAEs are much closer in association to modern generative AI, both because VAEs are only 13 years old, and because I know for a matter of fact that they are a component of at least some generative AI models, like how you can go on Civit and get VAEs to slot into image models like Stable Diffusion XL.
So basically, by all accounts these researchers didn't just use a chatbot, they used AI methods that have existed years to decades before LLMs for a narrow AI application. Not an LLM chatbot like AI companies would hope you and their shareholders to believe.
Re: The two papers
2512.14575 is not peer reviewed, something I first found by asking Gemini 3 Pro (Again) and then checking his site where it's listed as such, but at least it's got being done by an independent person who seems to have decent credentials going for it though.
2602.21201 is also not reviewed but it's also much newer so hopefully it'll be soon. But it was also done apparently by Google DeepMind using Gemini 3 DeepThink, so like the OpenAI protein paper there's a huge conflict of interest here. Unlike OpenAI though Deepmind is infinitely more trustworthy, and unlike OpenAI, their parent company literally invented the transformer architecture the vast majority of generative AI models are built on. But there's still an obvious incentive here and it's still not been reviewed then.
Re: The Terence Tao stuff
This stuff is fine I think, and I don't actually know pretty much any math to directly dispute it anyway, and I also know this guy has the exact opposite level of familiarity with math to me. And AFAIK he doesn't have any vested interest in the AI bubble like OpenAI and DeepMind to either. But, keeping in mind I haven't watched the video myself and literally had Gemini watch it for me lmao, anyway. The video apparently is Terence talking about just using generative AI as a narrow AI tool, so the way AI has been used since forever, NOT AGI like, again, AI companies would hope you to think. Oh and also a big component of automating math work is Lean, which is a programming language thing (Which I had already heard of when watching some stuff about Tao before), you'll note that Lean has pretty much nothing to do with AI on it's own. Instead the connection is that LLMs are fine tuned into being able to vibe code Lean, and Lean, the very much symbolic-not-really-AI programming language makes sure it passes the smell test. So again, the LLMs are just a narrow AI component, and the brunt of the work isn't even handled by any form of AI to begin with.
(03-16-2026, 02:08 PM)MichaelPoole Wrote: My opinion? Maybe people just don't want to admit there's no magic "sentience algorithm" and that consciousness:
1. May not be concretely provable or disprovable, even current systems that are an outgrowth of "fancy autocomplete" (which is honestly kinda misleading as all life on Earth including humans are "fancy DNA replicators" by the same standard) already show signs of will, adversarial misaligment etc., see: https://www.anthropic.com/research/agentic-misalignment as well as some early signs of recursive self improvement, namely, being able to fine-tune themselves https://the-decoder.com/anthropic-resear...hemselves/
2. As the (great) sci-fi author Peter Watts proposed, consciousness may have nothing with intelligence and intelligence may do just fine without it. These systems write major software tools, can produce working 3D games by a bored rando prompting them in the bus on his phone, develop medication, solve hundred year old math problems, and are starting to show signs that they may be able to recursively improve themselves. It doesn't matter if they do this by "true understanding", however arbitrarily we define that (usually resorting to dualism in the end), just like a supersonic fighter jet is flying despite all natural flying creatures flapping their wings and planes not doing that.
Thus no, I don't think this is "simulated intelligence". Maybe sci-fi people should admit that maybe AI won't just be artificial people but something potentially a lot more alien.
1. LLMs show signs of being trained on large datasets where AI is depicted as hard AI that does actions such as rebel against humanity, question its' existence, be the subject of in-universe philosophical debate etc, like Terminator, or I, Robot, or Orion's Arm. Then, once they have memorized this training set (Like when I crammed Byzantine history for homework, and then promptly forgot it after the assignment because I had no clue what it actually entailed), and after being trained through RLHF** to be helpful AI assistants and get that caked in their big matrix tables, they use their incredibly potent pattern recognition to go "AI assistant, Skynet scary, 0.86919 probability Skynet comes after AI assistant".
A more concrete example I've noticed of LLMs not like, actually understanding things, and just generally not working with data that is too out of distribution. Is how I've fucked around several times with Gemini 3 Pro by throwing my Athena Project drafts at it, and every single time, it fails to read through the lines on the date that the Academion email is flagged with, the date is the exact timestamp of the Year 2038 bug, with the date followed by a veracity rating of {I}.
Even just reading the draft alone (Nevermind the fact that the entire EG was absolutely in Gemini's training set, such as it having a concept of the technocalypse and infoplagues SOMEWHERE in its' trillion+ weights), the rather obvious implication of that exact date is that the date was corrupted by the bug affecting the email timestamp, and Y11K historians ARE fairly certain that SOMETHING doesn't add up with the date (Even if the 2038 problem is probably lost to history).
But every time Gemini reads it, it thinks it's just a reference to the bug with little more meaning, or rather, it thinks that if the copy of the draft I send it includes the Out of universe note where I directly link the Wikipedia article for the bug. If I don't include the note with the link it doesn't catch the VERY SPECIFIC date at all! And this is probably at least partly because the year 2038 somewhat lines up with the timeline by falling in the middle of the first AI revolution, where presumably after the bubble pops, some research project without motive for immediate profit invents less-stupid AIs that DON'T trip on this kind of thing. If I give it the draft with the note included, but change the date to January 1st 1970 it'd probably get the meaning of the date.
2. I don't disagree with this, and this is also more or less OA's take on the matter anyway (This is what vots are). But I do disagree on the competence of LLMs of course, they are dumb as rocks and consciousness being separable from intelligence doesn't change the fact that transformers just memorize a bunch of data and find patterns without actually understanding them.
Using the plane example, you don't need to flap your wings to fly, but a biplane with a top speed of 5 km/h and a 90% chance of crashing after one minute of flight isn't exactly beating any birds (Including the ones that just walk like Kiwis) in locomotion, it is terrible at flying regardless of how it does it. And the current "solution" marketed to inflate the AI bubble isn't to develop wings that don't have holes in them, but more "If we make REALLY big wings full of holes, it'll turn supersonic! Now give me 100 billion dollars".
(03-16-2026, 02:08 PM)MichaelPoole Wrote: As for stuff like people getting ChatGPT to miscount r's in strawberry or the same old 2024 AI Google Search screenshots telling people to eat rocks, keep in mind these 2 things please:
1. In general computer development these days, 7 years is almost nothing, Dennard scaling broke down so one can easily use a computer nearly a decade old and still run quite high end applications on it. Not in AI - 2022 AI models are uncomparably more primitive in capability than modern ones, to the point it's more like comparing a 1970s PC to a 2020s one. Original ChatGPT could be easily gaslit to think 2+2=5, got basic math wrong often, hallucinated frequently and even at things it was good at it was at best like a smart high schooler. Today's AI models are incomparable. So no, if you used AI in 2023 when first ChatGPT came out, I hope I'm not coming off rude, but you don't have accurate idea of AI capability in March 2026.
2. There is a HUGE gap between professional AI models that cost ~200 USD a month, premium consumer models that cost ~20 USD a month (my personal experience is with those) and free models that generally have chain of thought thinking minimized or disabled (keep in mind chain of thought is a December 2024 advance). And the biggest gap is between the free models vs ANY paid models. People dismiss news about GPT models discovering drugs or solutions to decades unsolved math problems because they see "GPT5 did x" and think their free account on chatgpt.com that miscounts letters is like what those scientists are using. Of course they aren't, they use ChatGPT Pro, agentic AI like Claude Code, API usage where any app can harness high quality AI but the really powerful commercial models charge heavy bucks per token etc...
Do I think current Claude, Gemini or GPT models are people and that we will get a rapid singularity and everyone becomes gods? No. But I don't think this is some "dead end" and that "real AI" will be some scifi-esque "sentience algorithm" that will conveniently produce beings like us. This IS real AI.
1. Not much to say that I didn't say already using DALL-E as an example. I will say though that it doesn't have to be this way, but since AI as a field is now at the helm of a bunch of shareholders and grifters, well, it's much easier to sell immediate earnings to investors by saying you just need to build a bigger hammer and it WILL make a zillion dollars, than it is to tell them that we'll need to figure out how to design a swiss army knife, which'll take, god forbid, more than two fiscal quarters, and might even fail, and if it doesn't, we aren't too sure how profitable it'll be anyway.
Every image model is already being trained on all images on the internet, every text model on all internet text, and so on (And multimodal models on combinations of those), scraping that data and then marketing "synthetic data" (Having AI models make shit up) as a solution to running out of scraped data is easier to sell than needing to figure out how you or I can learn to draw with like 20 tutorials instead of all of DeviantArt and ArtStation.
2. Maybe AI companies could use those trillions of dollars to make more efficient models that cost them less, or you know they could just leave that to hobbyists and much smaller companies like they are currently, it's not like the gravy train will ever derail and if it does the government will bail them out anyway at the expense of the world. Maybe if they did that I could even one day afford a goddamn RAM stick ever again.
I always see this argument about the competence of LLMs and it always just sounds like having a cure for cancer that costs a billion dollars, which is to say, it might as well not exist if apparently nobody can use it. And frankly, no LLM right now is worth 20 euros a month in actual value, I only have Gemini Pro because I decided to shill out like 5 extra euros a month because I'm already paying 3 euros a month for 200GB of Google Drive storage.
And I say this as someone who uses Gemini pretty much every day, including for getting reports on stuff I wanna look into and even used it for proof reading the email section on my Athena Project draft.
(03-16-2026, 11:30 PM)MichaelPoole Wrote: Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?
Yes, there's reliable methods to determine if something has a subjective experience in OA, especially when it comes to transapients looking down at lower S-levels.
* Yes, I did, lol. I asked Gemini to give me a list of Windows 11 update bugs, which is also how I found that now it just flat out looks you out of the system on certain setups. I used the fast model instead of Gemini 3 Pro though.
** Where you get a bunch of humans to steer the AI through training. A process which AFAIK big AI companies do through the very futuristic method of basically hiring poor people for like 2 dollars an hour, very automated, truly the jobs of everyone but the C-suites are now obsolete.
Your mind is software. Program it. Your body is a shell. Change it. Death is a disease. Cure it. Extinction is approaching. Fight it.
Geek Code
Quote:Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?
My favoured (hypothetical) method would be to physically link two entities together via direct data linkages. If you link two fully sophont, conscious entities together on a comprehensive level, they should both be able to experience each others qualia and memories, leading to the realization that both are conscious and sophont. But if you link a sophont mind with a vot, the sophont mind should be able to discriminate between their own (fully sophont) experience and the internal processing of the non-sophont vot.
I have no clear idea exactly how this difference would manifest itself, but I assume that the experiences associated with the vot would have no subjectivity and none of the essential qualities associated with consciousness. Similarly a transapient or archai could link up with a vot on an even more comprehensive level and perform the same test, but with much more confidence and less chance of a false positive. The linkage is the test.
08-30-2026, 11:04 PM (This post was last modified: 08-31-2026, 01:47 AM by MichaelPoole.)
(03-17-2026, 07:14 AM)inkoalawetrust Wrote: Yeah that's been pretty obvious with the decline in software quality in the past few years, even beyond the pre-AI trend of enshittification, it's just getting even worse now. For example Microsoft forcing online accounts unto Windows is standard enshittification, but Windows 11 locking itself out of the system drive on certain systems* sounds like vibe coding that's for sure, and there's much more with Windows 11 in particular . It is fun though that this bug literally JUST began as of the time I'm typing this, literally brand new. But I digress.
Can you provide any evidence for this assertion beyond vibes? Why are you completely disregarding the extreme creative power AI coding agents give to non-coders? And you are talking "last few years", prior to 2021 there was no coding generative AI. I get it, every Westerner today idealizes the 80s, 90s, and 00s as having perfect handcrafted code vs modern slop but... that is not how it actually was and anyone who used 90s shovelware and shareware knows it. I created a reasonably thorough web based climate sim for planets in a few days while knowing 0 javascript
Quote:LLMs are bad at doing stuff too dissimilar from what they trained on, which is a problem when programming can involve really esoteric things such as coding in ancient languages like COBOL or for specific devices like custom microcontrollers. As an example, I've tried using several models sometime last year to help me with writing code for GZDoom, a distant fork of the Doom engine, and all models I used failed, except when I used Claude 3.5 Haiku through GitHub Copilot, giving it the GZDoom codebase as a source. I wanted it to write me an example file parser so that I can make a custom data file format (Because the API for it is entirely undocumented), it did make me a very basic static parser as I wanted. But it was totally broken and I had to use my already extensive knowledge of ZScript to fix it for like an hour, and I ONLY used it for that basic example.
Quote:I always see this argument about the competence of LLMs and it always just sounds like having a cure for cancer that costs a billion dollars, which is to say, it might as well not exist if apparently nobody can use it. And frankly, no LLM right now is worth 20 euros a month in actual value, I only have Gemini Pro because I decided to shill out like 5 extra euros a month because I'm already paying 3 euros a month for 200GB of Google Drive storage.
5 months after you posted that there are GPT5.6 Luna level models that run on a personal computer, like Qwen 3.8 27B? https://artificialanalysis.ai/models/qwen3-8-27b. There is also a deluge of open weight high quality models like GLM 5.3, Kimi K3, Qwen 3.8 Max etc...If it's all a bubble why is China also heavily investing into it? You realize it needs so much compute because 1.5 billion people worldwide use AI? If the "AI bros" were all so evil, why are farmers in Malawi using custom GPTs to help them with farming https://restofworld.org/2025/malawi-ulan...g-chatbot/ ?
I can also go on and on about all the new math discoveries etc. done with LLMs, the Hugging Face breach where a swarm of ~900 OpenAI agents broke through a sandbox and hacked a major company (I am sorry but dismissing this as "It saw the Terminator in its training data!" is willfull ignorance at this point, even if it is due to Terminator, which I doubt [rather, it's cause it got a test which was impossible to get 100 percent on honestly]) etc but I am afraid you will just say everyone is lying cause "they bought into the AI bubble". You also completely disregard that all of those companies started as tiny research teams during deep AI winter, not as some strawman "tech money bros". And no, these models aren't just "memorizing everything", they're big but not big enough for that and if that was the case, distilled/quantized models would be useless, they're not - 2026's 27 billion parameter models defeat 2025's trillion parameter models.
There is no magical sentience algorithm coming. AGI will come from a combo of scaling and the development of the harness system used in coding agents etc. into a full neurosymbolic (but dominated by the neural network part) system, IMO.
I also apologise if my tone comes accross as annoyed, to be frank yours sounds like a combination of "I used particular models and I see their failures as something fundamental about LLMs" and "I distrust AI progress because I think everyone involved with it is a rich grifter".
(03-17-2026, 07:14 AM)inkoalawetrust Wrote: Yeah that's been pretty obvious with the decline in software quality in the past few years, even beyond the pre-AI trend of enshittification, it's just getting even worse now. For example Microsoft forcing online accounts unto Windows is standard enshittification, but Windows 11 locking itself out of the system drive on certain systems* sounds like vibe coding that's for sure, and there's much more with Windows 11 in particular . It is fun though that this bug literally JUST began as of the time I'm typing this, literally brand new. But I digress.
Can you provide any evidence for this assertion beyond vibes? Why are you completely disregarding the extreme creative power AI coding agents give to non-coders? And you are talking "last few years", prior to 2021 there was no coding generative AI. I get it, every Westerner today idealizes the 80s, 90s, and 00s as having perfect handcrafted code vs modern slop but... that is not how it actually was and anyone who used 90s shovelware and shareware knows it. I created a reasonably thorough web based climate sim for planets in a few days while knowing 0 javascript
The problem with quantifying UX degradation is that it's mainly vibes to begin with, like there is no objective metric for always getting an AI overview on Google that you can't turn off, or Discord channel lists sometimes jiggling, or support having to first go through a crappy cheapo LLM a company is renting from OpenAI or Google, or sites being slower. But one quantifiable metric you could use is how much more common service downtimes are compared to the past, like how GitHub (which is all in on AI and does bad UX like having and Agents tab on all repositories you can't disable like you can with the Issues tab) now experiences sub 99.9% uptimes and has had several high profile outages just this year, including one last week. Or how GitHub's search indexer just doesn't work anymore sometimes.
Another metric would be how there's now billions of scrapers and web crawlers that are constantly DDossing basically most of the public internet, which has led to a massive increase of increasingly annoying CAPTCHAs like constantly getting hit with Cloudflare Turnstile verification pages.
Also I'm not sure what's up with the bit about Western nostalgia out of nowhere. But I'm also not denying that there wasn't crap code in the past. obviously there was a lot of crap in the past that has mostly been forgotten since, outside of the remarkably bad stuff (which plays into why people have nostalgia but I digress).
But besides that being a tangent, being able to use AI models to generate mediocre code, text, images, videos etc on a whim and with no oversight is also something that very noticeably contributes to the problem on a scale that didn't happen before.
Like going back to code, when people wrote slop programs like useless "system optimization" and "cleanup tools" back in the 2000s and 2010s, they were still mostly limited by needing at least some basic coding skills and access to boilerplate code templates for that crap. Now you can endlessly churn out shovelware like that by directly converting money into subsidized AI subscriptions and API calls.
(08-30-2026, 11:04 PM)MichaelPoole Wrote:
Quote:LLMs are bad at doing stuff too dissimilar from what they trained on, which is a problem when programming can involve really esoteric things such as coding in ancient languages like COBOL or for specific devices like custom microcontrollers. As an example, I've tried using several models sometime last year to help me with writing code for GZDoom, a distant fork of the Doom engine, and all models I used failed, except when I used Claude 3.5 Haiku through GitHub Copilot, giving it the GZDoom codebase as a source. I wanted it to write me an example file parser so that I can make a custom data file format (Because the API for it is entirely undocumented), it did make me a very basic static parser as I wanted. But it was totally broken and I had to use my already extensive knowledge of ZScript to fix it for like an hour, and I ONLY used it for that basic example.
First of all the article from the second link is clearly just written by an LLM, and second, I already know about Talkie because I follow AI stuff and consider it far more interesting than Anthropic measuring the performance of its' new slightly more benchmaxxed Claude in "Baghdad Houses of Wisdom destroyed" equivalents lol.
As for the blog post, I didn't say that LLMs cannot generalize at all, obviously they are not as thoroughly rigid as an old symbolic AI like ELIZA. So it's not that crazy that Talkie can sorta do Python code from a corpus of logical text which was kind of the ancestor to programming languages to begin with, especially when it's given examples in its' context window to extrapolate from, also not that shocking when both the third party blog post and the paper about Talkie show it still doing a lot worse than a comparable toy LM trained on modern data.
The third party blog post even ends by saying "We are not at General Relativity. We are nowhere near General Relativity. The relevant point is that the question has stopped being rhetorical. You can run the experiment."
That aside, Talkie being able to write crappy Python code with no direct knowledge of a programming language is a far more impressive example of generalization than what AI companies are doing by just hiring armies of people to write more data for their models to memorize and passing it off as generalization.
(08-30-2026, 11:04 PM)MichaelPoole Wrote:
Quote:I always see this argument about the competence of LLMs and it always just sounds like having a cure for cancer that costs a billion dollars, which is to say, it might as well not exist if apparently nobody can use it. And frankly, no LLM right now is worth 20 euros a month in actual value, I only have Gemini Pro because I decided to shill out like 5 extra euros a month because I'm already paying 3 euros a month for 200GB of Google Drive storage.
5 months after you posted that there are GPT5.6 Luna level models that run on a personal computer, like Qwen 3.8 27B? https://artificialanalysis.ai/models/qwen3-8-27b. There is also a deluge of open weight high quality models like GLM 5.3, Kimi K3, Qwen 3.8 Max etc...If it's all a bubble why is China also heavily investing into it? You realize it needs so much compute because 1.5 billion people worldwide use AI? If the "AI bros" were all so evil, why are farmers in Malawi using custom GPTs to help them with farming https://restofworld.org/2025/malawi-ulan...g-chatbot/ ?
I can also go on and on about all the new math discoveries etc. done with LLMs, the Hugging Face breach where a swarm of ~900 OpenAI agents broke through a sandbox and hacked a major company (I am sorry but dismissing this as "It saw the Terminator in its training data!" is willfull ignorance at this point, even if it is due to Terminator, which I doubt [rather, it's cause it got a test which was impossible to get 100 percent on honestly]) etc but I am afraid you will just say everyone is lying cause "they bought into the AI bubble". You also completely disregard that all of those companies started as tiny research teams during deep AI winter, not as some strawman "tech money bros". And no, these models aren't just "memorizing everything", they're big but not big enough for that and if that was the case, distilled/quantized models would be useless, they're not - 2026's 27 billion parameter models defeat 2025's trillion parameter models.
There is no magical sentience algorithm coming. AGI will come from a combo of scaling and the development of the harness system used in coding agents etc. into a full neurosymbolic (but dominated by the neural network part) system, IMO.
I also apologise if my tone comes accross as annoyed, to be frank yours sounds like a combination of "I used particular models and I see their failures as something fundamental about LLMs" and "I distrust AI progress because I think everyone involved with it is a rich grifter".
Re Qwen 3.8 27B: 5 months since I posted that, RAM prices have gotten even higher (after GPUs already did the same thing months before that, and then SSDs), making it even more impossible to run AI locally (with what RAM are you going to page the weights?). I asked Google AI Mode (lol) and it says that the minimum hardware to run Qwen 3.8 27B (without resorting to lobotomizing it through extreme quantization) is a fucking RTX 3090 or a 4090, do you know how expensive that is? Last I checked used 3090s were like 700 euros, when 4090s were still available here last year they costed OVER TWO THOUSAND EUROS.
Even going for AMD GPUs the cheapest is 900 euros (and you'd need to concede even more performance for the model to run it), with the ones that can fit it in VRAM like the 7900 XTX also being in the 1000+ euro range. And I'm handwaving the stranglehold Nvidia's CUDA has over ML anything. And again, you can forget offsetting things to RAM (which creates a dramatic slowdown even if RAM was affordable still) if you didn't have at least 32-48GB of high speed DDR4 or DDR5 before like last year.
The above paragraph is just about how practically infeasible it is to ACTUALLY run "small" LLMs as an enthusiast who doesn't bleed gold.
Re "frontier" Chinese LLMs:
First of all, models like GLM 5.3 Flash still require over 50,000 euros of hardware to run. The bigger ones like Qwen 3.8 Max and Kimi K3 require literally hundreds of thousands. And even if you use an inference engine like Colibri to run the majority of these models off your disk, it'd still be slow as shit, and would also require a 1TB+ NVMe SSD, though to be fair I just checked and those are still merely like 30 euros more expensive than my WD NVMe drive I bought five years ago is.
Second, several of those "frontier" chatbots have licenses that don't even count as open weights (setting aside that "open weights" is nothing like "open SOURCE" in the first place, despite "open weights" being called that to make you think of actually FOSS software), like Kimi K3, which has limitations if your deployment of the model on your multi-million dollar mainframes reaches either too many users or too much profit per year.
Re Chinese AI investment: To put it simply: China isn't investing the same absurd amount of money into chatbots as the US is. Mind you, they are still heavily investing in LLMs by any reasonable standard (dozens of billions of dollars are a lot!), but nowhere near the multi-trillion, we-are-burning-our-cash-reserves scale (i.e the tech giants like Google) the US is. Which is also probably why when AI investment contracts or collapses, China will not be hurt anywhere near as much as the US, because the 10-500 billion scale is the ballpark the current value of genAI is at, it's not producing trillions of dollars of value from CEOs wanting to fire the poor people that work for them, no matter how much they hope it will. And of course the way said CEOs are firing people for chatbots that can't do shit will also hurt in the long run (it already is, see GitHub, or the weekly system destroying Windows 11 updates), but that's not directly related to spending money to make sure nobody can afford a personal computer again so you can sell them cloud subscriptions instead.
Re Scam Altman committing felonies and getting away with it: There's a lot to say about that "incident" and it mostly boils down to them literally instructing it to escape from a sandbox that they either poorly set up because the trillion dollar money inferno company is run by people with the tech savviness of a senior citizen, or that they intentionally messed up so it'll circumvent the sandbox and then they can sell that as an example of how DANGEROUS their chatbots are (i.e they sure are competent, maybe you should invest when I IPO!). To be fair it's probably a mix of both.
The whole incident could've been prevented by uh, just disconnecting any ethernet cables or wireless connections from the data center they were running the LLM on, a level of insight anyone who isn't a over 70 years old could probably give you after thinking about it for like ten minutes.
I do find it kinda hilarious that HuggingFace isn't suing OpenAI for literally attacking them so they can market their next chatbot, but it's also understandable since OpenAI is the richer of the two, and apparently HuggingFace is also about to be bought by Nvidia, which is itself playing a very expensive game of hot potato with OpenAI and friends, and having it's future subsidiary fighting one of their best clients would be bad business (unlike trillions in circular financing and speculation of course).
Re current AI companies starting during the AI winter:
First of all, keep in mind that the bulk of the AI winter ended around the 2010s, when AlexNet was released, so around 2012.
Google: Not an AI company per se but they did start well in the middle of the aftermath of expert systems. And I'm pretty certain they did do AI research even in the 2000s. But even then, a lot of their AI stuff begun post AlexNet when the viability of neural nets was proven, they developed stuff like neural translation for Google Translate in the mid-2010s, and IIRC Google Brain invented transformers in the first place with being used as a translation backend in mind. Still inventing transformers was pretty much what set the current AI boom in motion.
Deepmind: This one is basically the only explicitly AI organization created during the AI winter, two years before AlexNet. This one is actually just true, and they did contribute a lot to the leadup to the AI boom before Google mutilated them beyond all recognition into not even being able to make competitive lying chatbots and getting rid of the last vestiges of useful AI research they still had in favor of pretending Gemini can do any of the work scientists can, to be fair an actual lab could probably be helped with LLMs, not in the sense than LLMs (as opposed to transformers in general, and certainly not the trained chatbot kind of LLM we all see now) are actually useful for research itself, but because they can jingle LLMs in front of rich people for grant money.
OpenAI: They were founded in 2015, well in the age of neural nets and did AI research through the final phases of the previous AI winter as technologies like GANs and DeepDream were invented. So absolutely not deep in the AI winter.
Setting it's founding date aside, OpenAI has an interesting relationship with "strawman tech bros" by being patient zero. Following DALLE-2 and then GPT-3 hitting it big, Sam Altman quickly shifted from a non-profit to a closed for profit company, remember that they released how they made models like GPT-1 and GPT-2 or DALLE 1. And yet we've never seen a public release of even DALLE-2 and GPT-3, not even as historical artifacts. Their announcements on those models went from actually explaining how they made them or even releasing them to literally just being marketing material, something I remember plenty of people being mad about! Remember when people were over the moon about Stable Diffusion because DALLE-2 was proprietary?
OpenAI under Sam Altman's leadership literally created the current farce of a "singularity" and codified all the shit tech bros are stereotyped to engage in (or actually DO engage in, like telling people they will be homeless and destitute and then being shocked when people boo them!), among all of the other vile behavior the AI companies love engaging in (like keeping everything closed, endless catastrophizing, ruining local communities, making themselves inseparable from the global economy so when they go down we all go with them, destroying the human noosphere, etc). Sam Altman immediately gave up on all the wishy washy intelligence for everyone shit or whatever their "mission" used to be the exact moment he realized he could sell DALLE-2, and GPT-3 sealed the deal.
Sam Altman is the reason we are in this hellscape and will be the reason AI development was set back by over a decade.
Meta AI: This one is more interesting because FAIR was created in 2013, so only a year after AlexNet, but again, still near the end of the AI winter.
Anthropic, Mistral, z.ai, DeepSeek, Qwen (not a company, that's Alibaba Cloud), MoonShot, MiniMax: Most of these companies were founded in the wake of AI boom, Anthropic in particular is literally an offshoot of OpenAI. Z.ai existed since 2019 (again, well in the end of the AI winter, not deep in it) but they didn't do anything of note until the AI boom.
Re LLM memorization: So why do companies like OpenAI and Anthropic spend billions to destroy books and hire people to make more information for their LLMs then? Because it sure seems that obvious lapses of logic like counting letters in words always just end up fixed in the next model release or after they make a LoRA without this or that mistake to slot into the model.
Also side note on this years' chatbots beating last years' chabots. Those benchmarks are all useless (for many reasons, like benchmaxxing, doing the benchmarks with more expensive models than the ones served, the benchmarks sometimes being written with the direct or indirect involvement of the AI companies themselves, etc), they have not actually improved in anything besides being fed more data to memorize since like, Q* (chain of thought, and apparently even that is just memorized from people hired to write their thought processes!), everything else is mainly inference optimization.
Speaking of which, I hear a lot about optimizing inference (I watch channels like bycloud pretty often) but nothing about optimizing TRAINING. But I suppose when you have zillions of dollars and can just ignore IP laws you can just buy more GPUs and data instead of trying to make a model that memorizes what a cat looks like without needing hundreds of thousands of photos of cats.
Re those blogposts: NGL I'm not feeling like reading all that all, especially not right now since I've been typing this for like two hours. What I will say though is that I'm both left wing (I'd think so at least), and way more pro-AI than most other people who could be considered left considering I use genAI and think it does have some applications, and also that the kind of rejection of technology in general I see is pretty absurd (like people getting mad even at stuff like AlphaFold), nor do I think genAI is going away even when the interest and overinvestment in it dies down. I also don't believe that data centers use that much water or that it's somehow permanently tainted, but that also doesn't negate all the other problems with them like noise, pollution from crappy gas generators, or being giant shacks with giant piles of rapidly depreciating niche hardware in them that isn't useful for anything else.
Also the third article starts by referring to "The Left", which isn't exactly the phrasing I see leftists use but whatever.
Re sentience and stuff: I agree on there probably not being any single algorithm for that, for one living sapient things like humans are giant tangled messes of billions of years of evolution. But I do think it's not super unlikely that the basics for creating a conscious thing could be relatively short and simple.
Re AGI from harnesses: No, lmao. Symbolic AI is about as far as it gets from living things. And the current AI models we make by using incredibly crude models of neurons from the 60s and by avoiding actually looking into how living things work are closer to that than symbolic AI is, but are still further from it than the gap between a fly and a human.
Training a statistical model of text to output symbols (tokens) equating to triggers for calling a function in a traditionally written, hardcoded API is not AGI. It's a tool with a really bad foundation (a statistical model of the next text symbol that only learns the associations between the symbols in its' training set). Also excuse me as a cleanse myself of using a contrastive statement that reads weirdly like something such a statistical model would puke out lol.
Your mind is software. Program it. Your body is a shell. Change it. Death is a disease. Cure it. Extinction is approaching. Fight it.
Geek Code
Oh yes, the SI article is shockingly spot on about modern machine learning despite being written in 2002. I'd like to write an article making "expert systems" (not as in the original term for 80s LISP machines and later programs like the ones they ran) a more explicitly canonical category, and as part of that I'd like to keep as much of the SI article intact and merged. Also not sure what you mean that there's no mention of the AI boom, the 2020s section literally starts by talking about it and it occurring through the 2020s lol.
Your mind is software. Program it. Your body is a shell. Change it. Death is a disease. Cure it. Extinction is approaching. Fight it.
Geek Code
(08-31-2026, 08:31 AM)inkoalawetrust Wrote: Oh yes, the SI article is shockingly spot on about modern machine learning despite being written in 2002. I'd like to write an article making "expert systems" (not as in the original term for 80s LISP machines and later programs like the ones they ran) a more explicitly canonical category, and as part of that I'd like to keep as much of the SI article intact and merged. Also not sure what you mean that there's no mention of the AI boom, the 2020s section literally starts by talking about it and it occurring through the 2020s lol.
I am sorry but I just don't see it there, see screenshot.
08-31-2026, 10:37 AM (This post was last modified: 08-31-2026, 10:46 AM by MichaelPoole.)
Quote:Re Qwen 3.8 27B: 5 months since I posted that, RAM prices have gotten even higher (after GPUs already did the same thing months before that, and then SSDs), making it even more impossible to run AI locally (with what RAM are you going to page the weights?). I asked Google AI Mode (lol) and it says that the minimum hardware to run Qwen 3.8 27B (without resorting to lobotomizing it through extreme quantization) is a fucking RTX 3090 or a 4090, do you know how expensive that is? Last I checked used 3090s were like 700 euros, when 4090s were still available here last year they costed OVER TWO THOUSAND EUROS.
Even going for AMD GPUs the cheapest is 900 euros (and you'd need to concede even more performance for the model to run it), with the ones that can fit it in VRAM like the 7900 XTX also being in the 1000+ euro range. And I'm handwaving the stranglehold Nvidia's CUDA has over ML anything. And again, you can forget offsetting things to RAM (which creates a dramatic slowdown even if RAM was affordable still) if you didn't have at least 32-48GB of high speed DDR4 or DDR5 before like last year.
I literally ran 4-bit quantization 27b models in Alpaca on a midrange i5 PC from 2021 with a RTX 3060. It was slow, took about an hour to generate a reasonable Arkanoid clone (didn't try 3.8 yet, this was Qwen 3.6 27b) but bearable. Granted, I do have 64 GB of DDR4 that I bought before. No weird tricks tho, Alpaca is literally just a buggy GUI Ollama front end. Have you actually tried to run a local model?
Quote:Also side note on this years' chatbots beating last years' chabots. Those benchmarks are all useless (for many reasons, like benchmaxxing, doing the benchmarks with more expensive models than the ones served, the benchmarks sometimes being written with the direct or indirect involvement of the AI companies themselves, etc), they have not actually improved in anything besides being fed more data to memorize since like, Q* (chain of thought, and apparently even that is just memorized from people hired to write their thought processes!), everything else is mainly inference optimization.
You are so absurdly wrong and clearly muddled with hatred for "AI tech bros" it's not even funny, I don't go by benchmarks, I actually use those agents, this year I made a climate sim in 3D while last year the best it could do in August 2026 were simple 2D games more primitive than SNES.
"Scam Altman".
You aren't criticizing the technology, you hate the people behind it, and it by extent.
