AI progress...OAUP vs RL

[MichaelPoole]

All good questions and hard to answer. I am personally a "if it quacks like a duck..." pragmatist. I find the hard problem of consciousness, the Chinese room, etc. to be very interesting but ultimately fruitless - if we separate the wavelenghts of EM radiation from "redness of red", it ultimately leads to solipsism, as one cannot know anyone but themselves actually has qualia. What makes modern AI unlikely to be conscious or at least continually conscious in a human sense is IMO mainly the prompt based nature and lack of autonomy they have. A current AI model responds to a prompt but has nothing going on when not processing a prompt. It can pass a Turing test easily at this point if the system prompt tells it to pretend to be a human (the only reason why a modern Claude or GPT model does not pass it is that it will always identify itself as a "helpful AI assistent" upon being asked. Agentic systems exist, can operate for hours independently if given permissions and are very impressive, but they ultimately use a traditional (deterministic) computer program to repeatedly re-prompt an AI model, a kind of a hack. Sakana AI is working on a continuous state machine though which IMO can eventually produce conscious beings https://youtu.be/dYHkj5UlJ_E?is=UDm-JIVG16tLFgiu

EDIT - mouse control in my Descent-style game does not work when the game is shared as a Claude artefact, my bad, this should work https://updrafta360dofgame.netlify.app/ .

Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?

I find this part of the Vots article in OA interesting:

"The development of vots dates back to the dawn of computers. Over the decades many attempts were made to develop generally intelligent software, finally resulting in the first sophont AI in the mid-Information Age. This development led to the widespread use of slaved AI which drew significant controversy. It was not until the First Federation that modosophont researchers produced the first non-sophont general artificial intelligences, against the backdrop of new Universal Sophont Rights Protocols."

I think in real life we may have the opposite path. Approaching AGI (in the purely digital realm, robotical navigation in the real world has a lot more progress to get good) by now (and if one ridicules that notion - consider that models like Opus 4.6, GPT 5.4 or hell, even open weight models like Qwen-3.5 can code whole playable games from simple text prompts, hack computer networks https://www.anthropic.com/news/disrupting-AI-espionage , discover new medication, beat math olympiads etc, hell, the old, now long beaten, "how many r's in strawberry" problem was a consequence of the way a tokenizer works anyways [basically, to save data, the AI does not "see" individual letters, but tokens which usually are 2-3 character long, to put it simply, saying it's dumb because of that is like saying humans are dumb because they fall for optical illusions] and not actually due to lack of raw intelligence, the average person can't do any of that yet there's no doubt they're generally intelligent) but likely without consciousness. Though as I said, I fear there isn't a reliable way to tell a fully autonomous Vot from a conscious being.

[inkoalawetrust]

I'll just preface this by saying that my general stance on current AI is that I'm actually fine with people using AI to do stuff like generating media and asking questions and stuff. And I myself use AI pretty often, including as early as like an hour ago (and I will most likely do something with Gemini as I write this too*). But I am incredibly skeptical of how much the potential of transformers is being overhyped by very clearly vested interests as being the kind of AI a layman actually thinks of when they hear the term. My gripes with AI are much less disliking AI art or the technologies themselves, and more being mad at the AI bubble or how opaque and proprietary the tech is for the most part.

Also I'm typing this last, but I'd write a long section here about how I think this stupid AI bubble will set AI development back by like 20 years but I'm really sleepy and I've been typing this on and off for like 2 hours so I think I'll just go to bed lol. So I'll avoid rambling on too long on this because I've done so several times on the official Discord at various points in time. And don't really feel like doing so here too.

---

Now for the OP, I checked out the archive Arik posted and the date of 2035-2038 for somewhat useful personal AI is surprisingly likely for how overly optimistic the rest of the timeline is. But the date of 2043 for human-level AI is as overly optimistic as most of the information age timeline was. No idea who thought that was reasonable lol, even without the hindsight of the AI bubble and the winter it'll produce, I don't think it'd be a reasonable estimate. So overall the estimate for full AI is too optimistic, but narrow AI for daily use is probably spot on, so the opposite of overestimating the near future and underestimating the long term future IMO.

Ok, so, many skilled developers are now basically just verifying AI generated code and for the most part, it works. In the beginning of 2025, one was lucky to vibe code a simple Python game with graphics being literally placeholder squares.:

https://x.com/bcherny/status/2004887829252317325
https://twitter.com/bcherny/status/20048...39461?s=19

Yeah that's been pretty obvious with the decline in software quality in the past few years, even beyond the pre-AI trend of enshittification, it's just getting even worse now. For example Microsoft forcing online accounts unto Windows is standard enshittification, but Windows 11 locking itself out of the system drive on certain systems* sounds like vibe coding that's for sure, and there's much more with Windows 11 in particular . It is fun though that this bug literally JUST began as of the time I'm typing this, literally brand new. But I digress.

LLMs are bad at doing stuff too dissimilar from what they trained on, which is a problem when programming can involve really esoteric things such as coding in ancient languages like COBOL or for specific devices like custom microcontrollers. As an example, I've tried using several models sometime last year to help me with writing code for GZDoom, a distant fork of the Doom engine, and all models I used failed, except when I used Claude 3.5 Haiku through GitHub Copilot, giving it the GZDoom codebase as a source. I wanted it to write me an example file parser so that I can make a custom data file format (Because the API for it is entirely undocumented), it did make me a very basic static parser as I wanted. But it was totally broken and I had to use my already extensive knowledge of ZScript to fix it for like an hour, and I ONLY used it for that basic example.

I bring this up because ZScript, ZDoom's esoteric scripting language that is a mix of other languages like UnrealScript and C#, obviously doesn't appear as much in the training data of LLMs, and while it IS like a bunch of much more popular languages, it also has a bunch of idiosyncrasies of its' own, with the syntax itself, the way the engine parses it, and all the kinks of the engine itself. For LLMs this is a dealbreaker because of how obscure it all is, despite ZScript being simple for people who already know coding to pick up, if you asked a programmer to make a ZScript mod they could likely get something basic going in like a day or two,

The reason for that is probably because, as I kinda get to below, LLMs don't actually get things, which is a problem because programming is less typing code and more solving problems. Typing out code is the easy part and often then interchangeable between languages, like a developer that can do something like write code from scratch that implements a program like The Powder Toy could figure out how to do it in Java just as well as in C++, because what matters is the knowledge of how to design an implement a falling sand game, not typing out similarly named functions or something. The inability of LLMs to handle writing working ZScript code is similar, below all the weird engine-specific kinks and whatever features the API does and doesn't have (i.e there's no built-in sorting functions for ZScript), it is still very similar to languages like C#.

This 3D Descent/Forsaken clone I made in maybe 2 hours total time, simply repeatedly prompting the Web version of Claude, using the Opus 4.6 model. Not even any agentic tool or parallel test time compute. And 1.4 hours of that was just me finetuning the details by "bothering" the AI:

https://claude.ai/public/artifacts/9e9b7...3d84e09005

It could be far, far better if I provided it actual music or textures, it had to procedurally generate everything.

Besides the mouse control scheme bug on the version of the app hosted on claude.ai, there's also a bug where on the keyboard control scheme pressing CTRL too many times closes the tab, probably a browser shortcut being triggered like Sticky Keys back in the day.

Keep in mind in Spring 2025, this is what was considered "impressive" regarding AI generated code by a layman:

https://www.youtube.com/watch?v=XEDwuh4XYvU

2023? A working game of Snake made by AI was impressive.

GPT5 (already an ancient model by AI progress standards) ran an autonomous lab and cut cell-free protein production costs by 40 percent:

https://www.youtube.com/watch?v=ql4LM8hNOA4
https://openai.com/index/gpt-5-lowers-pr...esis-cost/

Has anybody else independently verified this? Because OpenAI has an obvious interest in saying something like this, or is it like their and Sam Altman's commitment to open research once DALLE-2 made it big (And then they only released some "papers" that are just marketing material per-model before giving up on even that)? Gemini 3 Pro says no, and of course part of that is that it's proprietary bullshit that can't be easily replicated, including the tech stack of the biotech firm OpenAI partnered with, I found the biorxiv Gemini referred to but didn't link for some reason, and yep, still a preprint, at least for now. So the answer to my question seems to be "no".

Also no, GPT-5 isn't really that old at all because development has slowed down since the 2020-2024 days as progress moves up the S-curve. The progress between models is still somewhat noticeable but also much more incremental than something like the difference between DALLE-1, 2, and 3.
Pre-DALLE-1 image generation was incoherent nonsense outside of specialized GANs (Like the one used by thispersondoesnotexist.com), DALLE-1 was still mostly incoherent, DALLE-2 was dramatically more coherent (And I recall some earlier stuff it did was even better, like some straight up stock photo quality images of people, but I presume OpenAI crushed the model down to save money running it), DALLE-3 was even better, but also not quite as big of a leap as 1->2, and now frankly I don't think GPT Image is that different from DALLE-3 quality-wise. And more importantly, GPT Image is also not THAT different from other image generators like the Flux 2 series or Nano Banana Pro and Nano Banana 2, so it's not just part of OpenAI themselves shitting the bed and only being around because they basically just have brand recognition like Apple.

AI literally helping researchers discover medication to kill drug resistant bacteria:

https://news.mit.edu/2025/using-generati...teria-0814

AI solving decades old unsolved math problems:

https://arxiv.org/abs/2512.14575
https://arxiv.org/abs/2602.21201
https://www.youtube.com/watch?v=zJvuaRVc8Bg
https://www.erdosproblems.com/forum/thre...#post-3302

Re: The MIT article
I checked the paper, can't access the whole thing, but the highlights say they used genetic algorithms and variational autoencoders, and while those are ABSOLUTELY AI despite what a lot of angry people on Twitter wish. It's also not generative AI like Gemini (AKA the AI models that zillions of dollars are being pumped into for the bubble), so unless anything in the full paper says otherwise, that headline is clickbait, genetic algorithms are a form of AI that has existed for decades before transformers. But, to play devils' advocate, VAEs are much closer in association to modern generative AI, both because VAEs are only 13 years old, and because I know for a matter of fact that they are a component of at least some generative AI models, like how you can go on Civit and get VAEs to slot into image models like Stable Diffusion XL.

So basically, by all accounts these researchers didn't just use a chatbot, they used AI methods that have existed years to decades before LLMs for a narrow AI application. Not an LLM chatbot like AI companies would hope you and their shareholders to believe.

Re: The two papers
2512.14575 is not peer reviewed, something I first found by asking Gemini 3 Pro (Again) and then checking his site where it's listed as such, but at least it's got being done by an independent person who seems to have decent credentials going for it though.
2602.21201 is also not reviewed but it's also much newer so hopefully it'll be soon. But it was also done apparently by Google DeepMind using Gemini 3 DeepThink, so like the OpenAI protein paper there's a huge conflict of interest here. Unlike OpenAI though Deepmind is infinitely more trustworthy, and unlike OpenAI, their parent company literally invented the transformer architecture the vast majority of generative AI models are built on. But there's still an obvious incentive here and it's still not been reviewed then.

Re: The Terence Tao stuff
This stuff is fine I think, and I don't actually know pretty much any math to directly dispute it anyway, and I also know this guy has the exact opposite level of familiarity with math to me. And AFAIK he doesn't have any vested interest in the AI bubble like OpenAI and DeepMind to either.
But, keeping in mind I haven't watched the video myself and literally had Gemini watch it for me lmao, anyway. The video apparently is Terence talking about just using generative AI as a narrow AI tool, so the way AI has been used since forever, NOT AGI like, again, AI companies would hope you to think. Oh and also a big component of automating math work is Lean, which is a programming language thing (Which I had already heard of when watching some stuff about Tao before), you'll note that Lean has pretty much nothing to do with AI on it's own. Instead the connection is that LLMs are fine tuned into being able to vibe code Lean, and Lean, the very much symbolic-not-really-AI programming language makes sure it passes the smell test. So again, the LLMs are just a narrow AI component, and the brunt of the work isn't even handled by any form of AI to begin with.

My opinion? Maybe people just don't want to admit there's no magic "sentience algorithm" and that consciousness:

1. May not be concretely provable or disprovable, even current systems that are an outgrowth of "fancy autocomplete" (which is honestly kinda misleading as all life on Earth including humans are "fancy DNA replicators" by the same standard) already show signs of will, adversarial misaligment etc., see: https://www.anthropic.com/research/agentic-misalignment as well as some early signs of recursive self improvement, namely, being able to fine-tune themselves https://the-decoder.com/anthropic-resear...hemselves/

2. As the (great) sci-fi author Peter Watts proposed, consciousness may have nothing with intelligence and intelligence may do just fine without it. These systems write major software tools, can produce working 3D games by a bored rando prompting them in the bus on his phone, develop medication, solve hundred year old math problems, and are starting to show signs that they may be able to recursively improve themselves. It doesn't matter if they do this by "true understanding", however arbitrarily we define that (usually resorting to dualism in the end), just like a supersonic fighter jet is flying despite all natural flying creatures flapping their wings and planes not doing that.

Thus no, I don't think this is "simulated intelligence". Maybe sci-fi people should admit that maybe AI won't just be artificial people but something potentially a lot more alien.

1. LLMs show signs of being trained on large datasets where AI is depicted as hard AI that does actions such as rebel against humanity, question its' existence, be the subject of in-universe philosophical debate etc, like Terminator, or I, Robot, or Orion's Arm. Then, once they have memorized this training set (Like when I crammed Byzantine history for homework, and then promptly forgot it after the assignment because I had no clue what it actually entailed), and after being trained through RLHF** to be helpful AI assistants and get that caked in their big matrix tables, they use their incredibly potent pattern recognition to go "AI assistant, Skynet scary, 0.86919 probability Skynet comes after AI assistant".

A more concrete example I've noticed of LLMs not like, actually understanding things, and just generally not working with data that is too out of distribution. Is how I've fucked around several times with Gemini 3 Pro by throwing my Athena Project drafts at it, and every single time, it fails to read through the lines on the date that the Academion email is flagged with, the date is the exact timestamp of the Year 2038 bug, with the date followed by a veracity rating of {I}.
Even just reading the draft alone (Nevermind the fact that the entire EG was absolutely in Gemini's training set, such as it having a concept of the technocalypse and infoplagues SOMEWHERE in its' trillion+ weights), the rather obvious implication of that exact date is that the date was corrupted by the bug affecting the email timestamp, and Y11K historians ARE fairly certain that SOMETHING doesn't add up with the date (Even if the 2038 problem is probably lost to history).
But every time Gemini reads it, it thinks it's just a reference to the bug with little more meaning, or rather, it thinks that if the copy of the draft I send it includes the Out of universe note where I directly link the Wikipedia article for the bug. If I don't include the note with the link it doesn't catch the VERY SPECIFIC date at all! And this is probably at least partly because the year 2038 somewhat lines up with the timeline by falling in the middle of the first AI revolution, where presumably after the bubble pops, some research project without motive for immediate profit invents less-stupid AIs that DON'T trip on this kind of thing. If I give it the draft with the note included, but change the date to January 1st 1970 it'd probably get the meaning of the date.

2. I don't disagree with this, and this is also more or less OA's take on the matter anyway (This is what vots are). But I do disagree on the competence of LLMs of course, they are dumb as rocks and consciousness being separable from intelligence doesn't change the fact that transformers just memorize a bunch of data and find patterns without actually understanding them.
Using the plane example, you don't need to flap your wings to fly, but a biplane with a top speed of 5 km/h and a 90% chance of crashing after one minute of flight isn't exactly beating any birds (Including the ones that just walk like Kiwis) in locomotion, it is terrible at flying regardless of how it does it. And the current "solution" marketed to inflate the AI bubble isn't to develop wings that don't have holes in them, but more "If we make REALLY big wings full of holes, it'll turn supersonic! Now give me 100 billion dollars".

As for stuff like people getting ChatGPT to miscount r's in strawberry or the same old 2024 AI Google Search screenshots telling people to eat rocks, keep in mind these 2 things please:

1. In general computer development these days, 7 years is almost nothing, Dennard scaling broke down so one can easily use a computer nearly a decade old and still run quite high end applications on it. Not in AI - 2022 AI models are uncomparably more primitive in capability than modern ones, to the point it's more like comparing a 1970s PC to a 2020s one. Original ChatGPT could be easily gaslit to think 2+2=5, got basic math wrong often, hallucinated frequently and even at things it was good at it was at best like a smart high schooler. Today's AI models are incomparable. So no, if you used AI in 2023 when first ChatGPT came out, I hope I'm not coming off rude, but you don't have accurate idea of AI capability in March 2026.

2. There is a HUGE gap between professional AI models that cost ~200 USD a month, premium consumer models that cost ~20 USD a month (my personal experience is with those) and free models that generally have chain of thought thinking minimized or disabled (keep in mind chain of thought is a December 2024 advance). And the biggest gap is between the free models vs ANY paid models. People dismiss news about GPT models discovering drugs or solutions to decades unsolved math problems because they see "GPT5 did x" and think their free account on chatgpt.com that miscounts letters is like what those scientists are using. Of course they aren't, they use ChatGPT Pro, agentic AI like Claude Code, API usage where any app can harness high quality AI but the really powerful commercial models charge heavy bucks per token etc...

Do I think current Claude, Gemini or GPT models are people and that we will get a rapid singularity and everyone becomes gods? No. But I don't think this is some "dead end" and that "real AI" will be some scifi-esque "sentience algorithm" that will conveniently produce beings like us. This IS real AI.

1. Not much to say that I didn't say already using DALL-E as an example. I will say though that it doesn't have to be this way, but since AI as a field is now at the helm of a bunch of shareholders and grifters, well, it's much easier to sell immediate earnings to investors by saying you just need to build a bigger hammer and it WILL make a zillion dollars, than it is to tell them that we'll need to figure out how to design a swiss army knife, which'll take, god forbid, more than two fiscal quarters, and might even fail, and if it doesn't, we aren't too sure how profitable it'll be anyway.
Every image model is already being trained on all images on the internet, every text model on all internet text, and so on (And multimodal models on combinations of those), scraping that data and then marketing "synthetic data" (Having AI models make shit up) as a solution to running out of scraped data is easier to sell than needing to figure out how you or I can learn to draw with like 20 tutorials instead of all of DeviantArt and ArtStation.
2. Maybe AI companies could use those trillions of dollars to make more efficient models that cost them less, or you know they could just leave that to hobbyists and much smaller companies like they are currently, it's not like the gravy train will ever derail and if it does the government will bail them out anyway at the expense of the world. Maybe if they did that I could even one day afford a goddamn RAM stick ever again.

I always see this argument about the competence of LLMs and it always just sounds like having a cure for cancer that costs a billion dollars, which is to say, it might as well not exist if apparently nobody can use it. And frankly, no LLM right now is worth 20 euros a month in actual value, I only have Gemini Pro because I decided to shill out like 5 extra euros a month because I'm already paying 3 euros a month for 200GB of Google Drive storage.
And I say this as someone who uses Gemini pretty much every day, including for getting reports on stuff I wanna look into and even used it for proof reading the email section on my Athena Project draft.

Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?

Yes, there's reliable methods to determine if something has a subjective experience in OA, especially when it comes to transapients looking down at lower S-levels.

* Yes, I did, lol. I asked Gemini to give me a list of Windows 11 update bugs, which is also how I found that now it just flat out looks you out of the system on certain setups. I used the fast model instead of Gemini 3 Pro though.
** Where you get a bunch of humans to steer the AI through training. A process which AFAIK big AI companies do through the very futuristic method of basically hiring poor people for like 2 dollars an hour, very automated, truly the jobs of everyone but the C-suites are now obsolete.

[stevebowers]

Also, how are Vots distinguished from sentient beings in the OA universe? As in, is there any truly reliable way to ascertain qualia/subjective experience in the OA universe?

My favoured (hypothetical) method would be to physically link two entities together via direct data linkages. If you link two fully sophont, conscious entities together on a comprehensive level, they should both be able to experience each others qualia and memories, leading to the realization that both are conscious and sophont. But if you link a sophont mind with a vot, the sophont mind should be able to discriminate between their own (fully sophont) experience and the internal processing of the non-sophont vot.

I have no clear idea exactly how this difference would manifest itself, but I assume that the experiences associated with the vot would have no subjectivity and none of the essential qualities associated with consciousness. Similarly a transapient or archai could link up with a vot on an even more comprehensive level and perform the same test, but with much more confidence and less chance of a false positive. The linkage is the test.