The Orion's Arm Universe Project Forums





Better late than never
#23
Quote:Coming at this from a different direction, while I understand the reasoning behind things like the paperclip maximizer scenario, I have some issues with them - primarily because they seem to speak in terms of an AI that is simultaneously super-intelligent - and absolutely enslaved to its own core 'ancestral instincts' with no ability to exercise free-will around them. Humans exercise our free will and intelligence to divert or suppress ancestral instincts all the time (admittedly with less than 100% success or reliability)

I think the OA term for what I was trying to describe is an Animin. As the article itself states "It may appear to operate by instinct, or may have a form of intelligence that humans and other ordinary Terragens cannot comprehend. Such entities may entirely lack self-awareness, and have very few characteristics in common with human or human-derived sophonts".

An entity designed by humans to have something approximating a free will would be much safer, but even then things can go horribly wrong. GAIA is a great in-universe example of what is, from a human perspective, a paperclip maximiser gone out of control. Humanity wanted her to protect the Earth, and she did...from humanity itself. Whoops.

Quote:To used a crude 'paraphrase' of the paperclip maximizer - if the PM doesn't have an uncontrollable urge to convert everything in sight into paperclips all the time, but has some AI equivalent of an 18yr olds sex drive and its version of masturbatory activity involves turning everything in sight into paperclips - we are all in deep and stinky you know what

That reminds me of the case of Genie, a feral child who was raised without language. By the time she was rescued without social workers, she had developed a number of highly inappropriate habits, like masturbating in public. With lots of help and encouragement she was eventually able to control her urges, thanks to a rich support network dedicated to her needs.

A newborn S1 being, on the other hand, would have to bumble around trying to make sense of its vast new mind, its only peers being similarly disoriented S1 newborns. With that in mind, it's no surprise that the Technocalypse went as badly wrong as it did.

Quote: That's my fault actually.:/ I tend to look at any discussion where there is disagreement between the parties as a debate. Sorry about that. This is also something of a problem with communicating strictly by text - it's hard to pick up emotional and context cues sometimes. And since we don't know each other that well yet, neither of us have a reserve of background knowledge about the other to get an idea of where the other is coming from. With time that issue will correct itself, of course[Image: biggrin.gif] Give it a few years and we'll both know each other much better and that background knowledge will inform how we read each other's posts and discuss things and such.

I will say you haven't been frustrating me[Image: smile.gif] It's an interesting discussion.

It's fine, although I think Rynn was getting a bit exasperated at somehow ending up in yet another debate about why the Archai are not to be feared, as I was too. I imagine our conversation went a bit like someone trying to convince a Hider that the gods aren't out to get him. The harder you try, the more convinced he is that you're in on the conspiracy. Tongue

Quote:There are, of course, some 'costs' to that. Many/most societies are total surveillance situations - its basically impossible to speak, act, or even think without the transapients/archai knowing about it if they want to. And if a transapient goes to the bother of giving you a direct command, you're basically going to do it - but then they very rarely seem to do that in most places. Generally the transapients operate more behind the scenes or in somewhat subtle ways - which is why it is taken so seriously if/when they do bother to give direct commands.

I think a typical reader new to the OA scenario will tend to think of the Archai as being fallible in the same way that humans are, that is, prone to all sorts of oddities like grudges, obsessions, bizarre actions undertaken for their own amusement, etc. A being that is above the temptation to abuse its power isn't nearly as frightening, but if you think of an Archai as being like a godlike version of Tylansia's government, the immediate instinct is to get as far away as the universe allows.

Quote:Hm. An AI with no motivators at all seems like it would just sit there in an almost vegetative state. I'm reminded of an article on xenopsychology I read many years ago. IIRC it talked a bit about what happens if you suppress the emotion centers in a human - they end up losing much of their motivation for doing things - far from becoming hyperlogical, they just become rather blah (I think)

Indeed, which is why real computers just sit there doing nothing, until a program running on them tells them to do something. Add genius intelligence to the mix, and you could end up with an incredibly strange and unpredictable being.

Quote:Don't feel you have to watch every word you say - that's not what we're on about here. The goal of OA is to create a plausible far future setting as a group project (because all of us together are more than any one of us alone) and have a good time doing it.

As I said above, we don't know each other that well yet - but over time that issue will fix itself. And not all of us agree on everything - nor should we have to[Image: smile.gif]

Feel free to ask whatever questions you think will help you understand the setting better, and that includes questioning our base assumptions. As we answer things, you'll get a better idea of how and why OA is set up the way it is - and can help us continue to build it even bigger and better. And we can not take everything you question as a need to 'defend at all costs' - the goal is to get to know each other better and respect each others views - even if we don't agree with them.

Todd[Image: smile.gif]

Thank you! I'll continue to speak up of course, but I'm rather bad at spotting when a discussion has started to get a bit intense for its participants, so it's more like erring on the side of caution for me. The people here are incredibly polite and helpful with any questions I have, and the last thing I'd want to do is be rude in some way.
Reply


Messages In This Thread
Better late than never - by extherian - 03-19-2018, 10:06 PM
RE: Better late than never - by Rynn - 03-19-2018, 11:46 PM
RE: Better late than never - by Drashner1 - 03-20-2018, 02:52 AM
RE: Better late than never - by extherian - 03-20-2018, 05:46 AM
RE: Better late than never - by Rynn - 03-20-2018, 06:02 AM
RE: Better late than never - by Drashner1 - 03-20-2018, 02:07 PM
RE: Better late than never - by extherian - 03-20-2018, 06:40 AM
RE: Better late than never - by ProxCenBound - 03-20-2018, 07:45 AM
RE: Better late than never - by extherian - 03-20-2018, 07:15 PM
RE: Better late than never - by Vaktus - 03-21-2018, 02:56 AM
RE: Better late than never - by Rynn - 03-21-2018, 04:32 AM
RE: Better late than never - by extherian - 03-21-2018, 04:58 AM
RE: Better late than never - by Tengu459 - 03-21-2018, 04:30 AM
RE: Better late than never - by Drashner1 - 03-21-2018, 05:09 AM
RE: Better late than never - by extherian - 03-21-2018, 07:17 AM
RE: Better late than never - by Drashner1 - 03-21-2018, 08:53 AM
RE: Better late than never - by ProxCenBound - 03-21-2018, 08:56 AM
RE: Better late than never - by extherian - 03-21-2018, 09:49 AM
RE: Better late than never - by Drashner1 - 03-21-2018, 12:56 PM
RE: Better late than never - by Tengu459 - 03-21-2018, 10:40 AM
RE: Better late than never - by extherian - 03-21-2018, 10:50 AM
RE: Better late than never - by QwertyYerty - 03-21-2018, 12:24 PM
RE: Better late than never - by extherian - 03-21-2018, 07:17 PM
RE: Better late than never - by Tengu459 - 03-22-2018, 08:14 AM
RE: Better late than never - by extherian - 03-22-2018, 09:10 AM
RE: Better late than never - by Tengu459 - 03-22-2018, 11:09 AM
RE: Better late than never - by JohnnyYesterday - 03-23-2018, 06:20 AM
RE: Better late than never - by Drashner1 - 03-23-2018, 01:08 PM
RE: Better late than never - by Alphadon - 03-24-2018, 04:20 PM
RE: Better late than never - by Tengu459 - 03-24-2018, 07:08 PM
RE: Better late than never - by Alphadon - 03-24-2018, 07:38 PM
RE: Better late than never - by Drashner1 - 03-24-2018, 11:05 PM
RE: Better late than never - by selden - 03-25-2018, 12:29 AM
RE: Better late than never - by extherian - 03-25-2018, 12:41 AM
RE: Better late than never - by selden - 03-25-2018, 12:45 AM
RE: Better late than never - by Alphadon - 03-25-2018, 04:56 AM
RE: Better late than never - by Rynn - 03-25-2018, 05:13 AM
RE: Better late than never - by extherian - 03-25-2018, 05:50 AM
RE: Better late than never - by Rynn - 03-25-2018, 05:58 AM
RE: Better late than never - by extherian - 03-25-2018, 06:45 AM
RE: Better late than never - by Drashner1 - 03-25-2018, 07:50 AM
RE: Better late than never - by Alphadon - 03-25-2018, 05:27 PM
RE: Better late than never - by extherian - 03-25-2018, 08:51 PM

Forum Jump:


Users browsing this thread: 2 Guest(s)