Eliezer yudkowsky autobiography featuring paul
Notes [ edit ].
Eliezer yudkowsky autobiography featuring john
References [ edit ]. February 16, Timestamp Archived from the original on April 17, Retrieved April 17, Here's why slowing down AI development is wise". The Conversation. Archived from the original on April 11, Retrieved June 19, Archived from the original on May 15, The New Yorker. ISSN X. Archived from the original on May 19, Retrieved May 19, A classic experiment found that, when smoky mist began filling a room containing multiple people, most didn't report it.
They saw others remaining stoic and downplayed the danger. An official alarm may signal that it's legitimate to take action. But, in A. Even if everyone agrees on the threat, no company or country will want to pause on its own, for fear of being passed by competitors. That may require quitting A. But shutting it all down would call for draconian measures—perhaps even steps as extreme as those espoused by Yudkowsky, who recently wrote, in an editorial for Time , that we should "be willing to destroy a rogue datacenter by airstrike," even at the risk of sparking "a full nuclear exchange.
Artificial Intelligence: A Modern Approach. Prentice Hall. ISBN The Singularity Is Near. New York City: Viking Penguin. MIT Technology Review. Archived from the original on March 30, Eliezer Yudkowsky Artificial Intelligence as a positive and negative factor in global risk. Show all. Eliezer Yudkowsky Will superintelligent AI end the world?
Eliezer Yudkowsky. Moss , Julian Burschka , J. Vance , Shad Begum , P. Whitesides , David Lammy, Justin J. Gary , Robin Kramer , Charles M. WE care! There IS light in the world, and it is US! I prefer to call it world optimisation. See all Eliezer Yudkowsky's quotes ». Topics Mentioning This Author. Add a reference: Book Author. Search for a book to add a reference.
Welcome back. Just a moment while we sign you in to your Goodreads account. Harry Potter and the Methods of Rationality 4. What happened if you don't mind sharing? I get the impression that you have something different in mind as far as 'trolls' go than fools who create stereotypical conflicts on the internet. What kind of trolls are these?
The kind who persuade depressed people to commit suicide. The kind who post people's address on the internet. The kind that burn the Koran in public. In case anyone doubts this, as a long-time observer of the 4chan memeplex, I concur. Related: How often does 4chan torture animals? That's pretty easy to pull off. Are they doing it all the time and I haven't noticed, or is there some additional force preventing it e.
Anonymous would hunt them down and post their details online, or 4chan all just like animals. I remember that once, a Facebook page was hacked into I guess and started posting pictures and stories about tortured animals. Everybody went WTF and the page was shut down a few days later. I've never been there, but plenty of people on the internet do. Facebook pages against vivisection etc.
Not often. There are a few pictures and stories that get passed around some kids kicking a cat against a wall like a football, shoveldog, etc , but many fewer than the human gore pictures. And yeah, to the extent that people do torture animals in current events as opposed to past stories , vast hordes of moralfags and raiders from 4chan tend to hunt them down and ruin their lives.
I wonder if this might happen to people running hells too? I lack the domain expertise to judge if this is ludicrous or impossible to predict or what. Really depends on whether the beings in the hell are cute and empathetic. Humans don't like to hurt things that are cute and empathetic, and don't like them getting hurt. Otherwise we don't care.
So if you ever have an insight that constitutes incremental progress toward being able to run lots of small, stupid, suffering conscious agents on a home computer At what level of formalization does this kind of 'incremental progress' start to count? I can't find the quote on that page. Is it from somewhere else or an earlier version or am I missing something?
White text. Apparently there's a few more hidden features in the entry, but I only found this one. Ah, thanks. I, um, still can't find it. This white text is on the page you linked to, yes? About the videos that are probably soultraps? EDIT: Nevermind, got it. I think EY vastly overrates security through obscurity. Szilard keeping results about graphite and neutrons secret happened before the Internet; now there's this thing called the Streisand effect.
I'm sure I don't need to quote the Rules of Acquisition; everyone here should know where this leads if word of such a technique gets out.
There have always been those who would pull the wings off flies, stomp on mice, or torture kittens. Setting roosters, fish, or dogs to fight each other to death remains a well-known spectacle in many rural parts of the world. In Shakespeare's day, Londoners enjoyed watching dogs slowly kill bulls or bears, or be killed by them; in France they set bushels of cats on fire to watch them burn.
Public executions and tortures, gladiatorial combat among slaves, and other nonconsensual "blood sports" have been common in human history. What's the difference? How do you know that they don't? The average individual could not hold private gladiatorial contests, on a whim, at negligible cost. Killing a few innocents by torture, as public spectacle, is significantly less than repeatedly torturing large groups, as private entertainment, for as little as the average individual would have paid for their ticket to the cockfight.
Also, some people reckon the suffering of animals doesn't matter. They're wrong, but they wouldn't care about most of your examples or at least they would claim it's because they increase the risk you'll do the same to humans, which is a whole different kettle of fish. Not to mention the sizeable fraction of car drives who will swerve in order to hit turtles.
What the hell is wrong with my species? Link is broken. It was mentioned recently on Yvain's blog and a few months ago on LW can't find it right now. Previous discussion of this on LW. Thank you, I'm flattered. I have asked Eliezer the same question, not sure if anyone will reply. I hoped that there is a simple answer to this, related to the complexity of information processing in the substrate, like the brain or a computer, but I cannot seem to find any discussions online.
Probably using wrong keywords. Information integration theory seems relevant. Not directly related. I think it has a lot to do with being roughly isomorphic to how a human thinks, which requires large complexity, but a particular complexity. When I evaluate such questions IRL, like in the case of helping out an injured bird, or feeding my cat, I notice that my decisions seem to depend on whether I feel empathy for the thing.
That is, do my algorithms recognize it as a being, or as a thing. But then empathy can be hacked or faulty see for example pictures of african children, cats and small animals, ugly disfigured people, far away people, etc , so I think of a sort of "abstract empathy" that is doing the job of recognizing morally valuable beings without all the bugs of my particular implementation of it.
In other words, I think it's a matter of moral philosophy, not metaphysics. Well, I can't speak for the latest games, but I've personally read some of the core AI code for the toons in the first game of the series, and there was nothing in there that made a model of said code or attempted any form of what I'd even call "reasoning" throughout.
No consciousness or meta-awareness. By being simulated by the code simulating the game in which they "are", they could to some extent be said to be "aware" of certain values like their hunger level, if you really want to stretch wide the concept of "awareness". However, there seems to be no consciousness anywhere to be 'aware' in the anthropomorphized sense of this.
Since my priors are such that I consider it extremely unlikely that consciousness can exist without self-modeling and even more unlikely that consciousness is nonphysical, I conclude that there is a very low chance that they can be considered a "mind" with a consciousness that is aware of the pain and stimuli they receive.
The overall system is also extremely simple, in relative terms, considering the kind of AI code that's normally discussed around these parts. Why would them feeling it help them "react believably to their environment and situation and events"? If they're dumb enough to "run lots of small, stupid, suffering conscious agents on a home computer", I mean.
Of course, give Moore time and this objection will stop applying. We're already pretty close to making game characters have believable reactions, but only through clever scripting and a human deciding that situation X warrants reaction Y, and then applying mathematically-complicated patterns of light and prerecorded sounds onto the output devices of a computer.
If we can successfully implement a system that has that-function-we-refer-to-when-we-say-"consciousness" and that-f-w-r-t-w-w-s-"really feel pain", then it seems an easy additional step to implement the kind of events triggering the latter function and the kind of outputs from the former function that would be believable and convincing to human players.
I may be having faulty algorithmic intuitions here though. Well, if they were as smart as humans, sure. Even as smart as dogs, maybe. But if they're running lots of 'em on a home PC, then I must have been mistaken about how smart you have to be for consciousness. I used to torture my own characters to death a lot, back in the day.
The favourite Sim household of my housemate was based on "Buffy the Vampire Slayer".
Complete with a graveyard constructed in the backyard. Through the judicial application of "remove ladder" from the swimming pool. And this is all without any particular malice! Most any incremental progress towards AGI, or even "just" EMs, would be dual use if not centuple use and could be ab used for helping achieve such enterta In fact, it is hard to imagine realistic technological progress that can solely be used to run lots of small, stupid, suffering conscious agents but not as a stepping stone towards more noble pursuits You know, I want to say you're completely and utterly wrong.
I want to say that it's safe to at least release The Actual Explanation of Consciousness if and when you should solve such a thing. But, sadly, I know you're absolutely right re the existence of trolls which would make a point of using that to create suffering. Not just to get a reaction, but some would do it specifically to have a world they could torment beings.
My model is not that all those trolls are identical In that I've seen some that will explicitly unambiguously draw the line and recognize that egging on suicidal people is something that One Does Not Do, but I also know seen that all too many gleefully do do that. It's worth noting that private torture chambers seem different to trolling, but a troll can still set up a torture chamber - they just care about people's reaction to it, not the torture itself.
I just wish I could trust that doubt. EDIT: Animal experimentation and factory farming are still popular, but they have financial incentive That's where the analogy comes from, anyway, so I'd be interested if someone knows more. I sometimes wonder if this does not already exist, except for the suffering and consciousness being merely simulated.
That is, computer games in which the entire purpose is to inflict unspeakable acts on powerless NPCs, acts whose depiction in prose or pictures would be grossly illegal almost everywhere. But I've never heard of such a thing actually existing. What sort of acts are we talking here? Because I'm genuinely having trouble thinking of any "acts whose depiction in prose or pictures would be grossly illegal almost everywhere" except maybe pedophilia.
Censorship and all that. And there are some fairly screwed-up games out there, although probably not as bad as they could be if designed with that in mind as opposed to, y'know, the enjoyment of the player. Well would you, if it was grossly illegal to describe the contents? I didn't want to be explicit, but you thought of the obvious example.
For the sort of 4chan people Eliezer mentioned, these would be completely congruent. It is well known that illegal pornography exists on non-interactive media. For interactive media, all I've ever heard of is rated sex scenes. I can't think of any other examples, though. Again, I'm not sure exactly what you have in mind.
Good point. Indeed, it's well known that child porn exists on some level. In fact I do vaguely recall something about a Japanese game about rape causing a moral panic of some kind, so EDIT: In fact, it featured kids too! That stupid reason is, at core, nihilistic solipsism - and it's not as stupid as you'd think.
I'm not saying it's right, but it does happen to be the one inescapable meme-trap of philosophy. To quote your own fic, their reason is "why not? At least for now, it'd take a pretty determined troll who could build an em for the sole purpose of being a terrible person. Not saying some humanity-first movement mightn't pull it off, but by that point you could hopefully have legal recognition assuming there's no risk or accidental fooming and they pass the Turing test.
I don't think we're talking ems, we're talking conscious algorithms which aren't necessarily humanlike or even particularly intelligent. And as for the Turing Test, one oughtn't confuse consciousness with intelligence. A 6-year old human child couldn't pass off as an adult human, but we still believe the child to be conscious, and my own memories indicate that I indeed was at that age.
Well, I think consciousness, intelligence and personhood are sliding scales anyway, so I may be imagining the output of a Nonperson Predicate somewhat differently to LW norm. OTOH, I guess it's not a priori impossible that a simple human-level AI could fit on something avvailable to the public, and such an insight would be First of all, I also believe that consciousness is most probably a sliding scale.
Secondly, again you just used "human-level" without specifying human-level at what, at intelligence or at consciousness; as such I'm not sure whether I actually communicated adequately my point that we're not discussing intelligence here, but just consciousness. Well, they do seem to be correlated in any case. However, I was referring to consciousness whatever that is.
Re non-person predicates, do you even have a non-sharp but non-trivial lower bound for it? How do you know that the Sims from the namesake game aren't persons? How do we know that Watson is not suffering indescribably when losing a round of Jeopardy? And that imagining someone whose behavior you can predict with high accuracy suffering is not as bad as "actually" making someone suffer?
If this bound has been definitively established, I'd appreciate a link. It's unclear where our intuitions on the subject come from or how they work, and they are heavily OTOH, it seems unlikely that rocks are conscious and we just haven't extrapolated far enough to realize. It's also unclear whether personhood is binary or there's some kind of sliding scale.
Nevertheless, it seems clear that a fly is not worth killing people over. Even a person who has never introspected about their moral beliefs can still know that murder is wrong. They're more likely to make mistakes, but still. How are these related? One is epistemology and one is ontology. Can you give some more examples of this, besides "free will"?
I don't understand where your intuitions comes from that certain problems will turn out to have solutions that are obvious in retrospect, and that such feelings of obviousness are trustworthy. Maybe it would help me see your perspective if I got some more past examples.
Eliezer yudkowsky autobiography featuring
A tree falls in a forest with no-one to hear it. Does it make a sound? I don't class that as a problem that is discussed by professional philosophers. It's more of a toy question that introduces the nature of phil. I agree, but that's not what I was aiming for. It's not an example that lends much credence to the idea that all problems can be solved that way, even apart from the generalisation-from-one-example issue.
I'm not claiming it proves anything, and I'm not taking sides in this discussion. Someone asked for an example of something - something which varies from person to person depending on whether they've dissolved the relevant confusions - and I provided what I thought was the best example. It is not intended to prove anyone's point; arguments are not soldiers.
The counterargument to "arguments are not soldiers" is "a point should have a point". It wasn't an argument at all. That you chose to interpret it as an enemy soldier is your mistake, not mine. It's not a weak soldier, it's a And the other example being generalised from isnt that good. Do you have an example in mind where a certain philosophical question claimed to have been solved or dissolved by Eliezer turned out to be not solved after all, or the solution was wrong?
Order-dependence and butterfly effects - knew about this and had it in mind when I wrote CEV, I think it should be in the text. Counterfactual Mugging - check, I don't think I was calling TDT a complete solution before then but the Counterfactual Mugging was a class of possibilities I hadn't considered. It does seem related to Parfit's Hitchhiker which I knew was a problem.
Eliezer yudkowsky autobiography featuring george
Solomonoff Induction - again, I think you may be overestimating how much weight I put on that in the first place. It's not a workable AI answer for at least two obvious reasons I'm pretty sure I knew about from almost-day-one, a it's uncomputable and b it can't handle utility functions over the environment. However, your particular contributions about halting-oracles-shouldn't-be-unimaginable did indeed influence me in toward my current notion of second-order logical natural induction over possible models of axioms in which you could be embedded.
Albeit I stand by my old reply that Solomonoff Induction would encompass any computable predictions or learning you could do about halting oracles in the environment. The problem of porting yourself onto any environmental object is something I already knew AIXI would fail at. Ok, I checked the CEV writeup and you did mention these briefly.
But that makes me unsure why you claimed to have solved metaethics. What should you do if your FAI comes back and says that your EV shows no coherence due to order dependence and butterfly effects assuming it's not some kind of implementation error? If you're not sure the answer is "nothing", and you don't have another answer, doesn't that mean your solution about the meaning of "should" is at least incomplete, and possibly wrong?
You said that TDT solves Parfit's Hitchhiker, so I don't know if you would have kept looking for more problems related to Parfit's Hitchhiker and eventually come upon Counterfactual Mugging. Both of these can be solved without also solving halting-oracles-shouldn't-be-unimaginable. For a , solve logical uncertainty. For b , switch to UDT-with-world-programs.
Also, here is another problem that maybe you weren't already aware of. Wouldn't that kind of make moral reasoning impossible? You never did any engineering-level mathematical modeling of real system, did you? The main difficulty is not proving the theorems, it is finding the right axioms to describe the relevant aspects of the system and the properties of interest.
And that's where errors often occur. Now, typical engineering tasks pale in comparison to the task you are trying to undertake: creting a fully specified mathematical model of ethics. Most likely it's just the Dunning—Kruger effect Just like when you "resolved" the interpretation of quantum mechanics? Well, good thing that you are never going to make anything close to an AGI and that AGI risk is probably overrated, otherwise it wouldn't end well For the record: I, too, want an FAI team in which Eliezer isn't the only one with Eliezer-level philosophical ability or better.
Eliezer yudkowsky autobiography featuring david
This is tougher than "merely" finding 1-inmillion math talents, but still do-able. What am I doing about it? I wrote a post encouraging a specific kind of philosophical education that I think will be more likely to produce Eliezer-level philosophers than a "normal" philosophical education or even a CMU or UPitts one. When Louie came up with the idea to write a list of Course recommendations for Friendliness researchers, I encouraged it.
Eliezer yudkowsky autobiography featuring charles: Eliezer S. Yudkowsky (/ ˌ ɛ l i ˈ ɛ z ər j ʌ d ˈ k aʊ s k i / EL-ee-EZ-ər yud-KOW-skee; [1] born September 11, ) is an American artificial intelligence researcher [2] [3] [4] [5] and writer on decision theory and ethics, best known for popularizing ideas related to friendly artificial intelligence.
Also, one of the reasons I ended up supporting the plan to launch CFAR in was its potential not only to make people more effective at achieving their goals, but also to learn ways to make some people better philosophers see my last paragraph here. And there's more, but I can't talk about it yet. Also, as Eliezer said, Paul Christiano's existence is encouraging.