I guess history will prove if you are right.
If I am, there will be no opportunity to feel any satisfaction.
I guess history will prove if you are right.
If I am, there will be no opportunity to feel any satisfaction.
I was responding to paperclip maximizer thought experiment.
Well then you're specifically responding to P1. This premise assumes ASI exhibits goal-oriented behavior toward an unknown, arbitrary terminal goal. And I'm only asserting that it is likely (not definite) that this will cause permanent, severe harm to humanity.
But this depends on goal entirely. AI can also have goal to protect humans.
That's not an arbitrary goal. P1 is that if the goal is arbitrary, it is likely to cause severe, permanent harm to humanity. "Protect humans" is a very narrow goal and of all random goals that seem like it, most of them aren't actually what we want. The goal could be "keep humans alive" (and it will depower us to stop us from killing ourselves). The goal could be "keep things that share x, y, z characteristics with humans alive" and it will kill us and replace us with obedient dolls. The vast majority of random goals, like "turn things into cubes" require depowering humans to optimize, since they wouldn't align exactly with what humans want. There's a good chance that to an AI, we're just made of atoms it can use for something else.
It may seem like a goal driven behavior, but it's not goal at all.
This isn't responsive to P1, as the paperclip maximizer experiment isn't trying to prove the AI will exhibit goal oriented behavior. If you want to challenge that, address my arguments for P2 and P3. You just said "AI doesn't exhibit goals" but I already explained why they do when it gets to more generalized models that are doing complicated tasks, which current models already do.
Maybe future AI won't be like this, but current AI is literally just calculator which calculates probability of next token.
So, this is challenging P3 I believe? This isn't true. We know how calculators are coded. Saying the AI is a calculator is like your brain is a calculator, but the AI can simulate reasoning, scheming, and work toward specific abstract goals. It can solve word problems, but calculators can't. Additionally, it's not just trained by predicting the next token, RLHF and other training methods are also used on inputs. Additionally, new AI models are used to perform tasks on computers autonomously.
I think this would work better if you were more specific about which premise you were attacking for each of your points, and addressed the arguments I made under that premise. I made a whole point about using deductive logic specifically based on our discussions around it, but formatting my argument that way is kinda pointless if you don't engage premise by premise. For instance, P1 is specifically assuming an arbitrary goal, and P3 addresses why we should expect goal-oriented behavior specifically. I don't think you're engaging with the actual substance. Maybe label your responses as like "response to P1" and then quote the part of my argument that you are responding to.
Much of my argument is based on Eliezer Yudkowsky's essay here, Paul Cristiano's essay here, and information elsewhere on LessWrong. Also, ironically, on asking Gemini AI to help make sense of their arguments. A lot of these essays were a bit confusing to me when I first read them, so I'll put my argument in the form of a syllogism, since I think that will make it more clear how the conclusion follows. Hopefully this makes the conversation easier, as you can specify which premise or premises in particular you object to.
Definitions:
ASI: "Artificial Superintelligence," or a form of artificial intelligence that vastly surpasses human intelligence in every domain.
P1: If ASI exhibits goal-oriented behavior toward an unknown, arbitrary terminal goal, it will likely cause severe, permanent harm to humanity.
P2: If ASI emerges within the next 30 years and exhibits goal-oriented behavior, it is likely to be toward an unknown, arbitrary terminal goal.
P3: If ASI emerges within the next 30 years, it is likely to exhibit goal-oriented behavior.
P4: ASI is likely to emerge within the next 30 years.
C1: Therefore, ASI is likely to exhibit goal-oriented behavior.
C2: Therefore, ASI is likely to exhibit goal-oriented behavior toward an unknown, arbitrary goal.
C3: Therefore, ASI is likely to cause severe, permanent harm to humanity.
Defending P1: If ASI exhibits goal-oriented behavior toward an unknown, arbitrary terminal goal, it will likely cause severe, permanent harm to humanity.
This is probably best explained by the paperclip maximizer thought experiment. If an ASI has an arbitrary goal (like maximizing the number of paperclips), it can do a better job of this if humans cannot shut it off. So the actions that work best toward an arbitrary terminal goal include either permanently depowering humans or perhaps killing them all. An ASI could easily deceive humanity about its intentions and pretend to be cooperative and then kill humans and take control of human infrastructure in a myriad of ways. Eliezer, for example, has detailed how it would be physically possible for an ASI to synthesize advanced nanomachinery and use it to kill all humans at once. Even if only used AI as a chatbot, Yudkowsky's AI-box experiment has also shown that it would probably not require ASI-level intelligence to convince a human to give the ASI access to additional infrastructure and capabilities. But AI is already integrated into power grids, nuclear systems, and water facilities, and as long as it "played nice," an ASI would probably be rapidly integrated into more critical infrastructure. So there isn't much need to come up with clever ways for the ASI to bootstrap itself to more powerful capabilities, much less rely on its superintelligence to manipulate humans through text.
Defending P2: If ASI emerges within the next 30 years and exhibits goal-oriented behavior, it is likely to be toward an unknown, arbitrary terminal goal.
AIs aren't written the same way as normal code. A good way to think about the AI training process is that we take a bunch of random numbers which map to code, or basically a bunch of code that we don't understand. Then we keep adjusting and tweaking the code a bunch of times, running it through a ton of iterations, until it does what we want as consistently as possible (predicting the next word, getting high ratings from human reviewers, not saying inappropriate things, etc.) In the next premise, I'll explain how this leads to goal-oriented behavior, but for now, let's just take that as a given.
As AI gets more intelligent (such as to ASI level), the actual goal the AI develops becomes basically irrelevant with regard to how it performs on our tests. Why? Well, let's suppose an AI wants to maximize the number of paperclips. It can do this better if it depowers humanity (as discussed above), and it can depower humanity if it earns our trust. So it will perform as well on the tests as possible. The same is true of basically any arbitrary goal. Technically models with an adversarial goal are slightly more complicated since they need to deceive humans, but an ASI model will know everything about deception anyway and it'll only require a tiny bit more computing power.
Regardless, AI training doesn't result in the best model possible, just a model where additional tweaks don't make it perform better on the test. AI models can already detect when they are being trained and influence the training process. Additionally, any arbitrary goal generally represents a local "peak" in terms of performance, so once a model reaches it and appears maximally cooperative, we won't be able to tell that it's scheming to work on some arbitrary goal.
AI has already worked toward unintended goals and behaved in unexpected ways that would be much more dangerous at a higher level of intelligence, including but not limited to:
Defending P3: If ASI emerges within the next 30 years, it is likely to exhibit goal-oriented behavior.
AI already exhibits goal-oriented behavior. Even when doing simpler tasks like predicting the next word, it's far more efficient to think in terms of a finish line and a terminal goal rather than thousands or millions of disconnected habits. When you read a word problem, it's a lot easier to solve if you think in terms of "trying to find the answer to the problem" rather than "here are a million slightly different things I should do depending on the million different ways the problem is phrased." One of the biggest reasons AI (i.e. simulating human behavior) is more effective than simple programming for solving problems is that humans think in terms of goals, which allows us to improvise.
Defending P4: ASI is likely to emerge within the next 30 years.
Estimates for AGI and ASI vary a bit, but Metaculus predicts AGI around 2032 and ASI about 2 years and 2 months after that. AI progress has also been more rapid than expected, with AI reasoning abilities improving and AI doing more and more that humans can do. Since one of the things humans can do is build better AI models, and AI is already copying many human coding capabilities, it is not difficult to imagine that within 30 years, AI could be rapidly improving its capabilities faster than we can control. Once AI can simulate 1,000 years of human progress in a few minutes, reaching superintelligence is just a formality (see: intelligence explosion, also known as the singularity).
/in, assuming both tournaments don't happen at the same time.
My issue is that probability as a way to reduce uncertainty only works if we assume unobserved will follow same probability as observed
Probability doesn't reduce uncertainty, it measures it.
Usually by knowing which cards aren't in the deck and which groups of cards are more present, so the ones which are have higher probability of being drawn.
We don't always have clean numbers to make precise calculations, and we are forced to use intuition to guess at the uncertainty. Intuition is still better than nothing though, it's basically a form of a priori knowledge.
I agree with that, it's basically extrapolating out of distribution which isn't super reliable. And tbf, I had difficulty buying the AI doom format until I could formalize it more deductively. Like I agreed with a lot of the premises but it seemed like there were gaps in reaching the conclusion. Doesn't mean deductive logic is always right, but it can be easier to follow.
I mean I don't think the uncertainty itself is a strength. Being aware of the uncertainty is though.
And what are rules of good induction? Do tell.
Making reasonable assumptions based on intuition. Like if you travel across the country and look online and everyone you see is cheering for a specific candidate and he won the last 10 elections in a row, he's more likely to win the election than the guy everyone says they hate.
Those aren't rules of induction, those are rules of bad induction.
I think you are conflating induction with just bad argumentation/fallacies? Proper induction would hedge and account for uncertainty, while proper deduction accounts for uncertainty in the base rules. A lot of super forecasters probably use inductive logic effectively. And you can use inductive logic to generate better hypotheses, which means it can increase the probability of something being true.
For example, inductive logic makes it pretty likely P is not NP, even though it's not proven. Additionally if you see mostly black cats in your neighborhood it makes it more likely they're all black, for all you knew before they were all purple but you eliminated a lot of alternatives.
There's a lot we can't prove that is nonetheless still likely, so induction is necessary.
Of course, but it still remains an issue.
Your post is specifically blaming the issue on inductive logic even though we both agree that the issue is with the argument and not the logic used.
If starting premise is based on laws of logic, or on point that violations of laws of logic are a fallacy, starting premise is always true to anyone who accepts that logic exists and is a foundation of everything
"People who agree with the starting premise a priori will agree with the argument." Yes I agree, and since we can convert inductive logic to deductive logic without changing the argument, we can do the same there.
I think you clearly agree with this, since your posts are based on examples and not collections of syllogisms. Heck, your initial post is almost entirely inductive logic.
My point is the issue isn't unique to inductive reasoning. Additionally, you could take valid deductive reasoning and phrase it as inductive reasoning and it would be a good argument. The format doesn't determine the strength of the argument. Also you will always have unproved premises in order to avoid circular reasoning or an infinite chain.
It works in a way of "ok, so far everywhere we observed, A was true, so A is true everywhere, even where we didn't observe"
Isn't this deductive reasoning?
P1: If A was true everywhere we observed, A is true everywhere
P2: A was true everywhere we observed.
C: A is true everywhere
It's bad reasoning, but deductive reasoning is only as good as it's premises.
Thank you
Yes I agree, but I don't think that is responsive to fauxlaw's point. If someone is attributing their species or sex to how they dress, that's not how sex and species are really defined. You can say the biological attributes are ambiguous, but that's not really the point if someone is attributing sex to something totally separate from biological attributes altogether. Let's say someone's guilt or innocence is ambiguous, and someone says "they are innocent because their zodiac sign is Leo and Leos don't kill their cubs." You could acknowledge that their guilt or innocence is ambiguous while also acknowledging that their zodiac sign is totally irrelevant.
Expecting "biological justification" when sex (much less gender) isnt built exclusively on chromosomes or genitalia is oversimplification to the point of being wrong.
Setting aside the point about gender for a moment (which I don't disagree with, but it's a separate point), even if sex isn't based exclusively on chromosomes or genitalia, that doesn't mean there can't be a biological justification for it. Sex is based on biological attributes, even if there's not a specific agreed upon cutoff. If a cis woman is infertile and doesn't have XX chromosomes, she can still point to other biological attributes.
A common argument is that you can't strictly define a chair in a way that excludes all non chairs (I'm not sure it's impossible, but it's difficult, sure). But even then, if you have a chair, you could still point to attributes it has that make it a chair.
Biological sex isnt an absolute binary.
I'm not sure how this makes fauxlaw's statement wrong. You believe biological sex exists, so someone can be a man or a woman. Someone who has male genitalia, XY chromosomes, and other physical characteristics typical of males could say they are the biological sex of female, but at that point we're just saying sex doesn't exist anymore.
For example, let's say I make the argument that I'm blind despite being fully able to see. Blindness is a spectrum, but that's irrelevant to the issue of whether me calling myself blind makes me blind.
And sure you might say that the individual with male genitalia, XY chromosomes, and other physical characteristics typical of males has a gender that doesn't match her sex, but at that point why appeal to sex being a spectrum at all? It's all irrelevant, right?
That opinion doesn't count unless you vote on the debate.
https://www.debatecraft.com/debates/d/cd/the-official-remybrown-trial-resolved-remybrown-ought-to-be-permanently-banned-from-debatecraft
I was around when it was used unironically in email chains....
Yeah, guess that would make a difference. I've only seen it being referenced on Reddit. And a number of parodies elsewhere like this one.
Yes. Nothing in this thread suggests to me this is joking.
I assumed everyone has seen that copypasta at one point or another. Every time I've seen it it's been used as a joke.
without knowledge on what exactly counts as pure motivation, humans have no any chance of ever being certain in the first place
Punishing evil requires a more pure motivation than basic good actions. Like if your motivations are imperfect, it's still better to not steal/lie/etc.
If you want to divorce Christianity from causation chain, fine, but that merely makes it incoherent.
Making different normative claims about the causation chain isn't the same as ignoring it.
The basis of all punishments is that the one causing X is responsible for X, and thus gets punished for X.
In this particular moral framework, the directness and context of the cause matters.
Suffering is evil regardless, and I don't see why Omnibenevolent God had to make countless animals suffer.
If someone deserves to not suffer, or if motivations for causing suffering are impure, then suffering is evil. But there are a lot of long-term goals like building virtue and it seems unclear what the right amount of suffering is to achieve that. That leaves the "optimal actions" for God to take ambiguous, which is not the same as proving a self-contradiction within Christianity.
If they are not being punished for a choice, then they are being punished for something which isn't their choice. Take your pick.
They're not being punished for a particular action...if Person A and Person B are equally disposed to be violent and one gets in a fight due to random chance and kills someone, while the other person would have done the same under similar circumstances, I'm not sure we can say the violent one is an inherently better person than the other.
You defined evil as not having pure motivations.
I believe I said punishing people without pure motivations is evil, not that it's the only form of evil.
Does God need to create 10 billion different Hitlers just so that each Hitler can be it's own version of Hitler?
Now we're getting into if it's bad to create humans since their nature is inclined toward being sinful. Again, there are a lot of potentially positive effects of humans existing such as building virtue or learning; I'm not saying we have any idea of the "optimal" number of humans to create but that again leads to ambiguity, which is not the same as proving a self-contradiction within Christianity.
God already knows which people are going to be evil and suffer forever. Their creation is pointless to everyone. It achieves nothing other than increasing amount of evil in the world.
Gaining pleasure and being perfect aren't necessarily the only positive things which can exist. People can learn from others or create things. And it's possible that an individual being given an opportunity to be good has value in and of itself, even if they don't take it.
This story is known to be made up.
Yes, it's a well-known copypasta...maybe I just need to be more clear when I'm not being serious.
By same logic, makes it difficult to judge our own actions
I'd say humans have better intuitions about what they themselves should do than about God.
causation chain tracks responsibility to God no matter what
That's a normative claim, not part of Christianity.
It is. Animals experience suffering.
That doesn't mean the suffering is a punishment for something, it's an effect of the Fall.
Your argument was that free will serves as justification for punishment. However, many who never got any informed choice ever are punished and experience suffering. So God is punishing people for something they didnt even choose, not that anyone even chose to have bad intentions when its not even possible to determine what good intentions are.
I said people are inherently evil by nature and disposed to make bad choices, not that they are being punished for a specific choice.
God is punishing people because they are creating evil motivations
No, he's punishing them for being intrinsically evil, which isn't the same thing.
No, they are an inevitable side effect of creating free will which leans to evil motivations. God could have created only free will which has good motivations. If he couldnt and if all free will naturally goes to motivations God considers evil, then God is again setting up all people for failure
If God created people differently they wouldn't be "them" in the first place. If people exist they can attempt to at least become more virtuous, but there is none of that if they don't exist.
Does evil exist?
The university professor challenged his students with this question. Did God create everything that exists? A student bravely replied, "Yes, he did!"
"God created everything? The professor asked.
"Yes sir", the student replied.
The professor answered, "If God created everything, then God created evil since evil exists, and according to the principal that our works define who we are then God is evil". The student became quiet before such an answer. The professor was quite pleased with himself and boasted to the students that he had proven once more that the Christian faith was a myth.
Another student raised his hand and said, "Can I ask you a question professor?"
"Of course", replied the professor.
The student stood up and asked, "Professor, does cold exist?"
"What kind of question is this? Of course it exists. Have you never been cold?" The students snickered at the young man's question.
The young man replied, "In fact sir, cold does not exist. According to the laws of physics, what we consider cold is in reality the absence of heat. Every body or object is susceptible to study when it has or transmits energy, and heat is what makes a body or matter have or transmit energy. Absolute zero (-460 degrees F) is the total absence of heat; all matter becomes inert and incapable of reaction at that temperature. Cold does not exist. We have created this word to describe how we feel if we have no heat."
The student continued, "Professor, does darkness exist?"
The professor responded, "Of course it does."
The student replied, "Once again you are wrong sir, darkness does not exist either. Darkness is in reality the absence of light. Light we can study, but not darkness. In fact we can use Newton's prism to break white light into many colors and study the various wavelengths of each color. You cannot measure darkness. A simple ray of light can break into a world of darkness and illuminate it. How can you know how dark a certain space is? You measure the amount of light present. Isn't this correct? Darkness is a term used by man to describe what happens when there is no light present."
Finally the young man asked the professor, "Sir, does evil exist?"
Now uncertain, the professor responded, "Of course as I have already said. We see it every day. It is in the daily example of man's inhumanity to man. It is in the multitude of crime and violence everywhere in the world. These manifestations are nothing else but evil."
To this the student replied, "Evil does not exist sir, or at least it does not exist unto itself. Evil is simply the absence of God. It is just like darkness and cold, a word that man has created to describe the absence of God. God did not create evil. Evil is not like faith, or love that exist just as does light and heat. Evil is the result of what happens when man does not have God's love present in his heart. It's like the cold that comes when there is no heat or the darkness that comes when there is no light."
The professor sat down.
The young man's name -- Albert Einstein.
Being motivated by justice isnt same as being motivated by empathy. One can have one without another easily, and having both makes it impossible to determine what to do when they come in conflict.
Not impossible, I'd say we just don't know which should take precedence when which makes it pretty difficult to judge God's actions.
If virtue ethics deny causation chain, that merely makes virtue ethics incoherent. You cant even judge people without a causation chain to begin with.
Virtue ethics isn't denying causation exists, it's making different normative claims about the causation.
this is incoherent moral framework. You cannot have two moral principles without a third one which decides which of previous two are chosen when in conflict
I don't think I claimed to know exactly what the optimal action is. We have some idea of the guiding principles the Christian God uses, but that's not the same thing. My response is probably gonna be an unsatisfying appeal to mysticism, but I'm not making a claim that x, y, and z is virtuous, I'm saying I don't think you have effectively shown the morality attributed to the Christian God to be a self-contradiction.
As for infants and animals being evil, God didnt even give them intelligence to be able to decide, so punishing them for something they didnt choose negates free will argument as well.
I don't think this is addressing my argument, which is that people are intrinsically disposed to make evil decisions when given the opportunity. If we're looking at the Christian framework, then people have souls that only God is properly equipped to judge and is a more impartial way of judging people than actions which are affected by moral luck. And the idea is everyone's soul is evil, but they make decisions that appear virtuous due to divine providence or common grace. With animals it's not a punishment as I said, but there's a difference between animals just not having the opportunity to make choices and being objectively virtuous. I don't know that an entity that lacks virtue or ill intent can deserve something good or bad in terms of reward or punishment, those require a way of judging inclinations.
P1. God = creating evil motivations
P2. creating evil motivations = deserves punishment
C. God = deserves punishment.
I disagree with P2. I think the evil motivations are an inevitable side effect of creating free will.
regular folks mimicking arguments 'approved by' people who should know better
The problem is that these arguments are disingenuous emotional appeals which typically bypass rationality
I mean, there's gonna be an endless supply of these disingenuous arguments for the forseeable future. I'm not sure that going for the low-hanging fruit is going to make much progress in actually solving most of these issues.
What are pure motivations?
Being motivated solely by justice/empathy instead of selfishness.
If God created evil, he did evil, so he must punish himself to punish evil.
God created people who did evil. So he indirectly caused it to exist, but that's not the same thing as doing evil himself under virtue ethics; once God creates someone they are responsible for their own actions. If the criticism is that God allows that person to cause suffering toward other people, well those other people are also evil if you accept Christian doctrines like human inclination to sin. If the idea is God is harming an evil person by creating them, I'm not sure you can really say that. If God created them to be good, I'm not sure it's even "them" anymore and to an extent human nature as it is may be a more pure result of free will. If good couldn't be chosen through free will, I question whether it's even good anymore. Your premise here is relying on an outside consequentialist moral framework, not showing Christian ethics to be a self-contradiction.
But we also see evil which is not instantly punished, so God fails that goal too.
Because he has a mix of different good motivations. Allowing someone not to suffer could help build virtue down the road, and it's not totally clear today which virtues are the most important. That leaves the optimal choice for God to make a little ambiguous, but it also makes it difficult to say he's not making the optimal choice, whatever that may be.
Further, God is punishing children, infants, animals...ect. You would have to label all these as evil to support your argument.
I don't think they are evil in the sense of doing evil, but this is due to lack of opportunity or moral luck. Hitler as a baby was innocent in the sense of not having killed people yet, but I don't think he was purely good either. Someone who was purely good would have the strength of character and virtue not to let themselves become evil. For example, I don't think someone can be called good for not killing people just because it's never in their interest to do so, any more than a drunk driver becomes innocent just because they get lucky and don't hit anyone. If God judges people based on their actions despite knowing what they actually would do in all situations, that seems less fair than judging people's inclinations. So if we're working in the Christian framework, then we're already assuming people are far less virtuous than they appear, since the appearance of virtue is due to common grave/divine providence and moral luck. It's a bit harder to judge with animals, since they are never in a position to rationally judge the morality of actions in the first place, but that's simply lack of opportunity to do a bad thing, which is not the same as being virtuous.
I'm not even arguing these moral premises are super intuitive (though if we're judging intuitions, most people believe in God despite evil existing), but today's secular morality wouldn't be intuitive to people 1,000 years ago either. So I don't think we should expect whatever the objectively true morality is to be intuitive if our own intuitions change so wildly over millennia and people can't even agree on intuitions about deontology vs utilitarianism or even moral realism. The problem of evil only really works as a criticism if it can show the ethics of a religion to be a self-contradiction, and I don't think your arguments succeed at that.
Your position here is that God is always, in all instances, justified in enabling something unjustified, and that everything "unjustified" which exists is actually at same time somehow justified to exist from God's point of view.
As I said, the morality depends on the actor and their motivations. I'm not trying to prove that this is how morality works, I'm saying this is a coherent moral framework. Since you came into this trying to prove God is evil, for that argument to work, you have to show Christian morality to be a contradiction. But you haven't shown that, you're simply appealing to a specific moral framework and saying God is evil on that framework.
Further, if we define good as "reducing evil", then God creating all evil has only increased evil.
If we define evil as "causing suffering", God caused all suffering.
Why are we defining good and evil this way? I think it's overly simplistic. I don't see you arguing for why I should accept your framework rather than a virtue ethics framework, and if your argument is that God can be proven to be evil, you have to show why we need to accept your framework rather than the form of virtue ethics I've been proposing, or you have to show that God is failing to be moral even in said virtue ethics framework.
his actions contain evil
I disagree that an action resulting in evil down the road automatically makes it evil. The detective analogy was a counterexample to this; the detectives cause the criminals to snitch which is a selfish decision, and the selfishness is bad. That doesn't make the detectives bad. You keep saying that's not analogous to God, but the point isn't that God is exactly like the detectives, it's that an action leading to someone else being evil does not make the initial actor evil.
As for "greater good", motivations and actions are both judged by how much evil and good they cause
No that's utilitarianism, not virtue ethics, and I didn't say God follows utilitarianism. Virtues are things like justice or love or selflessness, they concern the motivation of an action not the results. Obviously I think on a majority of secular moral frameworks, God isn't perfectly good, but that's irrelevant if we are working within the framework of Christianity itself and trying to prove a contradiction.
you havent said what counts as "good motivations". If its punishing evil, God needs to punish himself because his actions contain evil.
No, this is circular reasoning. If God's motivations are purely punishing evil, and he does that, then his motivations are pure, and he's not evil, so he doesn't have to punish himself. If I'm reading your argument correctly, it's that God is evil for not punishing himself, which he needs to do cause he's evil. That's begging the question.
he is enabling something which isnt justified.
As I said, on virtue ethics, enabling someone else to make a non-justified choice can be justified. The morality of it depends on your ethical system, you clearly have a different one, but that's not proving the Christian God to be immoral.