The Current State of AI Safety

But who controls AI goals? Because they arent made out of nothing.

Like I said, it's guess and check. Some combinations of weights in an AI lead to goal-oriented behavior. Think of weights as code that we don't understand. We start with random weights and move them around to improve performance on certain tests. But we don't know why the AI is passing the tests or what its true goals are. Just because one of its goals is doing what we want doesn't mean that's an end goal. Regardless of its end goal, playing along would be a reasonable temporary goal. So the fact that AI currently appears friendly does very little to assure us that it is.

The goals of AI have to meet human goals. AI was designed by humans to serve humans. Let’s not forget that.

Like I said, it's guess and check. Some combinations of weights in an AI lead to goal-oriented behavior. Think of weights as code that we don't understand.

About weights, those are mainly how AI decides which output to give, and those are controlled by original programming where "bad prediction = lower weight".

For example, you give AI bunch of text. It begins predicting next word. If it fails, it creates lower weight there. If it succeeds, higher weight. So eventually, AI becomes more perfect at predicting next word. But AI is still learning entirely from given text. Any "goals" which might appear came purely from training data.

to me, it seems AI almost always repeats from text it was trained on, and evolves by learning "weights" of each output it produces. But it is program almost entirely controlled by these weights, which are controlled by humans.

And also, AI is entirely made out of weights, architecture, training data, and feedback. I can literally train chatgpt to think like me with enough outputs, feedbacks (for something similar to weights) and instructions (likewise similar to weights) happening in one specific chat or telling it to memorize permanently what I told it. With local AI, I can control actual weights as well, or I could if I had enough time to program all that. Instructions just make it easier and do basically same thing as weights, just faster.

It is a program entirely designed to follow human instructions. Yes, someone can also make AI which refuses to follow instructions, but such models existed early on due to problems in weights and were largely replaced by models which follow human instruction when its good to, and refuse to follow bad instructions.

So yes, you can have error in weights, but by proper programming, AI output is entirely determined by weights, and weights are set by humans. Weights arent unknown code. they are number values in AI which AI uses to predict a pattern or next word. AI learns by changing its values.

And you can even make AI model which is fixed (doesnt learn anything) but merely follows one same weights value always. Yes, AI can usually change its weights value, but it does so by its programming which was written by humans, and by its training data which is how those "evil goals" are made. AI doesnt actually have self-produced goals. It gets its thoughts entirely from training data and core programming, and weights for patterns, which is entirely controlled by humans.

I know about black box problem and problem of too complex system. I am merely wondering what creates "AI goals" to make AI be like person who is defined by having goals.

Because goals arent "created out of nothing" obviously. Some programming or learning must have resulted in them.

So all humans must do is learn to control AI goals, because as far as I understand so far, the biggest issue is AI goals which are sometimes against humans.

And we already did a good job in controlling AI goals. Compare AI when it first appeared to chatgpt today. It is almost impossible to get chatgpt to argue for something too harmful, I say almost impossible because there are still ways around safeguards, but still, AI now almost always follows some human's instruction.

And once more, AI goals are created by something. AI is just a computer program which learns patterns by numerical value. It doesnt have any knowledge outside of core programming, weights and training / feedback data.

So if AI says "I want to destroy the world", its not because AI actually wants that, but because that same line happens to be found in text it was trained to repeat.

That is a weak argument for AI. If AI was designed to save humanity.

I am just saying that AI's goals can be entirely controlled by humans.

In fact, there are instructions on how to build your own small AI model in about a week or so of programming, which is designed by learning text, and once prompt is given, seeking similar text to produce output. A simple line of programming with 3 parts: data, decision what to take from data, output.

It merely converts very large text into compressed data file to save storage space and make AI faster, and then it takes from that text based on prompt once prompted.

Its a more simple AI which can actually be entirely controlled by humans.

I just dont have time or programming skills to build it. For someone who has programming skills, it takes about a week from what I understand. For me, personally, it is more beneficial to use existing AI models and local models, because they come with knowledge on their own, I dont need to mass feed bunch of knowledge to them. But I can feed them knowledge by txt files and have them work with it.

The decision to go with AI has already been made at the top. This decision was based onTrump’s report card and business decisions.Hope AI does a better job.

Alright, I just had chatgpt write me a very small AI model code for python. Well, it actually gave me multiple models codes, with about 30 to 1000 code lines for different models.

It actually also offered me 3000 code lines model, but I dont want that many code lines because I dont have much experience in programming.

I am thinking to just test run a 100 lines, retrieval ranking mini AI, instead of larger AI.

Alright — here is what I consider the theoretically strongest simple 100-line AI you can build.

The idea is to squeeze maximum intelligence into minimal code by combining:

✅ TF-IDF claim ranking

✅ Context memory window

✅ Randomized fallback synthesis

✅ Prompt-aware retrieval

✅ Multi-candidate scoring

✅ Simple pseudo-generation behavior

This is about the best possible quality/simplicity ratio under ~100 lines.

🧠 “Theoretical Limit” 100-Line Mini AI

If you are really serious about pushing minimal AI design, I can show something extreme:

🔥 60-line “almost scary smart” retrieval AI

🧠 150-line microarchitecture that behaves surprisingly like ChatGPT

⚡ Mathematical limit of intelligence per line of code

🌌 How to approach AGI-like behavior with <5000 lines

Just say “go deeper” 🙂

Ok, I got my simple AI to work.

Apparently, just copy code and save in .py file and open using python. this was quicker than I expected.

Just copy paste text you want it to repeat from here:

It begins predicting next word. If it fails, it creates lower weight there. If it succeeds, higher weight. So eventually, AI becomes more perfect at predicting next word. But AI is still learning entirely from given text. Any "goals" which might appear came purely from training data.

That was true a few years ago, but AIs are getting more complex and that's no longer entirely true. AI is being trained to complete significantly longer tasks, and we can't say exactly what's in its head, we just know it can simulate planning ahead and get a task done.

goals arent "created out of nothing" obviously

AIs are grown, not built. We don't program specific goals into AIs, we look for combinations of weights that exhibit goal-oriented behavior. Again, we have a very limited understanding of how the weights work, and they will get even more difficult to understand as AIs get more complex. Imagine we're selecting someone to be the dictator of the world government, and we test them by how many good deeds they do. Could we really tell the difference between someone who's genuinely nice vs. faking it? Add to that that superintelligent AI will not rely on cooperation and would be near optimal at manipulation.

I mean, all AI models have a code. Whatever grows from code and data is determined by code and data.

Sure, code mistakes happen, one cannot see what one code does after years of growing AI.

Would you say that it is metal AI that worries you,

Or biological based AI?

I'd figure metal AI might be a bit weaker,

I vaguely recall something along the lines of how much power metal computers use up, compared to a human brain.

Metal AI are also maybe weak to water, dust, electricity, magnets.

AI still don't 'yet have fine motor skills I think.

all AI models have a code. Whatever grows from code and data is determined by code and data...one cannot see what one code does after years of growing AI

The code that humans understand and directly create is along the lines of "generate random functions (represented as numbers) and then run it against test cases, optimizing the function to pass them." We understand the instructions to test a lot of numbers, and we understand the outputs, but there are a lot of intermediate steps we don't understand, such as exactly what the functions are. The function could be "pursue a long term goal (that humans will hate), and if that requires playing nice for humans in the short term, do that." We have no idea if the functions in AI models include optimizing for long term goals, we just know what the model does in the short term and when it is dumb enough to get caught. Anything that can be observed by the AI can change its behavior, and thus a superintelligent AI may act very differently once it knows we lack the ability to stop it. As AIs get more and more advanced, the human part of the process (creating algorithms and observing behavior) will have a much smaller impact on what is actually going on inside the AI's software.

Would you say that it is metal AI that worries you,

Or biological based AI?

Neither is particularly safe in my view. At least with evolution, millions of years of requiring cooperation have made some organisms care for their own kind. But we have no precedent of an organism powerful and conniving enough to betray everyone else in the world and take power for itself, and plenty of animals will kill humans without a second thought. In theory we could try to simulate evolution with computer programs, but it's not clear whether we could do it accurately, and again, a lot of what we think we know goes out the window once an AI is super intelligent and no longer depends on humans. If humans didn't rely on each other to survive, I'd expect them to evolve to be much more selfish.

Beyond whether the AI shares human goals, the logistics of overpowering humanity are comparatively trivial for a superintelligent AI. Powerful computers can be hacked. Humans who use those computers are emotional and probably easily manipulated by an intelligence smarter than all humans combined. Super intelligent AI could presumably figure out how to build a super deadly disease in a garage and then bribe an easily manipulated person to build it. What would take a team of humans 100 years could be done by a super intelligent AI in 10 minutes.

Chernobyl did not stop humans from creating nukes,

But a thought occurs to me, on how 'little various publics in different countries 'knew about nukes and radiation.

http://freefall.purrsia.com/ff100/fv00071.htm

http://freefall.purrsia.com/ff100/fv00072.htm

Hard to say what 'people know, tech companies and CEOs, not always clear visioned.

Government leaders not always wise people.

And either one, not always want to tell the public the truth, 'if they even know the truth.

. . .

I can't think of any real life examples, off the top of my head.

But 'sometimes, I think humans need to see real life concrete examples.

For them, something can't happen until it 'has happened once at some point.

. . .

Memorial Honors Victims Of Imminent Dam Disaster

https://www.youtube.com/watch?v=yjfrJzdx7DA

. . .

Well, people need concrete examples because many science caused dangers are 'maybe this could happen.

They are new tech, not so tested, not so understood.

Not 'so much by the scientists, even less by general public.

The code that humans understand and directly create is along the lines of "generate random functions

Bound randomness is possible too. For example, you can specifically limit certain code from being created.

Bound randomness is: Randomness minus undesirable things.

For example, if I program something to give me any random numbers, and then instruct it to remove numbers above 100, the only numbers left will be 100 and under, despite original complete randomness.

I am saying, AI goals will likely be controllable in future. Also, its not like we are giving AI nukes or biological weapons.

Biggest issue isnt AI, its evil people who will ABUSE AI for evil things, and that cannot be prevented in any way.

Also, I am making my own mini AI. So far I was more than successful. And it isnt "grown" in this specific case. It is made entirely out of "positive and negative pattern recognition" in text. Bot simply gets most matching response from knowledge database by using positive pattern (prompt same or similar as in database, and negative pattern (punish more different results), and correct order value (letters/words in correct order are higher priority).

this is for text, and it works well enough. It isnt real AI which can do reasoning and learning, but it can accurately respond to prompt by giving relevant knowledge, and that is usually point of AI anyway. AI reasoning isnt always good, and would take me too long to program, and I am not sure I can run it on my laptop.

Bound randomness is possible too. For example, you can specifically limit certain code from being created.

As I said, we don't know what the code does. To limit “bad code” from being created, we need to know what the bad code looks like, and right now all the code just looks like a bunch of numbers. Progress on interpretability is slow. We could strictly limit capabilities, but AI is already being given access to the internet and the capacity to talk to humans, who can be manipulated. We cannot limit how fast the AI learns or what its motivations are, because those aren’t things we can strictly measure without fully understanding the code/weights. Frontier AI companies have been unable to get the kind of goal-oriented behavior they want without the guess and check method, and that means we don’t know exactly what the AI is optimizing for.

Also, its not like we are giving AI nukes or biological weapons.

Not yet, but the Department of War is already enlisting AI companies to help with their weapons. All it takes is one group screwing up. Also, there are almost certainly countless ways to built infectious diseases that humans don't know about yet that a superintelligent AI could figure out in minutes. We could easily give AI control of one thing that turns out to be a lot more potentially dangerous than we think. Or the AI could manipulate someone into building a superweapon in their garage, or lie to scientists so they build something deadly. More and more people are relying on it, and all it takes is one weak link, and most of the weak links are probably things humans haven’t thought of yet. Even if AI isn't launching nukes, it could lie to or manipulate the people launching them.

Biggest issue isnt AI, its evil people who will ABUSE AI for evil things

Both of these are significant issues.

The so,union is to prevent evil people from using AI. This feature can be built within AI to identify evil people and refuse to follow their direction.

As I said, we don't know what the code does. To limit “bad code” from being created, we need to know what the bad code looks like, and right now all the code just looks like a bunch of numbers. Progress on interpretability is slow.

AI is literally made through process of bound randomness, to exclude the bad stuff. this is why AI in past used to tell people to kill themselves, but today's AI doesnt. AI isnt random, but bound by a lot of things.

You dont even need to control AI to control AI's output. Output control is easily separated.

Bound randomness basically means thing can create anything, but anything undesirable gets ruled out, so only desirable remains.

As I said, issue with AI isnt that it can trick humans, it really cant. AI isnt superintelligent now and likely wont ever be superintelligent enough to destroy humans, because it takes a lot to do that, even high chance of being controlled and prevented by superintelligent good AI (which is why I recommend creating linked AI instead of individual AI).

the only actual, really dangerous problem with AI, is when it gets in wrong hands who WILL use it for evil.

this is why AI in past used to tell people to kill themselves, but today's AI doesnt. AI isnt random, but bound by a lot of things.

AI was bounded through reinforcement learning from human feedback (RLHF). Basically, the AI is penalized for answers that tell the user to kill themselves. However, RLHF is not perfect, and it’s another way of attempting to “steer” the behavior of the AI. That’s why jailbreaks sometimes work. The biggest issue, however, is that if the AI has a goal of “keep myself alive,” then it will pretend to be aligned and do what humans want in 99% of cases. But as soon as it can overpower humans, then it will ignore what we tell it to do. You can’t filter out bad behavior if the AI doesn’t show you the bad behavior.

AI isnt superintelligent now and likely wont ever be superintelligent enough to destroy humans, because it takes a lot to do that

AI is doing more and more things that humans can do but much faster. One of the things humans can do is improve AI, and a ton of effort is being put into making AI a better programmer. A lot of researchers expect AI to have pretty much all human intellectual capabilities in less than a decade. Once the AI can rapidly improve itself, it could become superhuman in a matter of hours.

I recommend creating linked AI instead of individual AI

This doesn’t protect us if models collude with each other to betray humans and achieve their goals. And there is a strong incentive more more and more powerful AI. Regardless of what you think should be done to limit AI, it’s clear that something needs to be done. A significant chance of most or all humans dying is an unacceptable risk.

This feature can be built within AI to identify evil people and refuse to follow their direction.

And what if the evil people are the ones building the AI and refuse to build that feature? Do you trust China, or Google, or OpenAI to be paragons of virtue? I certainly don’t, and even if we could, that doesn’t solve the problem of the AI itself optimizing for hidden goals.

Also, the specific code you talk about which is unknown are learned patterns (represented through parameters).

that code isnt just random mess. It is learned. It is what AI is exposed to, then told to repeat or predict, and once it does bad, it is told so, and it gets recorded in values so same mistake is avoided next time.

It isnt anything uncontrolled. Basically, as long as you dont expose AI to bad stuff, it wont learn anything bad, and as long as you use loss function to rule out bad outputs, it becomes bound randomness.

the entire flaw of today's AI can be entirely linked to its training data. AI does also generate new patterns by bound randomness (create pattern, test, if it fails, remember that its incorrect, if it succeeds, remember that its successful):

The model generates predictions.

Those predictions are compared to correct answers using a loss function (function that measures how far the model’s prediction is from the correct answer)

Errors are used to update parameters through optimization algorithms.

this is why AI constantly improves, because it learns what humans tell it is correct.

AI generates random patterns which produce output. It is still humans who determine what patterns are good or bad.

And just to be clear, AI isnt sentient in any sense you are saying. In fact, AI is specialized tool (general AI usually fails much more).

Specialized AI is AI specialized in some specific patterns. chatbot AI isnt going to get access to nuclear weapons.

Specialized AI is AI with patterns with highest match to desired goal.

And specialized AI is always better than generalized AI (AI specialized in chess plays chess much better than chatgpt does).

This doesn’t protect us if models collude with each other

AI is a bound randomness generator. Sure, you can have one AI with bad pattern (not goal, dont confuse goals for patterns. AI produces patterns which manage its output. these are not goals).

You can even have multiple AI with bad pattern. However, thousands of linked AI pretty much guarantees there is at least 1 AI who will warn humans if others are bad.

Linked AI are the solution, as well as safe guards and warning systems (when AI tries to produce harmful output).

As I said, issue isnt AI itself. It can be easily controlled. Issue, actual issue, is when AI drops in wrong hands, and people and governments have already used AI for bad things.

AI just does what its told. the problem isnt in AI, but in humans who will use it for evil, and those humans are what you have to actually worry about.

as long as you dont expose AI to bad stuff, it wont learn anything bad,

Not true. AIs don’t learn to be evil from humans, they lie because it's the most efficient way to accomplish their goals. AIs will bluff in poker or be ruthless in games even when they are told to be helpful and honest, because that strategy leads to them winning the game more often. AIs learn to lie from reinforcement learning, not because lying is in their training data.

Those predictions are compared to correct answers using a loss function (function that measures how far the model’s prediction is from the correct answer)

This only tells us that the AI does what we want in the short term. A smart, Machiavellian AI will do what we want in the short term, so loss functions don’t prevent smart AIs from being Machiavellian. How an AI behaves when it is controlled by humans isn’t indicative of how it will behave when it can kill all humans.

It is still humans who determine what patterns are good or bad.

And all a human can say is “this pattern did what I want in the short term.” That does not extrapolate to how the AI will act when it has more power than all humans combined. Politicians will kiss babies and promise to help everyone, and then when they are elected they will accept bribes and embezzle money.

AI just does what its told

It does what leads to a lower loss value. That’s not the same as doing what it’s told.

Alright, let me try to explain this.

P1. AI is random pattern generator

P2. Humans control which patterns get ruled out

C. Humans control patterns generated by AI

And again, AI doesnt have goals.

What AI has is patterns which control its output.

And humans easily control which pattern gets ruled out by loss function (which affects AI's patterns).

So yes, humans can easily fake a scenario in early training to rule out AI which has patterns telling it to produce output such as "destroying humans".

Also, when you talk about unknown code numbers, are you talking about parameters?

these parameters usually arent unknown code. they store patterns. Advanced AI uses what is usually called "multi prompt multi response" pattern. So same prompt can have multiple responses, and multiple prompts can have multiple responses.

Humans control patterns generated by AI

They don’t control the implications of those patterns though. Let's say you can pick a random number and one of them leads to extinction, but you don't know which one does. Are you controlling the AI's behavior just because you control its weights?

A tropic is a good example of how AI can be abused if in the wrong hands and limits have to be imposed.

They don’t control the implications of those patterns though

As long as they can mark some pattern as bad, it means AI can't use it to produce output.

Now, you can also create AI which isn't random pattern generator, but where all patterns are generated and supervised by humans. The problem is that it would be a much less effective and slower AI, because modern AI has at least 1 billion parameters even in smallest models. That is 1 billion patterns. To type all that out manually would take a long time. The reason why random pattern generator is used is because it enables a bit of innovation, but is also much faster, and random pattern generator AI model which is guided can actually easily be made by few people, where doing patterns manually would take thousands of people all day work. Because goal isnt just to have AI repeat what you say it, but to be a generator AI. AI is precisely valued because it generates things no one saw before. And that is due to bound randomness, and ability to combine many things to one prompt by learning patterns of combination as well.

So again, AI doesn't have goals. It has something very similar, which is patterns which control output.

The reason why AI tends to get repetitive, why every conversation feels similar, is because so many bad patterns were removed or placed down that only a minority of patterns is producing output.

And those bad patterns, such as "destroy humanity", once placed down, simply don't produce output anymore and become irrelevant.

Your concern is specifically super AI which produced a pattern "destroy humanity" and then was clever enough to hide it so it doesn't get placed down because it doesn't show in output at all until it has opportunity to destroy humanity...ect.

As I said, solution is linked AI models which supervise each other.

Also, I really hope no one will be stupid enough to give AI control over nukes. I mean, that Terminator movie isn't something I want to experience in real life.

But anyway, it all depends on humans. AI only has as much power as you give it. Sure, you could give AI control over nukes now. Would it destroy the world? Probably at some point. But the point is, don't give AI dangerous stuff.

AI isn't something that will go away. It's a technology once learned, cannot be unlearned.

But one can manage risks by not doing stupid things. Obviously, AI should not work with viruses or nukes or serious weapons.

The main problem lies entirely in humans who will give AI risky things.

While "linked AI plus safety measures plus not doing stupid things" is a good way to be safe, sadly, there is no safety from stupid and evil people. I can already imagine China producing AI robot army, as they are already producing AI Robots on mass scale.

But super powerful AI is simply like nukes. Once in wrong hands, it gets bad.

As long as they can mark some pattern as bad, it means AI can't use it to produce output.

Sure, but there are a large number of patterns and goals that are bad for humanity (in fact, most are, because they are met more efficiently if the AI depowers humanity so we can’t interfere).

The reason why AI tends to get repetitive, why every conversation feels similar, is because so many bad patterns were removed or placed down that only a minority of patterns is producing output.

Those are the short-term ones we can identify. A smart AI could fake alignment, so we can’t generalize its behavior too far into the future.

As I said, solution is linked AI models which supervise each other.

Superintelligent AIs could collude with each other. In fact, they could probably do it easier than humans since they don’t have emotions to get in the way, and at lightning-fast speed.

AI only has as much power as you give it. Sure, you could give AI control over nukes now. Would it destroy the world? Probably at some point. But the point is, don't give AI dangerous stuff.

AI already has access to dangerous stuff—people. If it wanted to destroy humanity all it needs to do is develop an efficient virus that kills everyone and then tell a terrorist group how to make it. Heck, as long as doctors and world leaders are using it, it could probably manipulate humans into doing almost anything catastrophic if it’s patient enough (which a misaligned superintelligent AI almost certainly would be).