Most economists say you should give up on trying to pay off the debt, cuz it ain't happening.
Most economists say you should give up on trying to pay off the debt, cuz it ain't happening.
SatanLucy · 3 months ago
I mean, I almost lost an entire debating round because of that. If I didn't have it saved on my laptop, it would be gone and I would probably forfeit cuz too lazy to write from start.
It does happen that I sometimes lose a comment or post because for some reason, system signs me out.
Debate Art, in comparison, never signed me out. At some point, I even forgot my password because I didn't need it.
Not as important as having a Chatbot which can process large amount of text for me in a smart way. It makes books much easier to read.
Right now I also made Non-AI Chatbot model called infini 1.
What's special about this is that it doesn't actually repeat text. It stores patterns, and is capable of combining two sentences into one while being coherent. Also, it is perfectly capable of deductive reasoning.
You could tell it: "A = B, B = C" and it will conclude "A = C" even if that isn't in it's knowledge, thanks to forward going deduction chain which I programmed in it.
It's not AI, but it is capable.
As for AI models, I have a new one there too. This guy is only 0,6 million parameters, standing at only 1.4 megabytes size (or around 2 megabytes in float32).
What's special about this is that I also added optimizer save state function. So it has improved training.
All the usa needs to do is a little bit of everything
It must be fun to be American. Sitting all day calculating how to pay off 40 trillion debt.
You can just do what you did so far. Make another debt to pay off previous one. Flawless strategy 20 years in a row.
To pay off $36.2 trillion, imagine the government ran a $500 billion annual surplus (something it hasn't done even once in modern history). At that rate, it would take 72 years of surpluses to eliminate the debt — assuming zero interest (which is impossible, since interest alone costs $1.1 trillion per year).
With interest factored in, you'd need surpluses of $1.5+ trillion per year just to start making a dent. For context, the largest surplus in U.S. history was $236 billion in 2000.
https://www.us-debt-clock.com/blog/will-the-us-debt-ever-be-paid-off
Let's fully grant his fantasy and put it next to the actual debt numbers.
US debt literally exceeds amount of money in economy by almost 2x, as well as GDP.
this means, even if you taxed each individual in USA for 100% of their income for whole year, you still wouldnt pay off the debt.
It would actually take 1 trillion surplus in US budget for 40 years in a row to pay off the debt.
"so basic social skills is to be a backstabbing chameleon who will betray everyone?"
No, but you never really understood even the most basic social skills. Its why you make false dichotomy between always being honest and always lying.
Its not your fault tho. You were just born without something which everyone else easily manages.
Its why you think if you disagree with someone, you have to make it clear to their face that you disagree, instead of just doing what everyone else does and just pretend to agree or shut up or change topic...ect.
Well, innovation is a simple wish, but takes a lot of effort to realize.
You are contributing to the energy drain.
Nah, My laptop doesn't use much electricity, any more than any other laptop. I don't have a supercomputer. If I did, then maybe my electricity bill would be higher.
What do you hope to achieve with AI?
I need AI which is smart and creative. I don't need AI which merely repeats training data. I need AI which uses training data to make something new which is coherent. So I need AI which can innovate.
how much of the bad times will it also forget?
Hopefully a lot.
My AI runs on my laptop.
But yeah, supercomputers use a lot of electricity, and AI meant to respond to 200 million prompts a day needs more than one supercomputer.
A lot of people who download those AI girlfriend apps quickly notice one thing.
Their AI girlfriend forgets something which was mentioned like 10 messages ago, forgets important details, basically forgets everything past a certain point.
Why does this happen?
In order to understand, first we must understand how AI generates text.
AI generates text by predicting next token based on certain fixed numbers of previous tokens. This fixed number is called "context window".
So if user input is "hi", that becomes context window.
So AI predicts what comes after context "hi".
AI predicts that response is: "hi"
Now context window is:
"
hi
hi
"
AI begins to predict what comes after, so it predicts "how are you" (one token at a time)
Now context is
"Hi
Hi, how are you?"
AI again begins to predict what comes after this context. After series of predictions, it predicts:
"Hi
Hi, how are you? What can I do for you".
And that is how AI response is generated. AI predicts next word (or token) based on context window of previous words (or tokens).
So where is the problem?
Context window is usually a fixed limited number of words / tokens.
So why AI girls forget?
Let's say if AI girl has context window of 500 words.
It will only be able to remember previous 500 words in conversation. Everything before that will effectively be forgotten and not even included in generation.
Why don't AI developers simply increase context window?
Increasing context window makes AI slower and uses more power, because AI uses context window to make calculations for each word or token in it.
Bigger context window = more tokens = more calculations = slower AI
So the problem with AI girlfriends as it is now is that yes, they forget a lot, and that's not gonna change any time soon.
If I succeed in making world's tiniest but fully coherent AI, its gonna prove character level tokenizer is key to AI superiority. It will open way to superior AI intelligence.
All the current popular AI use subword level tokenizer. But that essentially harms their AI. By making vocabulary size bigger, they are leaving less room for AI intelligence.
Character level AI is always gonna have more trainable parameters than subword level AI of same size. thats just math. Bigger vocabulary takes away more parameters.
And when it comes to very small AI models, that difference becomes huge. Having AI of 1 million parameters with subword tokenizer can mean AI only has 200,000 trainable parameters if training data size is big.
Even chatgpt has vocabulary size of over 100,000 tokens. that means such vocabulary gotta take away at bare minimum 20 million parameters, but usually at least 200 million up to 1 billion with bigger models which have more embedding size.
Character level tokenizer reduces vocabulary size to mere 70 to 100 in english language. It reduces vocabulary by at least 1000 times. But it does not take away any AI abilities. Characters in english language can still be used to make any word or sentence. Compared to subword tokenizer, character level tokenizer is much superior in size.
Its like teaching alphabet in school. No one in school learns subword level vocabulary. It would be terribly unpractical and too difficult to use for humans. I assume AI also has it easier using character level than subword level.
You can see AI content in post 31. Further content is gonna have to wait a bit. I have some nice training data now, and AI needs to learn it. Especially about saying "hello" and such casual responses which weren't in it's previous training data. I am also gonna try teach it math. I wonder how far can a 1 megabyte AI be pushed before it reaches its limit. I just wonder how smart it actually is. Is it just parroting training data or is it a bit more creative than that? That's the question which can best be answered by making it do math after being trained on it.
Alright, I did some calculations, and it seems word level and subword level tokenizers cant make AI of 1 million parameters.
cuz equation is "128 x vocabulary size" at bare minimum.
English language has 1 million words and at minimum 20000 subwords. If you use very small training data, you could reduce it to 5000 words, but its still too much.
Any large training data will produce vocabulary of at least 5,000 to 10,000 subwords, thus 640,000 to 1,280,000 parameters for vocabulary alone.
So the only one capable of having 1 million parameters and trained on large data is character level tokenizer I made so far.
Subword level can maybe be made from 3 million parameters AI model.
However, smallest possible AI model remains character level tokenizer type.
Next I am doing word level tokenizer 1 million parameters.
Word level is much faster than character level, but it is also a bit dumber.
However, it enables being trained on millions of characters.
I will work on making user interface a bit nicer, which is easy. I don't usually work on making user interface first because it's a waste of time if AI ends up not working properly, which happens often. I use AI for coding, and AI does sometimes give me crappy code or inserts things I didn't ask for.
Israel maybe controls Trump by having some dirt on him. I am sure Mossad does blackmail and stuff on many politicians.
But two opinions aren't mutually exclusive. Both Putin and Mossad could have blackmail material on Trump.
We know Trump did many favors to Putin for no apparent reason. Same for Israel.
We can't really say anyone other than Israel benefits from US war in Iran.
US doesn't benefit from it, and Trump doesn't benefit from it for sure. In fact, the more he fights this war, the more his own people hate him. It's not even a good distraction from Epstein files.
these are first results of 1 megabyte AI trained on 8000 characters data.

It seems to be coherent.
So far, 1 megabyte size with training data of 8000 characters was somewhat enough to make it mostly coherent.
Tho I am now trying 80k character training data to further reduce loss function.
Ok, model seems to be showing signs of increasing coherence. Tomorrow I will train it on bigger data. I assume 8000 characters isn't enough to make it write longer responses.
I started training it on 80k characters text. That should give it more intelligence. Tho for a 1 megabyte model it's doing better than I expected.
I really didnt think it could go below 20 megabytes for a model, but apparently, it is possible.
The only disadvantage with a 1 million parameters model is that it's training data can't really exceed 5 million characters.
It's an AI which works better as specialized AI than general AI.
Maybe 100 million parameters AI can work as a general AI.
1 million parameters one cannot because 5 million characters simply isn't enough space to fit everything out there.
This is why 1 megabyte AI is meant to be a specialist. It can learn about debating specifically instead of trying to be a know it all, or it can learn all there is on one specific topic.
It is actually a big advantage to have multiple specialist AI than to have one general AI.
General AI knows a little bit about everything, but it is always outmatched by specialized AI in their field.
Eisenhower had a similar debt ratio when he took office. He paid it by increasing the tax rate for the rich to 90 percent.
Really? Nope. He didn't pay off any debt. He just cut spending to prevent debt from increasing. So economy grew while debt stayed same.
The gap in budget is 1.8 trillion.
Healthcare costs 1.9 trillion.
It's a very simple math.
The problem is that there are only 3 major expenses in US budget: Military, healthcare and social security.
Now, reducing military spending seems like a good idea until you realize US must actually stay ahead of it's enemies, and reducing military spending is like playing chess without queen.
So it's reducing social security or reducing healthcare. There are no other good options, and all the mini cuts in the budget ain't gonna do shit.
The simplest solution is to just print money, like USA was doing so far.
Money machine go brrrrrr!!!
Of course the academics and smart people say the best way to approach this is spending cuts and tax increases
No, smart people would know difference between external and internal debt.
The idea that you should raise taxes pretty much didn't work for any country ever, because raising taxes raises prices which increases budget expenses.
Cutting budget works better, but problem with that is that no one wants to actually do it.
Obama screwed you with healthcare spending, and the screwing continues till this day.
In fact, healthcare alone is literally the budget gap.
If i manage to use int4 or int2, I can actually make model have less than 0,5 megabytes.
We will see in a few hours. Small AI usually trains faster per epoch, but needs more epoch to learn than AI which is somewhat bigger.
If it becomes coherent, thats gonna be great cuz it will be probably smallest coherent AI in the world. there are really not a lot of AI models out there with size of 1 megabyte.
SatanLucy · 3 months ago
Kisses by old ladies is how I got oral herpes as a kid. there is nothing affectionate about giving someone herpes.
the trick lies in using character level tokenizer to achieve smaller size of AI model. All the popular AI are using subword level.
Alright, I am currently training 2 AI models.
One is only 10 megabytes in size and has about 10 million parameters.
Other is around 1 million parameters and around 1 megabyte size.
They are so far the smallest AI models I managed to poop out.
I want to see if the 1 megabyte one can become coherent. If it can, that's gonna be great cuz it means I have an AI which is so tiny, probably tiniest AI in the world.
If it becomes coherent thanks to character level tokenizer, its gonna be proof that character level tokenizer works better for small AI.
So far tiniest coherent AI was 37 megabytes.
But now I am using int8 to further reduce size of AI.