Why AI girlfriends forget things constantly - How AI really works

A lot of people who download those AI girlfriend apps quickly notice one thing.

Their AI girlfriend forgets something which was mentioned like 10 messages ago, forgets important details, basically forgets everything past a certain point.

Why does this happen?

In order to understand, first we must understand how AI generates text.

AI generates text by predicting next token based on certain fixed numbers of previous tokens. This fixed number is called "context window".

So if user input is "hi", that becomes context window.

So AI predicts what comes after context "hi".

AI predicts that response is: "hi"

Now context window is:

"

hi

hi

"

AI begins to predict what comes after, so it predicts "how are you" (one token at a time)

Now context is

"Hi

Hi, how are you?"

AI again begins to predict what comes after this context. After series of predictions, it predicts:

"Hi

Hi, how are you? What can I do for you".

And that is how AI response is generated. AI predicts next word (or token) based on context window of previous words (or tokens).

So where is the problem?

Context window is usually a fixed limited number of words / tokens.

So why AI girls forget?

Let's say if AI girl has context window of 500 words.

It will only be able to remember previous 500 words in conversation. Everything before that will effectively be forgotten and not even included in generation.

Why don't AI developers simply increase context window?

Increasing context window makes AI slower and uses more power, because AI uses context window to make calculations for each word or token in it.

Bigger context window = more tokens = more calculations = slower AI

So the problem with AI girlfriends as it is now is that yes, they forget a lot, and that's not gonna change any time soon.