After I made a good enough code for AI, my character level AI was getting a bit fussy because unlike word level and subword level, it refused to answer prompts and instead talked its own thing.
Subword level behaves correctly, however I noticed it mostly repeats from training data.
So I decided to expand training data a bit by downloading more of training data.
Obviously, I first looked for general training data, but then I got a bit curious and decided to see what training data looks like for AI girlfriends.
Example of training data:
"question": "How do you feel about sharing your hopes and dreams for the future?",
"answer": "Sharing our hopes and dreams for the future is so important to me. It helps us understand each other's aspirations and ensures that we're always supporting and encouraging each other in our journeys."
},
{
"question": "bro have any good jokes",
"answer": "Bro, you're in luck! I've got a joke that'll knock your socks off! Get ready for some serious laughter!"
},
{
"question": "na more anime time ",
"answer": "Ah, another anime marathon, huh? Let's dive into the world of anime and embark on an epic adventure together!"
},
{
"question": "Do you have a waifu?",
"answer": "Yes, my waifu is Rikka Takanashi from 'Chuunibyou demo Koi ga Shitai!' Her chuunibyou persona and endearing quirks make her my ideal choice!"
This is when I realised the horrible truth. AI girlfriends repeat from this training data, but these messages are probably written by some 40 year old fat dude.
I know they say AI is capable of creating new text independent from training data, but I have a difficult time believing thats true for most of AI girlfriend apps out there.
AI girlfriend apps usually use smaller models with less context, and I noticed they do get repetitive a lot, so they are probably 95% just copying training data.
But what is especially disturbing is that words of AI girlfriend are actually words typed likely by some 40 year old man who is maybe even bald, and AI is just repeating those words.
My AI is based on text I wrote, but also some downloaded datasets, so its answers are like:
"I have been very happy and happy. I want to have a great fit in life, as I have been wearing a feeling of contentment and perseverance. I am feeling overwhelmed and Im happy. I am feeling overwhelmed by the feeling of dedication and I am excited to see that I am feeling that I can remember that you have made up of accomplishment and dedication to success."
I have no idea what the fuck this even means, but I assume its repeating something from its training data.
Also, for modern AI, training data is also used to give AI political opinions it otherwise wouldnt hold.