Average conversation with logic-people:
"You are using logic to disprove logic!!!"
- "I am using logic? So my arguments are logical?"
"No, your arguments aren't logical."
- "But logical means using logic"
"You are using logic incorrectly!!"
- "Is incorrect logic same as logic?"
"No, it's not same!!!"
- "I accept your concessions."
If Jesus is God, who is the Father then?
Would it make your concessions feel better?
Lucy's short term endocrinal urges and tacky discharges would be the stuff of legends
Yes, what I do with a toilet paper roll is legendary.
I am nota child anymore and less dependent on mom, but he support is helpful.
You sound like a 4 year old.
Ew gross. Sometimes its best to not know.
These questions are entirely too stupid to engage with.
I accept your concession.
So am I using logic or not?
You’re trying to, and clearly failing.
Did you just say I failed to use logic? So I didnt use logic? Eh thats another concession.
I accept all your concessions here. Great work, you conceded all your original points!!!
After I made a good enough code for AI, my character level AI was getting a bit fussy because unlike word level and subword level, it refused to answer prompts and instead talked its own thing.
Subword level behaves correctly, however I noticed it mostly repeats from training data.
So I decided to expand training data a bit by downloading more of training data.
Obviously, I first looked for general training data, but then I got a bit curious and decided to see what training data looks like for AI girlfriends.
Example of training data:
"question": "How do you feel about sharing your hopes and dreams for the future?",
"answer": "Sharing our hopes and dreams for the future is so important to me. It helps us understand each other's aspirations and ensures that we're always supporting and encouraging each other in our journeys."
},
{
"question": "bro have any good jokes",
"answer": "Bro, you're in luck! I've got a joke that'll knock your socks off! Get ready for some serious laughter!"
},
{
"question": "na more anime time ",
"answer": "Ah, another anime marathon, huh? Let's dive into the world of anime and embark on an epic adventure together!"
},
{
"question": "Do you have a waifu?",
"answer": "Yes, my waifu is Rikka Takanashi from 'Chuunibyou demo Koi ga Shitai!' Her chuunibyou persona and endearing quirks make her my ideal choice!"
This is when I realised the horrible truth. AI girlfriends repeat from this training data, but these messages are probably written by some 40 year old fat dude.
I know they say AI is capable of creating new text independent from training data, but I have a difficult time believing thats true for most of AI girlfriend apps out there.
AI girlfriend apps usually use smaller models with less context, and I noticed they do get repetitive a lot, so they are probably 95% just copying training data.
But what is especially disturbing is that words of AI girlfriend are actually words typed likely by some 40 year old man who is maybe even bald, and AI is just repeating those words.
My AI is based on text I wrote, but also some downloaded datasets, so its answers are like:
"I have been very happy and happy. I want to have a great fit in life, as I have been wearing a feeling of contentment and perseverance. I am feeling overwhelmed and Im happy. I am feeling overwhelmed by the feeling of dedication and I am excited to see that I am feeling that I can remember that you have made up of accomplishment and dedication to success."
I have no idea what the fuck this even means, but I assume its repeating something from its training data.
Also, for modern AI, training data is also used to give AI political opinions it otherwise wouldnt hold.
OMG, is SatanLucy a Trumper?????
No, I just don't believe in science, math and logic. These are usual characteristics of a Trump supporter, yes, but I am not a Trump supporter. I support Mao's China.
I don't use X. Terribly crappy site, and design is pure trash. There is literally nothing good about that site. Comments have awful design and it's nearly impossible to read them given how many times you have to click to load comments.
And then yeah, free speech, such thing doesn't exist on any social media. It's only a matter of time before you piss off enough people that they start posting your account everywhere and telling people to report it, and then your account is gone.
On Tik Tok, it has become a trend to post other people's accounts to report them on mass scale.
B. Photographic memory
To be honest, this sounds like awesome ability. I always wanted to be like Sheldon Cooper.
Can she show you a birth certificate? Is the "child" covered on her insurance?Can she claim it on her taxes? No. Because neither the government nor any business would recognize that it's a child yet.
I know this might sound like a strange logic to you, but women are baby factories and they are called that way because their vagina poops babies. However, in order for baby to pop out from vagina, there first has to be a fetus. And difference between 9 month fetus and 9 month baby is... what?
I know this is very difficult to understand, but the fetus in the womb is actually a baby. And aborting that fetus results in woman failing to be a baby factory.
Your reply is irrelevant. Its like trying to have a cup of water without water.
Yes I understood my reply,
Did you understand my reply?
Hmmm...You didn't quite get it.
Did you?
Women do not carry children in their wombs
I mean, how do you think pregnancy works?
In other words, I just pulled an answer out of my ass and because I didn’t think about my words at all I didn’t use logic…
This is not an argument, neither are rest of your "oh look how unserious you are" thingies.
Because “true” means that a statement matches to reality. Logic is not a statement that can be evaluated in such terms, logic is the tool you’re using to do the evaluating. It’s a totally different thing.
This sounds like unproved claim. Where is proof that logic is the tool to evaluate reality? Where is proof that logic is not a statement? Where is proof that this reality of yours exists?
No genius, you’re using the process to arrive at your conclusions, that doesn’t mean you are doing so free of errors.
Errors are not logical. So am I using logic or not? Make up your mind.
This is new code for improved character level tokenizer which uses random batches. It has proved to be better, mainly because training is made more effective and also faster irrelevant of text size.
import os
import math
import json
import torch
import torch.nn as nn
import torch.nn.functional as F
# ============================================================
# CHARACTER-LEVEL TOKENIZER
# ============================================================
class EncodedText:
def __init__(self, ids):
self.ids = ids
class CharacterTokenizer:
SPECIAL_TOKENS = ["[UNK]", "[PAD]", "[BOS]", "[EOS]"]
def __init__(self, vocab):
self.vocab = vocab
self.id_to_token = {
token_id: token
for token, token_id in vocab.items()
}
self.unk_id = self.vocab["[UNK]"]
@classmethod
def from_text(cls, text):
vocab = {
token: index
for index, token in enumerate(cls.SPECIAL_TOKENS)
}
for character in sorted(set(text)):
if character not in vocab:
vocab[character] = len(vocab)
return cls(vocab)
@classmethod
def from_file(cls, filename):
with open(filename, "r", encoding="utf-8") as file:
saved_data = json.load(file)
if saved_data.get("type") != "character":
raise ValueError("Existing tokenizer is not character-level.")
return cls(saved_data["vocab"])
def save(self, filename):
with open(filename, "w", encoding="utf-8") as file:
json.dump(
{
"type": "character",
"vocab": self.vocab
},
file,
ensure_ascii=False,
indent=2
)
def encode(self, text, add_special_tokens=False):
return EncodedText([
self.vocab.get(character, self.unk_id)
for character in text
])
def decode(self, ids, skip_special_tokens=False):
special_ids = {
self.vocab[token]
for token in self.SPECIAL_TOKENS
}
characters = []
for token_id in ids:
token_id = int(token_id)
if skip_special_tokens and token_id in special_ids:
continue
characters.append(
self.id_to_token.get(token_id, "[UNK]")
)
return "".join(characters)
def token_to_id(self, token):
return self.vocab.get(token)
def get_vocab_size(self):
return len(self.vocab)
# ============================================================
# SETTINGS
# ============================================================
KNOWLEDGE_FILE = "knowledge.txt"
MODEL_FILE = "model.pt"
TOKENIZER_FILE = "tokenizer.json"
VOCAB_SIZE = 8000
EMBED_SIZE = 192
NUM_HEADS = 4
NUM_LAYERS = 4
FF_SIZE = 768
CONTEXT_SIZE = 256
BATCH_SIZE = 16
TRAINING_STEPS = 1000
LEARNING_RATE = 0.0003
DROPOUT = 0.1
SAVE_INTERVAL = 100
MAX_NEW_TOKENS = 2000
TEMPERATURE = 0.8
TOP_K = 40
DEVICE = torch.device(
"cuda" if torch.cuda.is_available() else "cpu"
)
# ============================================================
# TOKENIZER
# ============================================================
def create_tokenizer():
if not os.path.exists(KNOWLEDGE_FILE):
raise FileNotFoundError(KNOWLEDGE_FILE)
with open(
KNOWLEDGE_FILE,
"r",
encoding="utf-8"
) as file:
text = file.read()
if not text:
raise ValueError("knowledge.txt is empty.")
if os.path.exists(TOKENIZER_FILE):
print("Loading existing character-level tokenizer...")
try:
tokenizer = CharacterTokenizer.from_file(TOKENIZER_FILE)
missing_characters = set(text) - set(tokenizer.vocab)
if not missing_characters:
return tokenizer
print(
"Training text contains new characters. "
"Recreating tokenizer..."
)
except (ValueError, KeyError, json.JSONDecodeError):
print(
"Existing tokenizer is incompatible. "
"Creating a character-level tokenizer..."
)
print("Creating character-level tokenizer...")
tokenizer = CharacterTokenizer.from_text(text)
tokenizer.save(TOKENIZER_FILE)
print("Character-level tokenizer created.")
return tokenizer
# ============================================================
# MODEL
# ============================================================
class CausalAttention(nn.Module):
def __init__(self):
super().__init__()
self.heads = NUM_HEADS
self.head_size = EMBED_SIZE // NUM_HEADS
self.qkv = nn.Linear(
EMBED_SIZE,
EMBED_SIZE * 3
)
self.projection = nn.Linear(
EMBED_SIZE,
EMBED_SIZE
)
self.dropout = nn.Dropout(DROPOUT)
def forward(self, x):
batch, sequence, channels = x.shape
q, k, v = self.qkv(x).chunk(3, dim=-1)
q = q.view(
batch,
sequence,
self.heads,
self.head_size
).transpose(1, 2)
k = k.view(
batch,
sequence,
self.heads,
self.head_size
).transpose(1, 2)
v = v.view(
batch,
sequence,
self.heads,
self.head_size
).transpose(1, 2)
attention = F.scaled_dot_product_attention(
q,
k,
v,
is_causal=True,
dropout_p=self.dropout.p if self.training else 0.0
)
attention = attention.transpose(
1, 2
).contiguous().view(
batch,
sequence,
channels
)
return self.dropout(
self.projection(attention)
)
class FeedForward(nn.Module):
def __init__(self):
super().__init__()
self.network = nn.Sequential(
nn.Linear(EMBED_SIZE, FF_SIZE),
nn.GELU(),
nn.Linear(FF_SIZE, EMBED_SIZE),
nn.Dropout(DROPOUT)
)
def forward(self, x):
return self.network(x)
class TransformerBlock(nn.Module):
def __init__(self):
super().__init__()
self.norm1 = nn.LayerNorm(EMBED_SIZE)
self.attention = CausalAttention()
self.norm2 = nn.LayerNorm(EMBED_SIZE)
self.feed_forward = FeedForward()
def forward(self, x):
x = x + self.attention(
self.norm1(x)
)
x = x + self.feed_forward(
self.norm2(x)
)
return x
class GPTModel(nn.Module):
def __init__(self, vocab_size):
super().__init__()
self.token_embedding = nn.Embedding(
vocab_size,
EMBED_SIZE
)
self.position_embedding = nn.Embedding(
CONTEXT_SIZE,
EMBED_SIZE
)
self.dropout = nn.Dropout(DROPOUT)
self.blocks = nn.Sequential(
*[
TransformerBlock()
for _ in range(NUM_LAYERS)
]
)
self.final_norm = nn.LayerNorm(EMBED_SIZE)
self.output = nn.Linear(
EMBED_SIZE,
vocab_size,
bias=False
)
# Share input and output embeddings.
self.output.weight = self.token_embedding.weight
self.apply(self.initialize_weights)
def initialize_weights(self, module):
if isinstance(module, nn.Linear):
nn.init.normal_(
module.weight,
mean=0.0,
std=0.02
)
if module.bias is not None:
nn.init.zeros_(module.bias)
elif isinstance(module, nn.Embedding):
nn.init.normal_(
module.weight,
mean=0.0,
std=0.02
)
def forward(self, x, targets=None):
batch, sequence = x.shape
positions = torch.arange(
sequence,
device=x.device
)
token_embeddings = self.token_embedding(x)
position_embeddings = self.position_embedding(
positions
)
x = token_embeddings + position_embeddings
x = self.dropout(x)
x = self.blocks(x)
x = self.final_norm(x)
logits = self.output(x)
loss = None
if targets is not None:
loss = F.cross_entropy(
logits.reshape(-1, logits.size(-1)),
targets.reshape(-1)
)
return logits, loss
@torch.no_grad()
def generate(
self,
input_ids,
max_new_tokens=MAX_NEW_TOKENS,
temperature=TEMPERATURE,
top_k=TOP_K
):
self.eval()
for _ in range(max_new_tokens):
input_context = input_ids[
:, -CONTEXT_SIZE:
]
logits, _ = self(input_context)
logits = logits[:, -1, :]
logits = logits / max(temperature, 0.01)
if top_k > 0:
k = min(top_k, logits.size(-1))
values, _ = torch.topk(
logits,
k
)
threshold = values[:, [-1]]
logits = logits.masked_fill(
logits < threshold,
float("-inf")
)
probabilities = F.softmax(
logits,
dim=-1
)
next_token = torch.multinomial(
probabilities,
num_samples=1
)
input_ids = torch.cat(
(input_ids, next_token),
dim=1
)
return input_ids
# ============================================================
# SAVE AND LOAD
# ============================================================
def save_checkpoint(model, optimizer, step):
temporary_file = MODEL_FILE + ".tmp"
torch.save(
{
"model": model.state_dict(),
"optimizer": optimizer.state_dict(),
"step": step,
"config": {
"vocab_size": model.token_embedding.num_embeddings,
"embed_size": EMBED_SIZE,
"num_heads": NUM_HEADS,
"num_layers": NUM_LAYERS,
"ff_size": FF_SIZE,
"context_size": CONTEXT_SIZE
}
},
temporary_file
)
os.replace(
temporary_file,
MODEL_FILE
)
print("Model saved.")
def load_checkpoint(model, optimizer):
if not os.path.exists(MODEL_FILE):
return 0
print("Loading existing model...")
checkpoint = torch.load(
MODEL_FILE,
map_location=DEVICE,
weights_only=False
)
expected_config = {
"vocab_size": model.token_embedding.num_embeddings,
"embed_size": EMBED_SIZE,
"num_heads": NUM_HEADS,
"num_layers": NUM_LAYERS,
"ff_size": FF_SIZE,
"context_size": CONTEXT_SIZE
}
if checkpoint["config"] != expected_config:
raise ValueError(
"Model configuration does not match the saved model. "
"Restore the original settings or use a new model filename."
)
model.load_state_dict(
checkpoint["model"]
)
if "optimizer" in checkpoint:
optimizer.load_state_dict(
checkpoint["optimizer"]
)
print("Existing model loaded.")
return checkpoint.get("step", 0)
# ============================================================
# LOAD TRAINING DATA
# ============================================================
def load_training_data(tokenizer):
if not os.path.exists(KNOWLEDGE_FILE):
raise FileNotFoundError(
"knowledge.txt was not found."
)
with open(
KNOWLEDGE_FILE,
"r",
encoding="utf-8"
) as file:
text = file.read()
if not text.strip():
raise ValueError(
"knowledge.txt is empty."
)
encoded = tokenizer.encode(text)
tokens = encoded.ids
eos_id = tokenizer.token_to_id("[EOS]")
tokens.append(eos_id)
if len(tokens) <= CONTEXT_SIZE:
raise ValueError(
"Not enough training tokens. "
"Add more text to knowledge.txt or reduce CONTEXT_SIZE."
)
return torch.tensor(
tokens,
dtype=torch.long
)
# ============================================================
# RANDOM TRAINING BATCHES
# ============================================================
def get_batch(data):
max_start = len(data) - CONTEXT_SIZE - 1
starts = torch.randint(
0,
max_start + 1,
(BATCH_SIZE,)
)
x = torch.stack([
data[i:i + CONTEXT_SIZE]
for i in starts.tolist()
])
y = torch.stack([
data[i + 1:i + CONTEXT_SIZE + 1]
for i in starts.tolist()
])
return (
x.to(DEVICE),
y.to(DEVICE)
)
# ============================================================
# TRAINING
# ============================================================
def train_model(model, optimizer, data, start_step):
model.train()
print("\nStarting training...")
for step in range(
start_step + 1,
start_step + TRAINING_STEPS + 1
):
x, y = get_batch(data)
optimizer.zero_grad(set_to_none=True)
_, loss = model(x, y)
loss.backward()
torch.nn.utils.clip_grad_norm_(
model.parameters(),
1.0
)
optimizer.step()
if step % 10 == 0:
print(
f"Step {step} | Loss: {loss.item():.4f}"
)
if step % SAVE_INTERVAL == 0:
save_checkpoint(
model,
optimizer,
step
)
save_checkpoint(
model,
optimizer,
start_step + TRAINING_STEPS
)
print("\nTraining completed.")
# ============================================================
# CHAT
# ============================================================
def chat(model, tokenizer):
print("\n================================")
print(" GPT CHAT")
print("================================")
print("Type /exit to quit.\n")
eos_id = tokenizer.token_to_id("[EOS]")
while True:
user_input = input("You: ").strip()
if user_input.lower() == "/exit":
break
if not user_input:
continue
prompt = (
"User: "
+ user_input
+ "\nAssistant:"
)
encoded = tokenizer.encode(
prompt,
add_special_tokens=False
)
input_ids = torch.tensor(
[encoded.ids],
dtype=torch.long,
device=DEVICE
)
output = model.generate(
input_ids,
max_new_tokens=MAX_NEW_TOKENS,
temperature=TEMPERATURE,
top_k=TOP_K
)
generated_ids = output[0][
len(encoded.ids):
].tolist()
if eos_id in generated_ids:
generated_ids = generated_ids[
:generated_ids.index(eos_id)
]
response = tokenizer.decode(
generated_ids,
skip_special_tokens=True
)
# Remove accidental continuation into another user turn.
if "\nUser:" in response:
response = response.split(
"\nUser:",
1
)[0]
print("\nAssistant:", response.strip(), "\n")
model.eval()
# ============================================================
# MAIN PROGRAM
# ============================================================
def main():
print("================================")
print(" SMALL GPT MODEL")
print("================================")
print("Device:", DEVICE)
tokenizer = create_tokenizer()
vocab_size = tokenizer.get_vocab_size()
print("Vocabulary size:", vocab_size)
model = GPTModel(
vocab_size
).to(DEVICE)
parameter_count = sum(
p.numel()
for p in model.parameters()
)
print(
"Parameters:",
f"{parameter_count:,}"
)
optimizer = torch.optim.AdamW(
model.parameters(),
lr=LEARNING_RATE,
weight_decay=0.01
)
start_step = load_checkpoint(
model,
optimizer
)
data = load_training_data(
tokenizer
)
print(
"Training tokens:",
f"{len(data):,}"
)
print(
"Previous training steps:",
start_step
)
train_model(
model,
optimizer,
data,
start_step
)
chat(
model,
tokenizer
)
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("\nProgram interrupted.")
except Exception as error:
print("\nERROR:", error)
input("\nPress Enter to close...")
I’ve already explained why it’s not. Look up the definition of foundational.
Different foundations exist, or maybe none exist. Your arguments are still circular.
You just did. I made a statement and you concluded it’s false. How did you arrive at that conclusion?
I used Meow-logic.
Category error. So you still don’t even understand what logic is . Got it.
Why would it be a category error?
Why would this be a problem? What is the issue with circular reasoning?
Circular reasoning is only okay when Meow says it is. So in this case, its not okay.
So *if* you respond using meow logic, *then* you do not have to utilize logic. I wonder if there is a name to describe what process if/then statements such as this are using…
Yes, they use Meow logic.
Dunning-Kruger at its finest.
Are you saying that my arguments against logic are using logic? So they are logically true then? Well, thats a concession from you, I guess.
Right now I am experimenting with random training batches.
This code I gave at start trains basically on entire text.
However, after talking with chatgpt, I also learned of the advantage of random batches. So model doesnt train on entire text each time, but on randomly selected batches.
If you want for AI to give you best possible code for AI very quickly, just run this prompt:
"Give full code for program which creates chatgpt 5 style model, or loads and updates one if it already exists, make model few million parameters, sub word level tokenizer. Program trains model on knowledge.txt file and then enables chatting with it."
You can use json instead of txt, but I prefer txt for now. Its best to start with few million parameters and then you can just increase parameters in code manually till you reach a point where your computer can no longer keep up.
I worship Satan, Venus, Tao and Meow.
Muslims should submit to Meow and worship the true God infallible Meowbah who is competent in animation and has many followers on YouTube.
Mathematically, there is no such number
Actually, mathematically, there is.
2 times 0 is 0.
So math thinks 0 is two times greater than 0.
Does that make sense? Meow thinks not.
I haven’t presented an argument for logic genius, all I’ve done so far is show you why your argument is childishly silly.
You used logic to disprove arguments against logic, so yeah it's circular reasoning and problem of circular dependency.
We rely on logic because it’s inescapably foundational.
False, I don't rely on logic.
To argue for logic, requires the use of logic. To argue against logic, requires the use of logic.
This statement is only true if logic is true, so it is another circular dependency. If all your arguments depend on logic, you have no non-circular way to disprove arguments against logic.
So you are presuming the validity of logic the second you engage in any kind of intended coherent thought.
This is circular reasoning. Really, I am amazed that someone can say "the moment you presume validity of logic, you presume validity of logic", and not realize how dumb he sounds.
The second your mind starts working to decide the best way to respond to this
Best way is to respond with Meow-logic. That way, I don't have to use logic.
Child she is carrying in her womb.
Also, more problems with math.
2 times -2 is -4.
By logic of math, -4 is two times greater than -2. But it's actually two times smaller. The negative numbers are by definition smaller than positive numbers the more they progress.
Math doesn't make sense because 2 times 0 is 0.
By logic of math, zero is two times bigger than zero.
It doesn't make any sense.
Math is like witchcraft.
AI model uses math and multiplication and non-linear calculations to transform input into output. So yeah, it's the work of a devil.
