Errata

Figures 2-4 and 2-5 show the same token ID for different tokens:

the -> 278
b -> 278

A token identifier must be unique and the text is aligned with this idea, however the diagrams show as counter examples of this property.

On Page 49, the last two points explaining the insights/observations of GPT-2 Tokenizer are exactly the same, now either by mistake the same point has been reprinted twice, or instead of one more valid insight/observation, it has been replaced with the previous one. Please check it out & fix it!

On Page 51, in the section explaining observations of GPT-4 Tokenizer, in the 3rd point of that, its mentioned that word "tokens" has been represented by using 1 token, however, if you see the output of GPT-4 Tokenizer given just above the observations at the same page, the word "_tokens" has been marked as a single word.

Now, either of the two cases are possible:
1. "_tokens" is marked as 1 word by mistake in the output of GPT-4 Tokenizer, in reality, it should be 2 tokens i.e., "_" & "tokens".
2. Observation explanation is wrong, instead of mentioning "tokens" is represented using 1 token, it should be written as "_tokens" is represented using 1 token.

Please check this out & fix this!

'This is an encoder that forcuses on code generation'
should be
'This is an decoder that forcuses on code generation'

In right upper fig. and right lower fig., token ID 50,000 should be 49,999.

'that capture abolute and relative token position information' should be 'that capture relative token position information'

Here, it suggests evaluating generalization on the validation set when hyperparameter tuning is done using the training and test sets, but this is not the standard practice. Hyperparameter tuning should use the validation set, while the test set should be used only once at the end to assess the final generalization performance of the trained model.

'The cosine similarity is the angle between two vectors' should be 'The cosine similarity is the cosine of the angle between two vectors'

The correct code is:

# Visualize topics and documents
fig = topic_model.visualize_document_datamap(
titles,
topics=list(range(20)),
reduced_embeddings=reduced_embeddings,
width=1200,
#label_font_size=11, # les paramètres label_font_size, label_wrap_width et use_medoids ne font pas partie des arguments officiels de visualize_document_datamap dans BERTopic
#label_wrap_width=20,
#use_medoids=True,
datamap_kwds={ # Use this dictionary for advanced settings instead parameters
"label_font_size": 11,
"label_wrap_width": 20,
"use_medoids": True,
},
)
plt.savefig("datamapplot.png", dpi=300)

'adding it to the `data` variable' should be 'adding it to the `text` variable'

The float 16-bit representation is incorrect.

Float 16-bit (Half Precision) in IEEE 754 Format:
- Bit 15: Sign
- Bits 14–10: Exponent (5 bits and not 8!, biased by 15)
- Bits 9–0: Mantissa (10 bits, with implicit leading 1 for normalized numbers)

`system_prompt` is not included in the template. And there is no need to include <s> token in the template because llama-cpp-python automatically add it.

llm_chain.invoke("a girl that lost her mother")

should be

llm_chain.invoke({"summary" : "A girl that lost her mother"})

'Conversation history' should be 'Current conversation'.

# Generate a conversation and ask for the name
llm_chain.invoke({"input_prompt": "Hi! My name is Maarten. What is 1 + 1?"})
llm_chain.invoke({"input_prompt": "What is my name?"})

should return

{'input_prompt': 'What is my name?',
'chat_history': ' Maarten introduces himself and asks for the sum of 1 + 1, to which the AI responds that it equals 2. The AI provides a brief explanation about addition being a basic arithmetic operation resulting from combining numbers, in this case adding one unit to another to get a total of two units or items.',
'text': " Your name was mentioned as Maarten when you introduced yourself; therefore, based on the current conversation, your name is Maarten.\nHere's an explanation for 1 + 1 = 2: Addition is one of the four fundamental arithmetic operations and involves combining quantities. When we add 1 unit to another 1 unit, we are essentially counting up by one from the first number (which is 1), arriving at a total of two units or items."}

'similarity scores between 1 and 5' should be 'similarity scores between 0 and 5'

`trainer.train()` is omitted.