Building LLMs for Production

Understanding Fine-Tuning

While pre-training gives large language models (LLMs) a general understanding of language, it falls short for instruction-following tasks. LLMs are trained to predict the next token in its training data (often web pages) and this doesn’t necessarily make it immediately capable of answering questions or following instructions. For example, if a user types in a regular Google search or a pre-trained LLM, like “In what country is Montreal?”, rather than generating an answer the LLM might instead generate lists of similar questions with similar meanings. Such a model would struggle to give good answers to requests like summarizing a web page or generating SQL queries. Fine-tuning is a method to address these limitations ...