Appendix IV — Custom Text Completion with GPT-2
This appendix, relating to Chapter 7, The Rise of Suprahuman Transformers with GPT-3 Engines, describes how to customize text completion with a GPT-2 model.
This appendix shows how to build a GPT-2 model, train it, and interact with custom text in 12 steps.
Open Training_OpenAI_GPT_2.ipynb, which is in the GitHub repository of this appendix. You will notice that the notebook is also divided into the same 12 steps and cells as this appendix.
Run the notebook cell by cell. The process is tedious, but the result produced by the cloned OpenAI GPT-2 repository is gratifying. We are not using the GPT-3 API or a Hugging Face wrapper.
We are getting our hands dirty to see how the model is built and trained. ...
Get Transformers for Natural Language Processing - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
