Prompt Engineering for Generative AI

Table of contents

1. 1. Five Principles of Prompting
   1. What is Prompt Engineering?
   2. 1. Give Direction
   3. 2. Specify Format
   4. 3. Provide Examples
   5. 4. Evaluate Quality
   6. 5. Divide Labor
2. 2. Intro to Image Generation Models
   1. What are Diffusion models?
   2. OpenAI DALL-E
   3. Midjourney
   4. Stable Diffusion
3. 3. Standard Practices for Text Generation
   1. Generating Lists
   2. Hierarchical List Generation
   3. Generating JSON
      1. YAML
   4. Filtering YAML Payloads
   5. Handling Invalid Payloads in YAML
   6. Explain It Like I’m Five
   7. Ask For Context
   8. Text Style Unbundling
   9. Identifying the Desired Textual Features
   10. Alternatively Extract Specific Relevant Textual Features
   11. Generating New Content with the Extracted Features
   12. Summarization
   13. Chunking Text
       1. Benefits of chunking text
       2. Poor Chunking Example
   14. Applying Prompt Engineering Principles
   15. Sentence Detection using SpaCy
   16. Building a simple chunking algorithm in Python
   17. Sliding Window Chunking
   18. Text chunking with Tiktoken
   19. Encodings
       1. Understanding the tokenization of strings
   20. Estimating Token Usage for Chat API Calls
   21. Sentinment Analysis
       1. Techniques for Fine-tuning Sentiment Analysis
       2. Limitations and Challenges in Sentiment Analysis
   22. Least To Most
       1. Planning the Architecture
       2. Coding Individual Functions
       3. Adding Tests
       4. Benefits of the Least To Most Technique
   23. Role Prompting
   24. Summary
4. 4. Standard Practices for Image Generation
   1. Format Modifiers
   2. Art Style Modifiers
   3. Reverse-Engineering Prompts
   4. Quality Boosters
   5. Negative Prompts
   6. Weighted Terms
   7. Inpainting
   8. Outpainting
   9. Meme Unbundling
   10. Meme Mapping
5. 5. Vector Databases
   1. What are Vector Databases?
   2. What are Embeddings?
   3. Document Loading
   4. Memory Retrieval with FAISS
   5. Hosted Vector Databases with Pinecone
6. 6. Advanced Techniques for Text Generation
   1. Meta Prompting
   2. GPT Best Practices
      1. Avoiding Hallucinations with Reference
      2. Give GPT’s “Thinking Time”
      3. The Inner Monologue Tactic
      4. Critique Previous LLM Responses
   3. Introduction to LangChain
      1. Environment Setup
      2. Making an OpenAI LLM Request
      3. Generating an OpenAI Chat Model Request
   4. Langchain Prompt Templates
   5. Output Parsers
   6. Creating Few-Shot Prompt Templates
      1. Constructing the Example Set
      2. Formatting the Examples
      3. Creating the FewShotPromptTemplate
      4. Selecting by Length
      5. Saving and Loading LLM Prompts
   7. Data Connection
      1. Document Loaders
   8. Text Splitters
   9. Text Segmentation with Recursive Character Splitting
   10. VectorStores
       1. Building a simple QA retrieval system using FAISS
       2. Similarity Search with Score
       3. Loading and Saving the Vector Store
       4. Filtering and Similarity Search
       5. Maximal Marginal Relevance (MMR)
   11. Retrievers
       1. Vector Databases
       2. Self Querying
       3. Alternative Retrieval Mechanisms
   12. Task Decomposition
   13. Prompt Chaining
       1. LLMChain
       2. Sequential Chain
       3. Document Chains
       4. Stuff
       5. Refine
       6. Map Reduce
       7. Map re-rank
   14. Chain of Thought
   15. Agents
       1. Reason and Act (ReAct)
       2. Tools
       3. Agent ToolKits
       4. Using LLMs as an API (OpenAI Functions)
   16. Memory
       1. Long Term Memory (LTM):
       2. Short Term Memory (STM):
       3. Short-Term Memory in QA Conversation Agents:
       4. Adding Memory to an Agent in LangChain
   17. Memory in LangChain
       1. Preserving the state:
       2. Querying the state:
       3. ConversationBufferMemory
   18. Popular Memory Types in LangChain
       1. Conversation Buffer Window Memory
       2. Conversation Summary Memory
       3. Conversation Summary Buffer Memory
       4. ConversationToken Buffer Memory
   19. Advanced Agent Frameworks
       1. Plan and Execute Agents
       2. Tree of Thoughts
   20. Callbacks
       1. Global (Constructor) Callbacks
       2. Request-specific Callbacks
       3. The Verbose Argument
       4. When to Use Which?
       5. Token Counting with LangChain
   21. Classification with LLMs
   22. Building A Classification Model
   23. Majority Vote For Classification
7. 7. Advanced Techniques for Image Generation
   1. Stable Diffusion and AUTOMATIC1111
   2. Img2Img
   3. Upscaling Images
   4. Interrogate CLIP
   5. SD Inpainting and Outpainting
   6. ControlNet
   7. Segment Anything Model (SAM)
   8. Textual Inversion
   9. Dreambooth Fine-Tuning