Building AI Intensive Python Applications

9

LLM Output Evaluation

Regardless of the form factor of your intelligent application, you must evaluate your use of large language models (LLMs). The evaluation of a computational system determines the system’s performance, gauges its reliability, and analyzes its security and privacy.

AI systems are non-deterministic. You cannot be certain what an AI system will output until you run an input through it. This means that you must evaluate how the AI system performs on a variety of inputs to have confidence that it performs in line with your requirements. To be able to change the AI system without introducing any unexpected regressions, you also need to have robust evaluations. Evaluations can help catch these regressions before releasing the ...