Use Llama-2 Models with Amazon SageMaker

Contributor Name: Federico Castanedo

Difficulty level: Easy

Llama-2 models are available through Amazon SageMaker JumpStart and will be also available soon on Amazon Bedrock. Amazon Bedrock is a fully managed service that makes large language models (LLMs) available through an API so you can choose the model that’s best suited for your application.

On the other hand, Amazon SageMaker JumpStart is a machine learning hub that can help to accelerate your machine learning journey. Through JumpStart, you can access LLMs but also built-in algorithms for different use cases and prebuilt solution templates. Available models and solutions can be customized and deployed into dedicated AWS instances.

Since SageMaker provisions infrastructure on your behalf, it gives you more flexibility than Bedrock but involves more environment setup and charges you for endpoints when they are running. Bedrock is focused on offering an API-driven and serverless experience where you are only charged for what you use.

In this Shortcut, we will focus on Llama-2 models using SageMaker. At the time of writing, Llama-2 models are available in the following AWS regions: us-east-1 (fine-tunable), us-east-2 (inference only), us-west-2 (fine-tunable), eu-west-1 (fine-tunable), and ap-southeast-1 (inference only).

Please ensure you have a SageMaker Domain ready to use for this Shortcut.

Once you log into your Amazon SageMaker console, you will see a JumpStart menu on ...