Monday, February 17, 2025

DeepSeek-R1-Distill Models Now Available On AWS Marketplace

You can now access DeepSeek-R1 models on AWS.

On December 2024, DeepSeek launched DeepSeek-V3. On January 20, 2025, DeepSeek published DeepSeek-R1, DeepSeek-R1-Zero, which had 671 billion parameters, and DeepSeek-R1-Distill, which had 1.5–70 billion parameters. On January 27, 2025, they introduced their vision-based Janus-Pro-7B model. The models are accessible to the general public and are said to be 90–95% less expensive and efficient than similar models. According to Deepseek, their model is notable for its capacity for reasoning, which was attained by utilising cutting-edge training methods including reinforcement learning.

DeepSeek-R1 models are now available for deployment in Amazon Bedrock and Amazon SageMaker AI.

For companies looking to quickly integrate pre-trained foundation models using APIs, Amazon Bedrock is the ideal option. For businesses that desire sophisticated deployment, training, and customisation together with access to the underlying infrastructure, Amazon SageMaker AI is perfect. Furthermore, DeepSeek-R1-Distill models can be cost-effectively deployed using AWS Trainium and AWS Inferentia through Amazon SageMaker AI or Amazon Elastic Compute Cloud (Amazon EC2).

You may utilise DeepSeek-R1 models with AWS to develop, test, and responsibly grow your generative AI concepts. This model is robust, affordable, and requires no infrastructure investment. You can also build on AWS services that are specifically tailored for security to confidently drive generative AI innovation. For generative AI applications that are accessible to both Amazon Bedrock and Amazon SageMaker AI users, we strongly advise integrating your DeepSeek-R1 model deployments with Amazon Bedrock Guardrails.

There are several options for deploying DeepSeek-R1 models on AWS right now: The DeepSeek-R1 model is available on Amazon Bedrock Marketplace, the DeepSeek-R1 model is available on Amazon SageMaker JumpStart, the DeepSeek-R1-Distill models are available on Amazon Bedrock Custom Model Import, and the DeepSeek-R1-Distill models are available on Amazon EC2 Trn1 instances.

Allow me to guide you through the many routes for launching DeepSeek-R1 models on AWS. These approaches offer adaptable beginning points depending on the needs and experience of your team, whether you’re developing your first AI application or scaling already-existing solutions.

The Amazon Bedrock Marketplace’s DeepSeek-R1 model

In addition to the existing selection of industry-leading models in Amazon Bedrock, the Amazon Bedrock Marketplace provides more than 100 well-known, up-and-coming, and specialised FMs. Models are simple to find in a single catalogue, subscribe to, and then install on controlled endpoints.

Navigate to the Amazon Bedrock console and choose Model catalogue from the Foundation models area to access the DeepSeek-R1 model in Amazon Bedrock Marketplace. You can search for DeepSeek or filter by model suppliers to locate it quickly.

You may immediately deploy the model by entering an endpoint name, selecting the number of instances, and selecting an instance type after reviewing the model description page, which includes the model’s capabilities and implementation suggestions.

Advanced configuration options, such as VPC networking, service role permissions, and encryption settings, allow you to alter the DeepSeek-R1 model’s architecture and security configuration. You should check these settings for production deployments to make sure they meet the security and compliance needs of your company.

You may independently assess model outputs and user inputs with Amazon Bedrock Guardrails. By screening out unwanted and hazardous information in generative AI apps, you may utilise your defined set of policies to manage how users interact with DeepSeek-R1. Only Bedrock’s ApplyGuardrail API can be used with the DeepSeek-R1 model in Amazon Bedrock Marketplace to assess user inputs and model replies for third-party and custom FMs that aren’t offered by Amazon Bedrock.

In order to create generative AI systems that are safer and more secure and compliant with responsible AI rules, Amazon Bedrock Guardrails can also be linked with other Bedrock technologies, such as Amazon Bedrock Agents and Amazon Bedrock Knowledge Bases.

For best results, use DeepSeek’s conversation template when utilising the DeepSeek-R1 model with the Playground Console and Bedrock’s InvokeModel API. For instance, < sentence beginning >< User>content for inference< Assistant>.

The Amazon SageMaker JumpStart DeepSeek-R1 model

With just a few clicks, you can deploy prebuilt machine learning (ML) solutions, FMs, and built-in algorithms with Amazon SageMaker JumpStart. You can find the DeepSeek-R1 model in SageMaker Unified Studio, SageMaker Studio, SageMaker AI console, or programmatically using the SageMaker Python SDK in order to implement it in SageMaker JumpStart.

Open SageMaker Studio or SageMaker Unified Studio in the Amazon SageMaker AI console. To find “DeepSeek-R1” in the All public models tab for SageMaker Studio, select JumpStart.

To create an endpoint with default parameters, pick the model and then select deploy. You can draw conclusions by submitting requests to the endpoint when it becomes inservice.

With Amazon SageMaker AI capabilities like Amazon SageMaker Pipelines, Amazon SageMaker Debugger, or container logs, you can determine model performance and ML operational controls. Data security is supported by the model’s deployment in an AWS secure environment and under your virtual private cloud’s (VPC) regulations.

Similar to Bedrock Marketpalce, you may isolate security measures for your generative AI applications from the DeepSeek-R1 model using the ApplyGuardrail API in SageMaker JumpStart. Regardless of the models you choose, the ability to apply guardrails without using FMs allows you to further integrate standardised and extensively proven corporate protections into your application flow.

Amazon Bedrock Custom Model Import for DeepSeek-R1-Distill models

Using a single serverless, unified API, Amazon Bedrock Custom Model Import enables you to import and utilise your customised models alongside pre-existing FMs without having to worry about maintaining underlying infrastructure. You may import DeepSeek-R1-Distill Llama models with 1.5–70 billion parameters using Amazon Bedrock Custom Model Import. The distillation method entails employing the bigger DeepSeek-R1 model with 671 billion parameters as a teacher model to train smaller, more effective models to replicate its behaviour and reasoning patterns.

Once these publicly accessible models have been stored in an Amazon Simple Storage Service (Amazon S3) bucket or an Amazon SageMaker Model Registry, import and deploy them using Amazon Bedrock in a fully managed and serverless environment by selecting Imported models under Foundation models in the Amazon Bedrock console. This serverless method offers enterprise-grade security and scalability without requiring infrastructure maintenance.

DeepSeek-R1-Distill models using AWS Inferentia and Trainium

From tiny CPU-only instances to the newest, most powerful multi-GPU instances, AWS Deep Learning AMIs (DLAMI) offer customised machine images that you may use for deep learning in a range of Amazon EC2 instances. For optimal price-performance, you can use AWS Trainuim1 or AWS Inferentia2 instances to deploy the DeepSeek-R1-Distill models.

To begin, launch a trn1.32xlarge EC2 instance with the Neurone Multi Framework DLAMI named Deep Learning AMI Neurone (Ubuntu 22.04) via the Amazon EC2 console.

Install vLLM, an open-source program for serving Large Language Models (LLMs), and download the DeepSeek-R1-Distill model from Hugging Face after connecting to your launched EC2 instance. The model server can be invoked and the model deployed using vLLM.

Things to be aware of

Here are some essential facts to be aware of.

  • Pricing: You are only billed for the infrastructure cost of the inference instance hours you choose for Amazon Bedrock Markeplace, Amazon SageMaker JumpStart, and Amazon EC2 for publically available models such as DeepSeek-R1. You are only billed for model inference for the Bedrock Custom Model Import, which is billed in 5-minute windows according to the quantity of copies of your custom model that are active. Visit the Amazon EC2 Pricing, Amazon Bedrock Pricing, and Amazon SageMaker AI Pricing pages to find out more.
  • Security: Security of data To help you keep your data and apps safe and private, Amazon Bedrock and Amazon SageMaker offer enterprise-grade security capabilities. This implies that your data is not used to enhance the models or shared with model providers. This holds true for all models, including the DeepSeek-R1 models on Amazon Bedrock and Amazon SageMaker, both proprietary and publicly accessible.

Currently accessible

DeepSeek-R1 is currently widely accessible on Amazon SageMaker JumpStart and Amazon Bedrock Marketplace. DeepSeek-R1-Distill models can also be used with Amazon EC2 instances running AWS Trainum and Inferentia chips, as well as Amazon Bedrock Custom Model Import.

Drakshi
Drakshi
Since June 2023, Drakshi has been writing articles of Artificial Intelligence for govindhtech. She was a postgraduate in business administration. She was an enthusiast of Artificial Intelligence.
RELATED ARTICLES

Recent Posts

Popular Post

Govindhtech.com Would you like to receive notifications on latest updates? No Yes