Amazon SageMaker Inference Recommender In AWS Console

Amazon SageMaker Inference Recommender, a capability within Amazon SageMaker, now offers new features. Firstly, users can access Inference Recommender directly from the AWS console for SageMaker, making it more convenient to use. Secondly, Inference Recommender provides recommendations on prospective instances to deploy a model during the model creation process. Users can view the recommended instance list programmatically using the Describe Model API or through the SageMaker console UI. To optimize cost or performance, users can run benchmarking or load testing jobs with custom sample input payloads.

Amazon Elastic File System (Amazon EFS) has increased the maximum per file system throughput on Amazon EFS Provisioned Throughput mode by 3 times. This allows users to achieve up to 10 GiB/s of read throughput and 3 GiB/s of write throughput. The enhanced throughput benefits workloads such as machine learning, data processing, analytics, and transcoding that require consistent and high levels of throughput performance.

Amazon EC2 Dedicated Hosts now support targeted allocations in AWS Outposts racks. Users can allocate Dedicated Hosts to specific AWS Outpost hosts, enabling control over workload deployment location for low-latency goals or workload distribution across multiple Outposts racks for improved resiliency. This feature allows users to specify the AssetID associated with a physical server in an Outposts rack when allocating a new Dedicated Host, and multiple target AssetIDs can be specified for allocating more than one host.

AWS Security Hub now has enhanced management capabilities with AWS CloudFormation. Users can utilize CloudFormation to deploy Security Hub, manage its standards and controls, enable default standards like AWS Foundational Security Best Practices and CIS Foundations Benchmark, and opt into Consolidated Control Findings capability. The updated integration also allows users to enable specific security standards and manage individual controls within them using the AWS::SecurityHub::Standard resource. AWS CloudFormation StackSets can be employed to manage Security Hub across accounts and regions in a single action.

Amazon MQ now supports cross-region data replication for ActiveMQ brokers. This feature enables asynchronous message replication from a primary broker to a replica broker in a standby AWS region, facilitating the building of regionally resilient messaging applications. Cross-region data replication simplifies failover processes and ensures operational continuity in case of regional failures. Pricing for cross-region data replication includes replication charges, broker instance and storage charges, and data transfer charges between regions. This feature is available in multiple AWS regions.

These announcements introduce new capabilities and enhancements to Amazon SageMaker, Amazon EFS, Amazon EC2 Dedicated Hosts, AWS Security Hub, and Amazon MQ, empowering users with more efficient and flexible services for their various use cases.

Introducing Amazon SageMaker Inference Recommender

When deploying machine learning models, Amazon SageMaker Inference Recommender assists you in selecting the ideal compute instance and configuration for the best inference cost and performance.

Weeks of trial and error may be required to choose a compute instance with the optimal pricing performance for implementing machine learning models. Based on the quantity of the input data and the resource needs of your models, you must first select the appropriate ML instance type from a list of more than 70 alternatives. The model must then be optimized for the chosen instance type. Finally, in order to execute load tests and optimize cloud architecture for cost and performance, you must provision and manage infrastructure. All of this may cause delays in the time to market and model rollout.

SageMaker Inference Recommender Cost

Amazon SageMaker Inference Recommender automatically selects compute instance type, instance count, container settings, and model optimizations for inference to optimize performance and cost. SageMaker Inference Recommender lets you deploy your ML model in minutes using SageMaker Studio, the AWS CLI, or the AWS SDK. You can then deploy your model to one of the suggested instances or run a fully managed load test on a range of instance types without worrying about testing infrastructure. You can use SageMaker Studio load test findings to pick the ideal deployment configuration by weighing latency, throughput, and cost.

SageMaker Inference Recommender Availability

Amazon SageMaker Inference Recommender is available in all SageMaker locations except AWS China.