AWS, Savings Plans

LLM cost optimization on AWS: How to cut GPU costs in 2026

Reduce AWS LLM infrastructure costs by choosing the right accelerators, improving GPU utilization, and applying Spot, Capacity Blocks, and commitments to the right workloads.

Priyanka

Content Strategist

Updated September 18, 2026

21 min read

Key takeaways

GPU selection should be based on measured cost per request or token, throughput, p95 latency, memory headroom, and workload requirements, not GPU generation alone.

AWS Inferentia and Trainium can also be worth evaluating where model and framework compatibility allow.

Spot Instances can offer discounts of up to 90% compared with On-Demand pricing for workloads that tolerate interruption.

Capacity Blocks provide scheduled accelerator capacity, but reservations cannot be cancelled after purchase.

Rightsize first. Commit only the recurring spend you can defend if traffic, architecture, or model requirements change.

For broader optimization principles, see our cloud cost optimization guide.

LLM cost optimization on AWS starts with infrastructure efficiency. Choose the right accelerator, improve utilization, use Spot for interruption-tolerant workloads, reserve capacity only when availability matters, and apply Savings Plans only to a stable compute baseline.

AWS also reduced prices for several NVIDIA GPU instance families in June 2025. If your LLM budget still uses older rates, your forecasts may overstate current infrastructure costs.

Start with the workload, not the discount

Different LLM workloads need different capacity and purchasing strategies.

Workload	Option to evaluate	Cost strategy	Main risk
Steady production inference	GPU or purpose-built accelerator	Rightsize, batch, then cover stable spend	Low utilization
Large training or fine-tuning	P-family or Trainium	Capacity Blocks, Spot, or stable commitment	Capacity availability
Fault-tolerant batch inference	Flexible EC2 fleet	Spot where practical	Interruption
Scheduled accelerator project	Capacity Blocks	Reserve the required window	Paying for an incorrect schedule
Compatible inference stack	Inferentia	Benchmark price-performance	Migration and framework compatibility

AWS recommends selecting EC2 infrastructure based on workload and budget before choosing the purchasing model. Its EC2 cost and capacity guidance also positions Trainium and Inferentia as purpose-built options for suitable AI workloads.

Benchmark before you buy

Do not compare accelerators using hourly price alone. Test equivalent workloads with the same model, precision or quantization, prompt mix, context length, batch configuration, and latency target.

Use this scorecard:

Metric	What it tells you
Cost per million output tokens	Effective serving economics
Tokens per second	Throughput capacity
p95 latency	User-facing performance
Accelerator memory headroom	Rightsizing opportunity
CPU and network utilization	Non-accelerator bottlenecks
Interruption recovery time	Suitability for Spot
Regional availability	Deployment feasibility

A higher hourly rate can still produce a lower cost per token if the instance delivers materially better throughput.

Recalculate budgets using the 2025 GPU price reductions

AWS reduced pricing for several NVIDIA GPU accelerated families in June 2025. Its official GPU pricing update lists Amazon Linux On-Demand reductions of up to 45% for P5, 26% for P5en, and 33% for P4d and P4de. Eligible Savings Plan purchases effective after June 4, 2025 also reflect the reduced pricing. Review the official AWS GPU pricing update

Consider a simplified P5 example where a configuration affected by the full 45% Amazon Linux On-Demand reduction previously cost $100,000 per month.

Estimated revised cost = $100,000 × (1 - 0.45) = $55,000

Illustrative difference = $45,000 per month

This is an example, not a guaranteed saving. Actual pricing depends on Region, operating system, instance configuration, utilization, purchase model, and existing commitments.

Match the pricing model to workload stability

Use Savings Plans for a durable baseline

AWS Savings Plans provide discounted eligible compute pricing in exchange for a fixed hourly spend commitment for one or three years. Compute Savings Plans can apply across EC2 instance families, sizes, operating systems, tenancy, and Regions.

The financial commitment cannot simply be reduced because demand falls, so commitment sizing should follow workload optimization, not precede it.

For commitment sizing principles, see our AWS Savings Plans guide.

For GPU fleets:

Defendable commitment floor = recurring eligible hourly spend expected to remain even under a downside scenario

Leave experimental, seasonal, or burst demand outside that floor.

Run a commitment-risk test first

Before committing, model at least three scenarios:

Traffic or inference demand declines.

The architecture moves to another GPU family, AWS accelerator, or managed service.

A training or fine-tuning program ends earlier than expected.

If the proposed hourly commitment is still consumed in all three scenarios, it is a stronger candidate for native commitment coverage.

Use Spot for interruption-tolerant workloads

AWS Spot Instances use spare EC2 capacity and can offer discounts of up to 90% compared with On-Demand pricing.

Good candidates include checkpointed training, batch inference, preprocessing, testing, and experimentation.

Avoid relying on Spot alone for latency-sensitive production endpoints or training jobs that cannot recover from interruption.

Our On-Demand vs Reserved vs Spot guide explains the broader purchasing trade-offs.

Use Capacity Blocks when capacity assurance matters

Capacity Blocks for ML reserve supported accelerator capacity for a specified window. AWS offerings can start in as soon as 30 minutes, can be reserved up to eight weeks ahead, and support eligible reservation durations extending up to 182 days.

Capacity Blocks cannot be cancelled after purchase, so confirm the instance configuration, Region, dates, and workload schedule before buying.

Check the current AWS Capacity Blocks documentation and pricing immediately before budgeting because offerings and rates can change.

Improve utilization before buying more capacity

Low utilization can make an inexpensive GPU costly per request.

Focus on three areas:

Batch compatible inference requests to improve accelerator utilization.

Test quantization where lower precision maintains acceptable model quality.

Set realistic context and concurrency limits to control memory requirements.

For NVIDIA EC2 instances, GPU metrics are not automatically present in every CloudWatch configuration. AWS requires the CloudWatch agent to be configured for NVIDIA GPU metrics, and the instance needs an NVIDIA driver.

AWS CloudWatch NVIDIA GPU metrics documentation explains the required configuration.

Practical callout: Do not use a long-term discount to compensate for an oversized fleet. Rightsize the workload first, then apply discounts to the stable remainder.

When not to choose NVIDIA GPUs

NVIDIA GPU instances are not automatically the lowest-cost option for every LLM workload.

Evaluate AWS Inferentia for compatible inference workloads and Trainium for compatible training or model-serving workloads. These purpose-built accelerators can provide another price-performance option when the software stack supports them.

The decision still requires benchmarking. Migration effort, AWS Neuron compatibility, framework support, model architecture, operator support, latency, and throughput can outweigh headline infrastructure savings.

Step-by-step AWS LLM cost optimization process

Separate training, fine-tuning, real-time inference, and batch workloads.

Configure accelerator utilization and memory monitoring.

Benchmark candidate instances with identical model and request settings.

Compare cost per token or request, latency, throughput, and memory headroom.

Apply batching, quantization, and context optimization where appropriate.

Test Inferentia or Trainium where compatibility allows.

Move interruption-tolerant workloads to Spot where practical.

Use Capacity Blocks when scheduled workloads require predictable capacity.

Calculate the defendable recurring hourly spend floor.

Apply Savings Plans only after testing downside scenarios.

LLM infrastructure optimization checklist

Workloads are classified by training, inference, batch, or experimentation.

Accelerator utilization and memory metrics are collected.

Current AWS pricing is reflected in the model.

Candidate instances are benchmarked with equivalent settings.

Inferentia or Trainium has been evaluated where relevant.

Spot suitability is tested.

How Usage.ai helps reduce AWS commitment risk

At Usage.ai, we focus on reducing the management burden and long-term exposure associated with eligible cloud commitments.

With eligible Flex Insured Commitments, we can help you capture up to 57% savings of a three-year commitment with none of the commitment risk, depending on the service, configuration, payment option, usage profile, and program eligibility, while reducing the long-term commitment exposure you would otherwise take on.

We provide cashback protection for eligible Flex Commitments that we optimize, manage, generate savings from, and bill under the program. Cashback protection does not automatically apply to existing customer-owned Savings Plans, Reserved Instances, or commitments outside the Flex Commitment Program.

Review our Flex Commitment eligibility requirements before applying to the program.

Our fee is based on realized savings rather than total cloud spend. You can review our current pricing model for current terms.

OPTIMIZE AWS AI COMMITMENTS

See how much LLM spend you can safely commit.

Review stable AI compute, coverage, and eligible AWS spend while reducing commitment exposure.

Book a Savings Assessment

Frequently asked questions

What is the best way to reduce LLM costs on AWS?

Start by measuring utilization and benchmarking suitable accelerators. Then improve batching and model efficiency, use Spot for interruption-tolerant work, use Capacity Blocks when capacity assurance is needed, and apply commitments only to stable recurring usage.

How should I compare AWS GPU instances for LLM inference?

Compare cost per million tokens or requests, tokens per second, p95 latency, accelerator memory headroom, CPU and network bottlenecks, and regional availability using the same model and workload configuration.

Can Spot Instances reduce LLM training costs?

Yes. AWS states that Spot can offer discounts of up to 90% compared with On-Demand pricing. It is best suited to checkpointed or restartable jobs that can tolerate interruption.

Should I evaluate Inferentia and Trainium instead of GPUs?

Yes, when your model and framework are compatible. Inferentia is designed for machine learning inference, while Trainium is designed for deep learning workloads. Benchmark them against GPU alternatives before choosing.

When should I use AWS Savings Plans for LLM workloads?

Use them when you have a recurring hourly compute baseline that is likely to remain throughout the commitment term. Rightsize first and test what happens if demand or architecture changes.

Disclosure: AWS pricing, accelerator availability, Spot discounts, Savings Plans rates, Capacity Block offerings, and Usage.ai program terms can change. Savings figures depend on workload characteristics, services, configurations, payment options, and eligibility. Verify current AWS documentation and Usage.ai program terms before making purchasing or commitment decisions.