Amazon ECS GPU Sharing: Run Multiple Tasks on One GPU with NVIDIA MPS

On 27 October 2026, AWS announced that Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS). You can now run several Amazon ECS tasks at the same time on a single NVIDIA GPU, instead of reserving a whole GPU for every container. For teams running small-model inference, embedding generation or multi-model serving, that means fewer idle GPUs and a smaller monthly bill, without changing application code.

GPU instances are among the most expensive resources teams run on AWS, and an idle GPU costs exactly the same as a busy one. This guide explains how Amazon ECS GPU sharing works, when to use it, and how to roll it out safely step by step.

Why GPU Sharing Matters

Until now, a GPU container on Amazon ECS asked for a whole number of GPUs. A small inference service that needed only a few gigabytes of GPU memory still locked an entire GPU, and the rest of that capacity sat idle.

AWS’s GPU sharing documentation puts it plainly: many GPU workloads, such as inference and small model serving, use only a fraction of a GPU’s compute and memory. NVIDIA’s own guidance on when to use MPS says the same thing: MPS is useful when each application process does not generate enough work to saturate the GPU, because it lets leftover capacity be used by CUDA kernels from other processes.

How Amazon ECS GPU Sharing Works

MPS is an NVIDIA CUDA feature that runs multiple processes concurrently on the same GPU. Amazon ECS manages it for you. It starts the MPS daemon on GPU instances, places containers onto GPUs based on available memory, and enforces per-container limits.

• Memory-based placement: ECS adds up the GPU memory requested by all GPU containers in a task and finds a single GPU with enough free memory. All GPU containers in one task land on the same physical GPU.
• Cross-task sharing: tasks from different services can share the same GPU as long as it has memory left. Shared and dedicated GPU tasks can run on the same instance, on different GPUs.
• Two limits per container: a required memory value in MiB, which is a hard ceiling, and an optional maxComputePercent from 1 to 100. The compute value is a ceiling, not a reservation, and ECS places tasks on memory only.
• Self-healing on Managed Instances: on Amazon ECS Managed Instances, ECS monitors the GPUs and the MPS daemon. If either fails, it drains and replaces the instance.

According to the AWS announcement, the feature is available in all AWS commercial and AWS GovCloud (US) Regions, which includes the Mumbai and Hyderabad Regions for Indian teams.

Good Fits and Poor Fits

AWS recommends GPU sharing for inference, embedding generation, small model serving, multi-model serving, and development and test environments. It is not recommended for large model training that needs the full GPU memory, or for workloads that need hardware-level isolation. MPS is software-level sharing, and if one container triggers a fatal GPU error, other containers on that physical GPU could be affected.

If you need hard isolation, AWS also documents g6f instances on ECS Managed Instances, which provide hardware-partitioned slices (1/8, 1/4 or 1/2) of an NVIDIA L4 GPU with dedicated memory and compute.

Step-by-Step: Enabling GPU Sharing on Amazon ECS

1. Measure what each model really uses

Before you pack anything, record the peak GPU memory of each model under realistic load. The memory value you set becomes a hard limit, so size it from peak usage plus a safety margin, not the average.

2. Pick a supported launch type and instance family

GPU sharing works on ECS Managed Instances and ECS on Amazon EC2. It is not supported on AWS Fargate, ECS Express Mode, ECS Anywhere or managed daemon tasks. Supported families are G4dn, G5, G6, G6e, G7, G7e, P3, P4d, P5, P5e, P5en and P6-B200.

• Managed Instances: no extra setup beyond the task definition. NVIDIA drivers and the CUDA toolkit come pre-installed.
• ECS on EC2: use the ECS AL2023 GPU-optimized AMI version 20261019 or later with ECS container agent 1.107.0 or later.

3. Add a sharing strategy to your task definition

In each GPU container’s resourceRequirements, replace the integer value with a sharingStrategy block:

"resourceRequirements": [
  {
    "type": "GPU",
    "sharingStrategy": {
      "nvidiaMps": {
        "memory": 8192,
        "maxComputePercent": 50
      }
    }
  }
]

Keep the validation rules in mind. A GPU requirement uses either value or sharingStrategy, never both, and all GPU containers in a task must use the same strategy. You cannot set sharingStrategy in a RunTask or StartTask container override, and ECS reserves the CUDA_MPS_PINNED_DEVICE_MEM_LIMIT and CUDA_MPS_ACTIVE_THREAD_PERCENTAGE environment variables, so do not set them yourself.

4. Run one GPU process per container

AWS recommends one GPU process per container with MPS. If an application spawns several GPU processes, size memory for their combined peak, because ECS uses that number for placement and a multi-process app can exceed it and cause memory contention.

5. Separate noisy or sensitive services

Tasks from different services on the same capacity provider can be placed on the same GPU. To keep a critical service away from experimental ones, give them separate capacity providers or separate clusters.

6. Verify and monitor

Call DescribeContainerInstances and check the gpuDevices field, which shows sharingMode, containerCount and remainingMemoryInMiB for each GPU. Note that per-task and per-container GPU utilization and memory metrics are not emitted for MPS-shared GPUs. Instead, track savings with instance-level Container Insights metrics: compare InstanceGPUUsageTotal with InstanceGPULimit, and watch RunningTaskCount to confirm higher task density.

Cost and Operations Tips

• Start in dev and test: shared GPUs are an easy win for staging and test environments, where isolation needs are lower.
• Use compute ceilings deliberately: set maxComputePercent on batch-style containers so they cannot starve latency-sensitive inference. Idle compute still flows to other containers up to their own ceilings.
• Right-size after consolidating: once services are packed, review whether you can drop to fewer or smaller GPU instances, and then revisit Savings Plans coverage.
• Codify it: define task definitions and capacity providers in CloudFormation or Terraform so limits are reviewed like any other code change.

GPU consolidation touches containers, capacity planning and cost governance together. Our cloud DevOps services team helps set up ECS clusters, capacity providers and CI/CD for GPU workloads, and our cost optimization and infrastructure management practice helps you measure GPU waste and turn it into real savings.

Frequently Asked Questions

Do I need to change my application code to use Amazon ECS GPU sharing?

No. AWS says you can pack more workloads onto each GPU instance without changing application code. You only change the task definition by adding a sharingStrategy with a GPU memory limit.

Does GPU sharing work on AWS Fargate?

No. It is supported on Amazon ECS Managed Instances and Amazon ECS on Amazon EC2, but not on Fargate, ECS Express Mode, ECS Anywhere or managed daemon tasks.

Is MPS sharing safe for multi-tenant workloads?

Be careful. MPS is software-level sharing, so a fatal GPU error in one container can affect others on the same GPU. For strict isolation, use dedicated GPUs, separate capacity providers, or hardware-partitioned g6f instances.

Is the feature available in India?

Yes. AWS says GPU sharing is available in all AWS commercial and AWS GovCloud (US) Regions, so it covers the Mumbai and Hyderabad Regions.

Want to Cut Your GPU Bill on AWS?

Shanav Technologies helps startups and growing businesses run AI and machine learning workloads on AWS efficiently, from ECS cluster design to GPU right-sizing and cost reviews. If your GPUs are sitting half-idle, talk to our team and we will help you plan a safe move to shared GPUs and estimate the savings for your workloads.