AWS SageMaker Lets Devs Own the Training Loop

Amazon Web Services on Thursday, October 22, introduced training session APIs for Amazon SageMaker AI, a service that lets developers write their own large language model training loop in Python on a laptop while AWS runs the GPUs behind it. The AWS Machine Learning Blog announcement pitches it as a way to keep full control of the algorithm without building or babysitting a GPU cluster.

Key takeaways

• Each GPU step, such as a forward-backward pass or an optimizer step, is an API call against a live session that keeps the model loaded between steps.
• Three open-weight models are supported at launch: GPT-OSS-20B, Gemma-4-31B-it and Qwen-3.6-27B, in up to three AWS Regions.
• Billing is per token processed, and AWS says GPU time is not charged while code runs locally, such as during reward scoring.

How SageMaker training session APIs work

Fine-tuning an LLM "usually forces a tradeoff between training loop control and operational simplicity," the four AWS authors wrote. Managed fine-tuning hides the algorithm; custom training jobs hand over the infrastructure. The new APIs split the difference: "Instead of abstracting away the algorithm, they abstract away the compute."

Developers work through three clients in the SageMaker Python SDK. A ServiceClient holds settings such as the IAM role and S3 output path and starts sessions. A TrainingClient runs forward, forward_backward and optim_step and saves checkpoints. A SamplingClient generates text and scores it with sample and compute_logprobs.

Built-in loss functions cover supervised fine-tuning (cross-entropy) and reinforcement learning (PPO, importance sampling and CISPO). Training uses LoRA adapters; AWS's example uses rank 32. AWS says the service handles distributed execution, gradient synchronization, fault recovery, tenant isolation and checkpoint storage.

Supported models, Regions and limits

According to the SageMaker developer guide, GPT-OSS-20B runs in US East (N. Virginia), US West (Oregon) and Asia Pacific (Mumbai). Gemma-4-31B-it and Qwen-3.6-27B are listed for Oregon and Mumbai only. Charges are split into prefill, sample and train tokens; the guide links to the pricing page rather than listing rates. Accounts start with a default quota of two concurrent sessions, which can be raised.

Every saved checkpoint is registered as a version in a model package group. It can be merged and deployed to a real-time endpoint or served as a LoRA adapter. An active session also exposes an OpenAI-compatible chat completions endpoint for quick testing.

What AWS's own tests show

In a GSM8K math example using GRPO on GPT-OSS 20B, 10 steps on 40 problems raised held-out accuracy from 56% to 75%, AWS reported. A longer coding run of 50 steps, with 128 problems per step and eight responses each, lifted accuracy on 200 held-out problems from 40% to 50.5%. These are vendor-run examples, not independent benchmarks.

Context: a familiar API shape

The design closely resembles Tinker, the fine-tuning API from Thinking Machines Lab. Tinker's TrainingClient reference lists the same core method names, including forward_backward, optim_step and save_weights_for_sampler. The AWS post does not mention Tinker. For AWS, the release adds a fourth rung to its SageMaker range, between serverless model customization and full training jobs or HyperPod clusters. It follows other recent AWS developer releases, including the open-source Strands harness.

What's next

AWS published a training sessions cookbook on GitHub with recipes for Tulu 3 and UltraChat fine-tuning, GSM8K and coding RL, and KTO preference alignment. The announcement does not say whether the service is in preview or generally available, and per-token prices were not published in the post. The short model list will be the main constraint for teams that want to train larger or different models.

FAQ

Do I need to pick GPU instances?
No. AWS describes the service as serverless with no instance selection. Code runs locally, and GPU operations run as API calls on AWS-managed hardware.

Which models can I fine-tune?
At launch, the developer guide lists GPT-OSS-20B, Gemma-4-31B-it and Qwen-3.6-27B.

How is it billed?
By tokens processed, split into prefill, sample and train. GPU time is not billed while local code runs.

References

• Write your own training loop, without managing GPUs, with Amazon SageMaker training session APIs — AWS Machine Learning Blog (Oct. 22, 2026)
• Training session APIs — Amazon SageMaker AI Developer Guide
• GitHub - aws/sagemaker-training-sessions-cookbook — AWS
• TrainingClient API reference — Thinking Machines Lab Tinker docs