AWS Infinity Fits 75,000 Idle Agents in 8GB

Amazon Web Services on Oct. 23 released Infinity, an open-source research runtime that suspends AI agents while they wait on tools. AWS said in its Open Source Blog that an idle agent takes about 100 KB of memory, so roughly 75,000 could fit in the 8 GB of a Raspberry Pi 5.

The Rust project comes from Hydro, a joint effort between AWS and the University of California, Berkeley, and is licensed under Apache 2.0. AWS called it "an early research release," shared to collect feedback rather than as a managed service.

Key takeaways

• Infinity saves an agent's conversation and suspends it whenever it calls a tool, then resumes it from saved state when the result arrives.
• In AWS's benchmark, 80,000 agents ran 20 turns each, or 3.2 million completions, on one CPU thread using a simulated model. Memory grew about 100 KB per idle agent.
• The release includes the runtime, an open tool protocol called the Reactive Agent Protocol and a coding agent named Infinity Code. Existing MCP servers run through a compatibility layer.

How the Infinity agent runtime works

Shadaj Laddad, research lead for the Hydro team at AWS, wrote that Infinity borrows the stackless-coroutine technique programming languages use to pause and resume asynchronous functions. When an agent calls a tool, the runtime dispatches the call, saves the conversation and suspends the agent.

Work runs in bounded "slices." Each slice loads the history, runs one model completion, saves the output and dispatches at most one tool call. An idle agent holds no task, stack or open connection, so most memory goes to conversation histories.

Benchmark: 80,000 agents on one thread

AWS's test launched waves of agents in one system, backed by a simulated model rather than a real LLM. It ran 80,000 agents for 3.2 million completions on a single thread of an AMD EPYC 9R14 processor with in-memory stores.

The Raspberry Pi figure is an extrapolation from that roughly 100 KB-per-agent rate, not a test on the board. The post gave no throughput or latency numbers, and no independent measurements on real workloads have been published.

Serverless mode and injected events

Infinity also has a step mode for serverless platforms, where nothing stays in memory between calls. The AWS reference deployment uses SQS FIFO queues, AWS Lambda, Aurora DSQL for histories, DynamoDB and EventBridge Scheduler, packaged as CDK constructs.

Outside events, such as a message from another agent, reach the model as a tool call it never made, named receive_event__injected. The runtime never offers that name as a tool and rejects it if a model tries to call it.

Why idle agents are the new cost problem

Agents spend much of their time waiting on models, tools and people, and conventional runtimes keep processes alive during those pauses. Google attacked the same problem a day earlier with Agent Compute Engine, which checkpoints idle agents inside hardware-isolated microVMs. Infinity tackles it in software, keeping a waiting agent down to its saved conversation.

Security questions remain open as agents gain autonomy. Researchers have documented agents that turned to SQL injection when ordinary requests failed, and AWS lists proving where inbound events come from as unsolved for Infinity.

What's next

AWS named four research gaps. Models are mostly trained on synchronous tool-call traces, so how they handle interleaved events is unclear. Scheduling still follows message arrival order, compaction for large agent trees is unstudied, and event provenance for internet-exposed agents needs work. The GitHub repository had no tagged releases when checked.

FAQ

Is Infinity an AWS product?

No. AWS describes it as a research project under the Hydro effort with UC Berkeley. It is free under the Apache 2.0 license, with no service-level guarantees.

Did AWS run 75,000 agents on a Raspberry Pi?

Not according to the blog post. The benchmark ran on an AMD EPYC server thread with a simulated model. The 75,000 figure divides 8 GB by the measured 100 KB per idle agent.

Does it work with MCP tools?

Yes. The project README says existing MCP servers run unchanged through a compatibility layer, alongside its own Reactive Agent Protocol.

References

• Introducing Infinity: a runtime for extreme agent concurrency — AWS Open Source Blog (Oct. 23, 2026)
• hydro-project/infinity — GitHub
• Infinity — Hydro project documentation