1d:I[70616,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"Breadcrumb"] 1e:I[90762,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"ArticleGrid"] 1f:T17a5,

Server sizing for large language models has followed one crude rule: count the parameters, multiply by the precision, buy that much GPU memory. DeepSeek's V4.1 Flash breaks the rule. Its weights total 763 billion, which at FP8 implies a 763GB floor of accelerator memory, yet the model can be served from roughly 567GB because 196 billion of those parameters are designed to run from ordinary system RAM instead.

Key takeaways

Roughly 196GB of one model's weights can be moved off the accelerator entirely, changing which server configuration a deployment requires.
The weights that can move are lookup tables read a few dozen entries at a time, so they never compete for the memory bandwidth that limits token generation.
The saving only holds if the tier beneath the GPU is fast, which shifts engineering effort from GPU budgets toward system memory and storage I/O planning.

Why bandwidth, not capacity, sets the ceiling

Generating a token requires reading a model's active weights out of memory. That read, repeated for every token, is why memory bandwidth rather than raw arithmetic usually caps throughput, and why accelerator memory commands the price it does. Capacity matters, but the reason weights sit on the GPU at all is the speed at which they must be streamed.

That logic only applies to weights that get streamed. Some do not. DeepSeek's conditional memory module holds its 196 billion parameters as N-gram lookup tables — learned associations over short runs of tokens — and consulting them costs a few dozen table reads per token rather than a sweep through the whole block. Weights accessed that sparsely have no claim on the fastest memory in the system.

The practical consequence is a split inventory. About 8 billion parameters stay active per token and must be fast. The lookup pool can be parked in system RAM, or on a storage array quick enough to answer without stalling. The Register's reading of the technical report describes the goal as separating what a model knows from what it must compute.

What this does to a procurement conversation

Roughly 200GB is not a rounding error. It is frequently the gap between one class of node and the next, and therefore between one capital plan and another. A buyer who sizes on parameter count alone over-provisions; a buyer who sizes on the offloaded figure without checking the memory tier under it under-provisions and pays in latency.

Neither error is easy to avoid today, because model cards publish one memory number rather than two. The useful disclosure is a split: how much must be resident on the accelerator, and how much may live a tier down. Expect that question to enter vendor evaluations before it enters documentation.

There is a second figure worth demanding. Weights are only part of what fills a GPU — key-value caches grow with context length and with the number of simultaneous users, and in high-throughput chat or agent workloads they often dominate. The same DeepSeek release attacked that separately, using reworked attention and a new causal encoder-decoder to bring cache use down to between 13 and 25 percent of the previous Flash model. In capacity terms that is four to eight times the concurrent sessions per unit of memory, which for most operators lands closer to home than the parameter arithmetic does.

How portable is the technique

The approach is not proprietary. Google's Gemma team built a related mechanism, Per-Layer Embedding, to fit useful models onto phones, where every tier of the hierarchy is constrained. DeepSeek published its N-gram variant in January, and Alibaba has since built a 51-billion-parameter lookup pool into an experimental Qwen model. The design is spreading because the economics are legible to anyone who has priced an accelerator against a stick of DRAM.

What remains unsettled is how far down the hierarchy it survives. System RAM is a proven destination. Storage is a claim, and one that depends heavily on the array. Teams evaluating this should benchmark their own inference stack rather than trust a headline figure, because the saving is real but conditional, and the condition is infrastructure they already own or do not.

It also adds a variable to change management. A provider that alters what runs behind an endpoint can alter the memory profile with it, a risk that surfaced this week when a pinned model name began serving different weights with no opt-out.

FAQ

Can any model's weights be offloaded to system RAM?

No. Only weights accessed sparsely enough to avoid bulk streaming can move without a latency penalty, which in practice means lookup-style parameters rather than the active weights used for every token. Offloading conventional dense weights reintroduces the bandwidth bottleneck the GPU exists to solve.

How much GPU memory does DeepSeek V4.1 Flash actually need?

Its 763 billion parameters at FP8 imply a 763GB minimum if everything is resident on the accelerator. Offloading the 196-billion-parameter lookup pool lowers that floor to about 567GB. Production serving requires additional headroom for key-value caches, which scale with context length and concurrent users.

Does offloading weights slow the model down?

It depends entirely on the tier receiving them. Lookups served from system RAM are fast enough that the architecture holds, while a slow storage array can stall a lookup and return the latency the design was meant to save. Benchmarking against your own hardware is the only reliable answer.

a:["$","div",null,{"className":"mx-auto max-w-7xl px-4 py-8 sm:px-6","children":[["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"BreadcrumbList\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https://ainewsway.com\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"#gpu-computing\",\"item\":\"https://ainewsway.com/tag/gpu-computing\"}]}"}}],["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"ItemList\",\"@id\":\"https://ainewsway.com/tag/gpu-computing#itemlist\",\"name\":\"#gpu-computing Articles\",\"numberOfItems\":1,\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"url\":\"https://ainewsway.com/articles/the-200gb-question-which-model-weights-actually-need-to-sit-on-a-gpu-mtwqf1n8\",\"name\":\"The 200GB Question: Which Model Weights Actually Need to Sit on a GPU\"}]}"}}],["$","$L1d",null,{"items":[{"label":"# gpu-computing"}]}],["$","h1",null,{"className":"mb-8 text-3xl font-bold text-zinc-900 dark:text-zinc-100","children":["#","gpu-computing"]}],[["$","$L1e",null,{"articles":[{"id":"c32f1c60-81de-4f7e-986a-1311cb8fbb54","slug":"the-200gb-question-which-model-weights-actually-need-to-sit-on-a-gpu-mtwqf1n8","title":"The 200GB Question: Which Model Weights Actually Need to Sit on a GPU","subtitle":"Memory hierarchy, not parameter count, is becoming the number that decides what an AI server costs","content":"$1f","excerpt":"DeepSeek V4.1 Flash needs 567GB of GPU memory rather than 763GB because 196 billion of its weights are built to run from system RAM instead.","coverImage":"https://cdn.ainewsway.com/uploads/2026/09/f4b1bd9f-043e-4686-a498-21cf8ac7a791.webp","coverImageCaption":"Server racks in a data center, where the memory tier beneath the GPUs now determines how much accelerator capacity a model actually needs","authorId":"6773f67d-d7fa-4b19-8f38-384f10ac761b","status":"PUBLISHED","publishedAt":"2026-09-11T09:07:52.926Z","viewCount":2,"likeCount":0,"readingTime":5,"isFeatured":false,"isBreakingNews":false,"metaTitle":"Which LLM Weights Really Need GPU Memory?","metaDescription":"Sizing AI servers by parameter count now over-provisions. One 763B model runs on 567GB of GPU memory by moving lookup weights to system RAM.","metaKeywords":["GPU memory sizing","LLM server cost","weight offloading","DeepSeek V4.1 Flash","KV cache","AI infrastructure planning","N-gram parameters"],"ogImage":null,"aiSummary":"Parameter count is losing its value as a proxy for AI server cost. DeepSeek's V4.1 Flash totals 763 billion parameters, implying a 763GB GPU memory floor at FP8, but serves from roughly 567GB because 196 billion of those weights are lookup tables that run from system RAM. The saving depends on the speed of the memory tier beneath the accelerator, pushing capacity planning toward a two-number disclosure that model cards do not yet provide.","aiTags":[],"sentiment":null,"articleType":"ANALYSIS","videoEmbed":null,"videoType":null,"seoScore":85,"isAutoGenerated":true,"isBackfill":false,"pipelineStage":null,"copyrightScore":null,"similarityScore":null,"matchedSourceId":null,"createdAt":"2026-09-11T09:06:27.477Z","updatedAt":"2026-09-15T11:36:09.915Z","contentUpdatedAt":"2026-09-15T11:36:09.915Z","deletedAt":null,"author":{"id":"6773f67d-d7fa-4b19-8f38-384f10ac761b","name":"Seung Jung","username":"Seung Jung","avatar":null},"categories":[{"id":"e2b5fffc-de32-42aa-86fe-c75f219d6b4b","name":"AI & Machine Learning","slug":"ai-ml"}],"tags":[{"id":"e9d9ed20-6211-4e4c-a2af-eb3c5eced7c7","name":"AI Infrastructure","slug":"ai-infrastructure"},{"id":"a29858a6-9ef9-4eb4-a7c1-749ecbd09380","name":"DeepSeek","slug":"deepseek"},{"id":"661a50e8-d3f4-456c-9773-9ffc9be024e7","name":"AI Inference","slug":"ai-inference"},{"id":"e2eff9f2-fedf-4bdf-8474-4ea5cdd1fe20","name":"GPU Computing","slug":"gpu-computing"},{"id":"bb8d3605-627b-4977-96e1-49ba5d3460e5","name":"Data Centers","slug":"data-centers"}],"translations":[{"id":"7bda80ba-9db1-4330-8f11-b031e57c0bc9","language":"ko","title":"200GB의 질문: 모델 가중치 중 정말 GPU에 올라가야 하는 것은 무엇인가","subtitle":"AI 서버 가격을 결정하는 숫자가 파라미터 수에서 메모리 계층으로 옮겨가고 있다","excerpt":"DeepSeek V4.1 Flash는 763GB가 아니라 567GB의 GPU 메모리로 돌아간다. 가중치 1960억 개가 시스템 RAM에서 실행되도록 설계됐기 때문이다.","metaTitle":"LLM 가중치, 정말 GPU 메모리에 올라가야 하나","metaDescription":"파라미터 수로 AI 서버를 잡으면 과잉 투자가 된다. 7630억 파라미터 모델이 룩업 가중치를 시스템 RAM으로 내려 567GB로 돌아간다."},{"id":"2519d748-d82d-4f10-a746-a04dbc0b10e5","language":"ja","title":"200GBの問い：モデルの重みのうち、本当にGPUに置くべきものはどれか","subtitle":"AIサーバーの価格を決める数字は、パラメータ数からメモリ階層へ移りつつある","excerpt":"DeepSeek V4.1 Flashが763GBではなく567GBのGPUメモリで済むのは、1960億の重みがシステムRAM上で動くよう設計されているからだ。","metaTitle":"LLMの重み、本当にGPUメモリに要るのはどれか","metaDescription":"パラメータ数でAIサーバーを見積もると過剰になる。7630億パラメータのモデルがルックアップ重みをシステムRAMへ移し567GBで動く。"},{"id":"6ba69600-0a03-4617-8cff-5650e4bc5cc1","language":"zh","title":"200GB之問：模型權重裡，究竟哪些必須待在GPU上","subtitle":"決定一臺AI伺服器價格的數字，正從引數量轉向記憶體層級","excerpt":"DeepSeek V4.1 Flash只需567GB視訊記憶體而非763GB，因為其中1960億引數本就是按跑在系統記憶體上設計的。","metaTitle":"LLM權重裡，哪些才真正需要佔用視訊記憶體","metaDescription":"只按引數量配AI伺服器會買多。一個7630億引數模型把查表權重挪到系統記憶體，用567GB視訊記憶體即可執行。"},{"id":"6b0505f7-b5a1-4093-98f3-a81a91471484","language":"es","title":"La pregunta de los 200GB: qué pesos de un modelo necesitan de verdad estar en la GPU","subtitle":"La jerarquía de memoria, y no el número de parámetros, se está convirtiendo en la cifra que decide cuánto cuesta un servidor de IA","excerpt":"DeepSeek V4.1 Flash necesita 567GB de memoria de GPU en lugar de 763GB porque 196.000 millones de sus pesos están hechos para ejecutarse desde la RAM del sistema.","metaTitle":"¿Qué pesos de un LLM necesitan memoria de GPU?","metaDescription":"Dimensionar servidores de IA por número de parámetros sobredimensiona. Un modelo de 763.000 millones corre con 567GB al pasar pesos de consulta a la RAM."}],"celebrities":[{"id":"1829636e-4136-46e9-b226-e1f9e580f7f9","nameKo":"딥시크","nameEn":"DeepSeek","slug":"deepseek","profileImage":null}]}]}],"$L20"]]}]

#gpu-computing

200GBの問い：モデルの重みのうち、本当にGPUに置くべきものはどれか

ニュースレター購読

#gpu-computing

200GBの問い：モデルの重みのうち、本当にGPUに置くべきものはどれか