05. Agentic AI Requires More HBM, Server DRAM, and Enterprise SSDs
AI INDUSTRY ESSAYS · 05
Inference needs HBM, execution needs server DRAM, and context reuse needs enterprise SSDs
As of October 13, 2026 KST · Review of official technical documentation, product materials, and joint research
Agentic AI that handles long contexts and multiple tasks concurrently in large data centers requires more HBM, server DRAM, and enterprise SSDs. HBM holds the context being used for inference, server DRAM supports running tasks and extended context, and SSDs hold context and data for reuse. HDD demand depends on how much of that data is retained for long periods and retrieved again.
The core demand is for HBM, server DRAM, and enterprise SSDs
Active inference needs HBM, tool execution and context expansion need server DRAM, and context reuse needs enterprise SSDs. Frequent updates and low latency drive TLC demand; large, read-intensive datasets drive QLC demand; and economical bulk retention drives HDD demand. This is an assessment of how demand develops as large-scale agent services expand. It does not mean that component usage across all services, or growth in these three markets, will rise at the same rate.
1. Classify DRAM and HBM on a consistent basis first
HBM is not a storage device that operates on a principle separate from DRAM. It is high-bandwidth memory that stacks multiple DRAM chips and provides wide interconnections. Adding “total DRAM + HBM” when aggregating industry demand can therefore count HBM twice. The title’s “server DRAM” and the article’s conventional DRAM, distinguished from HBM, refer to DDR5- and LPDDR-family system memory used in data centers. [1]
SSDs are nonvolatile storage devices that primarily use NAND flash. TLC stores 3 bits per cell, while QLC stores 4. HDDs record data on magnetic media. These are not a descending ranking from “good memory to bad memory,” but options that combine speed, capacity, power, endurance, and cost differently. The NAND type alone does not determine an SSD product’s performance. [7][8]
Representative roles in agentic AI · Actual placement varies by system
Category: HBM
Primary role: Model computation on accelerators; active KV cache
Performance to check first: Bandwidth, capacity per accelerator, power
Conditions that increase demand: More inference accelerators, long contexts, high concurrency
Category: Conventional DRAM
Primary role: CPU workspace, execution state, memory cache, context expansion
Performance to check first: Capacity per server, bandwidth, power, interconnection architecture
Conditions that increase demand: Growth in tool execution, search, sandboxes, and concurrent tasks
Category: Enterprise TLC SSDs
Primary role: Frequent reads and writes, fast reloading, performance-oriented caches
Performance to check first: Latency, sustained write performance, endurance
Conditions that increase demand: Repeated state updates and latency-sensitive data access
Category: Enterprise QLC SSDs
Primary role: Large, read-intensive datasets; high-density storage
Performance to check first: Usable capacity, read performance, write conditions, total cost
Conditions that increase demand: Growth in documents and data that are read often but rewritten less often
Category: High-capacity HDDs
Primary role: Data lakes, records and outputs, the capacity tier of a tiered cache
Performance to check first: Total cost per TB, aggregate throughput, recovery time
Conditions that increase demand: Long retention periods, large datasets, low urgency
Here, the KV cache consists of values stored to reuse computations on context the model has already read. It does not mean the user’s business records or an entire database. Ordinary business data and computational caches that can be regenerated can also have different retention periods and levels of protection. [3][6]
2. Concurrent tasks and context matter more than answer length
Imagine an agent searching for information, opening files, running code, and reviewing the results. Even if its final report is short, there are documents it has read, programs still running, and task states for multiple subagents along the way. Real agent frameworks also treat file systems, subagents, context summarization, and external storage as separate capabilities. [2]
Daily token output alone therefore cannot determine the required memory capacity. What matters is how many contexts are live at the same time, how long each is, and how long they remain live. One hundred agents processing tasks sequentially and one hundred processing them simultaneously have different peak memory requirements.
KV cache capacity for a conventional full-attention model ≈ 2 (K and V) × number of layers × number of KV heads × head dimension × bytes per value × context tokens × concurrent contexts
To illustrate the scale, assume 32 layers, 8 KV heads, a head dimension of 128, and 2 bytes per value. The KV cache is then 131,072 bytes, or 128KiB, per token. The table below extends this calculation. These are not measurements of a particular commercial model, and they exclude model weights, temporary computational memory, replication, and management overhead. [3]
Direct calculations assuming the same model architecture · 1GiB = 2³⁰ bytes
Context length: 32,768 tokens
Concurrent contexts: 1
KV cache per context: 4GiB
Total: 4GiB
Context length: 32,768 tokens
Concurrent contexts: 16
KV cache per context: 4GiB
Total: 64GiB
Context length: 131,072 tokens
Concurrent contexts: 16
KV cache per context: 16GiB
Total: 256GiB
Context length: 131,072 tokens
Concurrent contexts: 64
KV cache per context: 16GiB
Total: 1TiB
If context length quadruples and the number of concurrent contexts also quadruples, KV capacity in this example increases 16-fold. This must not be used as a growth rate for every model. Requirements vary substantially with the actual number of KV heads, compressed precision, sliding windows, latent-attention architectures, and cache sharing. The purpose of the calculation is to examine more specific demand drivers than simply the “number of AI users.”
3. Assess HBM capacity and bandwidth together
HBM keeps a large model’s weights and active context close to the accelerator and supplies data quickly for computation. Insufficient capacity limits the contexts and concurrent requests that can be handled, while insufficient bandwidth can leave compute units waiting for data. Explaining HBM demand only as “larger models need more memory” captures just half the picture. [1][4]
Long contexts and repeated inference in agentic AI can increase these pressures. However, inference is not always constrained solely by memory bandwidth. The bottleneck shifts among computation, memory, and communication depending on the stage that processes inputs together versus the stage that generates new tokens sequentially, as well as batch size and model architecture. What matters is the actual work completed at the target response time, rather than a product’s peak specification.
Moving part of the cache to DRAM or SSDs can make more efficient use of limited HBM. This does not mean SSDs replace an equal amount of HBM capacity: active data needs a path and time to move back up. If each accelerator handles more requests, the HBM burden per task can fall, while total HBM purchases can still rise if service usage expands sufficiently. These changes are not contradictory. [6]
4. Conventional DRAM is the workspace for AI execution
For AI to act after reasoning, CPUs and conventional software do the work. They process webpages and files, query search indexes, execute code, and manage progress across multiple tasks. As more of these execution environments operate concurrently, conventional DRAM workspace becomes more important. Not every agent needs a separate server or operating system, but more isolated execution environments and live data can increase the memory budget. [2]
Conventional DRAM also expands the context available for inference. Micron has analyzed an LPDDR tier that moves KV cache out of HBM, and Samsung has published KV-offloading experiments using a CXL-based DRAM pool. These examples show that memory outside HBM can also affect inference efficiency. However, the performance reported in these white papers depends on specific interconnection architectures, software, and cache-hit conditions. It does not mean that “adding CXL improves every server’s performance.” [5][12]
Commercialization is also progressing. In its September 30, 2026 earnings release, Micron announced sampling of 512GB DDR5 RDIMMs, qualification of high-speed server RDIMMs with multiple customers, and more than a doubling of server LPDDR SOCAMM revenue from the previous quarter. Sampling, qualification, and revenue growth are indicators of different stages. They cannot all be attributed to demand created by agentic AI, but they do provide evidence that high-capacity, efficient server memory is expanding into actual business. [9]
From an investment perspective, demand depends not only on server counts but also on memory capacity per server and power efficiency. Some systems benefit from high-capacity DDR5 modules, while others use LPDDR-family memory to reduce power and area. Sharing a memory pool among multiple servers can also reduce overprovisioning. Growth in AI servers therefore cannot be directly substituted for growth in conventional DRAM bit demand.
5. TLC and QLC SSDs serve different demand
SSD demand should be segmented according to how often data is reread and rewritten, rather than simply the idea that “AI creates a lot of data.” Sustained write performance and endurance matter where intermediate task states and caches are frequently updated. By contrast, high storage density and read performance become more valuable when document collections or datasets to be reread are large and writes are relatively infrequent.
NAND type is the starting point; product specifications and actual workloads determine the final choice
Comparison: Suitable demand
Areas to assess for TLC: Frequent state updates, write-heavy caches, tasks requiring low latency
Areas to assess for QLC: Large, read-intensive data; storage tiers for documents and vector data; reusable datasets
Comparison: Key metrics to check
Areas to assess for TLC: Sustained write speed, write endurance, small-I/O latency distribution
Areas to assess for QLC: Cost per usable TB, read latency and throughput, allowable writes and write patterns
Comparison: Common misconceptions
Areas to assess for TLC: Not every TLC product offers high endurance or the highest performance
Areas to assess for QLC: QLC is neither read-only nor inherently unsuitable for AI
Comparison: Purchasing decisions
Areas to assess for TLC: Compare host writes, amplification, spare space, and lifetime conditions for each product
Areas to assess for QLC: Check whether the capacity advantage actually reduces rack, power, and operating costs
Within the same manufacturing generation and similar designs, QLC increases density by storing more bits per cell, at the cost of greater challenges in write and endurance design. But controllers, parallelism, firmware, spare capacity, and product generation change the outcome. Comparing only one TLC SSD’s maximum sequential speed against another QLC SSD’s maximum capacity cannot determine a winner. Small random accesses and long sequential reads are different tests. [7][8]
Comparing endurance only by DWPD, which indicates how many full-drive writes are possible per day, is also insufficient. For example, hypothetically, 1 DWPD for a 30TB product and 0.3 DWPD for a 100TB product both represent a nominal host-write budget of 30TB per day. Applying this in practice requires checking the warranty period, random versus sequential write conditions, TBW limits, and write amplification. A lower DWPD does not necessarily mean a lower absolute write allowance.
Specifications for actual products also vary with write conditions. Micron’s September 2026 materials list its 6600 ION QLC products of 61.44TB and above at 1.0 DWPD for 128KB sequential writes and 0.075 DWPD for 4KB random writes. These are reference theoretical maximum endurance figures for the respective patterns and must be distinguished from warranty figures. Even the same SSD can have figures this different. [8]
In March 2026, NVIDIA presented shared flash as a separate tier for inference context through STX and CMX. This direction means SSDs handle reusable computational state, beyond simply storing model files. However, partner systems were scheduled for availability in the second half of the year at the time of the announcement; a design announcement must not be read as completed adoption by all customers. How much TLC and QLC each system uses also requires separate verification. [6][13]
Actual shipments can also be verified. In its September 30, 2026 announcement, Micron said its 7600 and 9650 SSDs were shipping for KV-cache use at major customers and that 6600 ION volumes had also ramped. This is evidence of shipments by a specific supplier. It does not mean that growth rates for the entire AI SSD market or the profitability of each product have been disclosed. [9]
6. HDDs also connect to AI demand
HDDs have a role in retaining large datasets for long periods at relatively low cost. As enterprise documents, source data, logs, outputs, and training and evaluation data accumulate, they need a large-capacity storage foundation. However, not all data is retained for the long term. Retention periods and the value of reuse determine the capacity purchased.
It is also inaccurate to treat HDDs solely as “backup devices unrelated to AI” in light of recent developments. A joint SK hynix and Seagate white paper tested KV-cache offloading using SSDs and HDDs together. Older KV data resides on HDDs, is loaded onto SSDs when reused, and subsequent requests are then served from SSDs. The disclosed configuration used 4 SSDs and 12 HDDs. [10]
This example does not demonstrate that HDDs replace HBM for active inference. It shows the possibility that HDDs can serve as the capacity tier of a larger cache store if the first-reaccess delay is acceptable and subsequent reuse is handled by a faster tier. The experiment used synthetic multi-turn workloads and distinguished the delay of the first HDD read from performance on subsequent SSD hits. This result alone cannot establish the same benefit for every real-time AI service. [10]
For HDDs, the key measures are therefore high-capacity data-center demand, shipped bytes and pricing, and actual storage-system costs, rather than PC unit shipments. QLC SSDs can compete with HDDs where they gain an advantage in total cost including power, space, and performance. The two complement each other when SSDs provide fast front-end access and improve utilization of the HDD capacity tier.
7. Which demand matters most?
HBM is the foundation for active inference; server DRAM and enterprise SSDs are where to look for the additional demand created as agents continue their tasks. Frequent updates and fast reloading are central to TLC, while large, read-intensive datasets are central to QLC. HDD purchases depend on the amount retained and whether the delay on first reaccess is acceptable. These are distinctions between the paths that create demand, not rankings of market growth or investment returns. [6][7][10]
Actual purchasing priorities depend on the service’s bottleneck. The following items are an analytical classification based on publicly described architectures.
Find the workload bottleneck first, then connect it to component demand
Growing workload: Real-time inference with long contexts
Demand to check first: HBM bandwidth and capacity; DRAM context expansion
Why this demand matters: Must hold active tasks and meet the target response time
Growing workload: Concurrent code and tool execution by multiple agents
Demand to check first: Conventional DRAM; SSDs with suitable write and latency performance
Why this demand matters: Execution environments and state updates can create bottlenecks beyond computation
Growing workload: Repeated searches across large document collections
Demand to check first: DRAM indexes and caches; enterprise SSD read performance
Why this demand matters: Requires both source-document capacity and actual search and retrieval speed
Growing workload: Tasks resumed after a long pause
Demand to check first: DRAM, SSD, and HDD tiers matched to reuse rates
Why this demand matters: Retention and reloading must cost less than recomputation
Growing workload: Long-term retention of large datasets and records
Demand to check first: Total-cost competition between HDDs and high-density QLC SSDs
Why this demand matters: Capacity and retention costs become more important when access is infrequent
The demand most directly linked to agentic AI is for concurrent workspace and the ability to deliver reusable context on time. This is broader than HBM alone and narrower than growth in all stored data. Adding the same SSD produces different results in a service short of storage space and one constrained by its network.
A better question than “Does NAND grow at the same rate as AI revenue?” is therefore “How much data remains per completed task, and how much of it is reread, how quickly?” Some buyers purchase more devices because throughput becomes insufficient before capacity does, while others delay new purchases by improving utilization of the same devices.
8. Token growth is not a proxy for memory growth
Factors that restrain demand also exist. Cache quantization reduces capacity needs by lowering the precision of stored values, and cache sharing can reduce the burden of storing duplicate content. Summarizing older context or retrieving information only when needed reduces the data that must be loaded at once. Conversely, when rereading is costly, retaining data for a period can be more advantageous. [2][3]
vLLM’s prefix caching reuses KV data for requests with the same prefix, reducing repeated input processing. Its official documentation distinguishes the time saved in the input-processing stage from the time spent generating new tokens, which this feature does not directly reduce. A high cache-hit rate alone cannot be interpreted as an equal proportional reduction in total inference time. [11]
Economic rationale for retaining a reusable cache = avoidable recomputation cost − retention, transfer, retrieval, and operating costs
Basis for long-term storage capacity = newly stored bytes per day × retention days × additional storage multiplier for protection
The second formula should use newly stored volume after compression and deduplication, with replication and deletion policies reflected separately. Assuming every generated token is stored permanently can overstate demand. If a model or execution configuration changes so that existing KV data cannot be reused, the value of retaining old computational state for a long time also falls.
Even if better memory efficiency reduces usage per task, total usage can grow faster as services become cheaper and easier to use. The opposite is also possible. Which effect dominates must be assessed from actual concurrent usage, context distributions, repeat usage, and growth in paid tasks, rather than the maximum context length in a product announcement.
9. Connecting component demand to corporate profits
A component’s growing importance does not mean its manufacturer’s profits grow at the same rate. Prices per bit can fall even as supply volumes rise, and yield, packaging, and development costs can weigh on earnings even as the share of high-performance products increases. Customer qualification and actual deployment, supply contracts, mass-production capability, and competitors’ capacity additions must be assessed together.
For DRAM manufacturers, assess capacity per system, pricing, and product mix separately for HBM and conventional DRAM. For NAND manufacturers, examine enterprise SSD revenue and shipped capacity, as well as changes in product performance, density, and profitability. For HDD manufacturers, connect data-center capacity shipments and capacity per drive to prices and costs. In every case, allocating revenue to individual companies solely from an announced AI market size has a weak evidentiary basis.
The metrics to check also differ by role. For HBM, look at actual accelerator supply and memory content; for conventional DRAM, capacity per server and the expansion of execution environments; for SSDs, enterprise shipments and growth in read and write workloads; and for HDDs, demand for bulk retention and growth in usable capacity. The same data may be replicated across multiple tiers, so it is also important not to add each tier’s capacity and call the total independent end-data demand.
Agentic AI makes use of memory and storage in more varied ways. The central change is not about unconditionally storing as much data as possible. It is about delivering currently needed data quickly, retaining reusable data economically, and deleting low-value data. Examining which components best achieve this balance helps reveal both the scale and the quality of demand.
Sources and verification basis
Roles were classified using official technical documentation and joint vendor experiments. Vendor product and performance claims are read within their stated conditions, not treated as measurements of the entire market. The KV-capacity and DWPD examples in the article were calculated directly from the stated assumptions. Verified: 2026-10-13 KST.
[1] SK hynix — Five things you need to know about SK hynix and the future of AI memory. HBM’s stacked-DRAM structure and the roles of accelerator memory, system memory, and storage.
https://news.skhynix.com/en/5-things-about-skhynix/
[2] LangChain — Deep Agents overview. Official implementation descriptions of execution environments, file-based context management, subagents, and long-term memory.
https://docs.langchain.com/oss/javascript/deepagents/overview
[3] Hugging Face — Unlocking Longer Generation with Key-Value Cache Quantization, 2024. KV-cache formulas and the relationships among precision, speed, and capacity. The example in this article does not reproduce the source model’s figures.
https://huggingface.co/blog/kv-cache-quantization
[4] Micron — Powers AI Everywhere at COMPUTEX 2026, 2026-06-01. Distinct roles of HBM, DDR, LPDDR, and data-center SSDs. Product availability and performance conditions follow the announcement’s main text and footnotes.
[5] Samsung — Optimizing KV Cache Offloading to CMM-D in a CXL Switch-based Memory Pool, white paper pp.4–14. Experiments under specific topology, kernel, and cache-hit conditions; they do not represent performance across all conventional servers.
[6] NVIDIA — CMX Context Memory Storage Platform, 2026-03-16. Distinctions among active HBM, system RAM, local SSDs, shared context tiers, and persistent storage.
[7] KIOXIA — QLC vs. TLC Flash Storage Technology: How to Choose, 2026-09-29. Bits per cell, trade-offs between density and writes or endurance, and workload-specific selection criteria.
https://blog-us.kioxia.com/post/2026/09/29/qlc-vs-tlc-flash-storage-technology-how-to-choose
[8] Official Micron product specifications: 6600 ION, Rev. C 09/2026, p.3 and 9650, Rev. B 05/2026, pp.2–3. G9 QLC and G9 TLC, respectively. The 6600 ION’s DWPD figures by write pattern are reference theoretical maxima and are distinct from warranty figures.
[9] Micron — FY2026 Fourth-Quarter Earnings Release, Product Highlights, 2026-09-30. Company statements on server DDR5 sampling and qualification, LPDDR SOCAMM revenue growth, 7600 and 9650 shipments for KV-cache applications, and the ramp in 6600 ION volumes.
[10] SK hynix·Seagate — Enabling inference at massive scale with hybrid storage for KV cache offloading, 2026, pp.7–12. Joint SSD/HDD experimental setup and the difference between first reaccess and subsequent SSD hits. The tested configuration must be distinguished from extrapolations to larger scales.
[11] vLLM — Automatic Prefix Caching. Distinction between input processing reduced by context reuse and the token-generation stage. The documentation displayed the date 2026-09-20 when checked.
https://docs.vllm.ai/en/stable/features/automatic_prefix_caching/
[12] Micron — LPDDR at Scale: Enabling Efficient LLM Inference Through High-Capacity Memory, white paper pp.1–4. LPDDR’s role in KV offloading. The actual measured configuration must be distinguished from estimates for larger capacities.
[13] NVIDIA — BlueField-4 STX Storage Architecture Announcement, 2026-03-16. Planned availability of partner systems in the second half of 2026. Announced adoption plans are distinguished from actual operation.