Memory Stocks Fall After Google's TurboQuant Debut

Memory and Storage Stocks Slide After the TurboQuant Announcement

On March 25, 2026, memory and storage stocks fell together on the U.S. market. According to Investing.com’s intraday coverage that day, SanDisk dropped 5.7%, Western Digital 4.7%, Seagate 4%, and Micron 3%. The move came even as the Nasdaq 100 was rising. The article pointed to Google’s compression algorithm, TurboQuant, as the backdrop for the decline.

I checked Korean stock screens myself around 18:00 on the day I was writing this piece. Semiconductor-related names like Samsung Electronics, SK Hynix, and Hanmi Semiconductor were all down. Still, it’s hard to attribute stock movements—domestic or foreign—to this one technology alone.

TurboQuant compresses the KV cache, which AI systems store to reuse prior computations. Because this cache is also stored in the GPU’s HBM¹, compression could affect HBM usage. But we need to treat two things separately: memory needed to process a single request going down, versus the market’s total memory purchasing going down.

The draft looks accurate and complete. No corrections needed.

What TurboQuant Actually Does

When an AI generates the next sentence, it reuses the results of calculations from everything fed in before. The key and value data stored for this purpose is called the KV cache². Given the same model and storage format, a cache that retains every token generally grows in proportion to the number of tokens. The longer the documents you handle, or the more requests you process simultaneously, the heavier the memory burden gets.

TurboQuant is a quantization³ technique that stores these numbers using fewer bits. The original paper first applies a mathematical transformation called random rotation, making the distribution of each number easier to predict. It then replaces each value with the nearest pre-computed representative value for storage. The advantage here is that you don’t need to learn new compression criteria for each input dataset.

Storing numbers in simplified form introduces error. This can be especially problematic in the dot-product operations AI uses to compute relationships between data points, where results can end up skewed in one direction. To address this, the paper proposes a QJL step that handles the residual error. By representing each component as a 1-bit sign, it’s designed to eliminate bias in the dot-product estimate. That doesn’t mean the error in every individual calculation disappears.

In the paper’s LongBench experiments, Llama-3.1-8B-Instruct’s average score was 50.06 for both the uncompressed baseline and the 3.5-bit setting. At 2.5 bits, it was 49.44. The 3.5-bit setting reported over 4.5x compression, and in an experiment testing retrieval of hidden information within long text, it matched the performance of the uncompressed model. These are results from the specific models and tasks tested.

The fact that compression criteria don’t need to be learned per dataset is also useful for vector search. The paper presents results showing reduced time to prepare search indexes. That said, index preparation time and the time a chatbot takes to generate a response are two different measurements.

Google’s blog post describes an experiment cutting the KV cache by at least 6x and boosting computation speed by up to 8x. Here, the 8x figure is the result for a specific step—computing attention logits on an H100. It shouldn’t be read as meaning the entire response-generation process got 8x faster. Both the paper’s “over 4.5x” and the blog’s “6x” need to be checked against their experimental conditions—model, bit-width, and so on. Before I get impressed by the big numbers in a tech announcement, I try to look first at what was actually measured.

What Compression Means for HBM Demand

If AI operators can handle the same workload using less memory, they may end up running on fewer machines than expected. That’s exactly why investors banking on future memory-chip revenue took notice of this announcement.

KV cache is the data being stored; HBM is the memory chip that holds it. In real systems, the two are directly connected. Depending on usage, KV cache can sit in HBM, or it can be shifted to CPU-side DRAM and SSDs. Cache compression affects not just how much storage is needed, but also how much data has to be transferred when moving it around.

HBM holds more than just KV cache — it also stores model weights and the data needed for computation, and it’s used during training too. So even if cache size shrinks to one-sixth, that doesn’t mean total GPU memory demand or the HBM market shrinks by the same proportion.

To gauge overall market demand, you have to weigh the savings against any increase in usage. The same GPU could now handle more requests, delaying additional purchases — or lower costs could drive more AI adoption overall. Which effect wins out depends on how widely this gets deployed and how much demand there actually is for the service.

Nvidia is also researching KV cache compression

KVTC (KV Cache Transform Coding), from an Nvidia research team and accepted at ICLR 2026, also tackles KV cache compression. It applies transform coding—the technique used in image compression—to keep reusable cache small both on and off the GPU. The paper reports compression of up to 20x while preserving accuracy, and more than 40x for some use cases. At higher compression ratios, some tradeoff with accuracy becomes necessary.

KVTC uses pre-calibration data to set transform bases, allocates bit counts by importance, then applies additional compression. You also have to factor in the cost of decompressing the stored cache when it’s reused. It’s hard to rank performance by simply placing TurboQuant’s and KVTC’s compression-ratio numbers side by side.

Both studies show that methods for lowering inference costs keep being developed. As with the Jevons paradox⁴ covered in a previous newsletter, efficiency gains could lead to more usage. But there’s no guarantee this effect will always outweigh the savings.

The draft matches the source well; no distortions, omissions, or number mismatches found. Only minor polish needed.

Oswarld’s Lens

Watching how the market reacted this time, I found myself thinking that what matters now isn’t just the scale of AI infrastructure spending, but how much money that spending actually generates. News of better performance can’t mean the same thing for a company that supplies the equipment as it does for a company running services on top of it.

The earnings themselves are strong. Nvidia’s fiscal 2026 revenue came in at $215.9 billion, up 65% year over year. Micron’s fiscal 2026 second-quarter revenue, announced on March 18, was $23.86 billion, with gross margin at 74.4% on a GAAP basis and 74.9% adjusted. Both figures are confirmed in the companies’ own disclosures.

But strong results don’t lock in future returns on investment. We need to look at how much service revenue grows relative to the capital expenditure AI companies keep piling on, and whether margins hold up even as prices fall. I think the debate over an AI investment bubble is tied directly to this question.

From where I sit on GTM strategy, you have to look at expanding capacity and making more efficient use of equipment already purchased as two sides of the same coin. TurboQuant and KVTC are examples of the latter. Custom chip efforts like Google’s TPU and Amazon’s Trainium can also be read as attempts to directly control both cost and performance. I think these shifts are changing the yardstick by which we measure AI infrastructure investment.

I don’t think the mere appearance of efficiency research means AI industry growth is over. Being able to lower costs should let more services adopt AI in the first place. But who captures the gains from efficiency will differ from company to company — a service operator cutting its running costs and a chipmaker growing its revenue are not the same outcome.

Demand forecasts made before the earnings are also worth noting. The BofA estimate cited in SK Hynix’s January 2026 market outlook put this year’s HBM market at $54.6 billion, 58% growth year over year. That reflects a high-growth expectation at the time it was made — it isn’t confirmed revenue that already accounts for every technological shift still to come.

Nor can we pin down in advance exactly when efficiency gains will show up in equipment orders. That depends on how fast compression techniques get applied to actual services, how much of the saved capacity gets redirected toward longer inputs and concurrent requests, and the delivery schedules of equipment already under contract. I plan to keep watching subsequent performance benchmarks and how operators’ actual orders change.

Closing

The TurboQuant paper was first published on April 28, 2025 — roughly 11 months before this recent Google blog post and the market coverage that followed. I found it striking how far apart the timing of a research release and investors’ attention to it can be.

TurboQuant is a technique that shrinks the KV cache to save the memory required for inference. Its effect is tied directly to HBM usage. What remains to be seen is whether the compression ratios achieved in the lab hold up in real-world services, and whether usage grows enough to offset the reduced cost per query. Both of these need to be confirmed before we can judge the impact on the memory market.

Rather than pinning the stock’s decline or the industry’s overall trajectory on a single paper, I plan to keep updating my own view as these follow-on developments unfold.

⚠️ This newsletter is not investment advice for any specific stock. Please make investment decisions based on your own analysis and consultation with a professional.

References & Further Reading

• Zandieh, A., Daliri, M., Hadian, M., & Mirrokni, V., TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate, 2025.04.28. You can check the compression method and experimental results, including LongBench.
• Google Research, TurboQuant: Redefining AI efficiency with extreme compression, 2026.03. This introduces the compression ratio and the speedup on specific operations.
• Staniszewski, K., & Łańcucki, A., KV Cache Transform Coding for Compact Storage in LLM Inference, ICLR 2026. This is research on compressing reusable caches for storage.
• Louis Juricic, MU, WDC, SNDK fall: Why Google’s TurboQuant is rattling memory stocks, Investing.com, 2026.03.25. The U.S. stock price decline figures cited in the piece come from this article’s intraday numbers.
• SK Hynix, 2026 Market Outlook, 2026.01.05. This introduces memory market forecasts from institutions including BofA.
• NVIDIA, Fiscal Year 2026 Earnings Announcement, 2026.02.25.
• Micron, Fiscal Year 2026 Q2 Earnings Announcement, 2026.03.18.

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

Footnotes

1. HBM (High Bandwidth Memory): high-bandwidth memory that stacks multiple DRAM chips vertically and provides wide data connections. It’s placed close to AI accelerators to store model weights and KV caches. ↩

2. KV cache (Key-Value Cache): a method that stores and reuses the key-value data a language model has computed from preceding tokens. It’s not a separate device that remembers conversations the way a human would. ↩

3. Quantization: a method that represents numbers with fewer bits to reduce storage and computational load. How much error and performance change depends on how much you reduce it by. ↩

4. Jevons Paradox: a phenomenon where increased efficiency in using a resource leads to greater usage overall, so total consumption actually rises. This isn’t a result that necessarily appears with every efficiency improvement. ↩