Kimi K3: 2.8T Parameters You Can't Run at Home

A 2.8-Trillion-Parameter Model: Downloadable, But You Can’t Run It on Personal Hardware

On July 17th, Beijing-based Moonshot AI unveiled Kimi K3. With 2,800,000,000,000 parameters, Moonshot itself calls it the largest open-weight¹ model in the world. Anyone can still download it from Hugging Face today. The problem starts after you download it. The weight files alone exceed 1.5 terabytes, and as the measurements below show, actually running the model requires memory and server configurations that go far beyond any ordinary personal computer.

These days, anyone can download the weights for a frontier-class open model, but the pool of people who can actually run one has narrowed down to cloud providers, corporations, and governments. Let me walk through the memory requirements and serving costs to show just how wide that gap really is.

The leaderboard’s top five spots have flipped to Chinese models in two weeks

Let’s start with the rankings. On the intelligence index from benchmarking firm Artificial Analysis, the top open-weight models right now are Kimi K3 (57 points), GLM-5.2 (51 points), and DeepSeek V4 Flash (50 points), in that order. The fourth and fifth spots go to a lighter version of Kimi and to MiniMax. In other words, every one of the top five slots is now a Chinese model.

This leaderboard was rewritten in just the last two weeks. Moonshot released Kimi K3 on July 17, and two days later Alibaba previewed Qwen3.8-Max — a 2.4 trillion-parameter model — with the self-assessment that it trails “only Fable 5.” In between, OpenAI cut pricing on its flagship model by 80%. DeepSeek shipped the official release of V4 Flash on July 31, and Alibaba released the official version of Qwen3.8-Max on August 3. The weights are slated for public release next week.

It’s not just performance climbing — the models themselves are getting bigger. GLM-5.2 has 753 billion parameters, DeepSeek V4-Pro has 1.6 trillion, Qwen3.8-Max has 2.4 trillion, and Kimi K3 has 2.8 trillion. The reason for this growth is a push to catch up with frontier-level performance. DeepSeek V4 Flash’s 50 points sits just 1 point behind OpenAI’s newest closed model, and Kimi K3 openly claims to match Fable- and Sol-class intelligence. These companies have chosen the strategy of scaling models up to reach top-tier performance. And the reason they can scale up this way is that virtually every large model today uses a Mixture-of-Experts (MoE)² architecture — a design in which only a subset of the total parameters is actually used to process any given token. Qwen3.8-Max activates just 95 billion of its 2.4 trillion parameters; DeepSeek V4 Flash activates just 13 billion of its 284 billion.

This is where the cost structure splits. Compute cost tracks active parameters, but memory requirements track total parameters. In a serving setup built for fast responses, the full set of weights has to sit in memory so that whichever expert gets called can be used immediately. That’s why API pricing can be so cheap (DeepSeek charges $0.28 per million output tokens), while running the model at home would require enough graphics memory to hold all 2.8 trillion parameters. The very design that makes the API cheap is the same design that makes it impossible to run locally.

To put 1.5 terabytes in perspective: the RTX 5090, the top-tier consumer graphics card, has 32 gigabytes of memory. Simply adding up memory capacity, you’d need roughly 48 of these cards — and that’s before even factoring in the KV cache³ needed for a 1-million-token context window. Let’s also compare growth rates. Consumer GPU memory went from 24 gigabytes in 2020 to 32 gigabytes in 2025 — a 33% increase over five years. Over a different span, the largest open model went from Llama 405B in 2024 to 2.8T in 2026 — a 7x increase in just two years. The size of models is growing far faster than the memory capacity of consumer GPUs.

A GPU-Optimization Company Ran Kimi K3 Themselves

So who’s actually running these models, and how? Last week a useful field report came out. Wafer, a startup that sells open-model inference services on AMD chips, tried serving Kimi K3 directly and wrote up what happened. You should read it knowing this is a company whose business is AMD serving, running its own benchmark — but the process is described in enough detail that it’s worth paying attention to.

Let’s start with the numbers. Kimi K3 doesn’t fit its weights and KV cache together on a standard Nvidia 8-GPU B200 node (192GB per GPU, 1.5TB total). The alternatives Wafer compared were a B300 node at 288GB per GPU and a setup linking two B200 nodes. But there’s one more chip with 288GB per GPU: AMD’s MI355X. In Wafer’s measurements, an 8-GPU MI355X node produced 952 tokens per second, and — factoring in GPU rental prices — its throughput per dollar came to 48 tokens, beating both the B300 (33 tokens) and the two-node B200 configuration (7 tokens). In this benchmark, the large memory capacity let a single node handle the whole job, and the difference in rental costs worked in AMD’s favor on price-performance.

Comparing this against a record from a month earlier makes the gap clear. When the same team served GLM-5.2 (753 billion parameters) in early July, the story was different. GLM-5.2 fits on a single Nvidia node, so even after quantization⁴ and framework modifications, AMD landed only as a decent value option: “80% of the performance, at half the price.” Since this compares different models and configurations, you can’t pin the difference on a single cause — but it does show that memory capacity is a decisive factor.

What caught my attention in this benchmark, though, wasn’t the ranking — it was the difficulty. Even this company, whose entire business is GPU optimization, hit a scheduler crash on its very first request, had to hand-write a missing function in PyTorch, and resorted to workarounds that dodge shape constraints in the attention computation before finally hitting its performance target. And by the team’s own account, this time was the easy one — earlier models had forced them to write entirely new custom kernels or fix two separate framework bugs. There are weaknesses too. Reading a 170,000-token document for the first time took 51 seconds, more than twice as slow as the Nvidia B300’s 23 seconds. And even after clearing all of this, node rental still runs around $20 an hour. Run it for a month and you’re looking at roughly ₩20,000,000 (~$14,500) — not a level an individual can afford.

Of course, there’s another side to this. DeepSeek V4 Flash is designed to be small from the ground up — 284 billion parameters total — and its 3-bit quantized version comes in at 103GB, meaning a high-end workstation with 128GB of RAM can run it. Alibaba has also previewed a small 27-billion-parameter version alongside Qwen3.8-Max. So it’s not that “open models you can run yourself” have disappeared. It’s just that those models sit a tier below the frontier — and the frontier itself is no longer something an individual can run on their own.

So Why Give the Weights Away for Free?

Then why are companies racing to release, for free, models that most individual developers can’t even run?

Wall Street’s analysis offers a clue. In an Alibaba report from early August, Citi laid it out this way: as models get refreshed every few weeks, companies are adopting a “model-agnostic” strategy, swapping out models task by task. The more that happens, the weaker any single model’s defensibility becomes, and the locus of competition shifts to the platform and infrastructure the model sits on. That’s the logic behind the buy rating — it favors Alibaba, which owns everything from chips to cloud to models to apps. You should factor in that this is a sell-side report with its own interests, but the direction lines up exactly with Alibaba’s actual behavior. On the same day it announced it would give away the weights for free, Alibaba also rolled out QwenWork, an enterprise agent product, bundled into DingTalk. The model is the hook that pulls in customers; what’s actually being sold is the cloud and the business tools built on top of it.

This connects to Liang Wenfeng’s calculation, which I quoted last month during the DeepSeek funding news.

“If you want to make 100x profit, open source would get in the way. But if you’re only after reasonable profit, it makes no difference at all.”

So the real buyers of frontier-grade open weights were never individuals to begin with. They’re clouds that host the weights and sell access on their own infrastructure, companies that can’t send data outside their walls, governments that want intelligence kept within their borders, and firms like Wafer that turn serving itself into a business. Enterprise demand, in particular, is very concrete. In industries like finance, healthcare, or defense — where data can’t be sent to an external API — companies can keep the weights inside their own environment, fine-tune them on their own data, and be assured the model won’t suddenly change overnight. These open weights are also the first thing governments reach for when building what they call “sovereign AI.” In the end, releasing the weights is really a way of outsourcing model deployment and serving to these players.

There’s one more beneficiary in all this. As the competition over model size turns into a competition over memory capacity, the factor determining chip performance is shifting from the compute unit itself to the high-bandwidth memory (HBM)⁵ stacked next to it. It was 12-layer HBM3E from Samsung Electronics and Micron that filled up the 288 gigabytes in the MI355X. Samsung, whose Nvidia qualification had been stuck in limbo for a long time, started supplying AMD first — and as Chinese models grow bigger, that supply demand is climbing. Even if few places can actually run these models themselves, demand for the companies that sell the memory keeps rising regardless.

Or you can go the other way and shrink the model

There was a domestic case in this same two-week window too. On July 27th, Kakao unveiled its second small-model series, Kanana-2. The setup: a 3 billion-parameter model trained from scratch, then pruned and distilled down to 1.3 billion parameters — and the 1.3B version is explicitly labeled for on-device deployment. Set next to Kimi K3, it’s 2,000 times smaller. That’s the gap between a model whose weights weigh in at 1.5 terabytes and one at 2.6 gigabytes, between something you run on a server rack and something you run on a smartphone.

The reversed direction here is a deliberate choice. A company without the capital to compete in the frontier-model race has one real asset: an app already used by tens of millions of people — and running server inference at that scale is a cost it can’t absorb. If the model runs on-device, server inference costs approach zero and data never leaves the phone. It’s essentially the same logic as enterprises keeping weights in-house, just implemented at the level of a personal device. The design itself was built around memory efficiency.

kakaocorp/kanana-2-1.3b-base · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.co

Kakao says it used an attention architecture that cuts the KV cache by up to 72.7%. In its own benchmarks, the model beats a comparably-sized Qwen by a factor of two on Korean knowledge Q&A, though it falls behind on math. It’s clearly not in the same league as frontier models — but if the actual goal is everyday tasks like summarization or classification, the math changes. Rather than waiting for large-model performance to trickle down, this is a strategy of focusing on what a small model can already do well.

Oswarld’s Lens

When I review free-tier strategies as part of GTM strategy consulting, there’s one thing I always check: who ends up bearing the cost of that “free,” and in what form. By that standard, today’s open weights share a name with the open source of the Linux era but work in a fundamentally different way. With Linux, the person who downloaded it was the same person who ran it — it ran fine even on an old 486 computer. Frontier-grade open weights, by contrast, structurally separate the person who downloads the model from the person who actually runs it. Everyone can access it, they say, but in practice it amounts to handing the model over to operators with massive infrastructure.

So whenever I see the line “open source AI democratizes intelligence,” I’d encourage you to question it once more. The party whose options actually expand isn’t the individual user — it’s the cloud providers and companies that take those weights and resell them as services.

That said, let me note the fact that cuts the other way too. The trickle-down of performance is real. The model running on an individual’s workstation today is, roughly, at the level of a frontier model from about a year ago. Recent examples show performance once reserved for large-scale models is now available on personal hardware — but in the meantime, the top-tier models have moved forward too. Naming both this lag and this performance gap, in my view, is what it means to use the word “open” honestly.

Closing

First, all top five spots on the open-weight intelligence index are now occupied by Chinese models, and within two weeks, model size grew to 2.8 trillion parameters.

Second, in an MoE architecture, only a fraction of the parameters are used per computation, which keeps API costs low—but running the model yourself still requires enough memory to hold all the parameters. That’s why you can download these models but struggle to actually run them.

Third, the real demand for frontier-grade weight releases doesn’t come from individuals but from cloud providers, enterprises, and governments, and the race to scale up models feeds directly into demand for memory chips like HBM.

If Alibaba actually releases its 2.4 trillion-parameter weights next week, I’ll dedicate a separate issue to who ends up bearing the cost of that release.

Has your team tried running an open model yourselves, Reader? I’d love to hear in the comments what B-scale was actually workable, and where you hit a wall and fell back to an API. I’ll gather these cases and fold them into a future issue.

---

📨 If you know a colleague wrestling with self-hosting decisions, please pass this issue along.

---

The English draft matches the Korean source accurately with no distortions, omissions, number mismatches, or glossary violations. No Hangul remains, all numbers are preserved as digits, and heading/footnote/link counts match exactly. Outputting as-is:

References & Further Reading

Primary sources

• Ian Ye, “Is memory the moat?”, Wafer Blog, 2026.7.31. ··· This is the backbone of today’s piece — actual measurements from Kimi K3. The kernel-patching process is laid out in full, so if you’re curious what “running an open model” actually takes, start here.
• Ian Ye, “Performance per dollar is getting faster and cheaper”, Wafer Blog, 2026.7.3. ··· Real-world GLM-5.2 numbers from a month earlier. This serves as today’s control group, showing the landscape at the moment a model first “fits” onto a single node.
• Artificial Analysis, “DeepSeek V4 Flash 0731 analysis”, 2026.8. ··· The source for open-weight intelligence rankings and pricing data.
• Qwen Team, “Qwen3.8 announcement”, 2026.8.3. ··· The primary source on the 2.4-trillion-parameter model and its weight-release plans.
• Citi Research, “Alibaba: assessment of Qwen3.8-Max’s official launch”, 2026.8. ··· The source of the “model agnosticism” framing. Worth reading with the caveat that this is a buy-rated report.
• Kanana LLM Team, “Kanana-2 SLM”, Kakao Tech Blog, 2026.7.27. ··· The primary source on the compression process that shrinks a 3B model down to 1.3B. Weights are available on Hugging Face.

Background

• Unsloth, “Guide to running DeepSeek-V4 locally”, 2026.8. ··· How to actually run the 103-gigabyte quantized version. Useful if you’re wondering where the current ceiling of “runnable open” sits.
• Seoul Economic Daily, “AMD: ‘MI350X uses Samsung HBM3E’”, 2025.6. ··· Background on the Korean memory chips inside the MI355X.

Past issues worth reading alongside this one

• Issue 158: Reading Jensen Huang’s defense of Chinese models through Nvidia’s own interests
• Issue 159: DeepSeek’s first funding terms and its founder’s grip on control

---

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

📝 Glossary

Footnotes

1. Open weights: A release model in which the trained model’s weight files are made available for anyone to download. It’s a narrower concept than full open source, which also opens up training data and code — though the two are increasingly used interchangeably these days. ↩

2. Mixture of Experts (MoE): An architecture that splits a model into multiple “expert” sub-modules and, for each token, computes using only the sub-modules that are needed. This saves on computation, but since you can’t know in advance which experts will be called, all of them still have to sit ready in memory. ↩

3. KV cache: The working memory a model stacks up so it doesn’t have to recompute earlier text while reading a long passage. The longer the context and the more concurrent users, the bigger this share of memory becomes. ↩

4. Quantization: A compression technique that lowers the numerical precision of weights to shrink file size and memory usage. Done well, it can cut capacity down to a fraction of the original with almost no loss in performance. ↩

5. HBM (High Bandwidth Memory): Memory stacked vertically right next to the GPU to exchange data at extremely high speed. This memory effectively determines the capacity and speed of an AI chip, and the market for it is split among three companies: Samsung Electronics, SK Hynix, and Micron. ↩