DeepSeek's new model sets a template for powerful LLMs that run lean

Chinese AI darling DeepSeek unveiled an updated version of its cost-and-latency-optimized Flash model on Thursday, with a new version 4.1 that includes architectural improvements more significant than you would expect in a point release because the changes might open the door to larger, smarter, and less resource-intensive models. At 763 billion parameters, the point release is more than 2.5x the size of the model it replaces. In fact, the model is larger than the V3 and R1 models that put DeepSeek on the map back in early 2025. Despite its ginormous parameter count, DeepSeek V4.1 Flash’s memory requirements aren’t nearly as high as you’d expect for a model of its size. Under the hood, DeepSeek's devs have made numerous architectural changes that see the LLM become smarter while dramatically reducing the resources necessary to serve it. DeepSeek has managed this through two key improvements. First, it made significant changes to how the model handles the key-value (KV) caches used to track model state across multiple sessions. These so-called KV caches can be quite memory-hungry, particularly in high-throughput applications like chatbots. Updates to the model’s various attention mechanisms and the introduction of a new causal encoder-decoder (CED) enabled the devs to improve prompt processing performance while cutting KV cache consumption to between 13 percent and 25 percent of DeepSeek V4 Flash's requirements. In other words, the V4.1 release can support four to eight times as many users in the same KV cache footprint. DeepSeek’s technical report goes into far greater detail on the architectural changes, but arguably the most interesting change is the introduction of a different kind of model weight. Of its 763 billion parameters, 196 billion are N-gram parameters that form what DeepSeek's developers refer to as a “conditional memory module.” The idea is that by decoupling memory from computation, DeepSeek can make its models smarter while also reducing the compute and memory resources required to serve them. What the heck is an N-gram? The big idea behind DeepSeek’s V4.1 Flash’s memory module is similar in many respects to Per-Layer Embedding (PLE) tech originally developed by Google’s Gemma team. The goal with PLE was to get LLMs to be smart enough to run usefully on devices with constrained bandwidth, memory, and compute – like smartphones. DeepSeek’s implementation, first detailed in a January research paper, trades PLE embeddings for N-grams. At a high level, N-grams are just groups of tokens. A three-gram would be three tokens in a row, a two-gram would be two, and so forth. As complicated as that might sound, it actually works a bit like word or phrase association. If you were asked: "Find the parameter of a right triangle when only two sides are known." For those of you for whom geometry isn't too distant of a memory, the phrases "use the pythagorean theorem" or "A2 + B2 = C2" or perhaps "the perimeter would be the sum of its sides" might immediately pop to mind. The N-gram parameters found in models like DeepSeek V4.1 Flash are similar in concept. The weights are a source of implicit knowledge or ingrained memory. Rather than just calculating which combination of tokens have the highest probability of answering the question, as LLMs have traditionally done, the N-gram weights supplement this by quickly surfacing relevant information through a cheap lookup. This is a gross oversimplification of what's going on under the hood. In fact, the model isn't looking up the prompt so much as a series of hashes. These are numbers representing "Find the parameter," "right triangle," and so on. Similarly, the contents of the lookup table aren't raw responses. They're another mathematical representation, called vectors, which get fed into the inference pipeline. However, the end result is the same: The model can provide smarter, more nuanced answers without the performance penalty normally associated with additional parameters. What makes N-grams so cheap The relationship between model size and intelligence is well established at this point. The reason DeepSeek’s n-gram parameters are so interesting is actually related to how the data is accessed. As a general rule, modern LLMs are autoregressive during decode. That means for every token a chatbot or agent generates, the entirety of the model’s active weights have to be read from memory, making bandwidth the limiting factor. As we’ve previously discussed, architectural changes, like the rise of mixture of expert models or ultra-low precision block-floating point datatypes, have helped to minimize this bottleneck. But the N-gram weights found in DeepSeek V4.1 Flash work a bit differently. They offer a way of effectively increasing the number of parameters available to the model during inference without a proportionate increase in memory pressure. These N-gram weights are essentially enormous look up tables (LUTs). This makes them fast and cheap to query, since unlike the rest of the model’s active parameters, they don’t need to be read in their entirety from memory each time a token is generated. It’s just a few dozen table lookups per token. This has a couple of implications for memory access, but the big one is that those n-gram weights don’t have to be crammed into GPU memory to maintain performance. They can be offloaded to system RAM or, possibly even a sufficiently speedy storage array. So what does that mean in practice? If you look at DeepSeek V4.1 Flash, the model would normally need a minimum of 763 GB of GPU memory to hold the weights at FP8. However, since those n-gram weights can be offloaded to cheaper system memory, we can get away with around 567 GB of GPU memory. We emphasize the minimum here, because in production those numbers are going to be substantially higher since we also need to take into account key-value caches, which scale with context length and concurrent users. During inference, those N-gram weights supplement the eight billion active parameters DeepSeek uses to process a prompt, in theory increasing the accuracy and quality of the output in the process. It’s important to note at this point that the n-gram weights don’t actually increase the active parameter count. Instead, they function more like an oddly specific encyclopedia that almost instantly opens relevant pages as the model processes prompts. The future of open LLMs While DeepSeek’s latest model may have one of the largest pools of N-gram parameters yet, it’s not the only model developer betting on tech to make deploying larger models more efficient. As mentioned earlier, Google is already employing a similar approach using PLE to offload less bandwidth-sensitive weights to local storage. So far, it's only been applied to tiny models — at least that we know of (it's not like Google is particularly transparent about its proprietary models). Meanwhile, late last month, Alibaba revealed its latest experimental model codenamed Qwen 3.8-Flash-Next. Much like DeepSeek V4.1 Flash, the 180 billion-parameter model featured a large 51 billion-parameter pool of N-gram weights for much the same reason. In fact, the model's N-gram implementation uses techniques from the same research published by the DeepSeek team back in January. According to Alibaba, Qwen 3.8-Flash-Next's architectural underpinnings will form the foundation of its next generation of Qwen 4 models when they arrive. Which means, like it or not, this probably won’t be the last time you hear about N-grams. ®