AI Summary27 Oct 202617 Jumada I 1448 AH
AI Summary
27 Oct 202617 Jumada I 1448 AH
Launch of EdgeRock-4B AI Model with Fixed Memory Usage
Rock Matrix, in collaboration with the Hong Kong University of Science and Technology and Carnegie Mellon University, has launched EdgeRock-4B, a small language model with fixed memory usage regardless of context length. The model is based on Alibaba's Qwen3.5-4B and was converted in stages while retaining 24 Gated DeltaNet layers.
EdgeRock employs a novel technique called Rock Memory Anchors, which keeps the top 1,024 tokens from each group of key-value heads, reducing the cache size to 72 MiB compared to 8 GiB for the base model.
Follow these topics
Sign in to follow the topics that matter to you
This summary is generated with AI and receives periodic editorial review. Refer to the original source for full details.
Launch of EdgeRock-4B AI Model with Fixed Memory Usage
Rock Matrix, in collaboration with the Hong Kong University of Science and Technology and Carnegie Mellon University, has launched EdgeRock-4B, a small language model with fixed memory usage regardless of context length. The model is based on Alibaba's Qwen3.5-4B and was converted in stages while retaining 24 Gated DeltaNet layers.
EdgeRock employs a novel technique called Rock Memory Anchors, which keeps the top 1,024 tokens from each group of key-value heads, reducing the cache size to 72 MiB compared to 8 GiB for the base model.
Follow these topics
Sign in to follow the topics that matter to you
This summary is generated with AI and receives periodic editorial review. Refer to the original source for full details.
0
0 reading now
Insight Score
Rate to unlock
You have not rated yet.
Sign in to react, rate, and save. Sign In
