model: add Nanbeige4.2 (Looped Transformer) support#99
Open
Andgihat wants to merge 1 commit into
Open
Conversation
Port the Nanbeige4.2 architecture from Nanbeige/llama.cpp (branch nanbeige42, commit 26cfdc4) onto BeeLlama v4.0. Looped transformer reuses layers via num_loops; no new ggml ops (pure graph/arch level). Adds: nanbeige arch enum + KV keys (num_loops, skip_loop_final_norm), model class + graph builder (src/models/nanbeige.cpp), GGUF conversion (conversion/nanbeige.py), and the larger graph-node budget in graph_max_nodes for the expanded looped layer stack.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for the Nanbeige4.2 architecture (Nanbeige/Nanbeige4.2-3B, Apache-2.0) — a Looped Transformer: the physical decoder layers are reused
num_loopstimes, so effective depth grows without adding parameters (22 physical layers ×num_loops=2→ 44 effective).What's included
LLM_ARCH_NANBEIGE+ KV keysnanbeige.num_loops/nanbeige.skip_loop_final_norm.src/models/nanbeige.cpp: standard RMSNorm + GQA + SwiGLU decoder, with the loop unrolled over logical layers. Physical weights are shared across loops via pointer copies (layers[i + j*n_phys] = layers[i]), while each logical slot keeps its own KV index. An optional loop-boundaryoutput_normis applied between passes unlessskip_loop_final_normis set.gguf-pyconstants + writer keys and aconversion/nanbeige.pyconverter.Integration with this fork's hparams
This fork carries a single
n_layerfield (rather than a separate physical/logical pair).load_arch_hparamsexpandsn_layerto the logical count (n_layer_phys * num_loops) before tensors/KV are allocated, and replicates the per-layer arrays (n_head,n_head_kv,n_ff, swa/recurrent flags) across the loop copies. This keeps the graph loop, KV cache, and out-ids handling correct without touching the base hparams API. Pure graph/arch code — no new ggml kernels.Verification
Built for Windows/CUDA 12.8 (
sm_120) and compared token-for-token against the authors' reference branchNanbeige/llama.cpp@nanbeige42on the same BF16 GGUF with greedy decoding: outputs are identical (48/48 tokens). Q8_0 / Q4_K_M / imatrix quants run and produce coherent output with the embedded chat template.GGUF quants (incl. an imatrix + mixed-precision Q4) and a prebuilt Windows CUDA release are available at Andgihat/Nanbeige4.2-3B-GGUF.