Что думаешь? Оцени!
We have one horrible disjuncture, between layers 6 → 2. I have one more hypothesis: A little bit of fine-tuning on those two layers is all we really need. Fine-tuned RYS models dominate the Leaderboard. I suspect this junction is exactly what the fine-tuning fixes. And there’s a great reason to do this: this method does not use extra VRAM! For all these experiments, I duplicated layers via pointers; the layers are repeated without using more GPU memory. Of course, we do need more compute and more KV cache, but that’s a small price to pay for a verifiably better model. We can just ‘fix’ an actual copies of layers 2 and 6, and repeat layers 3-4-5 as virtual copies. If we fine-tune all layer, we turn virtual copies into real copies, and use up more VRAM.
，详情可参考新收录的资料
More top storiesChild sex abuse allegations covered up by Church in Wales for decades, report reveals
全年规模以上工业中，农副食品加工业增加值比上年增长5.6%，纺织业增长3.0%，化学原料和化学制品制造业增长7.8%，非金属矿物制品业下降0.6%，黑色金属冶炼和压延加工业增长4.5%，通用设备制造业增长8.0%，专用设备制造业增长4.3%，汽车制造业增长11.5%，电气机械和器材制造业增长9.2%，计算机、通信和其他电子设备制造业增长10.6%，电力、热力生产和供应业增长2.2%。（见表3）
但由于名声在外，吸引了很多游客前来打卡，自然也引来了很多不怀好意的“入侵者”，有的地方已经被破坏。