Model Merging for MoEs and improved DPO: this is how Mixtral is being transformed by the Open Source community

Week 3 of Coding with Intelligence

📰 News

• Nous Research releases Nous-Hermes-2 Mixtral 8x7B: SFT+DPO

  It outperforms the Mixtral Instruct model on a few tasks. Available on Together AI through their API.

• New merge + DPO model tops HF 7B charts: NeuralBeagle14-7B

  You can try it out here.

• Invisible characters can be used for prompt injection

• Hazy Research & Together AI release Long-Context optimized Retriever model based on Monarch Mixer/BERT

📦 Repos

• Mentat: open source AI coding assistant

  By ex-DeepMind researcher Scott Swingle.

• DeepSeek release MoE model

• AnimateAnyone implementation by Moore Threads

  Includes HF space, demo videos, code & weights. Original here.

• LVE: community sourced red teaming for LLM security

• Phixtral: Phi-2 as MoE

• Library for training Mamba models

• WizardLM releases DeepSeek based WizardCoder model

  Interesting to see DeepSeek team come out ahead: they've already released a MoE based coding model

📄 Papers

• LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning

  Improvement on top of QLoRA

• Sleeper Agents by Anthropic

  Behavior from LLMs hidden behind certain activation sequences. Causing safety detections to fail to pickup on malicious generation patterns. You know, a bit like the Volkswagen scandal.

• Soaring from 4K to 400K: Extending LLM's Context with Activation Beacon

• Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

  By Princeton and Tsinghua researchers

• Bytedance: MagicVideo-V2

  It’s awesome to see text-2-video progressing 🔥

📚 Resources

• GitHub repo with links to AI papers covered in Latent Space podcast

• LlamaPacks: next-gen primitives for code-reuse/packaging in the RAG context

  By the folks from LlamaIndex

• Benchmarks and comparison of LLM models and hosting providers

• Unsloth integration with Hugging Face’s TRL: blog tutorial

• AI timelines revisited by a Google DeepMind researcher (long read)

---

Want more? Follow me on Twitter! @ricklamers