8x faster inference with Flash Decoding - OS models are getting faster

Week 42 of Coding with Intelligence

📰 News

• Hugging Face fine-tune of Mistral 7B

• Fuyu-8B: A Multimodal Architecture for AI Agents

• Finetuned Mistral support lands on Fireworks

• Together AI: Flash-Decoding for long-context inference

  8x inference speedup on long sequences. From the same author that brought Flash Attention, Tri Dao.

📄 Papers

• NEFTune: Noisy Embeddings Improve Instruction Finetuning

• Training Diffusion Models with Reinforcement Learning

• RAFA: A Framework for LLM Agents with Provable Sample Efficiency

• BitNet: Scaling 1-bit Transformers for Large Language Models

🛠️ Products

• Oterm: TUI client for Ollama

📚 Resources

• Building a “heavy metal quartet” of AI compilers

• Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments

• Fireside Chat with Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future

---

Want more? Follow me on Twitter! @ricklamers