MosaicML acquired for $1.3B by Databricks

Week 26 of Coding with Intelligence

📰 News

• Head of Deepmind Demis Hassabis claims Gemini might outperform ChatGPT

• MosaicML releases 30B foundation model Apache 2.0 Open Source, matches GPT-3, 8k context-window

• Databricks acquires MosaicML, a platform for training custom LLMs, for $1.3B

• Geohot, Pytorch lead and Bing AI Lead confirm GPT-4 architecture

  The interesting takeaway imo is that open source models are likely less behind, just not stacked as a mixture model. Furthermore, lots of ideas still to explore and not many of them seem to be necessary for GPT-4 level performance.

📦 Repos

• tinygrad: PyTorch alternative with focus on broad hardware support (AMD too!)

  Geohot just raised $5M to build this out. Supports CPU and accelerated backends like Metal, CUDA, and Triton. Full list in repo.

• LMFlow: a toolkit for fine-tuning Open Source LLMs

  If you’re considering fine tuning an LLM don’t start from scratch!

📱 Demos

• JPT: Python within ChatGPT as a browser extension using WASM/Pyodide

  This is a really clever use case for WASM Python in the browser!

🛠️ Products

• Midjourney 5.2 released and does not disappoint

  They call out painting “Zoom Out”, either way the results are stunning. Link to Twitter thread with curated examples.

📚 Resources

• Weaviate Embedded

  Use Weaviate directly from your client for lightweight use cases

• LLM Powered Autonomous Agents

  An overview to get more familiar with the components involved with LLM agents

• Beware Tunnel Vision in AI Retrieval

  I think Colin makes a compelling argument that vector databases are just one form of retrieval and not always the best solution for prompt building. The fact that vested interests push this narrative makes for even more reason to keep your critical thinking hat on.

---

Want more? Follow me on Twitter! @ricklamers