MosaicML acquired for $1.3B by Databricks
Week 26 of Coding with Intelligence
📰 News
Head of Deepmind Demis Hassabis claims Gemini might outperform ChatGPT
MosaicML releases 30B foundation model Apache 2.0 Open Source, matches GPT-3, 8k context-window
Databricks acquires MosaicML, a platform for training custom LLMs, for $1.3B
Geohot, Pytorch lead and Bing AI Lead confirm GPT-4 architecture
The interesting takeaway imo is that open source models are likely less behind, just not stacked as a mixture model. Furthermore, lots of ideas still to explore and not many of them seem to be necessary for GPT-4 level performance.
📦 Repos
tinygrad: PyTorch alternative with focus on broad hardware support (AMD too!)
Geohot just raised $5M to build this out. Supports CPU and accelerated backends like Metal, CUDA, and Triton. Full list in repo.
LMFlow: a toolkit for fine-tuning Open Source LLMs
If you’re considering fine tuning an LLM don’t start from scratch!
📱 Demos
JPT: Python within ChatGPT as a browser extension using WASM/Pyodide
This is a really clever use case for WASM Python in the browser!
🛠️ Products
Midjourney 5.2 released and does not disappoint
They call out painting “Zoom Out”, either way the results are stunning. Link to Twitter thread with curated examples.
📚 Resources
Use Weaviate directly from your client for lightweight use cases
An overview to get more familiar with the components involved with LLM agents
Beware Tunnel Vision in AI Retrieval
I think Colin makes a compelling argument that vector databases are just one form of retrieval and not always the best solution for prompt building. The fact that vested interests push this narrative makes for even more reason to keep your critical thinking hat on.
Want more? Follow me on Twitter! @ricklamers