Never run out of cloud GPUs again: Berkeley's SkyPilot to the rescue
Week 43 of Coding with Intelligence
This week’s CoWI contains two SkyPilot based projects. Honestly, if you’re working directly with GPUs for training, fine-tuning or inference there is no more exciting & useful project to check out. It makes running across clouds and even localizing available resources (GPUs in a certain region) in a single cloud, a total breeze.
📦 Repos
SkyPilot: automatically train Vicuna like Llama 2 across multiple clouds
This is an example of what the much more generic cloud computing tool SkyPilot allows you to do. Treat heterogeneous cloud resources as a simple building block. No more GPU availability problems, it automatically finds where the GPU resources are available. Created by students/staff at Berkeley University.
HiTZ/GoLLIE-34B: Code Llama fine-tune for data extraction
An instruction following data extraction LLM. This can be useful for sensitive data extraction in case you don't want to send data to OpenAI.
AgentTuning: Enabling Generalized Agent Abilities for LLMs
Tuning on agent interaction traces, interesting idea!
ModelFusion: TypeScript library for building multi-modal AI applications
First class support for audio streaming from services like ElevenLabs and LMNT.
📄 Papers
Contrastive Prefence Learning: Learning from Human Feedback without RL
Improving results from open source models with fine tuning can include various forms of RLHF or RLAIF. This new method from researchers at Stanford introduces a new approach that models regret instead of reward. It’s simpler because no reward or value function needs to be modeled and about 1.6x times faster.
Large Language Models can Learn Rules
DeepMind explores the idea of extracting rule libraries that work well from the LLM to use those during inference on new tasks, the goal is to improve model reasoning performance. They observe a 11%-27% percentage point improvement in task accuracy on selected benchmarks.
📚 Resources
SkyATC: LLM serving on SkyPilot
This is a recent presentation given at Berkeley about the SkyATC project. Hopefully soon to be released as a useable project on top of SkyPilot for cheap and efficient OSS LLM serving across cloud resources.
Instructor library launches Fine-Tuning and Distillation feature
A neat way of creating fine-tuning datasets.
The case for using YELLING IN ALL CAPS for getting LLMs to follow instructions
Hint: it's because of the text data present in the training data used during pre-training.
The article also links to an interesting prompting technique dubbed Chain of Density: https://arxiv.org/abs/2309.04269
Want more? Follow me on Twitter! @ricklamers