How Instruction Fine-Tuning Elevates Mixtral – Instruct Above Competitors

by Writings, Papers and Blogs on Text ModelsOctober 18th, 2024

Too Long; Didn't Read

Mixtral – Instruct is fine-tuned through supervised methods and Direct Preference Optimization, earning a top score of 8.30 on MT-Bench. Independent evaluations confirm its superiority over major competitors, including GPT-3.5 Turbo and Llama 2 70B, making it the leading open-weights model as of December 2023.

featured image - How Instruction Fine-Tuning Elevates Mixtral – Instruct Above Competitors

Table of Links

Abstract and 1. Introduction

2 Architectural details and 2.1 Sparse Mixture of Experts

3 Results

3.1 Multilingual benchmarks, 3.2 Long range performance, and 3.3 Bias Benchmarks

4 Instruction Fine-tuning

5 Routing analysis

6 Conclusion, Acknowledgements, and References

4 Instruction Fine-tuning

We train Mixtral – Instruct using supervised fine-tuning (SFT) on an instruction dataset followed by Direct Preference Optimization (DPO) [25] on a paired feedback dataset. Mixtral – Instruct reaches a score of 8.30 on MT-Bench [33] (see Table 2), making it the best open-weights model as of December 2023. Independent human evaluation conducted by LMSys is reported in Figure 6 [3] and shows that Mixtral – Instruct outperforms GPT-3.5-Turbo, Gemini Pro, Claude-2.1, and Llama 2 70B chat.