WizardLM can wave its magic prompting Wand and lead us to the new AI possibilities.
The major difference between Facebook LLAMA and WizardLM models lies in their purpose and training methodology. Facebook LLAMA is a foundational large language model designed for versatility and applicability to various use cases, while WizardLM is a large language model specifically designed to follow complex instructions and is created using a novel method called Evol-Instruct.
Facebook LLAMA Model
Facebook LLAMA (Large Language Model Meta AI) is a foundational large language model that is available in several sizes (7B, 13B, 33B, and 65B parameters). It is designed to be versatile and can be applied to many different use cases, as opposed to a fine-tuned model designed for a specific task. The model takes a sequence of words as input and predicts the next word to recursively generate text. It is trained on text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets. LLaMA 65B and LLaMA 33B are trained on 1.4 trillion tokens, while the smallest model, LLaMA 7B, is trained on one trillion tokens. Facebook has released the LLaMA model under a noncommercial license focused on research use cases.
WizardLM Model
WizardLM is a large language model designed to follow complex instructions. It has been developed in three versions: 30B, 13B, and 7B. The model is created using a novel method called Evol-Instruct, which uses LLMs instead of humans to automatically mass-produce open-domain instructions of various difficulty levels and skill ranges, to improve the performance of LLMs. The training data for WizardLM is contained in the alpaca_evol_instruct_70k.json file, which contains 70K instruction-following data generated from Evol-Instruct. The Evol-Instruct method rewrites an initial set of instructions step by step into more complex instructions. The generated instruction data is then mixed to fine-tune LLaMA, resulting in the WizardLM model.
Key Differences Between Facebook LLAMA and WizardLM Models
The primary difference between the two models is their purpose and training methodology. Facebook LLAMA is a foundational model designed for versatility and applicability to various use cases, while WizardLM is specifically designed to follow complex instructions. The training methodology for WizardLM is unique, as it uses the Evol-Instruct method to generate open-domain instructions of various difficulty levels and skill ranges, which are then used to fine-tune the LLaMA model, resulting in the WizardLM model. In contrast, the LLaMA model is trained on a large set of unlabeled data from multiple languages, making it a more general-purpose model.
