In the open source community, LLaMA had the effect of a technological leap, giving independent developers access to a large GPT-level language model. Today, Abu Dhabi's Institute of Innovation and Technology (IIT) unveils Falcon, an open source LLM that outperforms LLaMA.
What is Falcon?
Falcon is presented as the most powerful language model to date, with three possible variants: Falcon 1B, 7B and 40B. Smaller than LLaMA, with 40 billion parameters versus 65, it nevertheless outperforms the latter. According to Hugging Face’s evaluation criteria (IA2 Reasoning Challenge, HellaSwag, MMLU and TruthfulQA), Falcon 40B Instruct, a Falcon variant, and Falcon 40B are more powerful than LLaMA in terms of performance.
This model is multilingual, understanding English, German, Spanish and French, and Dutch, Italian, Romanian, Portuguese, Czech, Polish and Swedish.
To achieve this result, IIT used a dataset of 1,000 billion tokens and a pipeline capable of extracting verified content to ensure the quality of Falcon’s responses. This “refined-web” dataset is also open source, so developers can train their IA to produce programs as powerful as, or even better than, those currently available.
How useful will it be?
Unlike its predecessor, developers will be able to use Falcon for commercial purposes. Although LLaMA is open source, these weights remain private for Meta, which limits its commercialization. This is why Falcon’s models, which use a modified version of Apache 2.0, can suit the user’s needs.
Developers trained to design new artificial intelligence will then be able to use Falcon to create a generation of even more powerful AIs. That’s why, if you’ve enjoyed this article and are considering a career in Data Science, don’t hesitate to check out our articles or our training offers on DataScientest.
Source : huggingface.co
