Meta AI recently released Code Llama 70B, the largest and most capable model in their Code Llama family of open-source AI assistants for code generation. At 70 billion parameters, this new coding AI model marks a significant milestone – surpassing other leading closed-source models like GPT-4 to become one of the highest-performing systems available for programming tasks. With its multi-variant release under an accessible open-source license, this model has the potential to accelerate innovation in AI coding tools while lowering barriers to entry.
Today we’re releasing Code Llama 70B: a new, more performant version of our LLM for code generation — available under the same license as previous Code Llama models.
— AI at Meta (@AIatMeta) January 29, 2024
Download the models ➡️ https://t.co/fa7Su5XWDC
• CodeLlama-70B
• CodeLlama-70B-Python
• CodeLlama-70B-Instruct pic.twitter.com/iZc8fapYEZ
Table of Contents
Training and Fine-Tuning of Meta New Coding AI Model
To create this model, Meta took the powerful Llama 2 model and further trained it on code-specific datasets. Each version of Code Llama, including the 70B, is trained with billions of tokens of code and code-related data. The 70B version, in particular, is trained on an astounding one trillion tokens. This extensive training allows Code Llama 70B to excel in generating accurate and context-aware code suggestions.

The Three Versions of Code Llama 70B
Meta’s new coding AI model is available in three distinct versions, each tailored to cater to different coding needs:
1. CodeLlama-70B (Base Model)
It is the foundational model which serves as the base for the other versions. It is designed to provide powerful text completion for coding tasks. With its extensive language understanding capabilities, it can assist developers in writing code more efficiently.
- Base Model: codellama/CodeLlama-70b-hf
2. CodeLlama-70B-Python
For those working with Python, CodeLlama-70B-Python offers specialized support. This version is fine-tuned specifically for Python coding, enabling developers to write Python code with ease and accuracy. It provides targeted suggestions and improves the overall coding experience.
- Python Model: codellama/CodeLlama-70b-Python-hf
3. CodeLlama-70B-Instruct
Code Llama-70B-Instruct is trained to understand and generate code based on natural language instructions. It acts as an assistant, allowing users to provide instructions in plain English and receive code snippets as output. This version bridges the gap between natural language and coding, making it easier for both seasoned developers and coding beginners.
- Instruct Model: codellama/CodeLlama-70b-Instruct-hf
Code Llama 70B vs GPT-4: Surpassing Expectations
When it comes to performance, Code Llama 70B stands tall against GPT-4. On the HumanEval benchmark, CodeLlama-70B-Instruct achieves an impressive 67.8, surpassing the capabilities of GPT-4 at the time of its release. This places Meta’s new coding AI model at the forefront of state-of-the-art open-source solutions, on par with other leading models like ChatGPT. Moreover, it solidifies its position as one of the highest-performing open models available today.
How to Access Code Llama 70B
You can access the models via Hugging Face. You need to request access to Code Llama 70B with a simple request form on the Llama website. Fill out the necessary details, select the models you want, and accept the terms. You’ll receive an email with the download link, allowing you to explore the vast capabilities of this model
Embracing Openness and Accessibility
Meta embraces the spirit of open-source software and aims to support both research and commercial use with Code Llama 70B. The model is available under the same license as its predecessor, Llama 2, ensuring that developers and researchers can freely utilize its capabilities.
The Potential of Code Llama 70B
Code Llama 70B is a game-changer for programmers and learners alike. With Code Llama 70B, Meta significantly lowers the barriers to entry for individuals venturing into the coding realm, opening up a world of possibilities. By establishing Code Llama 70B as the new publicly available benchmark leader and championing open standards, Meta continues driving the field of AI programming tools despite competitors’ current performance claims.
| Also Read
- Meta Research Introduces Revolutionary Self-Rewarding Language Models Capable of GPT-4 Level Performance
- Meta Audiobox: Create Al-Generated Audios From Voice and Text Prompts
- Meta Generative AI is Coming to Facebook and Instagram
- Meta Unveils I-JEPA: A Groundbreaking Human-Like AI Image Creation Model






