Fb father or mother Meta unveils LLaMA 2 open-source AI mannequin for business use 

In a blockbuster announcement at the moment designed to coincide with the Microsoft Encourage convention, Meta introduced its new AI mannequin, LLaMA 2 (Giant Language Model Meta AI). Not solely is that this new massive language mannequin (LLM) now obtainable, it’s additionally open-source and freely obtainable for business use — not like the primary LLaMA, which was licensed just for analysis functions.

The information, coupled with Microsoft’s outspoken assist for LLaMA 2, means the fast-moving world of generative AI has simply shifted but once more. Now the numerous enterprises speeding to embrace AI, albeit cautiously, have an alternative choice to select from, and this one is solely free — not like chief and rival OpenAI’s ChatGPT Plus, or challengers like Cohere.

Rumors surrounding the brand new launch of LLaMA have been swirling within the business for not less than a month, as U.S senators have been questioning Meta in regards to the availability of the AI mannequin.

The primary iteration of LLaMA was obtainable for lecturers and researchers below a analysis license. The mannequin weights underlying LLaMA have been nonetheless leaked, inflicting some controversy resulting in the federal government inquiry. With LLaMA 2, Meta is brushing apart the prior controversy and transferring forward with a extra highly effective mannequin that can be extra broadly usable than its predecessor and doubtlessly shake up your complete LLM panorama.

Microsoft hedges its AI bets

The LLaMA 2 mannequin is being made obtainable on Microsoft Azure. That’s noteworthy in that Azure can also be the main dwelling for OpenAI and its GPT-3/GPT-4 household of LLMs. Microsoft is an investor each in Meta’s former firm Fb and in OpenAI.

Meta founder and CEO Mark Zuckerberg is especially obsessed with LLaMA being open-source. In a press release, Zuckerberg famous that Meta has an extended historical past with open supply and has made many notable contributions, significantly in AI with the PyTorch machine studying framework.

“Open supply drives innovation as a result of it permits many extra builders to construct with new expertise,” Zuckerberg acknowledged. “It additionally improves security and safety as a result of when software program is open, extra individuals can scrutinize it to establish and repair potential points. I consider it might unlock extra progress if the ecosystem have been extra open, which is why we’re open sourcing Llama 2.”

In a Twitter message, Yann LeCun, VP and chief AI scientist at Meta, additionally heralded the open-source launch.

“That is large: [LLaMA 2] is open supply, with a license that authorizes business use!” LeCun wrote. “That is going to alter the panorama of the LLM market. [LLaMA 2] is accessible on Microsoft Azure and can be obtainable on AWS, Hugging Face and different suppliers”

What’s inside LLaMA?

LLaMA is a transformer-based auto-regressive language mannequin. The primary iteration of LLaMA was publicly detailed by Meta in February as a 65 billion-parameter mannequin able to a big selection of widespread generative AI duties.

In distinction, LLaMA 2 has a variety of mannequin sizes, together with seven, 13 and 70 billion parameters. Meta claims the pre-trained fashions have been educated on an enormous dataset that was 40% bigger than the one used for LLaMA 1. The context size has additionally been expanded to 2 trillion tokens, double the context size of LLaMA 1.

Not solely has LLaMA been educated on extra information, with extra parameters, the mannequin additionally performs higher than its predecessor, in response to benchmarks offered by Meta.

Security measures touted

LLaMA 2 isn’t all about energy, it’s additionally about security. LLaMA 2 is first pretrained with publicly obtainable information. The mannequin then goes by a sequence of supervised fine-tuning (SFT) phases. As a further layer, LLaMA 2 then advantages from a cycle of reinforcement studying from human suggestions (RLHF) to assist present an additional diploma of security and accountability.

Meta’s analysis paper on LLaMA 2 offers exhaustive particulars on the great steps taken to assist present security and restrict potential bias as effectively.

“It is very important perceive what’s within the pretraining information each to extend transparency and to make clear root causes of potential downstream points, similar to potential biases,” the paper states. “This could inform what, if any, downstream mitigations to think about, and assist information applicable mannequin use.”