Kneron to launch AI chip this yr

On this planet of generative AI, it’s a battle of computing energy and getting the quickest and strongest chips. Now, AI edge firm Kneron introduced it should ship its new neural processing models (NPU) chips by the top of the yr. 

Kneron mentioned the NPU chips, known as the KL730, would make it cheaper to run massive language fashions (LLMs) because the processor is constructed particularly for machine studying and AI purposes. 

The KL730 is the following technology of earlier processors from Kneron. In 2021, the corporate shipped out the KL530 chips that supported transformer fashions that underpinned some generative AI fashions. 

Albert Liu, CEO of Kneron, tells The Verge that NPU chips are particularly designed for AI and aren’t forcing one thing initially made for processing graphics to work for it — an implicit dig at reigning AI chipmaker Nvidia.

“I’ll say that you probably have a fairly highly effective and light-weight chip like ours, then you may deliver a strong transformer mannequin like GPT to many sorts of gadgets,” Liu mentioned. 

Liu wouldn’t disclose the worth of the KL730 however notes that customers of its KL530 chip noticed a 75 % drop in working prices in comparison with GPU chips. 

Most AI corporations and cloud suppliers flock to Nvidia’s H100 Tensor Core GPU chips, as individuals imagine GPUs are essentially the most accessible processors able to compiling the calculations wanted to run generative AI fashions. However even with that energy, it normally takes loads of H100s to run one massive language mannequin, so customers need to “break up” the mannequin to get it to run. 

Even so, costs for the H100 soared to roughly $40,000 per chip as demand continued to develop. Nvidia already introduced plans to launch a extra highly effective AI chip within the second quarter of 2024. Opponents are already ready within the wings, with AMD planning to launch its personal AI chips within the fourth quarter of this yr. 

Kneron mentioned the KL730 “yields a 3 to 4 occasions leap” in power effectivity in comparison with earlier chips and has a base-level compute energy beginning at 0.35 tera operations per second. 

The corporate mentioned the brand new chip additionally permits customers to run LLMs absolutely offline with out the necessity to hook up with a cloud supplier and deal with information extra securely.