Добавить новость

В российском регионе за транзит нелегальных мигрантов осудили иностранного подданного

В Москве ожидаются +16 градусов и четыре температурных рекорда

Шаховской школьник — призер всероссийской олимпиады по литературе

Следователь не отпустил обвиняемого во взятке экс-главу «Фонбета» на СВО



News in English


Новости сегодня

Новости от TheMoneytizer

Groq is 'unleashing the beast' to chip away at Nvidia's CUDA advantage

Mark Heaps is the chief technology evangelist for Nvidia challenger Groq
  • Groq is taking a novel approach to competing with Nvidia's much-lauded CUDA software.
  • The chip startup is using a free inference tier to attract hundreds of thousands of AI developers.
  • Groq aims to capture market share with faster inference and global joint ventures.

There is an active debate about Nvidia's competitive moat. Some say there's a prevailing perception of a 'safe' choice when investing billions in a technology, in which the return is still uncertain.

Many say it's Nvidia's software, particularly CUDA, which the company began developing decades before the AI boom. CUDA allows users to get the most out of graphics processing units.

Competitors have attempted to make comparable systems, but without Nvidia's headstart, it has been tough to get developers to learn, try, and ultimately improve their systems.

Groq, however, is an Nvidia competitor that focused early on the segment of AI computing that requires less need for directly programming chips, and investors are intrigued. The 8-year-old AI chip startup was valued at $2.8 billion at its $640 million Series D round in August.

Though at least one investor has called companies like Groq 'insane' for attempting to dent Nvidia's estimated 90% market share, the startup has been building its technology exactly for the opportunity that is coming in 2025, Mark Heaps, Groq's "chief tech evangelist" said.

'Unleashing the beast'

"What we decided to do was take all of our compute, make it available via a cloud instance, and we gave it away to the world for free," Heaps said. Internally, the team called the strategy, "unleashing the beast". Groq's free tier caps users at a ceiling marked by requests per day or tokens per minute.

Heaps, CEO and ex-Googler Jonathan Ross, and a relatively lean team have spent 2023 and 2024 recruiting developers to try Groq's tech. Through hackathons and contests, the company makes a promise — try the hardware via Groq's cloud platform for free, and break through walls you've hit with others.

Groq offers some of the fastest inference out there, according to rankings on Artificialanalysis.ai, which measures cost and latency for companies that allow users to buy access to specific models by the token — or output.

Inference is a type of computing that produces the answers to queries asked of large language models. Training, the more energy-intensive type of computing, is what gives the models the ability to answer. So far, the hardware used for those two tasks has been different.

Heaps and several of his Nvidia-challenging cohorts at companies like Cerebras and SambaNova Systems said that speed is a competitive advantage.

After the inference service was available for free, developers came out of the woodwork, he said, with projects that couldn't be successful on slower chips. With more speed, developers can send one request through multiple models and use another model to choose the best response — all in the time it would usually take to fulfill just one request.

Roughly 652,000 developers are now using Groq API keys, Heaps said.

Heaps expects speed to hook developers on Groq. But its novel plan for programming its chips gives the company a unique approach to the most crucial element within Nvidia's "moat."

No need for CUDA libraries

"Everybody, once they deployed models, was gonna need faster inference at a lower cost, and so that's what we focused on," Heaps said.

So where's the CUDA equivalent? It's all in-house.

"We actually have more than 1800 models built into our compiler. We use no kernels, and we don't need people to use CUDA libraries. So because of that, people can just start working with a model that's built-in," Heaps said.

Training, he said, requires more customization at the chip level. In inference, Groq's task is to choose the right models to offer customers and ensure they run as fast as possible.

"What you're seeing with this massive swell of developers who are building AI applications — they don't want to program at the chip level," he added.

The strategy comes with some level of risk. Groq is unlikely to accumulate a stable of developers who continuously troubleshoot and improve its base software like CUDA has. Its offering may be more like a restaurant menu than a grocery store. But this also means the barrier to entry for Groq users is the same as any other cloud provider and potentially lower than that of other chips.

Though Groq started out as a company with a novel chip design, today, of the company's roughly 300 employees, 60% are software engineers, Heaps said.

"For us right now, there is a billions and billions of dollars industry emerging, that we can go capture a big share of market in, while at the same time, we continue to mature the compiler," he said.

Despite being realistic about the near-term, Groq has lofty ambitions, which board CEO Jonathan Ross has described as "providing half the world's inference." Ross also says the goal is to cast a net over the globe — to be achieved via joint ventures. Saudi Arabia is on the way. Canada and Latin America are in the works.

Earlier this year, Ross told BI the company also has a goal to ship 108,000 of its language processing units or LPUs by the first quarter of next year — and 2 million chips by the end of 2025, most of which will be made available through its cloud.

Have a tip or an insight to share? Contact Emma at ecosgrove@businessinsider.com or use the secure messaging app Signal: 443-333-9088

Read the original article on Business Insider

Читайте на 123ru.net


Новости 24/7 DirectAdvert - доход для вашего сайта



Частные объявления в Вашем городе, в Вашем регионе и в России



Smi24.net — ежеминутные новости с ежедневным архивом. Только у нас — все главные новости дня без политической цензуры. "123 Новости" — абсолютно все точки зрения, трезвая аналитика, цивилизованные споры и обсуждения без взаимных обвинений и оскорблений. Помните, что не у всех точка зрения совпадает с Вашей. Уважайте мнение других, даже если Вы отстаиваете свой взгляд и свою позицию. Smi24.net — облегчённая версия старейшего обозревателя новостей 123ru.net. Мы не навязываем Вам своё видение, мы даём Вам срез событий дня без цензуры и без купюр. Новости, какие они есть —онлайн с поминутным архивом по всем городам и регионам России, Украины, Белоруссии и Абхазии. Smi24.net — живые новости в живом эфире! Быстрый поиск от Smi24.net — это не только возможность первым узнать, но и преимущество сообщить срочные новости мгновенно на любом языке мира и быть услышанным тут же. В любую минуту Вы можете добавить свою новость - здесь.




Новости от наших партнёров в Вашем городе

Ria.city

За город или за океан? Где недорого отдохнуть на весенних каникулах

Утреннее чтиво: новый премьер Канады, блогер под арестом, индексация пенсий

Рейс в Москву из Волгограда задержали из-за ограничений в Шереметьево

В Туле впервые покажут работы художника Михаила Сотникова

Музыкальные новости

В Псковской области пройдет десятый этап культурно-образовательного кластера «Город мастеров»

«Семь Я» Стаса Пьехи при поддержке «Авторадио»

Лукашенко приедет в Россию на следующей неделе для переговоров с Путиным

«Белая Луна»: DONI, DJ DAVEED и VIVATS представили второй фит

Новости России

В России начались продажи кроссоверов Nissan X-Trail по цене от 3,25 млн рублей

На юго-западе столицы восстановят фасад дома 1957 года постройки

Утреннее чтиво: новый премьер Канады, блогер под арестом, индексация пенсий

Рейс в Москву из Волгограда задержали из-за ограничений в Шереметьево

Экология в России и мире

Green Drive - лучший корпоративный экопроект прошлого года

Весенние цветы дарят свердловчанкам медики Центра СПИДа за заботу о здоровье

Givenchy, коллекция осень-зима 2025

Sacai, коллекция осень-зима 2025

Спорт в России и мире

Мирра Андреева одержала победу над Кларой Таусон на турнире WTA 1000

Казанская теннисистка Полина Кудерметова вышла в третий круг «тысячника» в США

Ничушкин оформил хет-трик в НХЛ, Рублёв проиграл на старте «Мастерса». Главное к утру

Рублев не смог пройти в третий раунд турнира "Мастерс" в Индиан-Уэллсе.

Moscow.media

В честь 80-летия Победы в Великой Отечественной войне ветераны получат выплаты накануне праздника

Движение на проспекте Космонавтов в Екатеринбурге может затруднить ремонт дороги

Беспроводной сканер штрих-кодов SAOTRON P05i промышленного класса

В Москве и Московской области 103 тысячи многодетных матерей и матерей детей с инвалидностью досрочно получают пенсию











Топ новостей на этот час

Rss.plus






Суточная аренда жилья на майские праздники в РФ увеличилась на 18 процентов

В парках Королева Московской области запланировали познавательную неделю

В России начались продажи кроссоверов Nissan X-Trail по цене от 3,25 млн рублей

На юго-западе столицы восстановят фасад дома 1957 года постройки