Startup DreamersStartup Dreamers
  • Home
  • Startup
  • Money & Finance
  • Starting a Business
    • Branding
    • Business Ideas
    • Business Models
    • Business Plans
    • Fundraising
  • Growing a Business
  • More
    • Innovation
    • Leadership
Trending

What Time Does It Start?

May 31, 2025

Best and Worst States for Retirement? Here’s the Ranking

May 31, 2025

Why Conversational Commerce is the Future of Shopping

May 31, 2025
Facebook Twitter Instagram
  • Newsletter
  • Submit Articles
  • Privacy
  • Advertise
  • Contact
Facebook Twitter Instagram
Startup DreamersStartup Dreamers
  • Home
  • Startup
  • Money & Finance
  • Starting a Business
    • Branding
    • Business Ideas
    • Business Models
    • Business Plans
    • Fundraising
  • Growing a Business
  • More
    • Innovation
    • Leadership
Subscribe for Alerts
Startup DreamersStartup Dreamers
Home » Meta’s Next Llama AI Models Are Training on a GPU Cluster ‘Bigger Than Anything’ Else
Startup

Meta’s Next Llama AI Models Are Training on a GPU Cluster ‘Bigger Than Anything’ Else

adminBy adminOctober 31, 20240 ViewsNo Comments3 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email

Managing such a gargantuan array of chips to develop Llama 4 is likely to present unique engineering challenges and require vast amounts of energy. Meta executives on Wednesday sidestepped an analyst question about energy access constraints in parts of the US that have hampered companies’ efforts to develop more powerful AI.

According to one estimate, a cluster of 100,000 H100 chips would require 150 megawatts of power. The largest national lab supercomputer in the United States, El Capitan, by contrast requires 30 megawatts of power. Meta expects to spend as much as $40 billion in capital this year to furnish data centers and other infrastructure, an increase of more than 42 percent from 2023. The company expects even more torrid growth in that spending next year.

Meta’s total operating costs have grown about 9 percent this year. But overall sales—largely from ads—have surged more than 22 percent, leaving the company with fatter margins and larger profits even as it pours billions of dollars into the Llama efforts.

Meanwhile, OpenAI, considered the current leader in developing cutting-edge AI, is burning through cash despite charging developers for access to its models. What for now remains a nonprofit venture has said that it is training GPT-5, a successor to the model that currently powers ChatGPT. OpenAI has said that GPT-5 will be larger than its predecessor, but it has not said anything about the computer cluster it is using for training. OpenAI has also said that in addition to scale, GPT-5 will incorporate other innovations, including a recently developed approach to reasoning.

CEO Sam Altman has said that GPT-5 will be “a significant leap forward” compared to its predecessor. Last week, Altman responded to a news report stating that OpenAI’s next frontier model would be released by December by writing on X, “fakes news out of control.”

On Tuesday, Google CEO Sundar Pichai said the company’s newest version of the Gemini family of generative AI models is in development.

Meta’s open approach to AI has at times proven controversial. Some AI experts worry that making significantly more powerful AI models freely available could be dangerous because it could help criminals launch cyberattacks or automate the design of chemical or biological weapons. Although Llama is fine-tuned prior to its release to restrict misbehavior, it is relatively trivial to remove these restrictions.

Zuckerberg remains bullish about the open source strategy, even as Google and OpenAI push proprietary systems. “It seems pretty clear to me that open source will be the most cost effective, customizable, trustworthy, performant, and easiest to use option that is available to developers,” he said on Wednesday. “And I am proud that Llama is leading the way on this.”

Zuckerberg added that the new capabilities of Llama 4 should be able to power a wider range of features across Meta services. Today, the signature offering based on Llama models is the ChatGPT-like chatbot known as Meta AI that’s available in Facebook, Instagram, WhatsApp, and other apps.

Over 500 million people monthly use Meta AI, Zuckerberg said. Over time, Meta expects to generate revenue through ads in the feature. “There will be a broadening set of queries that people use it for, and the monetization opportunities will exist over time as we get there,” Meta CFO Susan Li said on Wednesday’s call. With the potential for revenue from ads, Meta just might be able to pull off subsidizing Llama for everyone else.

Read the full article here

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Articles

Is Elon Musk Really Stepping Back from DOGE?

Startup May 31, 2025

Esoteric Programming Languages Are Fun—Until They Kill the Joke

Startup May 30, 2025

DOGE Loses Battle to Take Over USIP—and Its $500 Million Headquarters

Startup May 29, 2025

Kentucky’s Bitcoin Boom Has Gone Bust

Startup May 28, 2025

A Silicon Valley VC Got Israel Starlink Access Within Days of October 7 Attack

Startup May 27, 2025

Bluesky Is Plotting a Total Takeover of the Social Internet

Startup May 26, 2025
Add A Comment

Leave A Reply Cancel Reply

Editors Picks

What Time Does It Start?

May 31, 2025

Best and Worst States for Retirement? Here’s the Ranking

May 31, 2025

Why Conversational Commerce is the Future of Shopping

May 31, 2025

Is Elon Musk Really Stepping Back from DOGE?

May 31, 2025

Astronomers Need Better Models To Explain Webb’s Puzzling Observations

May 30, 2025

Latest Posts

His Kids Kept Breaking Things Playing Indoors — So He Invented a Toy That’s Sold Over 3 Million Units

May 30, 2025

Esoteric Programming Languages Are Fun—Until They Kill the Joke

May 30, 2025

Today’s NYT Mini Crossword Clues And Answers For Thursd, May 29

May 29, 2025

Why Every Company Should Have a 90-Day Cash Flow Buffer

May 29, 2025

My Bootstrapped Business Led to $20M for Women Entrepreneurs

May 29, 2025
Advertisement
Demo

Startup Dreamers is your one-stop website for the latest news and updates about how to start a business, follow us now to get the news that matters to you.

Facebook Twitter Instagram Pinterest YouTube
Sections
  • Growing a Business
  • Innovation
  • Leadership
  • Money & Finance
  • Starting a Business
Trending Topics
  • Branding
  • Business Ideas
  • Business Models
  • Business Plans
  • Fundraising

Subscribe to Updates

Get the latest business and startup news and updates directly to your inbox.

© 2025 Startup Dreamers. All Rights Reserved.
  • Privacy Policy
  • Terms of use
  • Press Release
  • Advertise
  • Contact

Type above and press Enter to search. Press Esc to cancel.

GET $5000 NO CREDIT