software by Stafford Williamsstafford williams
home blog devlog notes links talks apps about

2024-04-19 10:27am

ai, meta, moat, llama

Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters - fascinating interview - the least robot-like I’ve ever seen Zuck, he’s getting that billion-dollar media training. Highlights include:

  • They got the edge in GPU race because in 2022 they realised they were short on GPUs for training their Reels recommendation system, so purchased double what they needed.
  • They foresee the bottleneck being energy production (not chips) both in the (regulatory) time and tech required to produce enough energy to power the chips
  • They have their own chips now, so they can lessen their reliance on more expensive Nvidia chips - they wont train llama4 with their own silicon but might train llama5
  • If this was helpful, please share:

  • Reddit icon
  • Y Combinator icon
  • Twitter icon
  • LinkedIn icon
  • software by Stafford Williams
about
  • LinkedIn icon
  • Icon Letter Mail image/svg+xml Openclipart icon_letter_mail 2010-01-29T13:59:32 https://openclipart.org/detail/29117/icon_letter_mail-by-jean_victor_balin jean_victor_balin icon letter mail mailing unchecked
{{language}}