-
- News
- Books
Featured Books
- design007 Magazine
Latest Issues
Current IssueBreaking High-speed Material Constraints
Do you need specialty materials for your high-speed designs? Maybe not. Improvements in resins mean designers of high-speed boards can sometimes use traditional laminate systems. Learn more in this issue.
Level Up Your Design Skills
This month, our contributors discuss the PCB design classes available at IPC APEX EXPO 2024. As they explain, these courses cover everything from the basics of design through avoiding over-constraining high-speed boards, and so much more!
Opportunities and Challenges
In this issue, our expert contributors discuss the many opportunities and challenges in the PCB design community, and what can be done to grow the numbers of PCB designers—and design instructors.
- Articles
- Columns
Search Console
- Links
- Events
||| MENU - design007 Magazine
Intel Gaudi, Xeon and AI PC Accelerate Meta Llama 3 GenAI Workloads
April 22, 2024 | Intel CorporationEstimated reading time: 2 minutes
Meta launched Meta Llama 3, its next-generation large language model (LLM). Effective on launch day, Intel has validated its AI product portfolio for the first Llama 3 8B and 70B models across Intel® Gaudi® accelerators, Intel® Xeon® processors, Intel® Core™ Ultra processors and Intel® Arc™ graphics.
“Intel actively collaborates with the leaders in the AI software ecosystem to deliver solutions that blend performance with simplicity. Meta Llama 3 represents the next big iteration in large language models for AI. As a major supplier of AI hardware and software, Intel is proud to work with Meta to take advantage of models such as Llama 3 that will enable the ecosystem to develop products for cutting-edge AI applications,” said Wei Li, Intel vice president and general manager of AI Software Engineering.
As part of its mission to bring AI everywhere, Intel invests in the software and AI ecosystem to ensure that its products are ready for the latest innovations in the dynamic AI space. In the data center, Intel Gaudi and Intel Xeon processors with Intel® Advanced Matrix Extension (Intel® AMX) acceleration give customers options to meet dynamic and wide-ranging requirements.
Intel Core Ultra processors and Intel Arc graphics products provide both a local development vehicle and deployment across millions of devices with support for comprehensive software frameworks and tools, including PyTorch and Intel® Extension for PyTorch® used for local research and development and OpenVINO™ toolkit for model development and inference.
About the Llama 3 Running on Intel:
Intel’s initial testing and performance results for Llama 3 8B and 70B models use open source software, including PyTorch, DeepSpeed, Intel Optimum Habana library and Intel Extension for PyTorch to provide the latest software optimizations. For more performance details, visit the Intel Developer Blog.
Intel® Gaudi® 2 accelerators have optimized performance on Llama 2 models – 7B, 13B and 70B parameters – and now have initial performance measurements for the new Llama 3 model. With the maturity of the Intel Gaudi software, Intel easily ran the new Llama 3 model and generated results for inference and fine tuning. Llama 3 is also supported on the recently announced Intel® Gaudi® 3 accelerator.
Intel Xeon processors address demanding end-to-end AI workloads, and Intel invests in optimizing LLM results to reduce latency. Intel® Xeon® 6 processors with Performance-cores (code-named Granite Rapids) show a 2x improvement on Llama 3 8B inference latency compared with 4th Gen Intel® Xeon® processors and the ability to run larger language models, like Llama 3 70B, under 100ms per generated token.
Intel Core Ultra and Intel Arc Graphics deliver impressive performance for Llama 3. In an initial round of testing, Intel Core Ultra processors already generate faster than typical human reading speeds. Further, the Intel® Arc™ A770 GPU has Xe Matrix eXtensions (XMX) AI acceleration and 16GB of dedicated memory to provide exceptional performance for LLM workloads.
What’s Next: In the coming months, Meta expects to introduce new capabilities, additional model sizes and enhanced performance. Intel will continue to optimize performance for its AI products to support this new LLM.
Suggested Items
Kevin O’Buckley to Lead Foundry Services at Intel
05/14/2024 | Intel CorporationIntel Corporation announced the appointment of Kevin O’Buckley as senior vice president and general manager of Foundry Services, the customer service and ecosystem operations division of Intel Foundry. O’Buckley starts today and becomes a member of Intel’s executive leadership team reporting to CEO Pat Gelsinger.
SAIC Awarded $232 Million U.S. Army Contract for Systems Engineering and IT Modernization Services
05/13/2024 | SAICScience Applications International Corp. has been awarded a $232 million contract to develop signals intelligence and electronic warfare systems for the U.S. Army. SAIC was awarded this contract under the Department of Defense Information Analysis Center’s (DoD IAC) multiple-award contract (MAC) vehicle.
I-Connect007 Editor’s Choice: Five Must-Reads for the Week
05/03/2024 | Nolan Johnson, I-Connect007This week’s most important news is strategic—and telling. When one puts together the IPC industry reports, we simply have to include the recent conversation with Shawn DuBravac and Tom Kastner. On the design side, check out the latest “On The Line With…” podcast featuring Brad Griffin from Cadence Design Systems, discussing SI and PI in the realm of intelligent system design.
First Two WorldView Legion Spacecraft Performing Well After Launch
05/03/2024 | BUSINESS WIREMaxar Intelligence, provider of secure, precise geospatial intelligence, today confirmed the first two WorldView Legion satellites are performing well after being launched on a SpaceX Falcon 9 rocket earlier today from Vandenberg Space Force Base, California.
Dubai Launches Global Blueprint for Artificial Intelligence
05/02/2024 | BUSINESS WIREDubai has launched a blueprint for Artificial Intelligence (AI), a yearly plan that will focus on harnessing the technology’s potential to improve quality of life around the world.