{"id":882,"date":"2020-09-01T14:04:50","date_gmt":"2020-09-01T14:04:50","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/09\/01\/showcasing-the-benefits-of-software-optimizations-for-ai-workloads-on-intel-xeon-scalable-platforms\/"},"modified":"2020-09-01T14:04:50","modified_gmt":"2020-09-01T14:04:50","slug":"showcasing-the-benefits-of-software-optimizations-for-ai-workloads-on-intel-xeon-scalable-platforms","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/09\/01\/showcasing-the-benefits-of-software-optimizations-for-ai-workloads-on-intel-xeon-scalable-platforms\/","title":{"rendered":"Showcasing the Benefits of Software Optimizations for AI Workloads on Intel\u00ae Xeon\u00ae Scalable Platforms"},"content":{"rendered":"
\n

By Huma Abidi & Haihao Shen, Intel<\/b><\/p>\n

Intel\u00ae Xeon\u00ae Scalable platforms provide the foundation for an evolutionary leap forward in data centric innovation. Intel\u00ae Deep Learning Boost<\/a> is a technology that has built-in AI acceleration to enhance artificial intelligence inference and training performance. Specifically, for the 3rd Generation Intel Xeon Scalable Processors that was announced<\/a> in June 2020, it was the industry\u2019s first x86 support of Brain Floating Point 16-bit (blfoat16) and Vector Neural Network Instructions (VNNI). The enhancements in hardware architecture, coupled with software optimizations deliver up to 1.93x<\/b> more performance in AI training and up to 1.9x<\/b> more performance in AI inference compared to 2nd<\/sup> Generation Intel Xeon Scalable processors1<\/sup>. <\/p>\n

The focus of this blog is to bring to light that continued software optimizations can boost performance not only for the latest platforms, but also for the current install base from prior generations. For each magnitude of performance increase achieved by new hardware, software optimizations have the potential to multiply those results. However, new hardware platforms come to market roughly every 12-18 months. In between those new platform introductions, we continue to innovate on the software stack in order to enhance performance for even the current generation of platforms. In addition, the open source software stacks such as TensorFlow and PyTorch are continuously driving innovations and changes into their implementation. Our software stack is aligned to address these rapid releases and accelerate our Intel-optimized versions. We are also at the forefront of enabling new models and use cases published by the AI research community in industry and academia. This means customers can continue to extract value from their current platform investments. <\/p>\n

To highlight this point, we measured the performance on popular deep learning workloads. The table below includes image classification (ResNet50 v1.5), natural language processing (Transformer\/BERT), and recommendation systems (Wide & Deep). It is important to note that the workloads were all run on 2nd<\/sup> generation Intel Xeon Scalable processors. The only variables are the different versions of software releases. Therefore, the performance gains in the resulting table are purely attributed to software optimizations.<\/p>\n

\u00a0<\/p>\n

Results<\/h3>\n

\u00a0
Deep Learning Workload Performance<\/b>2<\/b><\/sup><\/p>\n

\"Results\"<\/div>\n

\u00a0<\/p>\n

These impressive performance gains are due to software optimizations used in Intel-optimized deep learning frameworks (e.g.TensorFlow, PyTorch, and MXNet) as well as the Intel\u00ae Distribution of OpenVINOTM <\/sup>toolkit<\/a> for deep learning inference. Listed are some of our approaches to software optimization: <\/p>\n