Performance Index

ID Date Classification
615781 11/27/2024 Public
Document Table of Contents

4th Generation Intel® Xeon® Scalable Processors

Performance varies by use, configuration and other factors.

Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See configuration disclosure for details. No product or component can be absolutely secure.

Availability of accelerators varies depending on SKU. Visit the Intel Product Specifications page for additional product details.

Intel optimizations, for Intel compilers or other products, may not optimize to the same degree for non-Intel products.

Estimates of SPECrate®2017_​int_​base and SPECrate®2017_​fp_​base based on Intel internal measurements. SPEC®, SPECrate® and SPEC CPU® are registered trademarks of the Standard Performance Evaluation Corporation. See www.spec.org for more information.

Claim Processor Family System Configuration Measurement Measurement Period
4th Gen Intel Xeon Scalable processors with Intel vRAN Boost delivers up to double the vRAN capacity with additional approximately 20% compute power savings vs. previous Gen platform, effectively more than doubling the performance per watt. 4th Generation Intel® Xeon® Platinum processor

SPR-EE 6443N on 4th Generation Platform: 1-node, 1x SPR-EE 6443N (14 cores used, the remaining cores in C6, 59.98% CPU Utilization) on Intel reference platform with 256GB (8 slots/ 32GB/ 4800Mhz[run at 4400Mhz]) total DDR5 memory, microcode 0xfd010420, HT ON, Turbo ON, Ubuntu 22.04, Kernel: 5.15.0-1009-realtime, 1x <WDS100T2B0A 1000GB SSD>, FlexRAN v23.07, ICX: 2023.0.0.20221201 GCC: 11.4.0,

ICX-SP 6338N on 3rd Generation Platform with ACC100 card: 1-node, 1x ICX-SP 6338N (14 cores used, the remaining cores in C6, 52.46% CPU Utilization) and 1x ACC100 card on Intel reference platform with 128GB (8 slots/ 16GB/ 2933Mhz[run at 2666Mhz]) total DDR4 memory, microcode 0xd000389, HT ON, Turbo ON, Ubuntu 22.04, Kernel: 5.15.0-1009-realtime, 1x <SA400S3 960GB SSD>, FlexRAN v23.07, ICX: 2023.0.0.20221201 GCC: 11.4.0

vRAN performance and power Test by Intel on August 9, 2023.
[A1] Up to 4x end-to-end fine-tuning performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) for Document Level Sentiment Analysis (DLSA) Hugging Face Distilbert on SST-2.

Up to 3.5x end-to-end fine-tuning performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) for Document Level Sentiment Analysis (DLSA) Hugging Face Distilbert on IMDB.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, 1x INTEL SSDSC2KG01, test by Intel on 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, 1x INTEL SSDSC2BA80, test by Intel on 10/21/2022. Software: Distilbert-base-uncased (256MB : 66 Million Param) https://huggingface.co/distilbert-base-uncased, IMDB (25K for fine-tuning and 25K for inference): 512 Seq Length https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1764; SST-2 (67K for fine-tuning and 872 for inference) : 56 Seq Length https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1765, FP32, BF16, 8 instances, https://pytorch.org/, PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2. DLSA on HuggingFace Distilbert - Fine Tuning Test by Intel on October 21, 2022.
[A2] IMDB Dataset Up to 5.7x end-to-end real-time inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on Document Level Sentiment Analysis (DLSA) with Hugging Face (IMDB).

Up to 2.3x end-to-end real-time inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on Document Level Sentiment Analysis (DLSA) with Hugging Face (IMDB).

Up to 6.7x end-to-end batch inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on Document Level Sentiment Analysis (DLSA) with Hugging Face (IMDB).

Up to 2.4x end-to-end batch inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on Document Level Sentiment Analysis (DLSA) with Hugging Face (IMDB). SST-2 Dataset.

Up to 3.5x end-to-end real-time inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on Document Level Sentiment Analysis (DLSA) with Hugging Face (SST-2).

Up to 1.5x end-to-end real-time inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on Document Level Sentiment Analysis (DLSA) with Hugging Face (SST-2).

Up to 2.5x end-to-end inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on Document Level Sentiment Analysis (DLSA) with Hugging Face (SST-2).

Up to 1.5x end-to-end inference performance speedup using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on Document Level Sentiment Analysis (DLSA) with Hugging Face (SST-2).
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, 1x INTEL SSDSC2KG01, Bert-large-uncased (1.3GB : 340 Million Param) https://huggingface.co/bert-large-uncased, IMDB (25K for fine-tuning and 25K for inference): 512 Seq Length - https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1764; SST-2 (67K for fine-tuning and 872 for inference): 56 Seq Length - https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1765, FP32, BF16,INT8, 28 instances, https://pytorch.org/, PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, MKL 2022.1.0, test by Intel on 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, 1x INTEL SSDSC2BA80, Bert-large-uncased (1.3GB : 340 Million Param) https://huggingface.co/bert-large-uncased, IMDB (25K for fine-tuning and 25K for inference): 512 Seq Length - https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1764; SST-2 (67K for fine-tuning and 872 for inference): 56 Seq Length - https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1765, fp32, int8, 20 instances, https://pytorch.org/, PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, MKL 2022.1.0, test by Intel on 10/21/2022. DLSA on HuggingFace - Inference Test by Intel on October 21, 2022.
[A3] Up to 1.29x higher end-to-end recommendations/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on Deep Interest Evolution Network (DIEN) Recommendation System. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7TB, E2E DIEN - https://github.com/alibaba/ai-matrix/tree/master/macro_benchmark/DIEN_TF2, FP32,BF16, BS1024, Spark 3.2.1 - https://spark.apache.org/, Intel TensorFlow avx512 2.10.0 - https://pypi.org/project/intel-tensorflow-avx512/, oneDNN 2.6.0, Dataset: Amazon Books 2018: meta(2,935,525, 12)​; reviews(51,311,621,10), test by Intel on 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80 800GB, E2E DIEN, FP32, BS1024, Spark 3.2.1 - https://spark.apache.org/, Intel TensorFlow avx512 2.10.0 - https://pypi.org/project/intel-tensorflow-avx512/, oneDNN 2.6.0, Dataset: Amazon Books 2018: meta(2,935,525, 12)​; reviews(51,311,621,10), test by Intel on 10/21/2022. E2E DIEN RecSys Test by Intel on October 21, 2022.
[A4] Up to 2.3x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on end-to-end video streamer.

Up to 1.7x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on end-to-end video streamer.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Video Streamer, BS1, FP32/BF16/INT8, ssd-mobilenet, Intel-tensorflow-avx512 2.10.0, oneDNN v2.6.0, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams: 56, test by Intel on 10/18/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80 800G, E2E Video Streamer, BS1, FP32/BF16/INT8, ssd-mobilenet, Intel-tensorflow-avx512 2.10.0, oneDNN v2.6.0, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams: 26 fp32/40 int8, test by Intel on 10/17/2022. E2E Video Streamer (TF) Test by Intel on New: October 21, 2022.

Baseline: October 17, 2022.

[A5] Up to 6x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32) on end-to-end face recognition.

Up to 2.5x higher end-to-end frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with prior generation (int8) on end-to-end face recognition.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Face Recognition with SSD-MobileNet + ResNet50_​v1.5 inference, FP32/BF16/INT8, BS1, maintain 30fps per stream, Intel-tensorflow-avx512 2.10.0, ssd-mobilenet, Resnet50_​v1.5, oneDNN v2.6.0, Dataset: Chelsea_​celebration.mp4 (7969 frames), test by Intel on 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80, E2E Face Recognition with SSD-MobileNet + ResNet50_​v1.5 inference, FP32/BF16/INT8, BS1, maintain 30fps per stream, Intel-tensorflow-avx512 2.10.0, ssd-mobilenet, Resnet50_​v1.5, oneDNN v2.6.0, Dataset: Chelsea_​celebration.mp4 (7969 frames), test by Intel on 10/21/2022. E2E Face Recognition Test by Intel on October, 21 2022.
[A8] Up to 1.41x faster end-to-end Census workload performance using 4th Gen Intel Xeon Scalable processor compared to prior generation. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, Census, Python 3.9.13, Modin 0.16.2,Scikit-Learn Intelex (Intel Optimized) 2021.6.0, Daal4py 2021.6.0, XGBoost 1.6.2, Dataset: IPUMS USA:https://usa.ipums.org/usa/, datatypes: int64, float64, test by Intel on 10/18/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80 800G, Census, Python 3.9.13, Modin 0.16.2,Scikit-Learn Intelex (Intel Optimized) 2021.6.0, Daal4py 2021.6.0, XGBoost 1.6.2, Dataset: IPUMS USA:https://usa.ipums.org/usa/, datatypes: int64, float64, test by Intel on 10/17/2022. E2E Census Test by Intel on October,18 2022.

Test by Intel on October,17 2022.

[A9] Up to 1.57x faster end-to-end Census workload performance with 4th Gen Intel Xeon Scalable processor compared to prior generation. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, PLAsTiCC, Python 3.9.13, Modin 0.16.2, Scikit-Learn Intelex (Intel Optimized) 2021.6.0, Daal4py 2021.6.0, XGBoost 1.6.2, Kaggle: https://www.kaggle.com/c/PLAsTiCC-2018/data, int32/float32, test by Intel on 10/18/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80 800G, PLAsTiCC, Python 3.9.13, Modin 0.16.2, Scikit-Learn Intelex (Intel Optimized) 2021.6.0, Daal4py 2021.6.0, XGBoost 1.6.2, Kaggle: https://www.kaggle.com/c/PLAsTiCC-2018/data, int32/float32, test by Intel on 10/17/2022. E2E Plasstic Test by Intel on October,18 2022.

Test by Intel on October,17 2022.

[A10] Up to 2.5x faster Vision Transfer Learning with the 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with prior generation (fp32).

Up to 1.5x faster Vision Transfer Learning with the 4th Gen Intel Xeon Scalable processor with Intel AMX (fp32) compared with prior generation (fp32).
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, Vision Transfer Learning Pipeline, Resnet50 v1.5, 1,2-sockets used, OpenMPI 4.1.2, Horovod 0.25, FP32/BF16, Intel Tensorflow AVX512 2.10, Dataset: https://www.tensorflow.org/datasets/catalog/colorectal_histology. Size: 5000 Images, Shape: 224 * 224, test by Intel on 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 CPU on Intel WHITLEY with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2BA80, Vision Transfer Learning Pipeline, Resnet50 v1.5, 1-socket used, FP32, Intel Tensorflow AVX512 2.10, Dataset: https://www.tensorflow.org/datasets/catalog/colorectal_histology. Size: 5000 Images, Shape: 224 * 224, test by Intel on 10/21/2022. E2E Vision Based Transfer Learning Test by Intel on October,21 2022.
[A16] 3.5x to 10x higher PyTorch Training performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

4.9x higher average* PyTorch Training performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

2.3x to 5.5x higher TensorFlow Training performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

3.6x higher average* TensorFlow Training performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).
4th Generation Intel® Xeon® Platinum processor See A19 to A31 for individual claims. 3.5-10x & 4.9x: PyTorch geomean of ResNet50, Bert-Large, DLRM, MaskRCNN, SSD-ResNet34, RNN-T. 2.3-5.5x & 3.6x: TensorFlow geomean of ResNet50, Bert-Large, SSD-ResNet34, Transformer. Summary Training Performance
[A17] 5.7x to 10x higher PyTorch real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

7x higher average* PyTorch real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

2.5x to 4.8x higher PyTorch real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

3.6x higher average* PyTorch real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

1.8x to 9.6x higher TensorFlow real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

4.6x higher average* TensorFlow real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32) 2.1x to 4.7x higher TensorFlow real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

2.9x higher average* TensorFlow real-time inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).
4th Generation Intel® Xeon® Platinum processor See A19 to A31 for individual claims. 5.7-10x & 7x: PyTorch geomean of ResNet50, Bert-Large, MaskRCNN, SSD-ResNet34, RNN-T, Resnext101. 2.5-4.8x & 3.6x: PyTorch geomean of ResNet50, Bert-Large,SSD-ResNet34, Resnext101. 1.8-9.6x & 4.6x: TensorFlow geomean of ResNet50, Bert-Large, SSD-ResNet34, Transformer, 3D Unet, DIEN. 2.1-4.7x & 2.9x: TensorFlow geomean of ResNet50, Bert-Large, SSD-ResNet34, Transformer, 3D Unet. Summary Real Time Inference Performance
[A18] 5.8x to 9.6x higher PyTorch batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

7x higher average* PyTorch batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

2.6x to 4.5x higher PyTorch batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

3.7x higher average* PyTorch batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

1.8x to 9x higher TensorFlow batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

4.7x higher average* TensorFlow batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. prior generation (FP32).

2.7x to 4.1x higher TensorFlow batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).

3.2x higher average* TensorFlow batch inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. prior generation (INT8).
4th Generation Intel® Xeon® Platinum processor See A19 to A31 for individual claims. 5.8-9.6x & 7x: PyTorch geomean of ResNet50, Bert-Large, MaskRCNN, SSD-ResNet34, RNN-T, Resnext101, DLRM. 2.6-4.5x & 3.7x: PyTorch geomean of ResNet50, Bert-Large, MaskRCNN, SSD-ResNet34, RNN-T, Resnext101, DLRM. 1.8-9x & 5.7x: TensorFlow geomean of ResNet50, Bert-Large, SSD-ResNet34, Transformer, 3D Unet, DIEN. 2.7-4.1x & 3.2x: TensorFlow geomean of ResNet50, Bert-Large, SSD-ResNet34, Transformer, 3D Unet. Summary Batch Inference Performance
[A19] Real Time Inference Performance Up to 6.2x higher real-time Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.6x higher real-time Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.4x higher real-time Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 7.3x higher real-time Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 5.8x higher batch Natural Language Processing inference performance (BERT)on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.6x higher batch Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4x higher batch Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 5.3x higher batch Natural Language Processing inference performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 4x higher Natural Language Processing training performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.7x higher Natural Language Processing training performance (BERT) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], Inference: bs: fp32=1,56, amx bf16=1,16, amx int8=1,56, Trg: Wikipedia 2020/01/01 ( seq len =512), bs:fp32=28, amx bf16=56 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], Inference: bs: fp32=1,56, int8=1,56, Trg: Wikipedia 2020/01/01 ( seq len =512), bs:fp32=28, amx bf16=56 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: BERT-Large Test by Intel on October,24, 2022.
[A20] Real Time Inference Performance Up to 5.8x higher real-time Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher real-time Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.2x higher real-time Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 6.6x higher real-time Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 5.7x higher batch Natural Language Processing inference performance (BERT TensorFlow)on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.3x higher batch Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.2x higher batch Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 6.1x higher batch Natural Language Processing inference performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 3.8x higher Natural Language Processing training performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.9x higher Natural Language Processing training performance (BERT TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Bert-Large, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,16, amx bf16=1,128, amx int8=1,16, Training bs:fp32=128 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Bert-Large, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,16, int8=1,16, Training bs:fp32=128 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. TF: Bert-Large Test by Intel on October,24 2022.
[A21] Batch Inference Performance Up to 6.3x higher batch Recommendation System inference performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.5x higher batch Recommendation System inference performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.2x higher batch Recommendation System inference performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 8.3x higher batch Recommendation System inference performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 4x higher Recommendation System training performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3x higher Recommendation System training performance (DLRM) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), DLRM, Inf: bs=n [1socket/instance], Inference: bs: fp32=128, amx bf16=128, amx int8=128, Training bs:fp32/amx bf16=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, DLRM, Inf: bs=n [1socket/instance], Inference: bs: fp32=128, int8=128, Training bs:fp32=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: DLRM Test by Intel on October,24, 2022.
[A22] Real Time Inference Performance Up to 10x higher real time object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4.8x higher real time object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 6.9x higher real time object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 13.6x higher real time object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 9.6x higher batch object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4.4x higher batch object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 6.6x higher batch object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 12x higher batch object detection inference performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 5.4x higher object detection training performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.7x higher object detection training performance (SSD-ResNet34) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), SSD-ResNet34, Inference: bs=n [1socket/instance], bs: fp32=1,112, amx bf16=1,112, amx int8=1,112, Training bs:fp32/amx bf16=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, SSD-ResNet34, Inference: bs=n [1socket/instance], bs: fp32=1,112, int8=1,112, Training bs:fp32=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: SSD-ResNet34 Test by Intel on October,24, 2022.
[A23] Real Time Inference Performance Up to 5.8x higher real time object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher real time object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.2x higher real time object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 6.6x higher real time object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance .

Up to 5.7x higher batch object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.3x higher batch object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.2x higher batch object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 6.1x higher batch object detection inference performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 3.8x higher object detection training performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.9x higher object detection training performance (SSD-ResNet34 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), SSD-ResNet34, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, amx bf16=1,112, amx int8=1,112, Training bs:fp32/amx bf16=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, SSD-ResNet34, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, int8=1,112, Training bs:fp32=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. TF: SSD-ResNet34 Test by Intel on October,24 2022.
[A24] Real Time Inference Performance Up to 6.2x higher real time image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher real time image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.5x higher real time image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Up to 7.5x higher real time image classification.

n inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 7.2x higher batch image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.8x higher batch image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 5.2x higher batch image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 9.7x higher batch image classification inference (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 3.5x higher image classification training performance (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.3x higher image classification training performance (ResNet50) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), ResNet50 v1.5, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, Training bs: fp32,amx bf16=128 [1 instance, 1socket], ImageNet (224 x224), Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, ResNet50 v1.5, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, int8=1,116, Training bs: fp32=128 [1 instance, 1socket], ImageNet (224 x224), Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: ResNet50-v1.5 Test by Intel on October,24, 2022.
[A25] Real Time Inference Performance Up to 6x higher real time image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.3x higher real time image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.4x higher real time image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 6.6x higher real time image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 6.5x higher batch image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher batch image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.8x higher batch image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 9.5x higher batch image classification inference (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 3.4x higher image classification training performance (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.2x higher image classification training performance (ResNet50 TensorFlow) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), ResNet50, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, amx bf16=1,80, amx int8=1,116, Training bs:fp32=1024 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, ResNet50, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, int8=1,116, Training bs:fp32=1024 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. TF: ResNet50-v1.5 Test by Intel on October,24, 2022.
[A26] Real Time Inference Performance Up to 8.6x higher real time speech recognition inference performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 5.3x higher real time speech recognition inference performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Batch Inference Performance.

Up to 6.1x higher speech recognition inference performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4x higher speech recognition inference performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Training Performance.

Up to 10.3x higher speech recognition training performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 7.8x higher speech recognition training performance (RNN-T) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Resnext101 32x16d, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, ImageNet, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Resnext101 32x16d, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, int8=1,116, ImageNet, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: RNN-T Test by Intel on October,24, 2022.

[A27] Real Time Inference Performance Up to 5.7x higher real time image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher real time image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.3x higher real time image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 8.1x higher real time image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 7.8x higher batch image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4.5x higher batch image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 5.4x higher batch image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 11.8x higher batch image classification inference performance (Resnext101) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Resnext101 32x16d, bs=n [1socket/instance], Inference: bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Resnext101 32x16d, bs=n [1socket/instance], Inference: bs: fp32=1,64, int8=1,116, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: ResNeXT101_​32x16d Test by Intel on October,24, 2022.
[A28] Real Time Inference Performance Up to 6.2x higher real time image segmentation inference performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4.5x higher real time image segmentation inference performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Batch Inference Performance.

Up to 6.4x higher batch image segmentation inference performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 4.3x higher batch image segmentation inference performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Training Performance.

Up to 4.6x higher image segmentation training performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3.1x higher image segmentation training performance (MaskRCNN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), MaskRCNN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, amx bf16=1,112, Training bs:fp32/amx bf16=112 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, MaskRCNN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, Training bs:fp32=112 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT: MaskRCNN Test by Intel on October,24, 2022.
[A29] Real Time Inference Performance Up to 5.3x higher real time volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.1x higher real time volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 3.8x higher real time volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 3.8x higher real time volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 6.3x higher batch volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.7x higher batch volumetric image segmentation inference performance (3D-Unet)on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 4.4x higher batch volumetric image segmentation inference performance (3D-Unet)on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 4.7x higher batch volumetric image segmentation inference performance (3D-Unet) on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), 3D-Unet, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,6, amx bf16=1,6, amx int8=1,6, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, 3D-Unet, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,6, int8=1,6, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. TF: 3D-Unet Test by Intel on October,24, 2022.
[A30] Real Time Inference Performance Up to 1.87x higher real time recommendation system inference performance (DIEN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 1.21x higher real time recommendation system inference performance (DIEN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32 Batch Inference Performance.

Up to 1.85x higher batch recommendation system inference performance (DIEN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 1.35x higher batch recommendation system inference performance (DIEN) on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), DIEN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=16,64K, amx bf16=16,64k, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, DIEN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=16,64K, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. TF: DIEN Test by Intel on October,24, 2022.
[A31] Real Time Inference Performance Up to 3x higher real time Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 3x higher real time Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 2x higher real time Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 3.1x higher real time Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Batch Inference Performance.

Up to 3x higher batch Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32.

Up to 2.7x higher batch volumetric image segmentation inference performance (3D-Unet)on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 vs. prior generation with INT8.

Up to 2.3x higher batch volumetric image segmentation inference performance (3D-Unet)on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.

Up to 2.6x higher batch Transformer inference performance on 4th Gen Intel Xeon Platinum 8480+ with AMX INT8 compared to FP32 Training Performance.

Up to 2.3x higher Transformer on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 vs. prior generation with FP32 Up to 2.1x higher Transformer on 4th Gen Intel Xeon Platinum 8480+ with AMX BF16 compared to FP32.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Transformer, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,448, amx bf16=1,448, amx int8=1,448, Training bs:fp32=12k [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Transformer, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,448, int8=1,448, Training bs:fp32=12k [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. TF: Transformer Test by Intel on October 24, 2022.
[A32] 1.37x higher average* machine learning training and inference performance on 4th Gen Intel Xeon Scalable processor vs. prior generation. 4th Generation Intel® Xeon® Platinum processor 1.37x average ML performance Geomean of kmeans-fit, kmeans-infer, ridge_​regr-fit, ridge_​regr-infer, linear_​regr-fit, linear_​regr-infer, logistic_​regr-fit, logistic_​regr-infer, SVC-fit, SVC-infer, dbscan-fit, kdtree_​knn-infer, elastic-net-fit, elastic-net-infer, train_​test_​split-fit, brute_​knn-infer. 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0xab0000a0, HT OS disabled, Turbo on, CentOS Stream 8, 4.18.0-408.el8.x86_​64, scikit-learn 1.0.2, icc 2021.6.0, gcc 8.5.0, python 3.9.7, conda 4.14.0, oneDAL master(a8112a7), scikit-learn-intelex 2021.4.0, scikit-learn_​bench master (3083ef8), test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on WilsonCity with 512 GB (16 slots/ 32GB/ DDR4-3200) total memory, ucode 0xd000363, HT OS disabled, Turbo on, CentOS Stream 8, 4.18.0-408.el8.x86_​64, scikit-learn 1.0.2, icc 2021.6.0, gcc 8.5.0, python 3.9.7, conda 4.14.0, oneDAL master(a8112a7), scikit-learn-intelex 2021.4.0, scikit-learn_​bench master (3083ef8), test by Intel on 8/23/2022. Scikit Learn Test by Intel on October 24, 2022.

Test by Intel on August 23, 2022.

[A33] 4.2x to 7.7x higher performance per watt on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. 3rd Gen Intel® Xeon® Scalable Processor (FP32).

5.1x higher average* higher performance per watt on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. 3rd Gen Intel® Xeon® Scalable Processor (FP32).

2.6x higher average* higher performance per watt on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. 3rd Gen Intel® Xeon® Scalable Processor (INT8).
4th Generation Intel® Xeon® Platinum processor *geomean of ResNet50, Bert-Large, SSD-ResNet34, 3D, ResNext101, DLRM for real-time and batch inferencing PT-NLP Bert-Large: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,56, amx bf16=1,16, amx int8=1,56, Trg: Wikipedia 2020/01/01 ( seq len =512), bs:fp32=28, amx bf16=56 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,56, int8=1,56, Trg: Wikipedia 2020/01/01 ( seq len =512), bs:fp32=28, amx bf16=56 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-DLRM: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), DLRM, Inference: bs=n [1socket/instance], bs: fp32=128, amx bf16=128, amx int8=128, Training bs:fp32/amx bf16=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, DLRM, Inference: bs=n [1socket/instance], bs: fp32=128, int8=128, Training bs:fp32=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-ResNet34: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), SSD-ResNet34, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, amx bf16=1,112, amx int8=1,112, Training bs:fp32/amx bf16=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, SSD-ResNet34, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, int8=1,112, Training bs:fp32=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-ResNet50: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), ResNet50 v1.5, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, Training bs: fp32,amx bf16=128 [1 instance, 1socket], ImageNet (224 x224), Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, ResNet50 v1.5, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, int8=1,116, Training bs: fp32=128 [1 instance, 1socket], ImageNet (224 x224), Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-RNN-T: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Resnext101 32x16d, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, ImageNet, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Resnext101 32x16d, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, int8=1,116, ImageNet, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-ResNext101: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Resnext101 32x16d, bs=n [1socket/instance], Inference: bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, Resnext101 32x16d, bs=n [1socket/instance], Inference: bs: fp32=1,64, int8=1,116, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. PT-MaskRCNN: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), MaskRCNN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, amx bf16=1,112, Training bs:fp32/amx bf16=112 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KG960G8, MaskRCNN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, Training bs:fp32=112 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. Inference: Resnet50 v1.5: ImageNet (224 x224), SSD Resnet34: coco 2017 (1200 x1200), Bert Large: SQuAD1.1 (seq len=384), Resnext101: ImageNet, Mask RCNN: COCO 2017, DLRM: Criteo Terabyte Dataset, RNNT: LibriSpeech. Training: Resnet50 v1.5: ImageNet (224 x224), SSD Resnet34: COCO 2017, Bert Large: Wikipedia 2020/01/01 ( seq len =512), DLRM: Criteo Terabyte Dataset, RNNT: LibriSpeech, Mask RCNN: COCO 2017. PyTorch Performance/Watt Test by Intel on October 24, 2022.
[A34] Demo: Training and Inference on Stable Diffusion 4th Generation Intel® Xeon® Platinum processor -
[A35] Up to 26x higher inference performance and up to 16x higher performance/watt (DLRM) on 4th Gen Intel Xeon vs.1st Gen Intel® Xeon® processors 4th Gen Intel® Xeon® processors (Intel Xeon Gold 5420+) and 1st Gen Intel® Xeon® processors (Intel Xeon Gold 5120) Test by Intel as of 09/22/23 1-node, 2x Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz, 14 cores, HT On, Turbo On, Total Memory 768GB (12x64GB DDR4 2666 MT/s [2400 MT/s]), BIOS 3.8b, microcode 0x2007006, 2x Ethernet Connection X722 for 10GBASE-T, 1x 894.3G Micron_​7450_​MTFDKBA960TFR, Ubuntu 22.04.2 LTS, 5.15.0-78-generic 1-node, 2x Intel(R) Xeon(R) Gold 5420+, 28 cores, HT On, Turbo On, NUMA 2, Integrated Accelerators Available [used]: DLB 2 [0], DSA 2 [0], IAX 2 [0], QAT 2 [0], Total Memory 1024GB (16x64GB DDR5 5600 MT/s [4400 MT/s]), BIOS 2.0, microcode 0x2b0004d0, 2x Ethernet Controller 10-Gigabit X540-AT2, 1x Ethernet interface, 1x 1.7T SAMSUNG MZQL21T9HCJR-00A07, Ubuntu 22.04.2 LTS, 5.15.0-78-generic DLRM performance and perf/watt refresh comparison September 22, 2023
[A36] Up to 8x higher inference performance and 5x higher performance/watt (BERT-Large) on 4th Gen Intel Xeon Platinum vs. 2nd Gen Intel® Xeon® processors 4th Gen Intel® Xeon® processors (Intel Xeon Gold 5420+) and 2nd Gen Intel® Xeon® processors (Intel Xeon Gold 5218) Test by Intel as of 10/05/23 1-node, 2x Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz, 16 cores, HT On, Turbo On, Total Memory 768GB (12x64GB DDR4 2666 MT/s [2666 MT/s]), BIOS 3.8b, microcode 0x4003604, 2x Ethernet Connection X722 for 10GBASE-T, 1x 894.3G Micron_​7450_​MTFDKBA960TFR, Ubuntu 22.04.2 LTS, 5.15.0-78-generic 1-node, 2x Intel(R) Xeon(R) Gold 5420+, 28 cores, HT On, Turbo On, NUMA 2, Integrated Accelerators Available [used]: DLB 2 [0], DSA 2 [0], IAX 2 [0], QAT 2 [0], Total Memory 1024GB (16x64GB DDR5 5600 MT/s [4400 MT/s]), BIOS 2.0, microcode 0x2b0004d0, 2x Ethernet Controller 10-Gigabit X540-AT2, 1x Ethernet interface, 1x 1.7T SAMSUNG MZQL21T9HCJR-00A07, Ubuntu 22.04.2 LTS, 5.15.0-78-generic BERT-Large performance and perf/watt refresh comparison October 5, 2023
[A201] 3.9x higher average* machine learning training and inference performance on 4th Gen Intel Xeon Scalable processor vs. Nvidia A100 GPU. 4th Generation Intel® Xeon® Platinum processor 3.9x average ML performance 2.85x average ML training performance 5.42x average ML inference performance Geomean of kmeans-fit, kmeans-infer, ridge_​regr-fit, ridge_​regr-infer, linear_​regr-fit, linear_​regr-infer, logistic_​regr-fit, logistic_​regr-infer, SVC-fit, SVC-infer, dbscan-fit, elastic-net-fit, elastic-net-infer, train_​test_​split-fit, brute_​knn-infer. 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0xab0000a0, HT OS disabled, Turbo on, CentOS Stream 8, 4.18.0-408.el8.x86_​64, scikit-learn 1.0.2, icc 2021.6.0, gcc 8.5.0, python 3.9.7, conda 4.14.0, oneDAL master(a8112a7), scikit-learn-intelex 2021.4.0, scikit-learn_​bench master (3083ef8), test by Intel on 10/24/2022. A100: 1-node, 1x NVIDIA A100-PCIE-40GB GPU & 2x Intel® Xeon® Platinum 8280 on S2600WFT with 768 GB (12 slots/ 64GB/ DDR4-3200[2933]) total memory, ucode 0x5003102, HT enabled, Turbo on, Ubuntu 20.04.4 LTS, 5.4.0-104-generic, scikit-learn 1.0.2, icc 2021.6.0, 20200928 (Red Hat 8.4.1-1), python 3.9.7, conda 4.14.0, scikit-learn_​bench master (3083ef8), rapids 22.06, cuml 22.6.0, test by Intel on 9/2/2022. Scikit Learn Test by Intel on October 24, 2022.

Test by Intel on September 2, 2022.

[A202] 1.9x higher average* machine learning training and inference performance on 4th Gen Intel Xeon Scalable processor vs. AMD EPYC 7763. 4th Generation Intel® Xeon® Platinum processor 1.9x average ML performance 1.55 average ML training performance 2.27x average ML inference performance Geomean of kmeans-fit, kmeans-infer, ridge_​regr-fit, ridge_​regr-infer, linear_​regr-fit, linear_​regr-infer, logistic_​regr-fit, logistic_​regr-infer, SVC-fit, SVC-infer, dbscan-fit, kdtree_​knn-infer, elastic-net-fit, elastic-net-infer, train_​test_​split-fit, brute_​knn-infer. 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0xab0000a0, HT OS disabled, Turbo on, CentOS Stream 8, 4.18.0-408.el8.x86_​64, scikit-learn 1.0.2, icc 2021.6.0, gcc 8.5.0, python 3.9.7, conda 4.14.0, oneDAL master(a8112a7), scikit-learn-intelex 2021.4.0, scikit-learn_​bench master (3083ef8), test by Intel on 10/24/2022. 7763: 1-node, 2x AMD EPYC 7763 on MZ92-FS0-00 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT OS disabled, Turbo on, Red Hat Enterprise Linux 8.4 (Ootpa), 4.18.0-408.el8.x86_​64, scikit-learn 1.0.2, icc 2021.6.0, gcc 8.5.0, python 3.9.7, conda 4.14.0, oneDAL master(a8112a7), scikit-learn-intelex 2021.4.0, scikit-learn_​bench master (3083ef8), test by Intel on 9/1/2022. Scikit Learn Test by Intel on September 1, 2022.
[A204] Up to 2.4x higher end-to-end recommendations/second using 4th Gen Intel Xeon Platinum 8480+ with Intel AMX (bf16) compared to Nvidia A10 GPU (fp16) on Deep Interest Evolution Network (DIEN) Recommendation System. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7TB, E2E DIEN - https://github.com/alibaba/ai-matrix/tree/master/macro_benchmark/DIEN_TF2, FP32,BF16, BS1024, Spark 3.2.1 - https://spark.apache.org/, Intel TensorFlow avx512 2.10.0 - https://pypi.org/project/intel-tensorflow-avx512/, oneDNN 2.6.0, Dataset: Amazon Books 2018: meta(2,935,525, 12)​; reviews(51,311,621,10), test by Intel on 10/21/2022. A10: 1-node, 1x Nvidia A10 & 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG96 960GB, E2E DIEN, FP32,FP16, BS1024, Spark 3.2.1 - https://spark.apache.org/, TensorFlow 2.10.0 - https://pypi.org/project/tensorflow/, cuda 11.8, Dataset:Amazon Books 2018: meta(2,935,525, 12)​; reviews(51,311,621,10), test by Intel on 10/21/2022. E2E DIEN RecSys Test by Intel on October 21, 2022.
[A205] Up to 1.7x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared to AMD EPYC 7663 (fp32) on end-to-end video streamer.

Up to 1.38x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared to AMD EPYC 7663 (int8) on end-to-end video streamer.

Up to 4.2x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared to Nvidia A10 GPU (fp16) on end-to-end video streamer.

Up to 4.8x higher frames/second using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared to Nvidia A10 GPU (fp16) on end-to-end video streamer.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Video Streamer, BS1, FP32/BF16/INT8, ssd-mobilenet, Intel-tensorflow-avx512 2.10.0, oneDNN v2.6.0, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams: 56, test by Intel on 10/18/2022. 7763: 1-node, 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Video Streamer, BS1, FP32/INT8, ssd-mobilenet, Intel-tensorflow 2.10.0, oneDNN v2.6.0, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams: 15, test by Intel on 44852. A10: 1-node, 1x Nvidia A10 GPU & 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG96 960G, E2E Video Streamer, BS1, FP32/FP16, ssd-mobilenet, Intel-tensorflow-avx512 2.10.0, cudatoolkit 11.7.0, cudnn 8.4.1.50, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams 42 fp32/64 INT8, test by Intel on 10/14/2022. E2E Video Streamer (TF) Test by Intel on October 18, 2022.

Test by Intel on October 14, 2022.

[A206] Up to 12.6x higher throughput using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared with AMD EPYC 7763 (fp32) on end-to-end face recognition.

Up to 11x more streams while maintaining a 30fps SLA/stream with the 4th Gen Intel Xeon Scalable processor (amx bf16) vs. AMD EPYC 7763(fp32).

Up to 3.4x higher throughput using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared with AMD EPYC 7763 (int8) on end-to-end face recognition.

Up to 7x more streams while maintaining a 30fps SLA/stream with the 4th Gen Intel Xeon Scalable processor (amx int8) vs. AMD EPYC 7763(int8).

Up to 4.5x higher throughput using 4th Gen Intel Xeon Scalable processor with Intel AMX (bf16) compared to Nvidia A10 GPU (fp16) on end-to-end face recognition.

Up to 4.4x more streams while maintaining a 30fps SLA/stream with the 4th Gen Intel Xeon Scalable processor (amx bf16) vs. Nvidia A10 GPU (fp16).

Up to 6.5x higher throughput using 4th Gen Intel Xeon Scalable processor with Intel AMX (int8) compared to Nvidia A10 GPU (fp16) on end-to-end face recognition.

Up to 5.6x more streams while maintaining a 30fps SLA/stream with the 4th Gen Intel Xeon Scalable processor (amx int8) vs. Nvidia A10 GPU (FP16).
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Face Recognition with SSD-MobileNet + ResNet50_​v1.5 inference, FP32/BF16/INT8, BS1, maintain 30fps per stream, Intel-tensorflow-avx512 2.10.0, ssd-mobilenet, Resnet50_​v1.5, oneDNN v2.6.0, Dataset: Chelsea_​celebration.mp4 (7969 frames) https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1786, test by Intel on 10/21/2022. 7763: 1-node, 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Face Recognition with SSD-MobileNet + ResNet50_​v1.5 inference, FP32/INT8, BS1, maintain 30fps per stream, Intel-tensorflow-avx512 2.10.0, ssd-mobilenet, Resnet50_​v1.5, oneDNN v2.6.0, Dataset: Chelsea_​celebration.mp4 (7969 frames) https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1786, test by Intel on 44855. A10: 1-node, 1x Nvidia A10 & 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG96, E2E Face Recognition with SSD-MobileNet + ResNet50_​v1.5 inference, FP32/TF32/FP16, BS1, maintain 30fps per stream, Tensorflow-gpu 2.10.0, ssd-mobilenet, Resnet50_​v1.5, cudatoolkit 11.8.89, cudnn 8.6.0 (using NV container), Dataset: Chelsea_​celebration.mp4 (7969 frames) https://analyticsmarketplace.intel.com/find-data/metadata?id=DSI-1786, test by Intel on 10/21/2022. E2E Face Recognition Test by Intel on October 21, 2022.
[A209] Up to 1.32x faster end-to-end Census workload performance with 4th Gen Intel Xeon Scalable processor compared to AMD EPYC 7763. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, E2E Video Streamer, BS1, FP32/BF16/INT8, ssd-mobilenet, Intel-tensorflow-avx512 2.10.0, oneDNN v2.6.0, Dataset: Mall.avi (8344 frames), Throughput per stream SLA = 30 fps, streams: 56, test by Intel on 10/18/2022. 7763: 1-node, 2x AMD EPYC 7763 64-Core Processor on GIGABYTE with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0xa001144, HT off, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, Census, Python 3.9.13, Modin 0.16.2,Scikit-Learn Intelex (Intel Optimized) 2021.6.0, Daal4py 2021.6.0, XGBoost 1.6.2, Dataset: IPUMS USA:https://usa.ipums.org/usa/, datatypes: int64, float64, test by Intel on 10/18/2022. E2E Census Test by Intel on October 18, 2022.
[A210] 2-socket 4th Gen Intel Xeon Platinum 8480+ with integrated Intel AMX (bf16) delivers 9% faster performance than Nvidia A100 GPU on End-to-End Vision Based Transfer Learning workload. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b000041, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, INTEL SSDSC2KG01 1.7T, Vision Transfer Learning Pipeline, Resnet50 v1.5, 1,2-sockets used, OpenMPI 4.1.2, Horovod 0.25, FP32/BF16, Intel Tensorflow AVX512 2.10, Dataset: https://www.tensorflow.org/datasets/catalog/colorectal_histology. Size: 5000 Images, Shape: 224 * 224, test by Intel on 10/21/2022. A100: 1-node, 1x Nvidia A10 & 2x AMD EPYC 7742 64-Core Processor on NVIDIA DGXA100 920-23687-2530-000 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, ucode 0x830104d, HT on, Turbo on, Ubuntu 20.04.2 LTS, 5.4.0-81-generic, 1.8T SAMSUNG MZ1LB1T9HALS-00007, Vision Transfer Learning Pipeline, Resnet50 v1.5, FP32/FP16, Tensorflow 2.10, Dataset: https://www.tensorflow.org/datasets/catalog/colorectal_histology. Size: 5000 Images, Shape: 224 * 224, test by Intel on 10/26/2022. E2E Vision Based Transfer Learning Test by Intel on October 21, 2022.

Test by Intel on October 26, 2022

[A211] Up to 6x higher PyTorch DLRM training performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 17x higher batch DLRM inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), DLRM, Inference: bs=n [1socket/instance], bs: fp32=128, amx bf16=128, amx int8=128, Training bs:fp32/amx bf16=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 on Gigabyte with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT off, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; DLRM, Training bs:fp32=32k [1 instance, 1socket], Criteo Terabyte Dataset, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. DLRM Test by Intel on October 24, 2022.
[A212] Up to 6.1x higher real-time image classification [ResNet-50] inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 7.1x higher batch ResNet-50 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 4x higher real-time ResNet-50 INT8 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. AMD EPYC 7763 (INT8).

Up to 5.7x higher batch ResNet-50 INT8 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. AMD EPYC 7763 (INT8).
4th Generation Intel® Xeon® Platinum processor TF ResNet50: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), ResNet50, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,64, Intel AMX BF16=1,80, amx int8=1,116, Training bs:fp32=1024 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; ResNet50 v1.5, Inference: bs=1 [4cores/instance], bs=n [8 cores/instance], bs: fp32=1,128,int8=1,32 ImageNet (224 x224), Framework: https://developer.amd.com/zendnn/#download; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, ZenDNN 3.3, TensorFlow:2.9, test by Intel on 10/24/2022. ResNet-50 Test by Intel on October 24, 2022.
[A213] Up to 5.3x higher real-time natural language processing [BERT-Large] inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 6.2x higher batch BERT-Large inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).
4th Generation Intel® Xeon® Platinum processor PT-Bert-Large: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,56, amx bf16=1,16, amx int8=1,56, Trg: Wikipedia 2020/01/01 ( seq len =512), bs:fp32=28, amx bf16=56 [1 instance, 1socket], Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 [4cores/instance], bs=n [1socket/instance], Framework: https://github.com/intel/intel-extension-for-pytorch; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on January 2023. BERT Large Test by Intel on October 24, 2022
[A214] Up to 12.5x higher real-time object detection [SSD-ResNet34] inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 12.4x higher batch SSD-ResNet34 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 9.3x higher real-time SSD-ResNet34 INT8 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. AMD EPYC 7763 (INT8).

Up to 7.9x higher batch SSD-ResNet34 INT8 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (INT8) vs. AMD EPYC 7763 (INT8).
4th Generation Intel® Xeon® Platinum processor TF SSD-ResNet34: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), SSD-ResNet34, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,112, Intel AMX BF16=1,112, amx int8=1,112, Training bs:fp32/Intel AMX BF16=224 [1 instance, 1socket], Coco 2017, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; SSD-ResNet34, Inference: bs=1 [8cores/instance] Coco 2017, Framework: https://github.com/Intel-tensorflow/tensorflow; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, Intel TensorFlow 2.9, test by Intel on 10/24/2022. SSD-ResNet34 Test by Intel on October 24, 2022
[A215] Up to 2.1x higher real-time 3D-Unet inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32). 4th Generation Intel® Xeon® Platinum processor TF: 3D-Unet: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), 3D-Unet, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=1,6, Intel AMX BF16=1,6, amx int8=1,6, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT off, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; 3DUnet, Inference: bs=1 [256 cpres/instance], Framework: https://github.com/Intel-tensorflow/tensorflow; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, Intel TensorFlow 2.9, test by Intel on 10/24/2022. 3D Unet Test by Intel on October 24, 2022
[A216] Up to 9.9x higher real-time ResNext101 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).

Up to 10.5x higher batch ResNext101 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32).
4th Generation Intel® Xeon® Platinum processor PT-ResNext101: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), Resnext101 32x16d, bs=n [1socket/instance], Inference: bs: fp32=1,64, amx bf16=1,64, amx int8=1,116, Framework: https://github.com/intel-innersource/frameworks.ai.pytorch.private-cpu/tree/d7607bdd983093396a70713344828a989b766a66; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.13, IPEX: 1.13, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; Resnext101 32x16d, bs=1 [4 cores/instance], Inference: bs: fp32=1,2, Framework: https://github.com/intel/intel-extension-for-pytorch; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, PT:1.12, test by Intel on 10/24/2022. ResNext101 Test by Intel on October 24, 2022.
[A217] Up to 2.1x higher batch DIEN inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX (BF16) vs. AMD EPYC 7763 (FP32). 4th Generation Intel® Xeon® Platinum processor TF-DIEN: 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with 1024 GB (16 slots/ 64GB/ DDR5-4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, CentOS Stream 8, 5.15.0, 1x INTEL SSDSC2KW256G8 (PT)/Samsung SSD 860 EVO 1TB (TF), DIEN, Inference: bs=1 [4cores/instance], bs=n [1socket/instance], bs: fp32=16,64K, Intel AMX BF16=16,64k, Framework: https://github.com/intel-innersource/frameworks.ai.tensorflow.private-tensorflow/tree/spr_ww42_2022; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, TF: 2.11, OneDNN: v2.7, test by Intel on 10/24/2022. AMD Epyc 7763: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT off, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; DIEN, Inference: bs=1 [256 cores/instance], bs=2 [256 cores/instance], Framework: https://github.com/Intel-tensorflow/tensorflow; Modelzoo: https://github.com/IntelAI/models/tree/spr-launch-public, Intel TF 2.9, test by Intel on 10/24/2022. DIEN Test by Intel on October 24, 2022.
[A218] 1.8x higher average* PyTorch real-time BF16/FP16 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX vs. Nvidia A10.

1.7x higher average* PyTorch real-time INT8 inference performance on 4th Gen Intel Xeon Scalable processor with built in Intel AMX vs. Nvidia A10.
4th Generation Intel® Xeon® Platinum processor Bert-Large: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; Bert Large, Inf: SQuAD1.1 (seq len=384), bs=1 (full chip, 1 instance), bs=1024 [full chip, 1 instance], Framework: TensorRT 8.4.3; test by Intel on 10/24/2022. DLRM: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; DLRM, Inference: bs=262100 [full chip, 1 instance], Criteo Terabyte Dataset, Framework: NGC 22.08 https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_22-08.html#rel_22-08; Pytorch 1.12, test by Intel on 10/24/2022. DIEN: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; DIEN, Inference: bs=2048 [full chip, 1 instance], Framework: NGC 22.05 https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_22-05.html#rel_22-05; Pytorch 1.12, test by Intel on 10/24/2022. SSDResNet34: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; SSD-ResNet34, Inference: bs=1 [full chip, 1 instance] Coco 2017, Framework: Tensor RT 8.4.3; test by Intel on 10/24/2022. ResNet50: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; ResNet50 v1.5, Inference: bs=1 [full chip, 1 instance], bs=2048 [full chip, 1 instance], Framework: TensorRT 8.4.3, test by Intel on 10/24/2022. ResNext101 32x16d: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; Resnext101 32x16d, bs=1 [full chip, 1 instance], bs = 64 [full chip, 1 instance]; Framework: TensorRT 8.4.3, test by Intel on 10/24/2022. MaskRCNN: 1x NVIDIA A10: 1-node with 2x AMD Epyc 7763 with 1024 GB (16 slots/ 64GB/ DDR4-3200) total memory, HT on, Turbo on, Ubuntu 20.04,Linux 5.4 kernel, 1x 1.4TB NVMe SSD, 1x 1.5TB NVMe SSD; MaskRCNN 1088x800 input, bs=1 [full chip, 1 instance], bs = 64 [full chip, 1 instance]; Framework: TensorRT 8.4.3, test by Intel on 10/24/2022. IPEX vs. TensorRT Test by Intel on October 24, 2022.

[A219] 4th Gen Intel® Xeon® Scalable Processors with Intel® AMX deliver up to 5.9x higher performance for ResNet50 v1.5 TensorFlow Batch Image Classification compared to AMD EPYC 7763.

4th Generation Intel® Xeon® Platinum processor

1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), on pre-production Intel® platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0xf000380, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.8-1.el8.elrepo.x86_​​64, 1x 1.92T Intel® SSDSC2KG01, TF 2.9.1, AI Model=Resnet 50 v1_​​5, best scores achieved using BS1=1 core/instance , BS16=5 cores/instance, using physical cores, tested by Intel September 2022.

1-node, 2x AMD EPYC 7763 processor (64 core) on GIGABYTE R282-Z92 with 1024GB DDR4 memory (16x64 GB), microcode 0xa001144, SMT On, Boost On, NPS=1, Ubuntu 20.04.5 LTS, 5.4.0-125-generic, 1x 1.92TB INTEL_​​SSDSC2KG01, TF 2.9, ZenDNN=v3.3 (Ubuntu 20.04 required for ZenDNN v3.3) , AI Model=Resnet 50 v1_​​5, best scores achieved using BS1=2 cores/instance , BS16=8 cores/instance for INT8, BS16=4 cores/instance for fp32, using cores and threads, tested by Intel September 2022.

Measurement: ResNet50 v 1.5; TensorFlow batch image classification BS=16

ResNet-50 Test by Intel on January, 2023
[A220] Inference latency Optimization of German Bert model with Intel OpenVINO 2022.2.0 toolkit shows up to 4x gain for AMX_​BF16 model compared to PyTorch v1.12.0 FP32 non-optimized baseline and 2.57X gain for SPR OpenVINO AMX_​BF16 model compared to prior gen OpenVINO FP32 model on 8 physical cores. 4th Generation Intel® Xeon® Platinum processor OPTIMIZED (Config 2: 4th Gen Intel® Xeon® Scalable Processors Codenamed: Sapphire Rapids): Tested by Intel as of October 2022. pre-production platform with 2 socket Intel® Xeon® Platinum 8480+ Processor (Sapphire Rapids), 56 cores/socket, HT On, Turbo ON, Total Memory 512GB (16slots/32GB/4800 MT/s DDR4), BIOS: SE5C6200.86B.0022. D64.2105220049, ucode 0x2b000041, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, GCC 9.4.0 compiler, Inference Framework: Intel® Distribution of OpenVINO™ toolkit 2022.2.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace: German-SentimentBert model, 1 instance/2 socket, Multiple stream, Datatype: AMX_​BF16. German-Bert Sentiment Analyzer (for Analysis) Tested by Intel as of October 2022.
[A221] 4th Gen Intel® Xeon® Scalable processors can deliver fine-tuning time-to-train in less than 4 minutes. 4th Generation Intel® Xeon® Platinum processor DistilBert results SPR: Test by Intel as of 10/21/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ ,56 cores, HT On, Turbo On, Total Memory 1024 GB (16 slots/ 64 GB/ 4800 MHz [run @ 4800 MHz] ), EGSDREL1.SYS.8612.P03.2208120629, 0x2b000041, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, n/a, DLSA fine tuning pipeline with Distilbert-base-uncased (256MB: 66 Million Param), PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2 NVIDIA-A100: Test by Intel as of 10/26/2022. 1-node (DGX-A100), 2xAMD EPYC 7742 64-Core Processor, 64 cores, HT On, Turbo On,, Total 1024GB (16 slots/64GB/3200 MHz) [run @ 3200MHz] ), Nvidia A100 GPU, BIOS 1.1, 0x830104d ,Ubuntu 20.04.2 LTS, 5.4.0-81-generic, n/a, DLSA fine tuning pipeline with Distilbert-base-uncased (256MB: 66 Million Param), PyTorch 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2 BERT-Large results ICX: Test by Intel as of 10/21/2022. 1-node, 2x Intel® Xeon® Platinum 8380, 40 cores, HT Off, Turbo On, Total Memory 1024 GB (16 slots/ 64 GB/ 3200 MHz [run @ 3200 MHz] ), SE5C620.86B.01.01.0005.2202160810, 0xd000375, Ubuntu 22.04.1 LTS, 5.15.0-48-generic, n/a, DLSA fine tuning pipeline with Bert-large-uncased (1.3GB: 340 Million Param), PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2 SPR: Test by Intel as of 10/27/2022. upto 4-nodes, pre-production platform with 2x Intel® Xeon® Platinum 8480+ ,56 cores, HT On, Turbo On, Total Memory 1024 GB (16 slots/ 64 GB/ 4800 MHz [run @ 4800 MHz] ), Ethernet Controller E810-C for QSFP, SE5C7411.86B.8424.D04.2208040243 BIOS, 0x2a0000a0, Rocky Linux 8.6 (Green Obsidian), 4.18.0-372.26.1.el8_​6.x86_​64, n/a, DLSA fine tuning pipeline with Bert-large-uncased (1.3GB: 340 Million Param), PyTorch 1.12, IPEX 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2. NVIDIA-A100: Test by Intel as of 10/26/2022. 1-node (DGX-A100), 2xAMD EPYC 7742 64-Core Processor, 64 cores, HT On, Turbo On,, Total 1024GB (16 slots/64GB/3200 MHz) [run @ 3200MHz] ), Nvidia A100 GPU, BIOS 1.1, 0x830104d ,Ubuntu 20.04.2 LTS, 5.4.0-81-generic, n/a, DLSA fine tuning pipeline with Bert-large-uncased (1.3GB: 340 Million Param), PyTorch 1.12, Transformers 4.21.1, oneDNN 2.6.0, OneCCL 2021.5.2. HuggingFace DistilBert (BF16) fine-tuning Tested by Intel as of October 21, 2022.
[A222] 4th Gen Intel Xeon delivers an average inference performance gain of 4X using SSD-ResNet34 Object Detection, BERT-Large and Resnet50. 4th Generation Intel® Xeon® Platinum processor

SSD-ResNet34: 8468V: 1-node, 1x 4th Gen Intel® Xeon® Scalable processor (48 core) with Intel® Advanced Matrix Extensions (Intel AMX) on Intel Software Development Platform  with 512GB DDR5 memory (8x64 GB), microcode 0x2b000181, HT On, Turbo On, SNC Off, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 1.7T INTEL SSDSC2KG01, Framework=Intel TF 2.11.dev202242, Python 3.8, AI Model=SSD-ResNet34, best scores achieved using BS=4[1 cores/instance], Precision=INT8-AMX, step size=800, warmup=200, tested by Intel March 2023.

SSD-ResNet34: 9454:1-node, 1x AMD EPYC 9454 processor (48 core)  on Supermicro H13SSL-NT with  768GB DDR5 memory (12x64 GB), microcode  0xa10110e, SMT On, Boost On,  NPS=1, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 3.5T INTEL SSDSC2KG03, Framework=TF 2.10.1, ZenDNN=v4.0, Python 3.8, AI Model=Resnet 50 v1_​5,  best scores achieved using BS=2[1 cores/instance], Precision=INT8, step size=800, warmup=200, tested by Intel March 2023.

BERT-Large: 8468V: 1-node, 1x 4th Gen Intel® Xeon® Scalable processor (48 core) with Intel® Advanced Matrix Extensions (Intel AMX) on Intel Software Development Platform with 512GB DDR5 memory (8x64 GB), microcode 0x2b000181, HT On, Turbo On, SNC Off, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 1.7T INTEL SSDSC2KG01, Framework=Intel TF 2.11.dev202242, Python 3.8, AI Model=BERT-Large, best scores achieved using BS=128 [8 cores/instance], Precision=BF16-AMX, step size=30, warmup=10, tested by Intel March 2023.

BERT-Large: 9454: 1-node, 1x AMD EPYC 9454 processor (48 core)  on Supermicro H13SSL-NT with  768GB DDR5 memory (12x64 GB), microcode  0xa10110e, SMT On, Boost On,  NPS=1, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 1.7T INTEL SSDSC2KG01, Framework=TF 2.10.1, ZenDNN=v4.0, Python 3.8, AI Model=BERT-Large, best scores achieved using BS=64 [2 cores/instance], Precision=FP32, step size=30, warmup=10, tested by Intel March 2023

ResNet50v1.5: 8468V: 1-node, 1x 4th Gen Intel® Xeon® Scalable processor (48 core) with Intel® Advanced Matrix Extensions (Intel AMX),  on Intel Software Development Platform with 512GB DDR5 memory (8x64 GB), microcode 0x2b000181, HT On, Turbo On, SNC Off, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 1.7T INTEL SSDSC2KG01, Framework=Intel TF 2.11.dev202242, Python 3.8, AI Model=ResNet50v1.5, best scores achieved using BS=32[6 cores/instance], Precision=INT8-AMX, step size=50, warmup=10, tested by Intel March 2023.

ResNet50v1.5: 9454: 1-node, 1x AMD EPYC 9454 processor (48 core)  on Supermicro H13SSL-NT with  768GB DDR5 memory (12x64 GB), microcode  0xa10110e, SMT On, Boost On,  NPS=1, Ubuntu 22.04.2 LTS, 5.15.0-67-generic, 1x 1.7T INTEL SSDSC2KG01, Framework=TF 2.10.1, ZenDNN=v4.0, Python 3.8, AI Model=ResNet50v1.5,  best scores achieved using BS=2 [1 cores/instance], Precision=INT8, step size=50, warmup=10, tested by Intel March 2023

SSD-ResNet34 Object Detection

BERT-Large

ResNet50v1.5

Tested by Intel as of March 2023.
[D1] Up to 3x higher RocksDB performance with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. prior generation

Up to 66% latency reduction on Rocks DB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. prior generation.

Up to 2.1x higher RocksDB performance with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA compared with Zstd software.

Up to 51% latency reduction on RocksDB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA compared with Zstd software.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production Intel platform with 2x 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0xf000380, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel September 2022. 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel October 2022. RocksDB Tested by Intel in September 2022.

Tested by Intel October 2022.

[D2] Up to 1.59x higher ClickHouse DB performance with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. prior generation.

Up to 25% reduction in memory bandwidth per query on ClickHouse DB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. prior generation.

Up to 29% better data compress rate on ClickHouse DB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. prior generation.

Up to 1.2x higher ClickHouse DB performance with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. LZ4 software compression.

Up to 23% reduction in memory bandwidth per query on ClickHouse DB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. LZ4 software. compression.

Up to 29% better data compress rate on ClickHouse DB with 4th Gen Intel Xeon Platinum 8490H using integrated Intel IAA vs. LZ4 software compression.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production Intel platform with 2x 4th Gen Intel® Xeon® Scalable processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), Number of IAA device utilized=8(2 sockets active), HT On, Turbo On, SNC off, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0xf000380, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21, accel-config-v3.4.6.4, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel September 2022. 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel October 2022. ClickHouse DB Tested by Intel September 2022 Tested by Intel October 2022.
[D3] Up to 1.5x higher HammerDB MySQL OLTP transactions with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. prior generation.

Up to 2.48x higher HammerDB MySQL OLTP transactions with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. 5-year platform refresh.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8490H on ArcherCity with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, RHEL 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, x 1x 894.3G INTEL SSDSC2KG96, 5x 1.5T INTEL SSDPF21Q016TB, HammerDB 4.4, MySQL 8.0.30, test by Intel on 10/12/2022. 8380: 1-node, 2 (1 used)x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT on, Turbo on, RHEL 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, x 1x 894.3G INTEL SSDSC2KG96, 1x 1.5T INTEL SSDPF21Q016TB, 4x 2.9T INTEL SSDPF21Q032TB, HammerDB 4.4, MySQL 8.0.30, test by Intel on 10/12/2022. 8180: 1-node, 2 (1 used)x Intel® Xeon® Platinum 8180 on S2600WFD with GB (24 slots/ 32GB/ DDR4 2666) total memory, ucode 0xd000363, HT on, Turbo on, RHEL 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, x 1x 223.6G INTEL SSDSC2KB24, 5x 1.5T INTEL SSDPF21Q016TB, HammerDB 4.4, MySQL 8.0.30, test by Intel on 10/13/2022. HammerDB MySQL Test by Intel on October 12, 2022
[D7] Up to 3.2x Backup Time Improvement with 4th Gen Intel Xeon Scalable Platform with built-in Intel® QuickAssist Technology & SQL Server 2022. 4th Generation Intel® Xeon® Platinum processor 4th Gen Intel® Xeon® Scalable with built-in Intel QAT: Test by Intel as of 10/18/2022. 1-node, 2x Intel® Xeon® Platinum 8490H Processor on Archer City (ref. platform), 60 cores with 1024 GB (16 slots/ 64GB/ 4800[4800]) total DDR5 memory, ucode 0xAB0000C0, HT on, Turbo on, with 4th Gen Intel® QuickAssist Technology, 2.0.10.10 driver version, Windows Server 2022, 10.0.20348.587, 1x Intel® Ethernet Converged Network Adapter X550, SQL Server 2022 16.0.1000.6, Multiple Databases, Database Backup with Compression​. 3rd Gen Intel® Xeon® Scalable with software: Test by Intel as of 10/18/2022. 1-node, 2x Intel® Xeon® Platinum 8358 Processor on Wilson City (ref. platform), 32 cores with 1024 GB (16 slots/ 64GB/ 3200[3200]) total DDR4 memory, ucode 0xD00037B, HT on, Turbo on, Windows Server 2019,10.0.17763.1999, 1x Intel® Ethernet Converged Network Adapter X550, SQL Server 2022 16.0.1000.6, Multiple Databases, Database Backup with Compression​ ​ QAT on SQL Server Test by Intel on October 18, 2022.
[D8] Up to 1.34x higher Online Analytical Processing workload running Microsoft SQL Server 2022 on 4th Gen Intel® Xeon® Scalable Platform compared to prior generation. 4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on ArcherCity with 4096 GB (16 slots/ 256GB/ DDR5 4800) total memory, ucode 0xAB000310, HT on, Turbo on, Windows 2022 FE, Build 20348.1129, 1x 960GB Intel D3-S4510, 8x 7.68TB Intel SSD D7-P5510, Decision support workload, 10TB scale factor, SQL Server 2022 RTM (16.0.1000.6), test by Intel on 10/27/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Wilson City with 4096 GB (16 slots/ 256GB/ DDR4 3200) total memory, ucode 0x0D000375, HT on, Turbo on, Windows 2022 FE, Build 20348.1129, 1x 960GB Intel D3-S4510, 8x 7.68TB Intel SSD D7-P5510, Decision support workload, 10TB scale factor, SQL Server 2022 RTM (16.0.1000.6), test by Intel on 10/27/2022. Decision Support SQL Server Test by Intel on October 27, 2022
[D9] Up to 1.18x higher performance on HiBench Kmeans with 4th Gen Intel® Xeon® Scalable Platform using Intel oneAPI Data Analytics Library (oneDAL) compared to prior generation at same core count (32-cores).

Up to 1.53x higher performance on HiBench Kmeans with 4th Gen Intel® Xeon® Scalable Platform using Intel oneAPI Data Analytics Library (oneDAL) compared to OpenBLAS.
4th Generation Intel® Xeon® Platinum processor 8454H: 1+4-node, each with: pre-production platform with 2x Intel® Xeon® Platinum 8454H, 32-cores on QuantaGrid D54Q-2U with 512 GB (16 slots/ 32GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, RHEL 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, 4x INTEL P5510 SSDPF2KX038TZ 3.5TB, 1x Intel® Ethernet Network Adapter E810-CQDA2, HiBench v7.1.1, Intel oneAPI Base Toolkit 2022.2, OpenBLAS 0.3.15, Cloudera 7.1.7, Spark 3.1.0, OAP-mllib 1.4/Netlib-java 1.1, bigdata dataset, test by Intel on 10/15/2022. 8358: 1+4-node, each with: 2x Intel® Xeon® Platinum 8358, 32-cores on WHITLEY SDP with 512 GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT on, Turbo on, RHEL 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, 4x INTEL P5510 SSDPF2KX038TZ 3.5TB, 1x Intel® Ethernet Network Adapter E810-CQDA2, HiBench v7.1.1, Intel oneAPI Base Toolkit 2022.2, OpenBLAS 0.3.15, Cloudera 7.1.7, Spark 3.1.0, OAP-mllib 1.4/Netlib-java 1.1, bigdata dataset, test by Intel on 10/17/2022. HiBench Kmeans Test by Intel on October 15, 2022

Test by Intel on October 17, 2022 .

[D11] Up to 1.56x higher queries per hour with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. prior generation. 4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on ArcherCity with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, x 1x 894.3G INTEL SSDSC2KG96, 2x 3.5T INTEL SSDPF2KX038TZ, IBM Big Data Insights (BDI) v0.8 (heavy queries, 16 users, 3TB scale), Db2 v11.5.7, test by Intel on 10/10/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on M50CYP2SBSTD with GB (32 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, x 1x 894.3G INTEL SSDSC2KG96, 3x 3.5T INTEL SSDPF2KX038TZ, IBM Big Data Insights (BDI) v0.8 (heavy queries, 16 users, 3TB scale), Db2 v11.5.7, test by Intel on 10/10/2022. Big Data on IBM DB2 Test by Intel on October 10, 2022
[D12] up to 1.73x higher mix [80R/20W] throughput with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. prior generation on Cassandra NoSQL database. 4th Generation Intel® Xeon® Platinum processor 8490H: Test by Intel as of 10/18/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H, 60 cores, HT On, Turbo On, Total Memory 512GB (16x32GB 4800 MT/s [4800 MT/s]), BIOS EGSDCRB1.FEI.0088.D16.2209090804, microcode 0xababc0a0, 1x Ethernet Controller I225-LM, 4x Ethernet Controller X710/X557-AT 10GBASE-T, 1x 1.5T INTEL SSDSC2BB01, 8x 1.5T INTEL SSDPF21Q016TB, Ubuntu 22.04.1 LTS, 5.19.10-051910-generic, compiler gcc version 11.2.0, cassandra-stress version 4.0.6, jdk version 14 build 14+36-1461 8380: Test by Intel as of 10/17/22, 1-node, 2x Intel® Xeon® Platinum 8380 CPU @ 2.30GHz, 40 cores, HT On, Turbo On, Total Memory Installed 512GB (16x32GB DDR4 3200 MT/s [3200 MT/s]), Total Memory Used 256GB as 256GB blocked, BIOS SE5C620.86B.01.01.0005.2202160810, microcode 0xd000375, 2x Ethernet Controller X710 for 10GBASE-T, 1x 223.6G KINGSTON SA400M8240G, 4x 1.5T INTEL SSDPF21Q016TB, Ubuntu 22.04.1 LTS, 5.19.10-051910-generic, compiler gcc version 11.2.0, cassandra-stress version 4.0.6, jdk version 14 build 14+36-1461 Cassandra NoSQL Test by Intel on October 18, 2022.

Test by Intel on October 17, 2022

[D15] Using Intel® QuickAssist Technology accelerated hardware, SQL Server backup services can lead up to a 3.26x faster compression time. 4th Generation Intel® Xeon® Platinum processor Baseline (w/o accelerator): Test by Intel as of 10/18/2022. 1-node, 1x pre-production platform with 2x Intel® Xeon® Platinum 8490H processor, 60 cores, HT on, Turbo on, 1024 GB (16 slots/ 64GB/ 4800[4800]) total DDR5 memory, ucode 0xAB0000C0, Windows Server 2022, 10.0.20348.587, 1x Intel® Ethernet Converged Network Adapter X550, SQL Server 2022 16.0.1000.6, Multiple Databases, Database Backup with XPRESS software compression​. New: (with Intel® QAT accelerator): Test by Intel as of 10/18/2022. 1-node, 1x pre-production platform with 2x Intel® Xeon® Platinum 8490H processor, 60 cores, Intel® QAT device utilized=8(1 socket active), HT on, Turbo on, 1024 GB (16 slots/ 64GB/ 4800[4800]) total DDR5 memory, ucode 0xAB0000C0, 6x P4500 NVME drives, 2.0.10.10 driver version, Windows Server 2022, 10.0.20348.587, 1x Intel® Ethernet Converged Network Adapter X550, SQL Server 2022 16.0.1000.6, Multiple Databases, Database Backup with Intel QAT Compression​. SQL Server backup with compression Test by Intel as of October 18, 2022.

[D16] For Microsoft SQL Server, 4th Gen Intel Xeon Scalable processors can deliver up to 53% faster backup with Intel® QuickAssist Technology (Intel® QAT) enabled.

4th Generation Intel® Xeon® Platinum processor

New; Test by Intel as of 12/12/2022. 1-node,Pre-production platform with 2x Intel® Xeon® Platinum 8460Y+ processor on Quanta SDP QuantaGrid D54Q-2U, 32 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32GB/ 4800 MHz [run @ 4800MHz]) DDR4 memory, Intel® QAT device utilized=2 (2 socket active), ucode 0x2B000081, Windows 2022 Standard Edition 21H2, 10.0.20348, SQL Server 2022, 16.0.1000.6 (X64), database backup with QAT using Xpress software compression.

Baseline; Test by Intel as of 12/12/2022. 1-node,Pre-production platform with 2x Intel® Xeon® Platinum 8460Y+ processor on Quanta SDP QuantaGrid D54Q-2U, 32 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32GB/ 4800 MHz [run @ 4800MHz]) DDR4 memory, Intel® QAT device utilized=0 (2 socket active), ucode 0x2B000081, Windows 2022 Standard Edition 21H2, 10.0.20348, SQL Server 2022, 16.0.1000.6 (X64), database backup without QAT using Xpress software compression.

SQL Server backup with compression Test by Intel as of December 12, 2022.
[D17] For Microsoft SQL Server, 4th Gen Intel Xeon Scalable processors can deliver a relative performance gain of up to 22% more NOPM transactions and up to 19% faster query response time, compared to the previous generation. 4th Generation Intel® Xeon® Platinum processor

1 Node, 2x Intel® Xeon® Platinum 8460Y+ (32C, 2.3GHz, 300W) CPU, 1x Quanta SDP QuantaGrid D54Q-2U, Total Memory: 512GB (16 x 32 GB 4800MHz DDR5 DIMM), HT On, Turbo On, Storage (boot): 1 x Solidigm DC S4610, 960 GB, Storage (Data drive): 6x Solidigm® D7 P5510 Series (3.84TB) (NVMe), Storage (Log drive): 2 x Intel® SSD DC P5800X 400GB (Optane SSD), Network devices: 1 x 25 GbE Intel(R) Ethernet Network Adapter E810-C-Q2, Network speed: 25 GbE, 1 x 10 GbE Intel(R) Ethernet Converged Network Adapter X550-T2, Network Speed: 1 GbE,HammerDB v4.0, OS/Software: Windows 2022 standard Edition with SQL Server 2019 Enterprise Edition with CU18.

1 Node, 2x Intel® Xeon® Gold 6348 (28C, 2.6GHz, 235W) CPU, 1x Coyote Pass Intel Platform ,uCode: 0x8D05A260 Total Memory: 512GB (16 x 32 GB 3200Hz DDR4 DIMM), Intel® Hyper-Threading Technology: Enabled, Turbo: Enabled, Storage (boot):1 x Intel® SSD D3-S4510 Series (240GB, 2.5in SATA 6Gb/s, 3D2, TLC), Storage (Data drive): Data Drive: 6x Intel® SSD DC P5510 Series (3.8TB) (NVMe)​, Storage (Log drive): 2 x Intel® SSD DC P5800X 400GB (Optane SSD), Network devices: 1 x 10GbE Intel(R) Ethernet Converged Network Adapter X550,HammerDB v4.0, OS/Software: Windows 2019 Data Center with SQL Server 2019 with CU9.

HammerDB OLTP SQL server Test by Intel as of December 12, 2022.
[D201] Up to 1.9x higher throughput and 47% reduction in latency with 4th Gen Intel Xeon processor with built in Intel In-Memory Analytics accelerator compared with AMD EPYC 7763. 4th Generation Intel® Xeon® Platinum processor New: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0xf000380, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 64 core Processor on GIGABYTE R282-Z92 platform, SMT On, Boost On, NPS=1, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xa001144, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel September 2022. RocksDB Tested by Intel September 2022.
[D202] Up to 59% higher throughput on ClickHouse DB with 4th Gen Intel Xeon processor with built in Intel In-Memory Analytics accelerator compared with AMD EPYC 7763.

Up to 24% reduction in memory bandwidth/query on ClickHouse DB with 4th Gen Intel Xeon processor with built in Intel In-Memory Analytics accelerator compared with AMD EPYC 7763.

Up to 19% reduction in memory usage on ClickHouse DB with 4th Gen Intel Xeon processor with built in Intel In-Memory Analytics accelerator compared with AMD EPYC 7763.

Up to 29% better compression rate on ClickHouse DB with 4th Gen Intel Xeon processor with built in Intel In-Memory Analytics accelerator compared with AMD EPYC 7763.
4th Generation Intel® Xeon® Platinum processor New: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, SNC off, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0xf000380, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21, accel-config-v3.4.6.4, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 64 core Processor on GIGABYTE R282-Z92 platform, SMT On, Boost On, NPS=1, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xa001144, 1x 1.92TB INTEL SSDSC2KG01, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel September 2022. ClickHouse DB Tested by Intel in September 2022.
[G1] 1.53x average performance gain with 4th Gen Intel Xeon Platinum 8490H processor over prior generation. 4th Generation Intel® Xeon® Platinum processor Geomean of HP Linpack, STREAM Triad, SPECrate2017_​fp_​base (est), SPECrate2017_​int_​base (est). See [G2, G4, G6] at intel.com/processorclaims: 4th Gen Intel® Xeon® Scalable. Geomean SPECcpu rate, STREAM Triad, LINPACK Test by Intel in September 2022.
[G2] Up to 1.50x higher memory bandwidth on 4th Gen Intel Xeon Platinum 8490H processor vs. prior generation. 4th Generation Intel® Xeon® Platinum processor 8490H:1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX076TZ, STREAM Triad version 5.10, OneAPI 2022.1, test by Intel on 9/9/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-39-generic, 1x INTEL SSDPF21Q016TB, STREAM Triad version 5.10, OneAPI 2022.1, test by Intel on 8/13/2022. STREAM Triad Test by Intel on September 9, 2022.

Test by Intel on August 13, 2022.

[G3] Up to 50% Memory Bandwidth Improvement on 4th Gen Intel Xeon Scalable Processor vs. 3rd Gen Intel Xeon Scalable Processor. 4th Generation Intel® Xeon® Platinum processor Results have been estimated or simulated. Comparing 4th Gen Intel® Xeon® Scalable processor with 8 memory-channels running at up to 4800MT/s vs. 3rd Gen Intel® Xeon® Scalable processor [Ice Lake] with 8 memory-channels running at up to 3200MT/s. Theoretical Memory Bandwidth Test by Intel in August 2022.
[G4] Up to 1.6x higher integer throughput performance on 4th Gen Intel Xeon Platinum 8490H processor vs. prior generation.

Up to 1.65x higher floating point throughput performance on 4th Gen Intel Xeon Platinum 8490H processor vs. prior generation.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX076TZ, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 9/10/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-39-generic, 1x INTEL SSDPF21Q016TB, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 8/14/2022. Integer & floating point throughput - SPECcpu2017 rate [est] Test by Intel on September 10, 2022.

Test by Intel on August 14, 2022.

[G5] Up to 1.19x higher integer speed performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 1.28x higher floating point speed performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX076TZ, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 9/8/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-39-generic, 1x INTEL SSDPF21Q016TB, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 8/14/2022. Integer & floating point speed - SPECcpu2017 speed [est] Test by Intel on September 8, 2022.

Test by Intel on August 14, 2022.

[G6] Up to 1.38x higher LINPACK performance on 4th Gen Intel Xeon Platinum 8490H processor vs. prior generation. 4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-52-generic, 1x INTEL SSDPF2KX076TZ, HPLinpack 2.3, OneAPI 2022.1, test by Intel on 10/31/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-46-generic, 1x INTEL SSDPF21Q016TB, HPLinpack 2.3, OneAPI 2022.1, test by Intel on 8/25/2022. Intel® Distribution for LINPACK Test by Intel on October 31,2022.

Test by Intel on August 25,2022.

[G7] 15% IPC Improvement with 4th Gen Intel Xeon Scalable Processor vs. 3rd Gen Intel Xeon Scalable Processor. 4th Generation Intel® Xeon® Platinum processor Results have been estimated or simulated. Performance comparison between Sapphire Rapids and Ice Lake at same core count, same frequency, same compiler. IPC Test by Intel in August 2022.
[G8] 2x PCIe Bandwidth Improvement. 4th Generation Intel® Xeon® Platinum processor Results have been estimated or simulated. Comparing x16 bandwidth on 4th Gen Intel® Xeon® Scalable processor with PCIe Gen5 running at up to 128 GB/s vs. 3rd Gen Intel® Xeon® Scalable processor with PCIe Gen4 running at up to 64 GB/s Theoretical PCIe Bandwidth improvement Test by Intel in August 2022.
[G9] 2.4x average performance gain with 4th Gen Intel Xeon Platinum 8490H processor over Intel Xeon Platinum 8180 [5-year refresh]. 4th Generation Intel® Xeon® Platinum processor Geomean of HP Linpack, Stream Triad, SPECrate2017_​fp_​base est, SPECrate2017_​int_​base est. Up to 1.96x higher Linpack performance 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-52-generic, 1x INTEL SSDPF2KX076TZ, HPLinpack 2.3, OneAPI 2022.1, test by Intel on 10/31/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on S2600WFT with GB (12 slots/ 32GB/ DDR4 2933[2666]) total memory, ucode 0x2006d05, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-46-generic, 1x INTEL SSDSC2BB48, HPLinpack 2.3, OneAPI 2022.1, test by Intel on 4/21/2022. Up to 2.4x higher memory bandwidth 8490H:1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX076TZ, STREAM Triad version 5.10, OneAPI 2022.1, test by Intel on 9/9/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on S2600WFT with GB (12 slots/ 32GB/ DDR4 2933[2666]) total memory, ucode 0x2006d05, HT off, Turbo on, Ubuntu 22.04 LTS, 5.15.0-46-generic, 1x INTEL SSDSC2BB48, STREAM Triad version 5.10, OneAPI 2022.1, test by Intel on 4/21/2022. Up to 2.5x higher integer throughput performance & Up to 2.7x higher floating point throughput performance 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX076TZ, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 9/10/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on S2600WFT with GB (12 slots/ 32GB/ DDR4 2933[2666]) total memory, ucode 0x2006d05, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-46-generic, 1x INTEL SSDSC2BB48, SPEC CPU2017 v1.1.8, OneAPI 2022.1, test by Intel on 4/21/2022. Geomean SPECcpu rate, Stream Triad, Linpack Test by Intel on April 21, 2022.

Test by Intel on October 31, 2022.

Test by Intel on September 10,2022.

Test by Intel on September 9,2022.

[H1] Up to 1.56x higher average HPC performance on 4th Gen Intel Xeon Scalable platform vs. prior gen Up to 1.7x higher average Earth System Model HPC performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.

Up to 1.45x higher average FSI performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.

Up to 1.53x higher average Life & Material Science HPC performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.

Up to 1.52x higher average Manufacturing HPC performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.
4th Generation Intel® Xeon® Platinum processor See claims H3-H26: 1.56x: Geomean of HPL, HPCG, Stream Triad, YASK, WRF, NEMO, MPAS-A, ROMS, Monte Carlo, Black Scholes, Binomial Options, Quantum Espresso, DeePMD, GROMACS, VASP, LAMMPS, Altair AcuSolve, RADIOSS, ParSeNet, ANSYS Mechanical, LS-DYNA, Star-CCM+, ANSYS Fluent, Converge, ESI Group OpenFOAM, ANSYS Mechanical, Cosmoflow. 1.7x: Geomean of WRF, NEMO, MPAS-A, ROMS. 1.45x: Geomean of Monte Carlo, Black Scholes, Binomial Options. 1.53x: Geomean of Quantum Espresso, DeePMD, GROMACS, VASP, LAMMPS. 1.52x: Geomean of Altair AcuSolve, RADIOSS, ParSeNet, ANSYS Mechanical, LS-DYNA, Star-CCM+, ANSYS Fluent, Converge, ESI Group OpenFOAM, ANSYS Mechanical. Average HPC Test by Intel from September to October 2022.
[H3] Up to 1.5x higher Stream Triad performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Stream v5.10 . 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Stream v5.10 Stream Triad Test by Intel as of October 7, 2022.
[H4] Up to 1.37x higher HPL performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, DDR5 Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Ubuntu 22.04.1 LTS, Linux version 5.15.0-50-generic, HPL from MKL_​v2022.1.0 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, HPL from MKL_​v2022.1.0 HPL Test by Intel as of October 7, 2022.
[H5] Up to 1.58x higher HPCG performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, DDR5 Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Ubuntu 22.04.1 LTS, Linux version 5.15.0-50-generic, HPCG from MKL_​v2022.1.0 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, HPCG from MKL_​v2022.1.0 HPCG Test by Intel as of October 7, 2022.
[H6] Up to 1.64x higher OpenFOAM performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor OpenFOAM (Geomean of Motorbike 20M, Motorbike 42M) 8480+: Test by Intel as of 9/2/2022. 1-node, 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode revision=0xaa0000a0, CentOS Stream 8, Linux version 4.18.0-365.el8.x86_​64, OpenFOAM 8, Motorbike 20M @ 250 iterations, Motorbike 42M @ 250 iterations 8380: Test by Intel as of 9/2/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode revision=0xd000270, Rocky Linux 8.6, Linux version 4.18.0-372.19.1.el8_​6.crt1.x86_​64, OpenFOAM 8, Motorbike 20M @ 250 iterations, Motorbike 42M @ 250 iterations This offering is not approved or endorsed by OpenCFD Limited, producer and distributor of the OpenFOAM software via www.openfoam.com, and owner of the OPENFOAM® and OpenCFD® trademark ESI Group OpenFOAM Test by Intel as of September 2, 2022
[H7] Up to 1.54x higher ANSYS Fluent performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Ansys Fluent (Geomean: pump_​2m, sedan_​4m, rotor_​3m, aircraft_​wing_​14m, combustor_​12m, exhaust_​system_​33m) 8480+: Test by Intel as of 09/02/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT ON, Turbo ON, SNC4, Total Memory 512 GB, BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode 0xaa0000a0, CentOS Stream 8, kernel version 4.18.0-365.el8.x86_​64, Ansys Fluent 2022R1 8380: Test by Intel as of 08/24/2022. 1-node, 2x Intel® Xeon® Platinum 8380, HT ON, Turbo ON, Quad, Total Memory 256 GB, BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode 0xd000270, Rocky Linux 8.6, kernel version 4.18.0-372.19.1.el8_​6.crt1.x86_​64, Ansys Fluent 2022R1 ANSYS Fluent Test by Intel as of 9/2/2022

Test by Intel as of 08/24/2022

[H8] Up to 1.45x higher ANSYS Mechanical Iterative Solvers performance on 4th Gen Intel Xeon Scalable platform vs. prior gen Up to 1.66x higher ANSYS Mechanical Direct Solvers performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Ansys Mechanical (Geomean: (V22iter-1, V22iter-2, V22iter-3, V22iter-4), (V22direct-1, V22direct-2)) 8480+: Test by Intel as of 09/02/2022. 1-node, 2x Intel® Xeon® Platinum 8480+, HT ON, Turbo ON, SNC4, Total Memory 512 GB DDR5 4800 MT/s, BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode 0xaa0000a0, CentOS Stream 8, kernel version 4.18.0-365.el8.x86_​64, Ansys Mechanical 2022 R2 8380: Test by Intel as of 08/24/2022. 1-node, 2x Intel® Xeon® Platinum 8380, HT ON, Turbo ON, Quad, Total Memory 256 GB, BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode 0xd000270, Rocky Linux 8.6, kernel version 4.18.0-372.19.1.el8_​6.crt1.x86_​64, Ansys Mechanical 2022 R2 ANSYS Mechanical Test by Intel as of September 2, 2022.

Test by Intel as of August 24, 2022

[H9] Up to 1.62x higher YASK performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor YASK (Geomean: AWP, ISO3DFD, SSG) 8480+: Test by Intel as of 10/7/2022. 1-node, Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode revision=0xaa0000a0, CentOS Stream 8, Linux version 4.18.0-365.el8.x86_​64, YASK v3.05.07 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, YASK v3.05.07 YASK Test by Intel as of Octoberr 7, 2022
[H10] Up to 1.37x higher Altair AcuSolve performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Altair AcuSolve (HQ Model) 8480+: Test by Intel as of 09/28/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT ON, Turbo ON, SNC4, Total Memory 512 GB, BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode 0xaa0000a0, CentOS Stream 8, kernel version 4.18.0-365.el8.x86_​64, Altair AcuSove 2021R2 8380: Test by Intel as of 09/28/2022. 1-node, 2x Intel® Xeon® Platinum 8380, HT ON, Turbo ON, Quad, Total Memory 256 GB, BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode 0xd000270, Rocky Linux 8.6, kernel version 4.18.0-372.19.1.el8_​6.crt1.x86_​64, Altair AcuSolve 2021R2 Altair AcuSolve Test by Intel as of September 28, 2022
[H11] Up to 1.62x higher WRF performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor WRF (CONUS 2.5KM) 8480+: Test by Intel as of 10/12/2022. 1-node, Intel® Xeon® Platinum 8480+, HT On, Turbo On, NUMA configuration SNC4, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, WRF v4.4 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-ip -O3 -xCORE-AVX512 -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low -w -ftz -align array64byte -fno-alias -fimf-use-svml=true -inline-max-size=12000 -inline-max-total-size=30000 -vec-threshold0 -qno-opt-dynamic-align. 8380: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, WRF v4.4 and v3.9.1.1 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-ip -O3 -xCORE-AVX512 -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low -w -ftz -align array64byte -fno-alias -fimf-use-svml=true -inline-max-size=12000 -inline-max-total-size=30000 -vec-threshold0 -qno-opt-dynamic-align. WRF Test by Intel as of October 12,2022
[H12] Up to 1.77x higher ROMS performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor ROMS (benchmark3 (2048x256x30), benchmark3 (8192x256x30)) 8480+: Test by Intel as of 10/12/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, NUMA configuration SNC4, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, ROMS V4 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-ip -O3 -heap-arrays -xCORE-AVX512 -qopt-zmm-usage=high -align array64byte -fimf-use-svml=true -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low", ROMS V4 8380: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, ROMS V4 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-ip -O3 -heap-arrays -xCORE-AVX512 -qopt-zmm-usage=high -align array64byte -fimf-use-svml=true -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low", ROMS V4 ROMS Test by Intel as of October 12,2022
[H13] Up to 1.68x higher NEMO performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor NEMO (GYRE_​PISCES_​25, BENCH ORCA-1) 8480+: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, NUMA configuration SNC4, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, NEMO v4.2 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-i4 -r8 -O3 -fno-alias -march=core-avx2 -fp-model fast=2 -no-prec-div -no-prec-sqrt -align array64byte -fimf-use-svml=true". 8380: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, NEMO v4.2 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-i4 -r8 -O3 -fno-alias -march=core-avx2 -fp-model fast=2 -no-prec-div -no-prec-sqrt -align array64byte -fimf-use-svml=true" NEMO Test by Intel as of October 12,2022
[H14] Up to 1.73x higher MPAS-A performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor MPAS-A (MPAS-A V7.3 60-km dynamical core) 8480+: Test by Intel as of 10/12/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, NUMA configuration SNC4, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, MPAS-A V7.3 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-O3 -march=core-avx2 -convert big_​endian -free -align array64byte -fimf-use-svml=true -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low", MPAS-A V7.3 8380: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, MPAS-A V7.3 build with Intel® Fortran Compiler Classic and Intel® MPI from 2022.3 Intel® oneAPI HPC Toolkit with compiler flags "-O3 -march=core-avx2 -convert big_​endian -free -align array64byte -fimf-use-svml=true -fp-model fast=2 -no-prec-div -no-prec-sqrt -fimf-precision=low", MPAS-A V7.3 MPAS-A Test by Intel as of October 12,2022
[H15] Up to 1.5x higher DeePMD training performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor DeePMD (Multi-Instance Training) 8480+: Test by Intel as of 10/12/2022. 1-node, 2x Intel® Xeon® Platinum 8480+, Total Memory 512 GB, kernel 4.18.0-365.eI8_​3x86_​64, compiler gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), https://github.com/deepmodeling/deepmd-kit, Tensorflow 2.9, Horovod 0.24.0, oneCCL-2021.5.2, Python 3.9 8380: Test by Intel as of 10/20/2022. 1-node, 2x Intel® Xeon® Platinum 8380 processor, Total Memory 256 GB, kernel 4.18.0-372.26.1.eI8_​6.crt1.x86_​64, compiler gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), https://github.com/deepmodeling/deepmd-kit, Tensorflow 2.9, Horovod 0.24.0, oneCCL-2021.5.2, Python 3.9 DeePMD Test by Intel as of October 12,2022.

Test by Intel as of October 20,2022

[H16] Up to 1.58x higher Binomial Options performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.

Up to 1.42x higher Black Scholes performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.

Up to 1.36x higher Monte Carlo performance on 4th Gen Intel Xeon Scalable platform vs. prior gen.
4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Binomial Options v1.1, Black Scholes v1.4, Monte Carlo v1.2 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Binomial Options v1.1, Black Scholes v1.4, Monte Carlo v1.2 Binomial Options, Black Scholes, Monte Carlo Test by Intel as of October 7, 2022.
[H17] Up to 1.68x higher LAMMPS performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 9/29/2022. 1-node, 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, LAMMPS v2021-09-29 cmkl:2022.1.0, icc:2021.6.0, impi:2021.6.0, tbb:2021.6.0; threads/core:; Turbo:off; BuildKnobs:-O3 -ip -xCORE-AVX512 -g -debug inline-debug-info -qopt-zmm-usage=high; 8380: Test by Intel as of 10/11/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, LAMMPS v2021-09-29 cmkl:2022.1.0, icc:2021.6.0, impi:2021.6.0, tbb:2021.6.0; threads/core:; Turbo:on; BuildKnobs:-O3 -ip -xCORE-AVX512 -g -debug inline-debug-info -qopt-zmm-usage=high; LAMMPS (Atomic Fluid, Copper, DPD, Liquid_​crystal, Polyethylene, Protein, Stillinger-Weber, Tersoff, Water) LAMMPS Test by Intel as of September 29, 2022.

Test by Intel as of October 11 ,2022

[H18] Up to 1.52x higher LS-DYNA performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Ansys LS-DYNA (ODB-10M) 8480+: Test by Intel as of ww41'22. 1-node pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, LS-DYNA R11 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s DDR4), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, LS-DYNA R11 LS-DYNA Test by Intel as of October, 2022.

Test by Intel as of October 7, 2022.

[H19] Up to 1.53x higher Star-CCM+ performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Siemens Simcenter Star-CCM+ (civil, HlMach10AoA10Sou, kcs_​with_​physics, lemans_​poly_​17m.amg, reactor, TurboCharger7M) 8480+: Test by Intel as of 14-Sep-22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 1024 GB (16x64GB 4800MT/s, Dual-Rank), BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode revision=0xaa000090, CentOS Stream 8, Linux version 4.18.0-394.el8.x86_​64, StarCCM+ 17.04.007, reactor 9m @ 20 iterations, lemans_​poly_​17m @ 20 iterations, civil 20m @ 20 iterations, TurboCharger7M @ 20 iterations, HlMach10AoA10Sou 6.4m @ 20 iterations, kcs_​with_​physics 3m @ 20 iterations 8380: Test by Intel as of 25-Oct-22. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode revision=0xd000270, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, StarCCM+ 17.04.007, reactor 9m @ 20 iterations, lemans_​poly_​17m @ 20 iterations, civil 20m @ 20 iterations, TurboCharger7M @ 20 iterations, HlMach10AoA10Sou 6.4m @ 20 iterations, kcs_​with_​physics 3m @ 20 iterations Star-CCM+ Test by Intel as of September 14, 2022.

Test by Intel as of October 25, 2022.

[H20] Up to 1.35x higher Quantum Espresso performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Quantum Espresso (AUSURF112, Water_​EXX) 8480+: Test by Intel as of 9/2/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), ucode revision= 0x90000c0, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Quantum Espresso 7.0, AUSURF112, Water_​EXX 8380: Test by Intel as of 9/30/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Quantum Espresso 7.0, AUSURF112, Water_​EXX Quantum Espresso Test by Intel as of September 9, 2022.

Test by Intel as of September 30, 2022

[H21] Up to 2.57x higher Cosmoflow performance on 4th Gen Intel Xeon Scalable platform with built-in Intel AMX vs. prior gen. 4th Generation Intel® Xeon® Platinum processor CosmoFlow (training on 8192 image batches) 8480+ (AMX BF16): Test by Intel as of 10/18/2022. 1node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, >40 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MHz, DDR5), BIOS EGSDCRB1.86B.0083.D22.2206290535, ucode 0xaa0000a0, CentOS Stream 8, kernel 4.18.0-365.el8.x86_​64, https://github.com/mlcommons/hpc/tree/main/cosmoflow, AMX, BF16, Tensorflow 2.9.1, horovod 0.24.3, keras 2.9.0.dev2022021708, oneCCL 2021.5, Python 3.8 8380: Test by Intel as of 06/07/2022. 1-node, 2x Intel® Xeon® Scalable Processor 8380, 40 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 3200 MHz, DDR4), BIOS SE5C6200.86B.0022.D64.2105220049, ucode 0xd0002b1, OS Red Hat Enterprise Linux 8.5 (Ootpa), kernel 4.18.0-348.7.1.el8_​5.x86_​64, https://github.com/mlcommons/hpc/tree/main/cosmoflow, AVX-512, FP32, Tensorflow 2.9.0, horovod 0.23.0, keras 2.6.0, oneCCL-2021.4, oneAPI MPI 2021.4.0, Python 3.8 Cosmoflow Test by Intel as of October 18,2022.

Test by Intel as of June 7, 2022.

[H22] Up to 1.44x higher ParSeNet performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor ParSeNet (SplineNet) 8480+: Test by Intel as of 10/18/2022. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, HT On, Turbo On, Total Memory 512 GB (16x32GB 4800MT/s, Dual-Rank), BIOS Version EGSDCRB1.86B.0083.D22.2206290535, ucode revision=0xaa0000a0, CentOS Stream 8, Linux version 4.18.0-365.el8.x86_​64, ParSeNet (SplineNet), PyTorch 1.11.0, Torch-CCL 1.2.0, IPEX 1.10.0, MKL (20220804), oneDNN (v2.6.0) 8380: Test by Intel as of 10/18/2022. 1-node, 2x Intel® Xeon® Platinum 8380 CPU, HT On, Turbo On, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C6200.86B.0020.P23.2103261309, ucode revision=0xd000270, Rocky Linux 8.6, Linux version 4.18.0-372.19.1.el8_​6.crt1.x86_​64, ParSeNet (SplineNet), PyTorch 1.11.0, Torch-CCL 1.2.0, IPEX 1.10.0, MKL (20220804), oneDNN (v2.6.0) ParSeNet Test by Intel as of October 18, 2022.
[H23] Up to 1.62x higher Converge performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Converge (SI8_​engine_​PFI_​SAGE_​transient_​RAN) 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Converge CFD 3.0.17 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Converge CFD 3.0.17 Converge Test by Intel as of October 7, 2022.
[H24] Up to 1.42x higher RADIOSS performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor Altair RADIOSS (Geomean: Neon1M @ 80 ms, t10M @ 8 ms) 8480+: Test by Intel as of 10/7/2022. 1-node, 2x 4th Gen Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Altair RADIOSS 2022.2, Intel MPI 2021.7, Neon1M/80ms, t10M/8ms Intel® Xeon® CPU Max Series: Test by Intel as of 9/2/2022. 1-node, 2x Intel® Xeon® CPU Max Series, HT ON, Turbo ON, NUMA configuration SNC4, Total Memory 128 GB (HBM2e at 3200 MHz), BIOS Version SE5C7411.86B.8424.D03.2208100444, ucode revision=0x2c000020, CentOS Stream 8, Linux version 5.19.0-rc6.0712.intel_​next.1.x86_​64+server, Altair RADIOSS 2022.2, Intel MPI 2021.7, Neon1M/80ms, t10M/8ms 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Altair RADIOSS 2022.2, Intel MPI 2021.7, Neon1M/80ms, t10M/8msP RADIOSS Test by Intel as of October 7, 2022.

Test by Intel as of September 2, 2022.

[H25] Up to 1.61x higher VASP performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor VASP(Geomean: CuC, Si, PdO4, PdO4_​k221) 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Platinum 8480+, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, VASP6.3.2 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, VASP6.3.2 VASP Test by Intel as of October 7, 2022.
[H26] Up to 1.53x higher GROMACS performance on 4th Gen Intel Xeon Scalable platform vs. prior gen. 4th Generation Intel® Xeon® Platinum processor GROMACS (geomean: benchMEM​, benchPEP​, benchPEP-h, benchRIB​, hecbiosim-3m​, hecbiosim-465k​, hecbiosim-61k​, ion_​channel_​pme_​large​, lignocellulose_​rf_​large​, rnase_​cubic​, stmv​, water1.5M_​pme_​large​, water1.5M_​rf_​large) 8480+: Test by Intel as of 10/7/2022. 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Scalable Processor, HT On, Turbo On, SNC4, Total Memory 512 GB (16x32GB 4800MT/s, DDR5), BIOS Version SE5C7411.86B.8713.D03.2209091345, ucode revision=0x2b000070, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, GROMACS v2021.4_​SP 8380: Test by Intel as of 10/7/2022. 1-node, 2x Intel® Xeon® 8380 CPU, HT On, Turbo On, NUMA configuration SNC2, Total Memory 256 GB (16x16GB 3200MT/s, Dual-Rank), BIOS Version SE5C620.86B.01.01.0006.2207150335, ucode revision=0xd000375, Rocky Linux 8.6, Linux version 4.18.0-372.26.1.el8_​6.crt1.x86_​64, Converge GROMACS v2021.4_​SP GROMACS Test by Intel as of October 7, 2022.
[H27] Up to 2.7x more genomes/node/day vs. 4 years ago. 4th Generation Intel® Xeon® Platinum processor 3rd Generation Intel® Xeon® Scalable Processor Configuration: Test by Intel as of August 8, 2021. One front-end node and four compute nodes, all using Intel® Server Board M50CYP2SB-003. 1x front-end configuration: 2x Intel® Xeon® Gold 6348 processor (28 cores, 2.60 GHz); Intel® Hyper-Threading Technology = ON, Intel® Turbo Boost Technology = ON, total memory 256 GB (16 slots/16 GB/3200 MHz); BIOS version: 22D08; BMC 2.66, SDR 0.31, CPLD 3p0; Microcode: 0x0b000280; CentOS Linux installation ISO (minimal or full) 8 build 2011; storage - boot drive 1x Intel® SSD P4610 1.6 TB (3D NAND PCIe 3.1 x4, 3D1, TLC); high-performance network: 1x Intel® Ethernet Converged Network Adapter X550-T2 (10 GbE), model X550T2. 4x compute nodes configuration: 2x Intel® Xeon® Gold 6348 processor (28 cores, 2.60 GHz); Intel Hyper-Threading Technology = ON, Intel Turbo Boost Technology = ON, total memory 512 GB (16 slots/32 GB/3200 MHz); BIOS version: 22D08; BMC 2.66, SDR 0.31, CPLD 3p0; Microcode: 0x0b000280; CentOS Linux installation ISO (minimal or full) 8 build 2011; storage - scratch drive: 1x Intel SSD P4610 1.6 TB (3D NAND PCIe 3.1 x4, 3D1, TLC); high-performance network: 1x Intel Ethernet Converged Network Adapter X550-T2 (10 GbE), model X550T2. ​ 2nd Generation Intel® Xeon® Scalable Processor Configuration: Test by Intel as of November 14, 2019. One front-end node and four compute nodes, all using Intel® Server Board S2600WFT. Front-end node configuration: 2x Intel® Xeon® Gold 6252 processor (24 cores, 2.10 GHz); total memory 64 GB (4 slots/16 GB/2933 MHz); 1x 960 GB Intel® SSD D3-S4510 Series (2.5 in SATA 6 Gb/s, 3D2,TLC); 1x 1.6 TB Intel® SSD DC P4610 Series (2.5 in PCIe 3.1 x4, 3D2, TLC); Microcode: 0x500002c, BIOS: SE5C620.86B.02.01.0009.092820190230; CentOS Linux Installation ISO (minimal or full) 7.7 build 1910; Intel® oneAPI Runtimes 2019.4; Intel® Cluster Checker 2019.3.5; Intel® Select HPC Solution for RPM packages for EL7 2018.0; OpenHPC 1.3.8. 4x compute nodes configuration: 2x Intel® Xeon® Gold 6252 processor (24 cores, 2.10 GHz); total memory 384 GB (12 slots/32 GB/2933 MHz); 1x 960 GB Intel SSD D3-S4510 Series (2.5 in SATA 6 Gb/s, 3D2, TLC); 1x 1.6 TB Intel SSD DC P4610 Series (2.5 in PCIe 3.1 x4, 3D2, TLC); Network devices: 1x Intel® C620 Series Chipset Ethernet Connection; Intel® Ethernet Adapter X722 onboard 10 GbE; Microcode: 0x500002c, BIOS: SE5C620.86B.02.01.0009.092820190230; CentOS Linux Installation ISO (minimal or full) 7.7 build 1910; 1x distributed 10 GB Lustre 2.10 ZFS system, 6 OST, 3 OSS, Lnet Router with single 10 GB link for all I/O traffic clients to Lustre servers. ​ 1st Generation Intel® Xeon® Scalable Processor Configuration: Test by Intel as of October 15, 2018. Single-node (compute and front-end node combined) configuration: 2x Intel® Xeon® Gold 6152 processor (22 cores, 2.10 GHz); Intel® Server Board S2600WFT; total memory 192 GB (12 slots/16 GB/2666 MHz); boot storage: 2x 480 GB Intel® SSD DC S3520 Series; cache storage: 4x 4 TB Intel® SSD DC P4600 Series PCIe HHHL; capacity storage: 16 TB of 4x 4 TB Intel® SSD DC P4510 Series; Intel Hyper-Threading Technology = ON, Intel Turbo Boost Technology = ON; Microcode: 0x043; CentOS Linux installation 7.6. Genomics Test by Intel on August 8, 2021.

Test by Intel on August 14, 2019.

Test by Intel on October 15, 2018.

[H28] Up to 35% higher genomes/day, single node 4th Gen Intel Xeon SP vs. 3rd Gen.

Up to 35% lower cost per genome.
4th Generation Intel® Xeon® Platinum processor 3rd Generation Intel® Xeon® Scalable Processors: Test by Intel as of November 7,2022. Single-node (compute and front-end node combined) 2x Intel® Xeon® Platinum 8358 CPU @ 2.60GHz, 128 cores, HT On, Turbo On, Total Memory 512GB (16 slots/ 32GB/ 3200 MT/s), BIOS version WLYDCRB1.SYS.0021.P25.2107280557, Microcode 0xd000363, Rocky Linux 8.6, Kernel 4.18.0-372.32.1.el8_​6.x86_​64, Workload: WholeGenomeGermlineSingleSample 3.1.6, gatk 4.2.6.1, java-1.8.0-openjdk & java-11-openjdk, Cromwell v. 84, Picard 2.27.4, Samtools 1.11, bwa 0.7.17, VerifyBamID 2.0.1, MariaDB 10.3.35​. 4th Generation Intel® Xeon® Scalable Processors: Test by Intel as of November 7,2022. Single-node (compute and front-end node combined) 2x Intel® Xeon® Platinum 8460Y+, 160 cores, HT On, Turbo On, Total Memory 512GB (16 slots/ 32GB/ 4800 MT/s), BIOS version 3A05, Microcode 0x2b000070, Rocky Linux 8.6, Kernel 4.18.0-372.32.1.el8_​6.x86_​64, Workload: WholeGenomeGermlineSingleSample 3.1.6, gatk 4.2.6.1, java-1.8.0-openjdk & java-11-openjdk, Cromwell v. 84, Picard 2.27.4, Samtools 1.11, bwa 0.7.17, VerifyBamID 2.0.1, MariaDB 10.3.35. Genomics Tests by Intel on November 7, 2022.
[N1] 1.68x average performance gain on a range of broadly-deployed communication infrastructure workloads vs. prior gen. 4th Generation Intel® Xeon® Platinum processor See N2-N18, N50, N55. Geomean of L3 forward, NGINX RSA2K handshake, NGINX RSA4K handshake, NGINX SHA256, Snort w/ HyperScan, CDN Live, CDN VoD, FiB, vBNG, vCMTS, IPSEC. Geomean Tests by Intel from June through October, 2022.
[N2] Up to 2x better vBNG performance with the 4th Gen Intel Xeon Platinum 8470N processor compared to the previous gen processor. 4th Generation Intel® Xeon® Platinum processor 2x better vBNG performance with 8470N 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4800]) total memory, ucode 0xab000080, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSD, 1x Intel® Ethernet Network Adapter E810-CQDA2, vBNG-22.03, VPP 21.1.0, GCC 11.2.0, E810 firmware 3.00, packet size 504B D/128B U, test by Intel on 10/10/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 512 GB (16 slots/ 32GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSD, 1x Intel® Ethernet Network Adapter E810-CQDA2, vBNG-22.03, VPP 21.1.0, GCC 11.2.0, E810 firmware 3.00, packet size 504B D/128B U, test by Intel on 6/27/2022. Virtual Broadband Network Gateway (vBNG) Test by Intel as of October 10, 2022.

Test by Intel on June 27, 2022.

[N3] Up to 1.73x better Network Intrusion Prevention with the new 4th Gen Intel Xeon Platinum 8470N processor with Hyperscan enabled on Snort compared to prior generation. 4th Generation Intel® Xeon® Platinum processor 1.73x better Network Intrusion Prevention 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4800]) total memory, ucode 0xab000080, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, Snort 3.1.36, Hyperscan 5.5, GCC 11.2.0, EntMix Pcap, test by Intel on 10/10/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 512 GB (16 slots/ 32GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2BF240A4, Snort 3.1.36, Hyperscan 5.5, GCC 11.2.0, EntMix Pcap, test by Intel on 6/28/2022. SNORT with Hyperscan Test by Intel as of October 10, 2022.

Test by Intel on June 28, 2022.

[N4] Up to 2x better vCMTS performance with the new 4th Gen Intel Xeon Platinum 8470N processor compared to prior generation. 4th Generation Intel® Xeon® Platinum processor 2x better vCMTS performance with 8470N 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4800]) total memory, ucode 0xab000080, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, 1x Intel® Ethernet Network Adapter E810-CQDA2, vCMTS 22.10 beta, DPDK 22.03, GCC 11.2.0, DPDK 22.03, Collectd 5.12.0, Grafana 8.5.3, Prometheus 2.0.0, test by Intel on 9/20/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 512 GB (16 slots/ 32GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, 1x Intel® Ethernet Network Adapter E810-CQDA2, vCMTS 22.10 beta, DPDK 22.03, GCC 11.2.0, DPDK 22.03, Collectd 5.12.0, Grafana 8.5.3, Prometheus 2.0.0, test by Intel on 10/10/2022. Virtual Cable Modem Termination System (vCMTS) Test by Intel as of October 10, 2022.

Test by Intel on September 20, 2022.

[N5] Up to 1.89x higher throughput on VPP IPv4 FIB (64B) performance with the new 4th Gen Intel Xeon Platinum 8470N processor compared to the previous generation.

Up to 1.52x higher throughput on VPP IPv4 FIB (512B) performance with the new 4th Gen Intel Xeon Platinum 8470N processor compared to the previous generation.
4th Generation Intel® Xeon® Platinum processor 1.89x [64B] & 1.52x [512B] higher VPP IPv4 FIB throughput with 8470N 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 9/30/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on Wilson City with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-2CQDA2 (total 800Gbps), VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 10/5/2022. VPP IPv4 forward information base (FIB) Test by Intel on 9/30/2022

Test by Intel on 10/5/2022. .

[N6] Up to 2.04x higher VPP IPSec (1420B) throughput with the new 4th Gen Intel Xeon Platinum 8470N processor compared to the previous generation.

Up to 1.81x higher VPP IPSec (512B) throughput with the new 4th Gen Intel Xeon Platinum 8470N processor compared to the previous generation.
4th Generation Intel® Xeon® Platinum processor 1.81x & 2.04x higher IPSec throughput performance with 8470N 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, x 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 512B / 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 9/30/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on Wilson City with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-2CQDA2 (total 800Gbps), VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 512B / 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 10/5/2022. VPP IPSec Security Gateway Test by Intel on September 30, 2022.

Test by Intel on October 5, 2022.

[N7] Up to 25% better DPDK L3 forwarding throughput with the new 4th Gen Intel Xeon Gold 6428N processor compared to the previous generation. 4th Generation Intel® Xeon® Gold processor 1.25x higher L3fwd throughput performance with 6428N 6428N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Gold 6428N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4000]) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), DPDK 21.11​, L3fwd IPv4 LPM ​, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 10/17/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on Wilson City with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-2CQDA2 (total 800Gbps), DPDK 21.11​, L3fwd IPv4 LPM ​, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 10/5/2022. DPDK L3 forwarding Test by Intel on October 5, 2022.

Test by Intel on October 17, 2022.

[N8] 5G Core UPF up to 1.88X higher throughput with user plane functions in 5G wireless core network compared to prior gen. 30% improvement in 5G UPF performance compared to prior gen. 4th Generation Intel® Xeon® Platinum processor 1.88x high throughput with UPF compared to prior gen Baseline Configuration: 1-node, 2x Intel® Xeon® Gold 6338N CPU @ 2.20GHz, 32 cores, HT On, Turbo Off, Total memory 512GB (16x32GB DDR4 3200 MT/s [2666 MT/s]), BI/OS 1.4, microcode 0xd000375, 4x Intel® Ethernet Network Adapter E810-CQDA2, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 7.5.0, DPDK 20.11, FlexCore 5G UPF (April 2021), VPP 20.09, Test by Intel as of 10/18/22. New Configuration: 1-node, 2x pre-production Intel® Xeon® Platinum 8470N CPU, 32 cores, HT On, Turbo On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4800 MT/s]), BIOS EGSDCRB1.SYS.0090.D03.2210040200, microcode 0x2b0000c0, 3x Intel® Ethernet Network Adapter E810-2CQDA2 (total 600Gbps), 1x 223.6G INTEL SSDSC2KB240G8, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 7.5.0, DPDK 20.11, FlexCore 5G UPF (April 2021), VPP 20.09, Test by Intel as of 10/14/22. 30% improvement in 5G UPF compared to prior gen Baseline Configuration:: 1-node, 2x Intel® Xeon® Gold 6338N CPU @ 2.20GHz, 32 cores, HT On, Turbo Off, Total memory 512GB (16x32GB DDR4 3200 MT/s [2666 MT/s]), BI/OS 1.4, microcode 0xd000375, 4x Intel® Ethernet Network Adapter E810-CQDA2, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 7.5.0, DPDK 20.11, FlexCore 5G UPF (April 2021), VPP 20.09, Test by Intel as of 10/18/22. New Configuration: 4th Gen Gold 6428N: Test by Intel as of 10/17/22. 1-node, pre-production platform 2(1 used)x Intel® Xeon® Gold 6428N CPU, 32 cores, HT On, Turbo On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4000 MT/s]), microcode 0x2b0000c0, 2x Intel® Ethernet Network Adapter E810-2CQDA2 (total 400Gbps), 1x 223.6G INTEL SSDSC2KB240G8, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 5G User plane functions Test by Intel on October 18, 2022.

Test by Intel on October 14, 2022.

[N9] 4th Gen Intel Xeon Scalable Processor with Intel Inline vRAN Boost adds to the platform's outstanding performance-per-watt gain by also eliminating the need for an external accelerator card, resulting in approximately 20% additional compute power savings for operators and reduced bill-of-materials costs for OEMs. 4th Generation Intel® Xeon® Platinum processor Estimated as of 08/30/2022 based on scenario design power (SDP) analysis on pre-production 4th Gen Intel® Xeon® Scalable processor with Intel® vRAN Boost (integrated vRAN acceleration) and pre-production 4th Gen Intel® Xeon® Scalable processor at same core count and frequency with external vRAN accelerator card. Results may vary. Flex RAN Test by Intel as of August 30, 2022.
[N10] 4th Gen Intel Xeon Scalable Processor delivers up to twice the capacity at the same power envelope for vRAN workloads vs. 3rd Gen Intel Xeon Scalable processors, enabling Communications Service Providers up to double the performance-per-watt to meet their critical performance, scaling and energy efficiency requirements. 4th Generation Intel® Xeon® Platinum processor Estimated as of 8/30/2022 based on 4th generation Intel® Xeon® Scalable processor architecture improvements vs. 3rd generation Intel® Xeon® Scalable processor at similar core count, socket power and frequency on a test scenario using FlexRAN™ software. Results may vary. Flex RAN Test by Intel as of August 30, 2022.
[N12] Up to 1.52x higher TLS1.3 https throughput with 4th Gen Intel Xeon Platinum 8470N processor compared to 3rd Gen Intel Xeon Gold 6338N processor on TLS 1.3 AES128-GCM-SHA256. 4th Generation Intel® Xeon® Platinum processor 1.52x higher TLS 1.3 Webserver: AES128-GCM-SHA256 8470N: 1-node, pre-production platform with 2(1 active)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4800]) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, 6x Intel® Ethernet Network Adapter E810-CQDA2, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), test by Intel on 10/18/2022. 6338N: 1-node, 2(1 active)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-CQDA2, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), test by Intel on 10/18/2022. NGINX TLS 1.3 Webserver: AES128-GCM-SHA256 Test by Intel on October 18, 2022.
[N13] Up to 85% fewer cores to achieve ~7K TLS 1.3 ECDHE-X448-RSA4K connections/second with 4th Gen Intel Xeon Scalable Processors [XCC] with integrated QAT vs. previous generation.

Up to 1.42x higher connections/second with 4th Gen Intel Xeon Platinum 8470N processor compared to 3rd Gen Intel Xeon Gold 6338N processor on NGINX TLS 1.3 ECDHE-X448-RSA4K handshake.

Up to 83% fewer cores to achieve ~11K TLS 1.3 ECDHE-X448-RSA4K connections/second with 4th Gen Intel Xeon Scalable Processors [MCC] with integrated QAT vs. previous generation.
4th Generation Intel® Xeon® Platinum processor and Intel® Xeon® Gold processor 1.42x higher connections/second on TLS 1.3 ECDHE-X448-RSA4K webserver handshake [8470N vs. 6338N] and 85% fewer cores to achieve ~7K connections/second with integrated Intel QAT on 8470N 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab000060, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2BB240G4, x 1x Ethernet Controller I225-LM, 6x Ethernet Controller E810-C for QSFP, 1x Ethernet interface, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), Opt SW cps at 51C/102T, test by Intel on 9/15/2022. 6428N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Gold 6428N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4000]) total memory, ucode 0xab000060, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2BB240G4, x 1x Ethernet Controller I225-LM, 6x Ethernet Controller E810-C for QSFP, 1x Ethernet interface, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), Opt SW cps at 31C/62T, test by Intel on 9/19/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, x 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller 10G X550T, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), Opt SW cps at 31C/62T, test by Intel on 9/19/2022. NGINX Webserver Handshake with Integrated QAT2.0 TLS 1.3 ECDHE-X448-RSA4K Test by Intel as of September 15, 2022.

Test by Intel as of September 19, 2022

[N14] Up to 1.31x higher connections/second with 4th Gen Intel Xeon Platinum 8470N processor compared to 3rd Gen Intel Xeon Gold 6338N processor on NGINX TLS 1.3 ECDHE-X25519-RSA2K handshake with optimized software. Intel® Xeon® Platinum processor 1.31x higher connections/second on TLS 1.3 ECDHE-X25519-RSA2K webserver handshake [8470N vs. 6338N] 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab000060, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2BB240G4, x 1x Ethernet Controller I225-LM, 6x Ethernet Controller E810-C for QSFP, 1x Ethernet interface, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), Opt SW cps at 51C/102T, test by Intel on 9/15/2022. 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on SuperMicro X12DPG-QT6 with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-27-generic, 1x INTEL SSDSC2KB240G8, x 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller 10G X550T, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), Opt SW cps at 31C/62T, test by Intel on 9/19/2022. NGINX TLS 1.3 ECDHE-X25519-RSA2K handshake Test by Intel as of September 15, 2022.

Test by Intel as of September 19, 2022

[N15] Up to 47% fewer cores to achieve same connections/second with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. prior generation on NGINX Key handshake.

Up to 38% fewer cores to achieve same connections/second with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. QAT software on NGINX Key handshake.

Up to 84% fewer cores to achieve same connections/second with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. out-of-box software on NGINX Key handshake.
4th Generation Intel® Xeon® Platinum processor QAT Configuration HW/SW on 8490H: 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel QuickAssist Accelerator (Intel QAT), Number of QAT device utilized=4(1 socket active), 1024GB (16x64 GB) total DDR5 memory, microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (1x100GbE), QAT engine v0.6.14, QAT v20.l.0.9.1, NGINX 1.20.1, OpenSSL 1.1.1l, IPP crypto v2021_​5, IPSec v1.1, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel September 2022. QAT Configuration SW on 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (1x100GbE), QAT engine v0.6.14, NGINX 1.20.1, OpenSSL 1.1.1l, IPP crypto v2021_​5, IPSec v1.1, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel October 2022. OOB Configurations: 8490H: 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Scalable Processor (60 cores), with 1024GB (16x64 GB) total DDR5 memory, microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (1x100GbE), NGINX 1.20.1, OpenSSL 1.1.1l, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel September 2022. 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (1x100GbE), NGINX 1.20.1, OpenSSL 1.1.1l, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel October 2022. NGINX TLS 1.3 ECDHE-X25519-RSA2K Tested by Intel September 2022.

Tested by Intel October 2022.

[N16] Up to 95% fewer cores and 2x higher level 1 compression throughput with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. prior generation.

Up to 96% fewer cores and 1.37x higher level 1 compression throughput with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. ISA-L software.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x 4th Gen Intel® Xeon Scalable Processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), QAT device utilized=8(2 sockets active), with Total 1024GB (16x64 GB) DDR5 memory, microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, QAT v20.l.0.9.1, QATzip v1.0.9, ISA-L v2.3.0, tested by Intel September 2022. 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01,QAT v1.7.l.4.16, QATzip v1.0.9, ISA-L v2.3.0, tested by Intel October 2022. QATzip Level 1 Compression with QAT Tested by Intel September 2022.

Tested by Intel October 2022.

[N17] Up to 33% fewer cores with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. prior generation for IPSec encryption performance at 200Gbps SLA target.

Up to 33% fewer cores with 4th Gen Intel Xeon Platinum 8490H using integrated Intel QAT vs. software based QAT IPSec encryption performance at 200Gbps SLA target.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), QAT device utilized= 4(1 socket active), 1024GB DDR5 memory (16x64 GB), microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (2x100GbE), 2x100GbE, QAT v20.l.0.9.1, DPDK v21.11, IPsec v1.1, VPP 22.02, nasm v2.14.02, AES 128 GCM, VAES instructions, tested by Intel September 2022. 8380: 1-node, 2x 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (2x100GbE), 2x100GbE, DPDK v21.11, IPsec v1.1, VPP 22.02, nasm v2.14.02, AES 128 GCM, VAES instructions, tested by Intel October 2022. VPP IPSec AES_​128_​GCM with QAT Tested by Intel September 2022.

Tested by Intel October 2022.

[N18] Generational Gain Up to 1.6x higher IOPs and up to 37% latency reduction for large packet sequential read with 4th Gen Intel Xeon Platinum 8490H using integrated Intel DSA vs. prior generation.

Up to 1.1x higher IOPs and up to 10% latency reduction for small packet random read with 4th Gen Intel Xeon Platinum 8490H using integrated Intel DSA vs. prior generation. DSA hardware vs. software.

Up to 1.7x higher IOPs for large packet sequential read with 4th Gen Intel Xeon Platinum 8490H using integrated Intel DSA vs. ISA-L software.

Up to 1.21x higher IOPs and up to 18% latency reduction for small packet random read with 4th Gen Intel Xeon Platinum 8490H using integrated Intel DSA vs. ISA-L software.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0xf000380, HT On, Turbo On, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, Intel® Ethernet Network Adapter E810-2CQDA2 (2x100GbE), FIO v3.30, SPDK 22.05, tested by Intel September 2022. 8380: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on Coyote Pass platform, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo On, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, 4x 1.92TB Samsung PM1733, Intel® Ethernet Network Adapter E810-2CQDA2 (2x100GbE), FIO v3.30, SPDK 22.05, tested by Intel October 2022. SPDK NVMe-o-TCP with DSA Tested by Intel October 2022.
[N19] 4th Gen Intel Xeon Scalable processor with Intel Optane SSD reaches 120 Million 512B IOPS and delivers 1.5x more IOPS than previous generation while maintaining average 25 microsecond latency (Intel Optane SSD on 3rd Gen Intel Xeon Scalable processors) using 12-cores.

4th Gen Intel Xeon Scalable processor with Intel Optane SSD reaches 37 Million 4KB IOPS and delivers 1.5x more IOPS than previous generation while maintaining average 82 microsecond latency (Intel Optane SSD on 3rd Gen Intel Xeon Scalable processors) using 4-cores.

Performance scales linearly up to 8 cores until SSD throughput saturates on 4th Gen Intel Xeon Scalable platform with Intel Optane SSD for 512B random read.
4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/22/22. 1-node, 2x Intel® Xeon® 8480+, 56 cores, HT On, Turbo On, Total Memory 1 TB (16 slots/ 64 GB/ 4800 MHz [run @ 4800 MHz] DDR5), 24x Intel® Optane™ SSD P5800X, 80 PCIe Gen 5 lanes. BIOS version: 3A05, ucode version: 0x2b000070, Ubuntu 22.04 LTS, 5.15.0-41-generic, Workload SPDK v22.05 NVMe perf, GCC (Ubuntu 11.2.0-19ubuntu1), run method warm, 3 iterations average, raw results IOPS and usecs, operating frequency 3.3, CPU utilization 100%. 8380: Test by Intel as of 08/12/22. 1-node, 2x Intel® Xeon® 8380, 40 cores, HT On, Turbo On, Total Memory 1 TB (16 slots/ 64 GB/ 3200 MHz [run @ 3200 MHz] DDR4), 16x Intel® Optane SSD P5800X, 64 PCIe Gen 4 lanes. IOPS throughput measured at QD=128 for Optane SSD. BIOS version: SE5C620.86B.01.01.0005.2202160810, ucode version: 0xd000363, Ubuntu 22.04 LTS, 5.15.0-41-generic, Workload SPDK v22.05 NVMe perf, GCC (Ubuntu 11.2.0-19ubuntu1), run method warm, 3 iterations average, raw results IOPS and usecs, operating frequency 3.3, CPU utilization 100%. Local Storage Performance Test by Intel as of October 10, 2022.

Test by Intel as of 08/12/22.

[N20] 4th Gen Intel Xeon Scalable processor provides more storage virtualization VM density compared to previous generation (at same SLA of 10K IOPS/VM), up to 20% more VMs with NAND SSD and up to 40% more VMs with Optane SSD.

Up to 33% more IO intensive VMs with the 4th Gen Intel Xeon Scalable platform vs. prior generation [4KB random read].

Up to 30 million IOPs [4KB random read] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with P5800X.

Up to 67% more IOPs [4KB random read] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with P5800X vs. prior generation with NVMe.

Up to 43% more IOPs [4KB random read] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with NVMe vs. prior generation with NVMe.

Up to 33% more IO intensive VMs with the 4th Gen Intel Xeon Scalable platform vs. prior generation [4KB random read70/write30].

Up to 30 million IOPs [4KB random read70/write30] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with P5800X.

Up to 2.2x higher IOPs [4KB random read70/write30] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with P5800X vs. prior generation with NVMe.

Up to 52% more IOPs [4KB random read70/write30] in virtualized storage solution with the 4th Gen Intel Xeon Scalable platform with NVMe vs. prior generation with NVMe.
4th Generation Intel® Xeon® Platinum processor 8480+: Test by Intel as of 10/22/22. 1-node, 2x Intel® Xeon® 8480+, 56 cores, HT On, Turbo On, Total Memory 1 TB (16 slots/ 64 GB/ 4800 MHz [run @ 4800 MHz] DDR5), 24x Kioxia® SSD KCM61VUL3T20 3.2TB, 24x Intel® Optane™ SSD P5800X, 80 PCIe Gen 5 lanes. BIOS version: 3A05, ucode version: 0x2b00070, Ubuntu 22.04 LTS, 5.15.0-41-generic, SPDK v22.05, GCC (Ubuntu 11.2.0-19ubuntu1), FIO 3.30, QEMU v7.0.0, VM: 1 vCPU(Explicit core usage enforced using "taskset -a -c" command), 4GB RAM (Pre-allocated from Hugepages, NUMA aligned to vCPU), Fedora 35 with Kernel 5.11 Workload: 4KB Rand Read & 70/30 Rand Read/Write using FIO run method warm, 3 iterations average, raw results IOPS and usecs, operating frequency 3.3. 8380: Test by Intel as of 08/12/22. 1-node, 2x Intel® Xeon® 8380, 40 cores, HT On, Turbo On, Total Memory 1 TB (16 slots/ 64 GB/ 3200 MHz [run @ 3200 MHz] DDR4), 16X Kioxia® SSD KCM61VKioxia® SSD, 64 PCIe Gen 4 lanes. Latency measured at QD=1. BIOS version: SE5C620.86B.01.01.0005.2202160810, ucode version: 0xd000363, Ubuntu 22.04 LTS, 5.15.0-41-generic, SPDK v22.05, GCC (Ubuntu 11.2.0-19ubuntu1), FIO 3.30, QEMU v7.0.0, VM: 1 vCPU(Explicit core usage enforced using "taskset -a -c" command), 4GB RAM (Pre-allocated from Hugepages, NUMA aligned to vCPU), Fedora 35 with Kernel 5.11 Workload: 4KB Rand Read & 70/30 Rand Read/Write @ QD=1 using FIO run method warm, 3 iterations average, raw results IOPS and usecs, operating frequency 3.3. Storage Virtualization Test by Intel as of October 22, 2022.

Test by Intel as of August 12, 2022

[N21] 3.01x higher AI inference performance on Intel® Xeon® Gold 6448Y processor with built-in AI acceleration engine, Intel® AMX (INT8), compared to prior generation Intel® Xeon® Gold 6348 processor for Image Classification workload [ResNet-50 v1.5 inference]. 4th Generation Intel® Xeon® Gold processor New: 1-node, 2x pre-production Intel® Xeon® Gold 6448Y on ArcherCity with 512 GB (16 slots/ 32GB/ 4800) total DDR5 memory, ucode 0x2b000111, HT ON, Turbo ON, Ubuntu 20.04.5 LTS, 5.16.0-051600-generic, 1x Samsung SSD 870 1TB, ResNet-50 v1.5 inference (Image Classification) AI Workloads with Batch_​size1, int8 precision, test by Intel on Dec 5 2022. Baseline: 1-node, 2x Intel® Xeon® Gold 6348 CPU on WilsonCity with 512 GB (16 slots/ 32GB/ 3200) total DDR4 memory, ucode 0xd000375, HT ON, Turbo ON, Ubuntu 20.04.5 LTS, 5.4.0-135-generic, 1x Crucial MX500 1TB, ResNet-50 v1.5 inference (Image Classification) AI Workloads with Batch_​size1, int8 precision, test by Intel on Dec 2 2022. ResNet-50 - FPS Test by Intel on December 2, 2022.
[N22] 4.25x higher AI inference performance on Intel® Xeon® Gold 6448Y processor with built-in AI acceleration engine, Intel® AMX (INT8), compared to prior generation Intel® Xeon® Gold 6348 processor for Object Detection workload [SSD-RN34 inference]. 4th Generation Intel® Xeon® Gold processor New: 1-node, 2x pre-production Intel® Xeon® Gold 6448Y on ArcherCity with 512 GB (16 slots/ 32GB/ 4800) total DDR5 memory, ucode 0x2b000111, HT ON, Turbo ON, Ubuntu 20.04.5 LTS, 5.16.0-051600-generic, 1x Samsung SSD 870 1TB, SSD-RN34 inference (Object Detection) AI Workloads with Batch_​size1, int8 precision, test by Intel on Mon Dec 5 2022. Baseline: 1-node, 2x Intel® Xeon® Gold 6348 CPU on WilsonCity with 512 GB (16 slots/ 32GB/ 3200) total DDR4 memory, ucode 0xd000375, HT ON, Turbo ON, Ubuntu 20.04.5 LTS, 5.4.0-135-generic, 1x Crucial MX500 1TB, SSD-RN34 inference (Object Detection) AI Workloads with Batch_​size1, int8 precision, test by Intel on Fri Dec 2 2022. SSD-ResNet34 - FPS Test by Intel on December 2, 2022.
[N23] 1.33x higher performance on Intel® Xeon® Gold 6448Y processor, compared to prior generation Intel® Xeon® Gold 6348 processor on integer throughput. 4th Generation Intel® Xeon® Gold processor New: 1-node, 2x pre-production Intel® Xeon® Gold 6448Y on ArcherCity with 512 GB (16 slots/ 32GB/ 4800) total DDR5 memory, ucode 0x2b000111, HT ON, Turbo ON, CentOS Stream 8, 5.15.0-spr.bkc.pc.12.7.15.x86_​64, 1x Samsung SSD 870 1TB, SPEC CPU 2017 (n-copy) (est.), ic2022.1, test by Intel on Nov 25 2022. Baseline: 1-node, 2x Intel® Xeon® Gold 6348 CPU on WilsonCity with 512 GB (16 slots/ 32GB/ 3200) total DDR4 memory, ucode 0xd000375, HT ON, Turbo ON, CentOS Stream 8, 5.15.0-spr.bkc.pc.12.7.15.x86_​64, 1x Crucial MX500 1TB, SPEC CPU 2017 (n-copy) (est.), ic2022.1, test by Intel on Nov 23 2022. Integer throughput Test by Intel on Nov 25, 2022.

Test by Intel on Nov 23, 2022.

[N24] Running next-generation firewall clear-text inspection, the 4th Gen Intel Xeon Gold 6428N has 1.26x higher, 4th Gen Intel Xeon Platinum 8470N has 1.71x higher performance than 3rd Gen Intel Xeon Gold 6338N, and 4th Gen Intel Xeon Platinum 8490H has 1.5x higher performance than 3rd Gen Intel Xeon Platinum 8380. 4th Generation Intel® Xeon® Platinum processor and Intel® Xeon® Gold processor 1.26x and 1.71x higher performance than prior gen running next-generation firewall workload. 4th Gen Platinum 8470N: Test by Intel as of 11/2022. 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Intel Corporation M50FCP, 52 cores, HT On, Turbo Off/On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4800 MT/s]), microcode 0x2b000310, Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2KB240G8, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 11.3, NGFW22.09-1, VPP : v22.06.0-16, Snort:3.1.36.0, DAQ: 3.0.9, LuaJIT: 2.1.0-beta3, OpenSSL: 1.1.11F 31 Mar 2020, Libpcap: 1.10.1(with TPACKET_​V3), PCRE:8.45 2021-06-15, ZLIB: 1.2.11, Hyperscan: 5.4.0 2021-01-26, LZMA: 5.2.5 3rd Gen Gold 6428N: Test by Intel as of 11/2022. 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Gold 6428N on Intel Corporation Archer City, 32 cores, HT On, Turbo Off/On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4000 MT/s]), microcode 0x2b000111, Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2KB240G8, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 11.3, NGFW22.09-1, VPP : v22.06.0-16, Snort:3.1.36.0, DAQ: 3.0.9, LuaJIT: 2.1.0-beta3, OpenSSL: 1.1.11F 31 Mar 2020, Libpcap: 1.10.1(with TPACKET_​V3), PCRE:8.45 2021-06-15, ZLIB: 1.2.11, Hyperscan: 5.4.0 2021-01-26, LZMA: 5.2.5 Baseline configuration: 3rd Gen Gold 6338N: Test by Intel as of 11/2022. 1-node, 2(1 used)x Intel® Xeon® Gold 6338N CPU @ 2.20GHz on Supermicro X12DPG-QT6, 32 cores, HT On, Turbo Off/On, Total Memory 512GB (16x32GB DDR4 3200 MT/s [2666 MT/s]), BIOS 1.4, microcode 0xd000375, Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2BW240H6, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.35, GCC 9.4, NGFW22.09-1, VPP : v22.06.0-16, Snort:3.1.36.0, DAQ: 3.0.9, LuaJIT: 2.1.0-beta3, OpenSSL: 1.1.11F 31 Mar 2020, Libpcap: 1.10.1(with TPACKET_​V3), PCRE:8.45 2021-06-15, ZLIB: 1.2.11, Hyperscan: 5.4.0 2021-01-26, LZMA: 5.2.5. 1.5x higher performance than prior gen running next-generation firewall workload: 4th Gen Platinum 8490H: Test by Intel as of 11/2022. 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8490N on Intel Corporation M50FCP, 60 cores, HT On, Turbo Off/On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4800 MT/s]), microcode 0x2b000111, Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2KB240G8, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 11.3, NGFW22.09-1, VPP : v22.06.0-16, Snort:3.1.36.0, DAQ: 3.0.9, LuaJIT: 2.1.0-beta3, OpenSSL: 1.1.11F 31 Mar 2020, Libpcap: 1.10.1(with TPACKET_​V3), PCRE:8.45 2021-06-15, ZLIB: 1.2.11, Hyperscan: 5.4.0 2021-01-26, LZMA: 5.2.5 3rd Gen Platinum 8380: Test by Intel as of 11/2022. 1-node, 2(1 used)x Intel® Xeon® Platinum 8380 CPU @ 2.30GHz on Supermicro X12DPG-QT6, 40 cores, HT On, Turbo Off/On, Total Memory 256GB (16x16GB DDR4 3200 MT/s [3200 MT/s]), BIOS 1.4, microcode 0xd000375, Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2BW240H6, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.35, GCC 9.4, NGFW22.09-1, VPP : v22.06.0-16, Snort:3.1.36.0, DAQ: 3.0.9, LuaJIT: 2.1.0-beta3, OpenSSL: 1.1.11F 31 Mar 2020, Libpcap: 1.10.1(with TPACKET_​V3), PCRE:8.45 2021-06-15, ZLIB: 1.2.11, Hyperscan: 5.4.0 2021-01-26, LZMA: 5.2.5 Next-gen Firewall (NGFW) Test by Intel as of November 2022.
[N25] 2.1x perf gain for VPP Fib 512B and 1.3x perf gain for VPP IPsec 1420B 1000 tunnels (DDIO On vs. DDIO off). 4.1x perf/memory bandwidth gain for VPP Fib 512B and 1.67x perf gain for VPP IPsec 1420B 1000 tunnels (DDIO On vs. DDIO off).

For NGFirewall clear-inspection, memory bandwidth reduction to 0.67x with DDIO enabled. Perf/memory bandwidth gain is 1.48x with DDIO On vs. DDIO Off.

For 5G UPF, comparing DDIO On vs. Off, performance gain is 4% with DDIO enabled; with DDIO enabled memory bandwidth is 0.86x, and performance/memory bandwidth gain is 1.21x with DDIO On.
4th Generation Intel® Xeon® Platinum processor 5G UPF: Test by Intel as of Dec 5 04:42:20. 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N, 52 cores, HT On, Turbo On, Total Memory 512GB (16x32GB 4800 MT/s [4800 MT/s]), microcode 0x2b000111, 6x Ethernet Controller E810-C for QSFP, 1x Ethernet interface, 1x Ethernet Controller I225-LM, 1x 745.2G INTEL SSDSC2BA800G3, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.0-27-generic, gcc 7.5.0, VPP 20.09, FlexCore 5G UPF PMA4, DDI/O ON/OFF NGFW: Test by Intel as of 11/28/22. 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N, 52 cores, HT On, Turbo Off, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4800 MT/s]), microcode 0x2b000111, 1x Ethernet Controller E810-C for QSFP, 1x 223.6G INTEL SSDSC2KB240G8, Ubuntu 22.04 LTS, 5.15.0-27-generic, gcc 11.2, NGFW 22.09-1, DDI/O ON/OFF VPP FIB : 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release FIB ipv4 router, Dataset size 512B​, IxNetwork 9.00.1900.17​, DDI/O ON/OFF test by Intel on 11/17/2022 VPP IPsec 8470N: : 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release 1000 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 1420B​, IxNetwork 9.00.1900.17​, DDI/O ON/OFF, test by Intel on 11/17/2022. Intel DDIO Test by Intel as of November 2022.
[N26] PTM improves clock synchronization accuracy and precision under heavy load. 4th Generation Intel® Xeon® Platinum processor 8470N: Precision Time Measurement Test by Intel on 11/21/2022. 1-node, 1x Pre-production platform with 1x Intel® Xeon® Platinum 8470N on Archer City, 52 cores, HT On, Turbo On, 256GB Memory (8x32GB 4800 MT/s [4800 MT/s]), microcode 0x2b0000c0, Intel® Ethernet Network Adapter I225-T1, 1x 223.6G INTEL SSDSC2KB240G8, Ubuntu 22.04 LTS, 5.4.209_​tgpio_​spr, BI/OS MUX configuration precision time measurement Test by Intel on November 21, 2022.
[N27] Up to 24 Percent Lower Off-Peak Power with New Data Monitoring Instructions on 4th Gen Intel Platinum 8470N compared to not using the instructions. 4th Generation Intel® Xeon® Platinum processor Up to 24 Percent Lower Off-Peak Power with New Data Monitoring Instructions vCMTS: Test by Intel as of 11/18/22. 1node, pre-production platform with 2 (1 used)x Intel® Xeon® Platinum 8470N, 52 cores on Archer City, HT On, Turbo OFF, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MHz [run @ 4800 MHz] ), microcode 0x2b0000c0, Ubuntu 22.04 LTS, 5.15.0-27-generic, gcc 11.2.0, vCMTS 22.10, DPDK 22.03, Collectd 5.12.0, Grafana 8.5.3, Prometheus 2.0,0, PMD Pwr Mgmt enabled/disabled, Cable iMix 1KB packet size, 24 instances vCMTS with UMWAIT Test by Intel on November 18, 2022.
[N28] Up to 2.02x on 3rd Gen Intel Xeon Gold 6338N, 2.79x on 4th Gen Intel Xeon Gold 6428N, 3.08x on 4th Gen Intel Xeon Platinum 8470N higher performance vs. 2nd Gen Intel Xeon Gold processor 6252N for VPP FIB Router.

Up to 1.92x on 3rd Gen Intel Xeon Gold 6338N, 2.63x on 4th Gen Intel Xeon Gold 6428N, 3.91x on 4th Gen Intel Xeon Platinum 8470N higher performance vs. 2nd Gen Intel Xeon Gold processor 6252N for VPP IPsec Secure Gateway.
4th Generation Intel® Xeon® Platinum processor and Intel® Xeon® Gold processor Up to 2.02x on 3rd Gen Intel® Xeon® Gold, 2.79x on 4th Gen Intel® Xeon® Gold, 3.08x on 4th Gen Intel® Xeon® Platinum higher performance vs. 2nd Gen Intel® Xeon® Gold processor for VPP Fib Router Up to 1.92x on 3rd Gen Intel® Xeon® Gold, 2.63x on 4th Gen Intel® Xeon® Gold, 3.91x on 4th Gen Intel® Xeon® Platinum higher performance vs. 2nd Gen Intel® Xeon® Gold processor for VPP IPsec Secure Gateway VPP Fib Router: 4th Gen Gold 6428N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Gold 6428N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4000]) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 512B​, IxNetwork 9.00.1900.17​, test by Intel on 10/17/2022. 3rd Gen Gold 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on Wilson City with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-2CQDA2 (total 800Gb/s), VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 512B​, IxNetwork 9.00.1900.17​, test by Intel on 10/5/2022 4th Gen Platinum 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 9/30/2022. 2nd Gen Gold 6252N: 1-node, 2(1 used)x Intel® Xeon® Gold 6252N on Supermicro platform with 192 GB (12x16GB DDR4 2933 MT/s [2934 MT/s]) total memory, ucode 0x500320a, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 2x Intel® Ethernet Controller X550, 6x Intel® Ethernet Controller E810, VPP 22.06-release FIB ipv4 router, GCC 9.4​, Dataset size 64B / 512B​, IxNetwork 9.00.1900.17​, test by Intel on 12/2022 VPP IPsec: 4th Gen Platinum 8470N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Platinum 8470N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800) total memory, ucode 0xab0000c0, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 9/30/2022 4th Gen Gold 6428N: 1-node, pre-production platform with 2(1 used)x Intel® Xeon® Gold 6428N on Archer City with 512 GB (16 slots/ 32GB/ DDR5-4800[4000]) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 5x Intel® Ethernet Network Adapter E810-2CQDA2 (total 1000Gbps), VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 10/17/2022. 3rd Gen Gold 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N on Wilson City with 256 GB (16 slots/ 16GB/ DDR4-3200[2666]) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 4x Intel® Ethernet Network Adapter E810-2CQDA2 (total 800Gb/s), VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 10/5/2022 2nd Gen Gold 6252N: 1-node, 2(1 used)x Intel® Xeon® Gold 6252N on Supermicro platform with 192 GB (12x16GB DDR4 2933 MT/s [2934 MT/s]) total memory, ucode 0x500320a, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.35, 1x INTEL SSDSC2KB240G8, 2x Intel® Ethernet Controller X550, 6x Intel® Ethernet Controller E810, VPP 22.06-release 1 tunnel per VF port, GCC 9.4​, Intel-ipsec-mb libraries v1.1, Dataset size 1420B​, IxNetwork 9.00.1900.17​, test by Intel on 12/2022 VPP IPv4 forward information base (FIB) and VPP IPsec Test by Intel on October 17, 2022.

Test by Intel on December 2022.

Test by Intel on September 30 ,2022.

Test by Intel on October 5, 2022.

[N29] Up to 1.39x higher connections/sec with QAT and optimized software on 4th Gen Intel Xeon Gold 6428N processor compared to optimized SW only for NGINX webserver TLS 1.3 ECDHE-X25519-RSA2K. 4th Generation Intel® Xeon® Gold processor Up to 1.39x NGINX TLS 1.3 connections/s with QAT and optimized software vs. optimized software on 4th Gen Intel® Xeon® Gold 6428N CPS with and without QAT: Test by Intel as of Oct 18 08:11:32. 1-node, pre-production platform with 2x Intel® Xeon® Gold 6428N, 32 cores on Archer City, HT On, Turbo On, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4000 MT/s]), microcode 0xab000060, 1x Ethernet Controller I225-LM, 6x Intel® Ethernet Network Adapter E810-CQDA2, 1x 223.6G INTEL SSDSC2BB240G4, 1x 223.6G INTEL SSDSC2KB240G8, 1x 240M Disk, Ubuntu 22.04 LTS, 5.15.0-27-generic, NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, ECDHE-X25519-RSA2K, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), QAT driver QAT.20.L.0.9.5 NGINX Webserver Handshake with Integrated QAT2.0 TLS 1.3 ECDHE-X25519-RSA2K Test by Intel as of October 18, 2022.
[N30] Up to 1776 for Deflate L9 compression, 749 for Deflate L1 compression, and 303 for Deflate decompression cores saved using 4th Gen Intel Xeon Platinum 8470N with Intel QAT vs. Lzbench software. 4th Generation Intel® Xeon® Platinum processor Up to 1776 for Deflate L9 compression, 749 for Deflate L1 compression, and 303 for Deflate decompression cores saved using Intel QAT vs. zlib software Test by Intel as of 12/07/22. 1-node, pre-production platform with 2 (1 used) x Intel® Xeon® Platinum 8470N, 52 cores, HT On, Turbo Off, Total Memory 512GB (16x32GB 4800 MT/s [4800 MT/s]), microcode 0x2b000161, 1x Ethernet Controller I225-LM, 2x Ethernet Controller XXV710 for 25GbE SFP28, 1x 240M Disk, 1x 223.6G INTEL SSDSC2KB240G8, 1x 223.6G INTEL SSDSC2BW240H6, Ubuntu 22.04.1 LTS, 5.15.0-56-generic, QAT20.L.1.0.0-00021, Lzbench (zlib-1.2.11), GCC 11.3.0, QAT L1 (160 Gbps),9 (100 Gbps)/SW L1,3,4 for compression, 320 Gbps for decompression, 64KB packets Compression offload with Intel QuickAssist Technology Test by Intel as of December 7, 2022.
[N31] Demo: Intel® Data Streaming Accelerator 4th Generation Intel® Xeon® Platinum processor 1-node, 2x pre-production 4th Gen Intel Xeon Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, 10GbE x540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022. 1-node, 2x production 3rd Gen Intel Xeon Scalable Processors (40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), HT On, Turbo On, SNC Off, microcode 0xd000375, 10GbE x540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022. SPDK large media files and database requests vs. OOB Test by Intel as of November 2022.
[N32] Up to 1.79x higher connections/second with 4th Gen Intel Xeon Gold 6428N w/QAT processor compared to 3rd Gen Intel Xeon Gold 6338N processor with Lewisburg PCH on NGINX TLS 1.3 ECDHE-X25519-RSA2K handshake with optimized software. 4th Generation Intel® Xeon® Gold processor 1.79x higher connections/second on TLS 1.3 ECDHE-X25519-RSA2K webserver handshake [6428N w/QAT vs. 6338N+LBG] 6428N: Test by Intel as of Sep 19, 2022. 1-node, pre-production platform 2(1 used)x Intel® Xeon® Gold 6428N, 32 (24C48T used) cores, HT On, Turbo Off, Total Memory 512GB (16x32GB 4800 MT/s [4000 MT/s]), microcode 0xab000060, 1x Ethernet Controller I225-LM, 6x Ethernet Controller E810-C for QSFP, 1x Ethernet interface, 1x 223.6G INTEL SSDSC2BB240G4 Ubuntu 22.04 LTS, 5.15.0-27-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0. NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), QAT driver QAT.20.L.0.9.5 (2 QAT HW) 6338N: 1-node, 2(1 used)x Intel® Xeon® Gold 6338N CPU @ 2.20GHz, 32 (24C48T used) cores on Supermicro SYS-740GP-TNRT, HT On, Turbo Off, Total Memory 256GB (16x16GB DDR4 3200 MT/s [2666 MT/s]), BIOS 1.4, microcode 0xd000375, 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller 10G X550T, 1x 223.6G INTEL SSDSC2KB240G8, Ubuntu 22.04 LTS, 5.15.0-27-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, LBG 62X Chipset (3 QAT), NGINX (async mode nginx 0.4.7), GCC 11.2.0, openssl 1.1.1m, qatengine v0.6.14 (Optimized SW and QAT HW), IPsecmb v1.2 (Optimized SW), IPP-Crypto ipp-crypto_​2021_​5 (Optimized SW), QAT Driver (CPM 1.7): QAT.L.4.18.1-00001, test by Intel on 09/19/2022. NGINX TLS 1.3 ECDHE-X25519-RSA2K handshake Test by Intel as of September 19, 2022.
[N33] New 4th Gen Intel Xeon Scalable processors now provide the industry's first 1 Tbps of performance for the 5G UPF workload with a single dual socket server. 4th Gen Intel® Xeon® Scalable processors 1-node, 2x Intel® Xeon® Platinum 8470N CPU, 52 cores(104 Total), HT On, Turbo Off, Total Memory 1024GB (16x64GB DDR5 4800 MT/s [4800 MT/s]), BIOS EGSDCRB1.SYS.0093.D22.2211170057, microcode 0x2b000130, 6x Intel E810-2CQDA2 (CVL, Chapman Beach, Total - 6x100G ports), 1x Intel E810-CQDA2 (CVL, Tacoma Rapids, Total - 2x100G ports) 1x 447.1G INTEL SSDSCKKB8, 1x 931.5G CT1000MX500SSD1, Ubuntu 22.04 LTS, 5.15.0-53-generic, UPF(GCC 9.4.0/Clang9.0.0,DPDK 22.07,VPP 20.09). Tested by Intel on January 27, 2023. 5G User Plane Network Functions Test by Intel on January 27, 2023.
[N34] New 4th Gen Intel® Xeon® Scalable processors with Intel® Infrastructure Power Manager for 5G Core deliver 93% performance-per-watt improvement compared to previous generation Intel Xeon Scalable processors without the new software. 4th Gen Intel® Xeon® Scalable processors (BASELINE) 1-node, 2x Intel® Xeon® Gold 6338N CPU, 32 cores, HT On, Turbo Off, Total Memory 256GB (8x32GB DDR4 2667 MT/), BIOS SE5C6200.86B.0020.P41.2109300305, microcode 0xd000311, 2x Intel E810-2CQDA2 (CVL, Chapman Beach, Total - 4x100G ports), 1x 931.5G CT1000MX500SSD1, Ubuntu 18.04 Bionic Beaver, 4.15.0-194-generic, GCC 7.5.0, DPDK 22.11 (NEW) 1-node, 2x Intel® Xeon® Gold 6438N CPU, 32 cores, HT On, Turbo Off, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4000 MT/s]), BIOS EGSDCRB1.SYS.0090.D03.2210040200, microcode 0x2b0000c0, 2x Intel E810-2CQDA2 (CVL, Chapman Beach, Total - 4x100G ports), 1x 223.6G INTEL SSDSC2KB240G8, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 7.5.0, DPDK 22.11. Tested by Intel on January 26, 2023. 5G User Plane Network Functions Test by Intel on January 26, 2023.
[N35] Intel® Infrastructure Power Manager for 5G Core software delivers an average power savings of 30% while maintaining key telco performance metrics by dynamically matching run-time CPU power consumption to traffic. 4th Gen Intel® Xeon® Scalable processors 1-node, 2x Intel® Xeon® Gold 6438N CPU, 32 cores, HT On, Turbo Off, Total Memory 512GB (16x32GB DDR5 4800 MT/s [4000 MT/s]), BIOS EGSDCRB1.SYS.0090.D03.2210040200, microcode 0x2b0000c0, 2x Intel E810-2CQDA2 (CVL, Chapman Beach, Total - 4x100G ports), 1x 223.6G INTEL SSDSC2KB240G8, 1x 745.2G INTEL SSDSC2BA800G3, Ubuntu 22.04 LTS, 5.15.0-27-generic, GCC 7.5.0, DPDK 22.11. Tested by Intel on January 26, 2023. 5G User Plane Network Functions Test by Intel on January 26, 2023.
[N36] Intel VROC achieves up to 5.7 million IOPS with RAID0 on mixed workloads

Intel® VROC performance is consistently better than RAID HBA, delivering up to 66% more IOPS, up to 40% lower latency and up to 62% better power efficiency.
4th Generation Intel® Xeon® Platinum processor 1-node, 2 x Intel® Xeon® Platinum 8468H on Intel Pre production M50FCP2SBSTD with 256GB (16 Slots/ 16GB/ 4800 ) total memory BIOS Version: SE5C7411.86B.8805.D02.2209220021 , ucode: 0x2b000081, Turbo On, HT On, OS: Red Hat Enterprise Linux Server 8.5, Kernel: 4.18.0-348.el8.x86_​64, Storage: 4 x 400GB Intel Optane P5810X PCIe Gen4 U.2 SSDs ,2 x 118GB Intel Optane P1600X M.2 SSDs , RAID Controller: 8x Intel, Firmware Package Build = 52.22.0-4544. OR Intel® VROC PreOS Version: 8.0.0.1336; RAID Configurations: 4-Disk RAID0 with Intel VROC and Intel RS3P4TF160F; FIO version: 3.30 , Power Efficiency calculated as performance (IOPS) per watt. testing by Intel as of March 21, 2023: FIO Performance, latency, and power Tested by Intel on March 21, 2023
[N37] Up to 3x higher Connections-Per-Sec with Intel Crypto acceleration and 5x higher Connections-Per-Sec with Intel QAT accelerator vs Native OpenSSL on Intel 4th Gen Xeon Scalable processor based Alibaba instance (g8i.4xlarge) with HAProxy TLS1.2 ECDHE-RSA-AES256-GCM-SHA384 load balancer handshakes 4th Generation Intel® Xeon® Platinum processor g8i.4xlarge - 1-node, 1x Intel(R) Xeon(R) Platinum 8475B CPU @ 2.70GHz, 8 cores, HT On, Turbo On, Total Memory 64GB (4x16GB RAM Unknown [Unknown]), Virtio ,1x 40G Alibaba Cloud Elastic Block Storage , BIOS-Alibaba ECS, 449e491, Ubuntu 22.04.2 LTS, Kernel 5.15.0-73-generic , HAProxy 2.7 , OpenSSL 1.1.1k, QAT_​engine 1.0.0, ipp_​crypto 2021.7.1, ipsec_​mb 1.3,QAT hardware driver QAT20.L.1.0.10-00005, Compiler gcc 11.3.0, Test by Intel as of 04/29/23 HAProxy software load balancer:ECDHE-RSA-AES256-GCM-SHA384 handshakes Test by Intel as of April 29, 2023
[N38] Up to 26x higher inference performance and up to 16x higher performance/watt (DLRM) on 4th Gen Intel Xeon vs.1st Gen Intel® Xeon® processors 4th Gen Intel® Xeon® processors (Intel Xeon Gold 5420+) and 1st Gen Intel® Xeon® processors (Intel Xeon Gold 5120) Test by Intel as of 09/22/23 1-node, 2x Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz, 14 cores, HT On, Turbo On, Total Memory 768GB (12x64GB DDR4 2666 MT/s [2400 MT/s]), BIOS 3.8b, microcode 0x2007006, 2x Ethernet Connection X722 for 10GBASE-T, 1x 894.3G Micron_​7450_​MTFDKBA960TFR, Ubuntu 22.04.2 LTS, 5.15.0-78-generic 1-node, 2x Intel(R) Xeon(R) Gold 5420+, 28 cores, HT On, Turbo On, NUMA 2, Integrated Accelerators Available [used]: DLB 2 [0], DSA 2 [0], IAX 2 [0], QAT 2 [0], Total Memory 1024GB (16x64GB DDR5 5600 MT/s [4400 MT/s]), BIOS 2.0, microcode 0x2b0004d0, 2x Ethernet Controller 10-Gigabit X540-AT2, 1x Ethernet interface, 1x 1.7T SAMSUNG MZQL21T9HCJR-00A07, Ubuntu 22.04.2 LTS, 5.15.0-78-generic September 9, 2023
[N39] Up to 8x higher inference performance and 5x higher performance/watt (BERT-Large) on 4th Gen Intel Xeon Platinum vs. 2nd Gen Intel® Xeon® processors 4th Gen Intel® Xeon® processors (Intel Xeon Gold 5420+) and 2nd Gen Intel® Xeon® processors (Intel Xeon Gold 5218) Test by Intel as of 10/05/23 1-node, 2x Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz, 16 cores, HT On, Turbo On, Total Memory 768GB (12x64GB DDR4 2666 MT/s [2666 MT/s]), BIOS 3.8b, microcode 0x4003604, 2x Ethernet Connection X722 for 10GBASE-T, 1x 894.3G Micron_​7450_​MTFDKBA960TFR, Ubuntu 22.04.2 LTS, 5.15.0-78-generic 1-node, 2x Intel(R) Xeon(R) Gold 5420+, 28 cores, HT On, Turbo On, NUMA 2, Integrated Accelerators Available [used]: DLB 2 [0], DSA 2 [0], IAX 2 [0], QAT 2 [0], Total Memory 1024GB (16x64GB DDR5 5600 MT/s [4400 MT/s]), BIOS 2.0, microcode 0x2b0004d0, 2x Ethernet Controller 10-Gigabit X540-AT2, 1x Ethernet interface, 1x 1.7T SAMSUNG MZQL21T9HCJR-00A07, Ubuntu 22.04.2 LTS, 5.15.0-78-generic October 10, 2023
[N50] Up to 1.8x higher performance with 2-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series vs. prior gen platform on CDN Video-On-Demand with Varnish Enterprise [20k connections]. 4th Generation Intel® Xeon® Gold processor 2S 6448N: Test by Intel as of 10/14/22. 1-node, pre-production platform with 2x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 4800 MT/s), 16x Intel® P5510, 4x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 2S 6338N: Test by Intel as of 10/14/22. 1-node, 2x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 3200 MT/s @ 2666 MT/s), 12x Intel® P5510, 4x Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN VOD w/Varnish Enterprise on NVMe Test by Intel as of October 14, 2022.
[N51] Up to 1.35x higher performance with 2-socket 4th Gen Intel Xeon Platinum 8480+, Intel Ethernet 800 Series vs. prior gen platform on CDN Video-On-Demand with Varnish Enterprise [4k connections]. 4th Generation Intel® Xeon® Platinum processor 2S 8380: Test by Intel as of 10/18/22. 1-node, 2x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 3200 MT/s), 12x Intel® P5510, 4x Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio.. 2S 8480+: Test by Intel as of 10/18/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 4800 MT/s), 16x Intel® P55510, 4x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. CDN VOD w/Varnish Enterprise - NVMe Test by Intel as of October 18, 2022.
[N52] Up to 1.25x higher performance with 1-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series vs. prior gen platform on CDN Video-On-Demand with Varnish Enterprise [20k connections]. 4th Generation Intel® Xeon® Gold processor 1S 6338N: Test by Intel as of 10/10/22. 1-node, 1x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s @ 2666 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. 1S 6438N: Test by Intel as of 10/10/22. 1-node, pre-production platform with 1x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. CDN VOD w/Varnish Enterprise - NVMe Test by Intel as of October 10, 2022.
[N53] Up to 1.36x higher performance with 1-socket 4th Gen Intel Xeon Platinum 8480+, Intel Ethernet 800 Series vs. prior gen platform on CDN Video-On-Demand with Varnish Enterprise [4k connections]. 4th Generation Intel® Xeon® Platinum processor 1S 8380: Test by Intel as of 10/18/22. 1-node, 1x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. 1S 8480+: Test by Intel as of 10/18/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. CDN VOD w/Varnish Enterprise - NVMe Test by Intel as of October 18, 2022.
[N54] Up to 1.26x higher performance with 1-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series and Intel Optane SSD P5800X vs. prior gen platform on CDN Live-Linear connection: close with Varnish Enterprise [4k connections]. 4th Generation Intel® Xeon® Gold processor 1S 6338N: Test by Intel as of 10/14/22. 1-node, 1x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s @ 2666 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 1S 6438N: Test by Intel as of 10/14/22. 1-node, pre-production platform with 1x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), RHEL 8.6, kernel 4.18.0-372.26.1.el8_​6.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), OpenSSL 1.1.1k FIPS 25 Mar 2021, varnish-plus-6.0.10r3 revision 4f67b6ec0d63f04560913cc7e195a3919bdf0366, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live Varnish P5800X Test by Intel as of October 14, 2022.
[N55] Up to 1.75x higher performance with 2-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series and Intel Optane PMem vs. prior gen platform on CDN Live-Linear connection: close with NGINX [40k connections]. 4th Generation Intel® Xeon® Gold processor 2S 6338N: Test by Intel as of 10/10/22. 1-node, 2x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 3200 MT/s), Total Persistent Memory 2048 GB (16 slots/ 128 GB/ 3200 MT/s, App-Direct-Interleaved), 4x Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 2S 6438N: Test by Intel as of 10/10/22. 1-node, 2x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On Total Memory 256 GB (16 slots/ 16 GB/ 4800 MT/s), Total Persistent Memory 2048 GB (16 slots/ 128 GB/ 4400 MT/s, App-Direct-Interleaved), 4x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX PMem Test by Intel as of October 10, 2022.
[N56] Up to 1.69x higher performance with 2-socket 4th Gen Intel Xeon Platinum 8480+, Intel Ethernet 800 Series and Intel Optane SSD P5800X vs. prior gen platform on CDN Live-Linear connection: close with NGINX [8k connections]. 4th Generation Intel® Xeon® Platinum processor 2S 8380: Test by Intel as of 10/04/22. 1-node, 2x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 3200 MT/s), Total Persistent Memory 2048 GB (16 slots/ 128 GB/ 3200 MT/s, App-Direct-Interleaved), 4x Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 2S 8480+: Test by Intel as of 10/04/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (16 slots/ 16 GB/ 4800 MT/s), Total Persistent Memory 2048 GB (16 slots/ 128 GB/ 4400 MT/s, App-Direct-Interleaved), 4x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX PMem Test by Intel as of October 4, 2022.
[N57] Up to 1.24x higher performance with 1-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series and Intel Optane Pmem vs. prior gen platform on CDN Live-Linear connection: close with NGINX [20k connections]. 4th Generation Intel® Xeon® Gold processor 1S 6338N: Test by Intel as of 10/10/22. 1-node, 1x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s @ 2666 MT/s ), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 3200 MT/s @ 2666 MT/s, App-Direct-Interleaved), 2x Intel® Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 1S 6438N: Test by Intel as of 10/10/22. 1-node, pre-production platform with 1x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 4400 MT/s, App-Direct-Interleaved), 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX PMem Test by Intel as of October 10, 2022.
[N58] Up to 1.27x higher performance with 1-socket 4th Gen Intel Xeon Platinum 8480+, Intel Ethernet 800 Series and Intel Optane Pmem vs. prior gen platform on CDN Live-Linear connection: close with NGINX [400 connections]. 4th Generation Intel® Xeon® Platinum processor 1S 8380: Test by Intel as of 10/04/22. 1-node, 1x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 3200 MT/s, App-Direct-Interleaved), 2x Intel® Mellanox MCX516A-CDAT, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 1S 8480+: Test by Intel as of 10/04/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+ Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 4400 MT/s, App-Direct-Interleaved), 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX PMem Test by Intel as of October 04, 2022.
[N59] Up to 1.27x higher performance with 1-socket 4th Gen Intel Xeon Gold 6438N, Intel Ethernet 800 Series and Intel Optane SSD P5800X vs. prior gen platform on CDN Live-Linear connection: close with NGINX [4k connections]. 4th Generation Intel® Xeon® Gold processor 1S 6338N: Test by Intel as of 10/10/22. 1-node, 1x Intel® Xeon® Gold 6338N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s @ 2666 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 1S 6438N: Test by Intel as of 10/10/22. 1-node, pre-production platform with 1x Intel® Xeon® Gold 6438N Processor, 32 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX 5800x Test by Intel as of October 10, 2022.
[N60] Up to 1.46x higher performance with 1-socket 4th Gen Intel Xeon Platinum 8480+, Intel Ethernet 800 Series and Intel Optane SSD P5800X vs. prior gen platform on CDN Live-Linear connection: close with NGINX [20k connections]. 4th Generation Intel® Xeon® Platinum processor 1S 8380: Test by Intel as of 09/28/22. 1-node, 1x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. 1S 8480+: Test by Intel as of 09/28/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+ Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5800X, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 93.3% cache hit ratio. CDN Live NGINX 5800x Test by Intel as of September 28, 2022.
[N61] Up to 1.31x higher performance with 1-socket 4th Gen Intel Xeon Platinum 8480+ and Intel Ethernet 800 Series vs. prior gen platform on CDN Video-On-Demand with NGINX [20k connections]. 4th Generation Intel® Xeon® Platinum processor 1S 8380: Test by Intel as of 09/28/22. 1-node, 1x Intel® Xeon® Platinum 8380 Processor, 40 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 3200 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS 1.4 (ucode 0xd000375), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. 1S 8480+: Test by Intel as of 09/28/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+ Processor, 56 cores, HT On, Turbo On, Total Memory 256 GB (8 slots/ 32 GB/ 4800 MT/s), 8x Intel® P5510, 2x Intel® Ethernet Network Adapter E810-2CQDA2, BIOS EGSDCRB1.SYS.0087.D13.2208261709 (ucode 0x2b000070), Ubuntu 22.04, kernel 5.15.0-48-generic, gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022), NGINX 1.22.0, wrk master 02/07/2021 (keep alive OR connection: close, 400 OR 4000 OR 20000 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 100% cache hit ratio. CDN VOD NGINX NVMe Test by Intel as of September 28, 2022.
[N62] Up to 2.08x better data integrity (CRC32 iSCSI) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 1.42x higher data protection (RAID6 P+Q) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 1.34x higher encryption (AES-GCM 128 encode) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 1.13x higher compression (L1 Silesia Corpus) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 1.04x higher hashing (Multibuffer MD5) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. prior generation.

Up to 3.61x better data integrity (CRC32 iSCSI) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. 5-year refresh platform.

Up to 1.25x higher data protection (RAID6 P+Q) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. 5-year refresh platform.

Up to 3.85x higher encryption (AES-GCM 128 encode) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. 5-year refresh platform.

Up to 1.16x higher compression (L1 Silesia Corpus) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. 5-year refresh platform.

Up to 1.13x higher hashing (Multibuffer MD5) performance on 4th Gen Intel Xeon Platinum 8480+ processor vs. 5-year refresh platform.
4th Generation Intel® Xeon® Platinum processor ISA-L Comparison based on cycle/Byte performance measured on single core. 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ Processor, 56 cores HT On Turbo OFF Total Memory 1,024 GB (16 slots/ 64GB/ 4800 MHz), Data protection (RAID6 P+Q), Data integrity (CRC32 iSCSI), Hashing (Multibuffer MD5),Data encryption (AES-GCM 128 encode), Data Compression (Level 1 Compression (Silesia Corpus)), BIOS: EGSDCRB1.86B.8901.P01.2209200239 (ucode: 0xab0000c0), Red Hat Enterprise Linux 9.0, 5.14.0-70.22.1.el9_​0.x86_​64, gcc 11.2.1 compiler, yasm 1.3.0, nasm 2.15.03, isal 2.30, isal_​crypto 2.24, OpenSSL 3.0.1, zlib 1.2.11, Test by Intel as of 10/21/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 Processor, 40 cores HT On Turbo OFF Total Memory 512 GB (16 slots/ 32GB/ 3200 MHz), BIOS: WLYDCRB1.SYS.0029.P30.2209011945 (ucode:0xd000331), Red Hat Enterprise Linux 9.0, 5.14.0-70.22.1.el9_​0.x86_​64, gcc 11.2.1 compiler, yasm 1.3.0, nasm 2.15.03, isal 2.30, isal_​crypto 2.24, OpenSSL 3.0.1, zlib 1.2.11, Test by Intel as of 10/24/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 Processor, 28 cores HT On Turbo OFF Total Memory 192 GB (12 slots/ 16GB/ 2666 MHz), BIOS: PLYXCRB1.86B.0626.P01.2209280333 (ucode:0x2006e05), Red Hat Enterprise Linux 9.0, 5.14.0-70.22.1.el9_​0.x86_​64, gcc 11.2.1 compiler, yasm 1.3.0, nasm 2.15.03, isal 2.30, isal_​crypto 2.24, OpenSSL 3.0.1, zlib 1.2.11, Test by Intel as of 10/31/2022. ISA-L Test by Intel as of October 21, 2022.

Test by Intel as of October 24, 2022.

Test by Intel as of October 31, 2022.

[N63] For exceptional storage performance, Intel® Virtual RAID on CPU (Intel® VROC) eliminates the need for a RAID host bus adapter (HBA), delivering up to 60% higher IOPS and 39% lower latency while reducing cost by up to 70%. 4th Generation Intel® Xeon® Platinum processor Platform: Intel Beta Fox Creek Pass M50FCP2SBSTD (chassis M50FCP2UR208BPP), pre-production platform with 2 x Intel® Xeon® Platinum 8468H @ 2.1GHz (XCC CPUs, QDF: Q242, Stepping: E5) (48 cores each) (EagleStream - Sapphire Rapids), 256GB RAM (16 x 16GB Micron MTC10F1084S1RC48BAW 4800 MT/s DDR5 Synchronous Registered (Buffered) DIMMs), BIOS Version: SE5C7411.86B.8805.D02.2209220021 (Microcode revision: 0x2b000081), BIOS Release Date: 09/22/2022, BMC version: 1.27-0-gfedbbf-3cc10000, ME version: 06.00.03.0248, FRU version: 0.02, CPLD version: 2.0; BIOS Settings: SpeedStep(Enabled), Turbo(Enabled), ProcessorC6(Enabled), PackageC-State(C0/C1 State), CPU_​PowerAndPerformancePolicy(Performance), HardwareP-States(NativeMode), WorkloadConfiguration(I/O Sensitive), Hyperthreading enabled; Storage: 4 x 400GB Intel Optane P5810X PCIe Gen4 U.2 SSDs (Model: SSDPF21Q400GA, Firmware: L0310351) connected to backplane which is connected via Broadcom SlimSAS to SlimSAS connections that connect to an Intel RS3P4TF160F RAID controller card on PCIe slot 1 on Riser card 2 on the 2nd CPU (NUMA Node 1). OS on 1 of the 2 x 118GB Intel Optane P1600X M.2 SSDs (Model: SSDPEK1A118GA, Firmware: U5110550) connected to M.2 sockets on the motherboard on the 1st CPU (NUMA Node 0), CPU affinitized on 2nd CPU (NUMA Node 1); RAID Controller: Intel RS3P4TF160F (x8) (equivalent to Broadcom MegaRAID 9560-16i) card with Broadcom firmware, Firmware Package Build = 52.22.0-4544, Firmware Version = 5.220.02-3691, Driver Version = 07.721.02.00, CLI Version = 007.1912.0000.0000 Nov 23, 2021, Added "scsi_​mod.use_​blk_​mq=y" to grub boot option for maximum throughput on the Broadcom card, When creating RAID volumes "pdcache=on, Write-Back, No Read Ahead, Direct I/O". OR Intel® VROC PreOS Version: 8.0.0.1336, mdadm version: mdadm - v4.2-rc2 - 2021-08-02, Installed kmod-iavmd-1.0.0.1600-rhel_​85.x86_​64, "initcall_​blacklist=vmd_​drv_​init" was added to grub boot option which disables inbox VMD and enables the kmod-iavmd driver, Added "pci=pcie_​bus_​perf" to grub boot option which sets MaxPayload to the maximum for each of the NVMe devices OS: Red Hat Enterprise Linux Server 8.5, Kernel: 4.18.0-348.el8.x86_​64 RAID Configurations: 4-Disk RAID0 with Intel VROC and Intel RS3P4TF160F FIO version: 3.30 (fio config files will not include the "iodepth_​batch_​complete_​min" parameter for all testing) Intel VROC MSRP: $349, RAID HBA: Broadcom MegaRAID 9560-16i Price: $11961 (Broadcom MegaRAID 9560-16i - storage controller (RAID), https://www.cdw.com/product/broadcom-megaraid-9560-16i-storage-controller-raid-sata-6gb-s-sas-1/6392393. Pricing captured on 11/29/2022) Intel VROC Test by Intel as of November 29, 2022.
[N64] Intel® Optane™ Persistent Memory 300 series on 4th Gen Intel® Xeon® Scalable processors can provide between 49% and 226% bandwidth increase across sequential and random workloads compared to the previous generation (Intel Optane Persistent Memory 200 series on 3rd Gen Intel Xeon Scalable processors). 4th Generation Intel® Xeon® Platinum processor BASELINE 1: Test by Intel as of 09/27/22. 1-node, 1x Intel® Xeon® Platinum 8380, 40 cores, HT On, Turbo On, Total DRAM 256 GB (8 slots/ 32 GB/ 3200 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 3200 MT/s, App-Direct-Non-Interleaved), WLYDCRB1.E9I.0027.P61.2209071746 (ucode 0xd000363), CentOS Stream 8, kernel 5.15.0-spr.bkc.pc.10.4.11.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), MLC v3.9a-RC2-internal, Single Optane Persistent Memory module under test through FSDAX interface, stride length 128B for random tests to force a non sequential data pattern at the DIMM. NEW-1: Test by Intel as of 09/27/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total DRAM 512 GB (8 slots/ 64 GB/ 4800 MT/s @4400 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 4400 MT/s, App-Direct-Non-Interleaved), EGSDCRB1.E9I.0087.D13.2209212048 (ucode 0x2b000070), CentOS Stream 8, kernel 5.15.0-spr.bkc.pc.10.4.11.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), MLC v3.9a-RC2-internal, Single Optane Persistent Memory module under test through FSDAX interface, stride length 256B for random tests to force a non sequential data pattern at the DIMM. Memory bandwidth Test by Intel as of September 27, 2022.
[N65] Intel® Optane™ Persistent Memory 300 series on 4th Gen Intel® Xeon® Scalable processors can provide an average of 56% more bandwidth for sequential workloads and an average of 214% more bandwidth for random workloads compared to the previous generation (Intel Optane Persistent Memory 200 series on 3rd Gen Intel Xeon Scalable processors) 4th Generation Intel® Xeon® Platinum processor BASELINE 1: Test by Intel as of 09/27/22. 1-node, 1x Intel® Xeon® Platinum 8380, 40 cores, HT On, Turbo On, Total DRAM 256 GB (8 slots/ 32 GB/ 3200 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 3200 MT/s, App-Direct-Non-Interleaved), WLYDCRB1.E9I.0027.P61.2209071746 (ucode 0xd000363), CentOS Stream 8, kernel 5.15.0-spr.bkc.pc.10.4.11.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), MLC v3.9a-RC2-internal, Single Optane Persistent Memory module under test through FSDAX interface, stride length 128B for random tests to force a non sequential data pattern at the DIMM. NEW-1: Test by Intel as of 09/27/22. 1-node, pre-production platform with 1x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total DRAM 512 GB (8 slots/ 64 GB/ 4800 MT/s @4400 MT/s), Total Persistent Memory 1024 GB (8 slots/ 128 GB/ 4400 MT/s, App-Direct-Non-Interleaved), EGSDCRB1.E9I.0087.D13.2209212048 (ucode 0x2b000070), CentOS Stream 8, kernel 5.15.0-spr.bkc.pc.10.4.11.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), MLC v3.9a-RC2-internal, Single Optane Persistent Memory module under test through FSDAX interface, stride length 256B for random tests to force a non sequential data pattern at the DIMM. Memory bandwidth Test by Intel as of September 27, 2022.
[N66] Up to 1.3Tbps CDN Live-Linear network throughput and up to 1.17Gbps per watt on 2-socket 4th Gen Intel Xeon Platinum 8480+ and Intel Ethernet 800 Series with Varnish Enterprise software stack. 4th Generation Intel® Xeon® Platinum processor Test by Intel as of 1/09/2023. 1 node, Supermicro SYS-621C-TN12R, 2x Intel® Xeon® Platinum 8480+ with integrated QAT 8 available [0 used], DLB 8[0 used], DSA 8[0 used], IAA 8[0 used], 56 core, HT on, Turbo on, Total Memory 512GB (16 slots / 32GB / 4800 MT/s [4800 MT/s]), BIOS 1.0a (ucode 0x2b0000a1), 6x Intel® Ethernet Network Adapter E810-CQDA2, 2x Intel® Ethernet Network Adapter E810-CQDA2 for OCP 3.0, RHEL 8.7, kernel 4.18.0-425.3.1.el8.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15), ldd (GNU libc) 2.28, OpenSSL 1.1.1k FIPS 25 Mar 2021, ice 10.1.1.2.2, varnish-plus-6.0.10r3 revision c1c313d10ba6ea6feb9a08e4d88de15187d69a69, wrk master 02/07/2021 (keep alive, 12800 total connections) Throughput measured with 100% Transport Layer Security (TLS) traffic with 98.8% target cache hit ratio. CDN Live-Linear Varnish Enterprise Test by Intel as of January 9, 2023
[N201] Up to 83% fewer cores to achieve ~65K connections/second with 4th Gen Intel Xeon processor with built-in Intel QuickAssist Technology accelerator compared with AMD EPYC 7763 4th Generation Intel® Xeon® Platinum processor New: 1-node, pre-production platform with 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel Quick Assist Accelerator (Intel QAT), on pre-production Intel® platform and software with DDR5 memory total 1024GB (16x64 GB), microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, QAT engine v0.6.14, QAT v20.l.0.9.1, NGINX 1.20.1, OpenSSL 1.1.1l, IPP crypto v2021_​5, IPSec v1.1, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 processor (64 core) on GIGABYTE R282-Z92 with 1024GB DDR4 memory (16x64 GB), microcode 0xa001144, SMT On, Boost Off, NPS=1, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, 1x Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, NGINX 1.20.1, OpenSSL 1.1.1l, AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, tested by Intel September 2022. NGINX Key Handshake Tested by Intel in September 2022.
[N202] Up to 96% fewer cores and 13% better compression with 4th Gen Intel Xeon processor with built-in Intel QuickAssist Technology accelerator compared with AMD EPYC 7763. 4th Generation Intel® Xeon® Platinum processor New: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 core) with integrated Intel Quick Assist Accelerator (Intel QAT), on pre-production Intel® platform and software with DDR5 memory Total 1024GB (16x64 GB), microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, QAT v20.l.0.9.1, QATzip v1.0.9, ISA-L v2.3.0, tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 processor (64 core) on GIGABYTE R282-Z92 with 1024GB DDR4 memory (16x64 GB), microcode 0xa001144, SMT On, Boost Off, NPS=1, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, QAT v1.7.l.4.16, QATzip v1.0.9, ISA-L v2.3.0, tested by Intel September 2022. QATzip Tested by Intel in September 2022.
[N203] Up to 66% fewer cores for 200 Gbps SLA target for IPSec encryption with 4th Gen Intel Xeon processor with built-in Intel QuickAssist Technology accelerator compared with AMD EPYC 7763. 4th Generation Intel® Xeon® Platinum processor New: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), on pre-production Intel ® platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0xf000380, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, 2x100GbE, Intel® Ethernet Network Adapter E810-2CQDA2 (total 200Gbps), QAT v20.l.0.9.1, DPDK v21.11, IPSec v1.1, VPP 22.02, nasm v2.14.02, AES 128 GCM, VAES instructions, tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 processor (64 core) on Supermicro AS-2124US-TNRP with 1024GB DDR4 memory (16x64 GB), microcode 0xa01173, SMT On, Boost Off, NPS=2, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel SSDSC2KG01, Intel® Ethernet Network Adapter E810-2CQDA2 (total 200Gbps), DPDK v21.11, IPSec v1.1, VPP 22.02, nasm v2.14.02, AES 128 GCM, tested by Intel September 2022. IPSec Tested by Intel in September 2022.
[N204] Up to 2.5x higher performance and up to 60% latency reduction for large packet sequential read with 4th Gen Intel Xeon processor with built-in Intel Data Streaming Accelerator [DSA] accelerator compared with AMD EPYC 7763.

Up to 1.9x higher performance and up to 49% latency reduction for small packet random read with 4th Gen Intel Xeon processor with built in Intel Data Streaming Accelerator [DSA] compared with AMD EPYC 7763.
4th Generation Intel® Xeon® Platinum processor New: 1-node, 2x pre-production 4th Gen Intel® Xeon ® Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), on pre-production Intel® platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0xf000380, HT On, Turbo On, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, Intel® Ethernet Network Adapter E810-2CQDA2 (total 200Gbps), FIO v3.30, SPDK 22.05, large: 128K QD64 sequential read, small: 16k QD256 random read, tested by Intel September 2022. Baseline: 1-node, 2x AMD EPYC 7763 processor (64 core) on Supermicro AS-2124US-TNRP with 1024GB DDR4 memory (16x64 GB), microcode 0xa01173, SMT On, Boost On, NPS=2, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, Intel® Ethernet Network Adapter E810-2CQDA2 (total 200Gbps), FIO v3.30, SPDK 22.05, large: 128K QD64 sequential read, small: 16k QD256 random read, tested by Intel September 2022. SPDK NVMe-o-TCP Tested by Intel on September 2022.
[P2] On the Tencent custom SKUs, compared with the previous generation platform, the Intel® AMX instruction set of the 4th Gen Intel® Xeon® Scalable processor improves the video enhancement performance by 1.86 times. 4th Generation Intel® Xeon® Platinum processor Xeon 83xxc: Test by Intel as of 12/12/22. 1-node, 2x Intel® Xeon® Platinum 83xxc Tencent Custom CPU@ 3.20GHz, 28 cores, HT On, Turbo On, Total Memory 512GB (16x32GB DDR4 3200 MT/s [3200 MT/s]); 1039104MB (4x259776MB Logical non-volatile device 3200 MT/s [3200 MT/s]), BIOS 05.01.01, microcode 0xd0002a0, 2x Intel® Ethernet Converged Network Adapter X550-T2, 2x 1.8T INTEL SSDPE2MX020T7, CentOS Linux release 8.5.2111, 4.18.0-348.7.1.el8_​5.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15), Video Preprocessing, PyTorch1.12/IPEX1.12.300 Xeon 84xxc: Test by Intel as of 12/12/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 84xxc Tencent Custom CPU, 48 cores, HT On, Turbo On, Total Memory 256GB (16x16GB DDR5 4800 MT/s [4800 MT/s]), BIOS EGSDCRB1.KEH.0082.D13.2207081008, microcode 0xaa000060, 1x I210 Gigabit Network Connection, 1x 447.1G INTEL SSDSCKKB48, CentOS Linux release 8.4.2105, 5.16.0, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15), Video Preprocessing, PyTorch1.12/IPEX1.12.300 Video enhancement/FPS Tested by Intel as of December 12, 2022.
[P3] On the Tencent custom SKUs, Intel® AMX instruction set of the 4th Gen Intel® Xeon® Scalable processor improves the YOLOv7 performance by 1.9 times​ over the 3rd Gen Intel® Xeon® Scalable processor. 4th Generation Intel® Xeon® Platinum processor Xeon 83xxc: Test by Intel as of 12/12/22. 1-node, 2x Intel® Xeon® Platinum 83xxc Tencent Custom CPU@ 3.20GHz, 28 cores, HT On, Turbo On, Total Memory 512GB (16x32GB DDR4 3200 MT/s [3200 MT/s]); 1039104MB (4x259776MB Logical non-volatile device 3200 MT/s [3200 MT/s]), BIOS 05.01.01, microcode 0xd0002a0, 2x Intel® Ethernet Converged Network Adapter X550-T2, 2x 1.8T INTEL SSDPE2MX020T7, CentOS Linux release 8.5.2111, 4.18.0-348.7.1.el8_​5.x86_​64, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15), Video Preprocessing, PyTorch1.12/IPEX1.12.300 Xeon 84xxc : Test by Intel as of 12/12/22. 1-node, 2x Intel® Xeon® Platinum 84xxc Tencent Custom CPU, 48 cores, HT On, Turbo On, Total Memory 256GB (16x16GB DDR5 4800 MT/s [4800 MT/s]), BIOS EGSDCRB1.KEH.0082.D13.2207081008, microcode 0xaa000060, 1x I210 Gigabit Network Connection, 1x 447.1G INTEL SSDSCKKB48, CentOS Linux release 8.4.2105, 5.16.0, gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15), Video Preprocessing, PyTorch1.12/IPEX1.12.300 YOLOv7/FPS Tested by Intel on December 12, 2022.
[P4] 2x further speedup in time-to-insight based on average performance on training mixed shallow and deep neural networks and ML models. 4th Generation Intel® Xeon® Platinum processor Aible: ​2x further speedup in time-to-insight based on average performance of these two metrics: Intel® Xeon® Platinum 8480+ w/AMX delivers up to 2.79x faster NN model training than Intel® Xeon® Platinum 8380 and pre-production platform with Intel® Xeon® Platinum 8480+ delivers up to 1.56x faster LightGBM model training than Intel® Xeon® Platinum 8380. Baseline: 3rd Gen Intel® Xeon® Scalable Processor: Test by Intel as of November 10, 2022. 1-node with 2x Intel® Xeon® Platinum 8380 CPU @ 2.30GHz, 40 cores/socket, 2 sockets, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 3200 MHz [run @ 3200 MHz] ), Dell® PowerEdge R750, 1.6.5, 0xd000375, Rocky Linux 8.6 (Green Obsidian)​, 4.18.0-372.32.1.el8_​6.x86_​64​, gcc 8.5.0, Sapphire Rapids AI DL Software Package Customer Preview III (NDA Release) Tensorflow 2.10, intel/intel-optimized-ml:xgboost 1.4.2, Python 3.8.10 [NN Models], Intel® Distribution for Python 3.7.10 [LightGBM Models] Intel Numpy1.22.4, LightGBM 3.3.3, Kubespray 2.20.0, Multus 3.8, Calico 3.23.3, containerd 1.6.8, Docker Registry 2.8.1, Kubernetes 1.24.6 (TopologyManager-Enabled), Kubeflow 1.6.1, DirectPV 3.2.0, Minio 4.5.2, Prometheus 2.39.1, Aible's Propietary AI Workload for Enterprise Insights - NN Models [HiddenLayers/Batchsize/Epochs=5/Probability=0.5], LightGBM Models [Num_​Estimators/Probability]. Model Training Time for Aible's NN Models [FP32]: 519s. ​ 4th Gen Intel® Xeon® Scalable Processor: Test by Intel as of November 10, 2022. 1-node with 2x Intel® Xeon® Platinum 8480+ CPU @ 2.00GHz, 56 cores/socket, 2 sockets, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 M Hz [run @ 4800 MHz] ), Quanta Cloud Technology Inc., QuantaGrid D54Q-2U, 3A06, 0x2b000081, Rocky Linux 8.6 (Green Obsidian)​, 4.18.0-372.32.1.el8_​6.x86_​64​, gcc 8.5.0, Sapphire Rapids AI DL Software Package Customer Preview III (NDA Release) Tensorflow 2.10, intel/intel-optimized-ml:xgboost 1.4.2, Python 3.8.10 [NN Models], Intel® Distribution for Python 3.7.10 [LightGBM Models] Intel Numpy1.22.4, LightGBM 3.3.3, Kubespray 2.20.0, Multus 3.8, Calico 3.23.3, containerd 1.6.8, Docker Registry 2.8.1, Kubernetes 1.24.6 (TopologyManager-Enabled), Kubeflow 1.6.1, DirectPV 3.2.0, Minio 4.5.2, Prometheus 2.39.1, Aible's Propietary AI Workload for Enterprise Insights - NN Models [HiddenLayers/Batchsize/Epochs=5/Probability=0.5], LightGBM Models [Num_​Estimators/Probability]. Model Training Time for Aible's NN Models [BFloat16]: 185.67s. Geomean of performance speedup for Aible LightGBM models on 4th Gen Intel® Xeon® Scalable Processor over 3rd Gen Intel® Xeon® Scalable Processor is 1.56. Aible LightGBM Models, Aible Neural Network Classifier Models Test by Intel as of November 10, 2022.
[P5] 4x faster gen-to-gen NLP inferencing. 4th Generation Intel® Xeon® Platinum processor Fujitsu: BASELINE(ICX): Tested by Intel as of October 2022. 2 socket Intel® Xeon® Platinum Ice Lake 8380 CPU @ 2.30GHz Processor(ICX), 40 cores/socket, HT On, Turbo ON, Total Memory 384GB (12slots/32GB/3200 MT/s DDR4), BIOS: SE5C6200.86B.0022.D64.2105220049, ucode 0xd000375​, Ubuntu 20.04.5 LTS, 5.4.0-126-generic ​,GCC 9.4.0 compiler, Inference Framework: Pytroch 1.12.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace :German-Sentiment-Bert model, Multiple streams, Datatype: FP32. Config1(ICX): Tested by Intel as of October 2022. 2 socket Intel® Xeon® Platinum Ice Lake 8380 CPU @ 2.30GHz Processor(ICX), 40 cores/socket, HT On, Turbo ON, Total Memory 384GB (12slots/32GB/3200 MT/s DDR4), BIOS: SE5C6200.86B.0022.D64.2105220049, ucode 0xd000375​, Ubuntu 20.04.5 LTS, 5.4.0-126-generic ​,GCC 9.4.0 compiler, Inference Framework: OpenVINO 2022.2.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace :German-Sentiment-Bert model, Multiple streams, Datatype: FP32. BASELINE(SPR): Tested by Intel as of October 2022. pre-production platform with 2 socket Intel® Xeon® Platinum 8480+(SPR), 56 cores/socket, HT On, Turbo ON, Total Memory 512GB(16slots/32GB/4800 MT/s DDR4), BIOS: SE5C6200.86B.0022.D64.2105220049, ucode 0x2b000041​, Ubuntu 22.04.1 LTS, 5.15.0-48-generic ​, GCC 9.4.0 compiler, Inference Framework: Pytorch 1.12.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace :German-Sentiment-Bert model, 1 instance/2 socket, Multiple stream, Datatype: FP32. Config1(SPR): Tested by Intel as of October 2022. 2 socket Intel® Xeon® Platinum 8480+(SPR), 56 cores/socket, HT On, Turbo ON, Total Memory 512GB(16slots/32GB/4800 MT/s DDR4), BIOS: SE5C6200.86B.0022.D64.2105220049, ucode 0x2b000041​, Ubuntu 22.04.1 LTS, 5.15.0-48-generic ​, GCC 9.4.0 compiler, Inference Framework: OpenVINO 2022.2.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace :German-Sentiment-Bert model, 1 instance/2 socket, Multiple stream, Datatype: FP32. OPTIMIZED(Config2:SPR): Tested by Intel as of October 2022. 2 socket Intel® Xeon® Platinum 8480+(SPR), 56 cores/socket, HT On, Turbo ON, Total Memory 512GB(16slots/32GB/4800 MT/s DDR4), BIOS: SE5C6200.86B.0022.D64.2105220049, ucode 0x2b000041​, Ubuntu 22.04.1 LTS, 5.15.0-48-generic ​, GCC 9.4.0 compiler, Inference Framework: Intel OpenVINO toolkit 2022.2.0, Sentiment analysis in NLP eCommerce Recommender, Topology: HuggingFace :German-Sentiment-Bert model, 1 instance/2 socket, Multiple stream, Datatype: AMX_​BF16. 2.55x Tested by Intel as of October 2022.
[P6] 62x higher gen-to-gen throughput with Numenta value-add based on 3rd Gen Intel® Xeon® Scalable Processor (Intel AVX-512) without Numenta optimization compared to 4th Gen Intel® Xeon® Scalable Processor with Numenta optimizations. 4th Generation Intel® Xeon® Platinum processor Numenta: BERT-Large: Sequence Length 64, Batch Size 1, throughput optimized 3rd Gen Intel® Xeon® Scalable: Tested by Numenta as of 11/28/2022. 1-node, 2x Intel® Xeon®8375C on AWS m6i.32xlarge, 512 GB DDR4-3200, Ubuntu 20.04 Kernel 5.15, OpenVINO 2022.3, Numenta-Optimized BERT-Large, Sequence Length 64, Batch Size 1 Intel® Xeon® 8480+: Tested by Numenta as of 11/28/2022. 1-node, pre-production platform with 2x Intel® Xeon® 8480+, 512 GB DDR5-4800, Ubuntu 22.04 Kernel 5.17, OpenVINO 2022.3, Numenta-Optimized BERT-Large, Sequence Length 64, Batch Size 1. Conversational AI/throughput Tested by Intel as of November 28, 2022.
[P7] 1.6x faster GNN training (FP32) for 4 th Gen Intel Xeon SP Katana Graph vs. NVIDIA A100. 4th Generation Intel® Xeon® Platinum processor Katana Graph:Distributed GNN Training: 8-node each with: 2x 4th Gen Intel® Xeon® Scalable processor (pre-production Sapphire Rapids >40cores) on Intel pre-production platform and software with 512 GB DDR5 memory, microcode 0x90000c0, HT on, Turbo off, Rocky Linux 8.6, 4.18.0-372.26.1.el8_​6.crt1.x86_​64, 931 GB SSD, 455 TB Luster filesystem with HDR fabric, Katana Graph 0.4.1 vs. DGL 0.9, test by Intel Corporation on 09/19/2022. Single node Graph Partitioning: 1-node, 2x 4th Gen Intel® Xeon® Scalable processor (pre-production Sapphire Rapids >40cores) on Intel pre-production platform and software with 1024 GB DDR5 memory, microcode 0x90000c0, HT on, Turbo off, Rocky Linux 8.6, 4.18.0-372.26.1.el8_​6.crt1.x86_​64, 894 GB SSD, 105 TB Luster filesystem with OPA fabric, DGL 0.9.0 random graph partition on single node, test by Intel Corporation on 08/17/2022. Distributed GNN Training with GPU: 8-node, 2x 3rd Gen Intel® Xeon® Scalable processor with 256 GB DDR4 memory, microcode 0xd000270, HT on, Turbo on, Rocky Linux4.18.0-372.26.1.el8_​6.crt1.x86_​64, 931 GB SSD, 455 TB Luster filesystem with HDR fabric, 2 A100-PCIE-40GB per node, DGL 0.9, test by Intel Corporation on 09/19/2022. Katana Graph/GNN training Test by Intel on September 19, 2022.

Test by Intel on August 17, 2022.

[P8] 3x higher average performance using Deci hardware-aware AI models on BERT-Large and Resnet50 w/accuracy gain. 4th Generation Intel® Xeon® Platinum processor 3x higher average based on geomean of BERT-large and ResNet-50 performance: ResNet50: Test by Intel as of 11/29/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MT/s),BIOS 3A03, ucode 0x2b000021, OS Ubuntu 20.04.5 LTS, kernel 5.15.0-52-generic, ImageNet Benchmark, IPEX==1.13.0, Resnet50, pytorch==1.13.0, intel-openmp==2022.2.1, score 9838 ips @ BS1, 13310 ips @ BS116 DeciNet: Test by Intel as of 11/29/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MT/s),BIOS 3A03, ucode 0x2b000021, OS Ubuntu 20.04.5 LTS, kernel 5.15.0-52-generic, ImageNet Benchmark, IPEX==1.13.0, DeciNet, pytorch==1.13.0, intel-openmp==2022.2.1, score 28998 ips @ BS1, 46288 ips @ BS116 BERT-Large: Test by Intel as of 11/29/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MT/s),BIOS 3A03, ucode 0x2b000021, OS Ubuntu 20.04.5 LTS, kernel 5.15.0-52-generic, SQuADv1.1 Benchmark, IPEX==1.13.0, BERT-Large, sequence length 384, pytorch==1.13.0, intel-openmp==2022.2.1, score 322 ips @ BS1, 380 ips @ BS56 DeciBERT: Test by Intel as of 11/29/22. 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+, 56 cores, HT On, Turbo On, Total Memory 512 GB (16 slots/ 32 GB/ 4800 MT/s),BIOS 3A03, ucode 0x2b000021, OS Ubuntu 20.04.5 LTS, kernel 5.15.0-52-generic, SQuADv1.1 Benchmark, IPEX==1.13.0, DeciBERT, sequence length 384, pytorch==1.13.0, intel-openmp==2022.2.1, score 1052 ips @ BS1, 1296 @ BS56 BERT-Large and Resnet50 Test by Intel as of November 29, 2022.
[P10] 35x better inference performance on short text sequences batch size 1 and 9x better inference performance on short text sequences batch size 8 with Numenta's optimized BERT-Large model running on the latest 4th Gen Intel® Xeon® Scalable processors vs NVIDIA A100 4th Generation Intel® Xeon® Platinum processor 1-node, pre-production platform with 2x Intel® Xeon® 8480+ processor, 512 GB DDR5-4800, Ubuntu 22.04 Kernel 5.17, OpenVINO™ Toolkit 2022.3, Numenta-Optimized BERTLarge, Sequence Length 128, BF16, Batch Size 1; NVIDIA: https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT#inference-performance-nvidia-dgx-a100-1x-a100-40gb Conversational AI/throughput Tested by Numenta as of March 22, 2023.
[P11] 70x better performance inference throughput and 123x better performance inference throughput with 10ms maximum latency with Numenta's optimized BERT-Large model running on the latest 4th Gen Intel® Xeon® Scalable processors vs AMD Milan 4th Generation Intel® Xeon® Platinum processor Numenta: BERT-Large: Sequence Length 64, Batch Size 1, throughput optimized 4th Gen Intel Xeon. AMD Milan: Tested by Numenta as of 11/28/2022. 1-node, 2x AMD EPYC 7R13 on AWS m6a.48xlarge, 768 GB DDR4-3200, Ubuntu 20.04 Kernel 5.15, OpenVINO 2022.3, BERT-Large, Sequence Length 64, FP32, Batch Size 1. 3rd Gen Intel® Xeon® Scalable: Tested by Numenta as of 11/28/2022. 1-node, 2x Intel® Xeon®8375C on AWS m6i.32xlarge, 512 GB DDR4-3200, Ubuntu 20.04 Kernel 5.15, OpenVINO 2022.3, BERT-Large, Sequence Length 64, FP32, Batch Size 1. Intel® Xeon® 8480+: Tested by Numenta as of 11/28/2022. 1-node, pre-production platform with 2x Intel® Xeon® 8480+, 512 GB DDR5-4800, Ubuntu 22.04 Kernel 5.17, OpenVINO 2022.3, Numenta-Optimized BERT-Large, Sequence Length 64, BF16, Batch Size 1. Conversational AI/throughput Tested by Intel as of November 28, 2022.
[P12] Working with Alibaba Cloud, MetaApp achieved 1.64x better AI training performance and 2.6x better AI training performance/cost by moving from cloud instances based on 2nd gen Intel Xeon scalable processors to 4th gen Intel Xeon scalable processors using the same number of virtual cores. 4th-generation-intel-xeon-scalable-processors Based on recommendation system training testing by MetaApp as of May 11, 2023. Configurations: Configuration 1: Alibaba Cloud Elastic Compute Service (ECS) ecs.r6.4xlarge instances with 16 vCPUs (Cascade Lake); memory capacity: 64 GB (DDR4-3200); deep learning (DL) framework: DeepRec; library: oneAPI Deep Neural Network Library (oneDNN); operating system (OS): Alibaba Cloud Linux 3; kernel: 5.10.134-15.al8.x86_​64; MetaApp training workload; instance cost: 2,544.00 RMB/month in China (Beijing) region. Configuration 2: Alibaba Cloud ECS ecs.c8i.4xlarge instances with 16 vCPUs; memory capacity: 32 GB (DDR5-4800); DL framework: DeepRec; library: oneDNN; OS: Alibaba Cloud Linux 3; kernel: 5.10.134-15.al8.x86_​64; MetaApp training workload; instance cost: 1,643.84 RMB/month in China (Beijing) region. MetaApp training workload May 11, 2023
[P13] Working with Alibaba Cloud to migrate their AI inference engine from instances based on 3rd gen Intel Xeon scalable processors to 4th gen Intel Xeon scalable processors, MetaApp reduced their virtual core footprint by 25%, lowering net cost by 22% and staying within 10% of original latency. 4th-generation-intel-xeon-scalable-processors Based on recommendation system inference testing by MetaApp as of April 18, 2023. Configurations: Configuration 1: Alibaba Cloud Elastic Compute Service (ECS) ecs.ebmc7.32xlarge instances with 128 vCPUs (Ice Lake); memory capacity: 256 GB (DDR4-3200); deep learning (DL) framework: DeepRec; library: oneAPI Deep Neural Network Library (oneDNN); operating system (OS): Alibaba Cloud Linux 3; kernel: 5.10.134-15.al8.x86_​64; MetaApp inference workload; instance cost: 12,524.51 RMB/month in China (Beijing) region. Configuration 2: Alibaba Cloud ECS ecs.c8i.24xlarge instances with 96 vCPUs (Sapphire Rapids); memory capacity: 192 GB (DDR5-4800); DL framework: DeepRec; library: oneDNN; OS: Alibaba Cloud Linux 3; kernel: 5.10.134-15.al8.x86_​64; MetaApp inference workload; instance cost: 9,863.05 RMB/month in China (Beijing) region. MetaApp inference workload April 18, 2023
[W2] Up to 1.61x higher throughput with the 4th Gen Intel Xeon Scalable Processor vs. prior generation on DeathStarBench - Hotel Reservation Microservices workload. 4th Generation Intel® Xeon® Platinum processor 8480+: 4 (1master, 3worker)-node, each-node, pre-production platform with 2x Intel® Xeon® Platinum 8360Y on Intel M50CYP2SBSTD with GB (32 slots/ 16GB/ DDR4 3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-52-generic with intel_​iommu=off, 1x 223.6G KINGSTON SA400M8, 8x 2.9T INTEL SSDPE2KE032T8, 2x Ethernet Controller E810-C for QSFP, 2x BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller, DeathStarBench hotelReservation 1.0 [ lianhao/dsbpp_​hotel_​reserve:1.0], Golang 1.17.3, GNU C Library 2.31-13+deb11u2, ice 5.15.0-52-generic, Kubernetes 1.23.6, Containerd 1.6.6 CRI-RM 0.7.0, Cilium: 1.11.7, gRPC-go 1.1, Consul 1.9.2, Memcached 1.6.8, MongoDB 4.4.3, Traffic generator open loop wrk2 included in DSB: mixed-workload_​type_​1.lua, 4 instance, 6 replica/instance, 4 wrk2 instance, 32 wrk2 thread/instance, 1920 wrk2 connection/instance, 44.5k wrk2 input rate/instance., test by Intel on 11/9/2022. 8360Y: 4 (1master, 3worker)-node, each-node, 2x Intel® Xeon® Platinum 8480+ on Intel ArcherCity with GB (32 slots/ 32GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-52-generic with intel_​iommu=off, 1x 54.9G INTEL SSDPEK1A058GA, 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller XXV710 for 25GbE SFP28, DeathStarBench hotelReservation 1.0 [ lianhao/dsbpp_​hotel_​reserve:1.0], Golang 1.17.3, GNU C Library 2.31-13+deb11u2, ice 5.15.0-52-generic, Kubernetes 1.23.6, Containerd 1.6.6 CRI-RM 0.7.0, Cilium: 1.11.7, gRPC-go 1.1, Consul 1.9.2, Memcached 1.6.8, MongoDB 4.4.3, Traffic generator open loop wrk2 included in DSB: mixed-workload_​type_​1.lua, 4 instance, 6 replica/instance, 4 wrk2 instance, 48 wrk2 thread/instance, 1920 wrk2 connection/instance, 70k wrk2 input rate/instance., test by Intel on 11/9/2022. https://github.com/delimitrou/DeathStarBench#publications Death Star Bench - Hotel Reservation Tested by Intel as of November 9, 2022.
[W4] Up to 1.5x higher performance on cloud native benchmark [CloudXPRT Web Microservices] with 4th gen Intel Xeon Platinum 8490H processor vs. prior gen.

Up to 2.4x higher performance on cloud native benchmark [CloudXPRT Web Microservices] with 4th gen Intel Xeon Platinum 8490H processor vs. 5-year old platform.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on Archer City with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x INTEL SSDPF21Q016TB, CloudXPRT v1.20, kubernetes​ v1.16.3​, Golang​ v1.13.1​, Ubuntu (Docker image)​ 20.04​, Redis (Docker image)​ 5.0.8-buster​, Cassandra (Docker image)​ 3.11.6​, Nginx (Docker image)​ v1.17​, test by Intel on 9/19/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x INTEL SSDPF21Q016TB, CloudXPRT v1.20, kubernetes​ v1.16.3​, Golang​ v1.13.1​, Ubuntu (Docker image)​ 20.04​, Redis (Docker image)​ 5.0.8-buster​, Cassandra (Docker image)​ 3.11.6​, Nginx (Docker image)​ v1.17​, test by Intel on 8/25/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on Purley with GB (12 slots/ 32GB/ DDR4 2666) total memory, ucode 0x2006d05, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x INTEL SSDPF2KX038TZ, CloudXPRT v1.20, kubernetes​ v1.16.3​, Golang​ v1.13.1​, Ubuntu (Docker image)​ 20.04​, Redis (Docker image)​ 5.0.8-buster​, Cassandra (Docker image)​ 3.11.6​, Nginx (Docker image)​ v1.17​, test by Intel on 4/21/2022. Intel contributes to the development of benchmarks by participating in, sponsoring, and/or contributing technical support to various benchmarking groups, including the BenchmarkXPRT Development Community administered by Principled Technologies. CloudXPRT - Web Microservices Tested by Intel as of September 19, 2022.

Tested by Intel as of April 21, 2022

[W5] Up to 2.5x higher throughput (RPS), 74% lower P99 latency and 12% lower cpu utilization on 4th Gen Intel Xeon Platinum 8480+ (8c/16t) with 2 QAT devices compared to solution with no acceleration on Istio Envoy Ingress.

Up to 1.7x higher throughput (RPS), 65% lower P99 latency and 36% lower cpu utilization on 4th Gen Intel Xeon Platinum 8480+ (8c/16t) with 1 QAT devices compared to so lution with no acceleration on Istio Envoy Ingress.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ with Intel QAT on Intel ArcherCity with GB (16 slots/ 32GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04.1 LTS, 5.17.0-051700-generic, 1x 54.9G INTEL SSDPEK1A058GA, 1x Ethernet Controller I225-LM, 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller XXV710 for 25GbE SFP28, Nighthawk, gcc version 11.2.0, Docker 20.10.17, Kubernetes v1.22.3, Calico 3.21.4, Istio 1.13.4. DLB SW v 7.8, qatlib is 22.07.1, Nighthawk POD's with response size: 25 PODs each with 1kB/10kB/1MB/mixed size, test by Intel on 10/27/2022. Istio Envoy Ingress with QAT Test by Intel on October 27, 2022.
[W6] Up to 96% lower latency at the same throughput (RPS) with 4th Gen Intel Xeon Scalable processor and Intel DLB vs. software for Istio ingress gateway working on 6Cores/12Threads.

Up to 9% higher throughput (RPS) and up to 25% lower latency (P99) at the same core count (4c/8t) with 4th Gen Intel Xeon Scalable processor vs. prior generation on HTTP/1.1 with Istio-Envoy Ingress in a service mesh environment.

Up to 21% higher throughput (RPS) and up to 56% lower latency (P99) at the same core count (8c/16t) with 4th Gen Intel Xeon Scalable processor vs. prior generation on HTTP/2 with Istio-Envoy Ingress in a service mesh environment.

Up to 63% lower latency and 37% lower CPU utilization at the same core count (8c/16t) and similar throughput (RPS @50ms SLA) with 4th Gen Intel Xeon Scalable processor vs. prior generation on HTTP/1.1 with Istio-Envoy Ingress in a service mesh environment.

Up to 89% lower latency and 57% lower CPU utilization at the same core count (8c/16t) and similar throughput (RPS @50ms SLA) with 4th Gen Intel Xeon Scalable processor vs. prior generation on HTTP/2 with Istio-Envoy Ingress in a service mesh environment.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ with Intel DLB on Intel ArcherCity with GB (32 slots/ 32GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-40-generic, 1x 54.9G INTEL SSDPEK1A058GA, 4x Ethernet Controller E810-C for QSFP, 2x Ethernet Controller XXV710 for 25GbE SFP28, 1x Ethernet Controller I225-LM, Nighthawk, gcc version 11.2.0, Docker 20.10.17, Kubernetes v1.22.3, Calico 3.21.4, Istio 1.13.4. DLB SW v 7.8, qatlib is 22.07.1, Nighthawk-worker run on 40 threads, 15-25 POD's with nighthawk-server and envoy sidecar proxy, 100Gb back-to-back connections between device, aRFS enabled - NIC interrupts pinned to the core with running applications, test by Intel on 10/27/2022. 8360Y: 1-node, 2x Intel® Xeon® Platinum 8360Y on Intel M50CYP2SBSTD with GB (32 slots/ 16GB/ DDR4 3200) total memory, ucode 0xd000375, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-50-generic, 2x 1.8T INTEL SSDPE2KX020T8, 1x Ethernet Controller E810-C for QSFP, 2x BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller, Nighthawk, gcc version 11.2.0, Docker 20.10.17, Kubernetes v1.22.3, Calico 3.21.4, Istio 1.13.4. DLB SW v 7.8, qatlib is 22.07.1, Nighthawk-worker run on 40 threads, 15-25 POD's with nighthawk-server and envoy sidecar proxy, 100Gb back-to-back connections between device, aRFS enabled - NIC interrupts pinned to the core with running applications, test by Intel on 11/3/2022. Istio Envoy Ingress with DLB Test by Intel on October 27, 2022.

Test by Intel on November 3, 2022.

[W7] Up to 2.33x [MTU 1500, 1500B] and 2.63x [MTU 9000, 2400B] higher single core throughput on 4th Gen Intel Xeon Scalable processor with Intel DSA memory copy compared with software memory copy for CNI ,Calico-VPP.

Save up to 3-cores to achieve 100Gbps throughput with 4th Gen Intel Xeon Scalable processor and Intel DSA with DSA memif vs. software memif at MTU 1500 and 1024B frame size for CNI ,Calico-VPP.

Save up to 2-cores to achieve 100Gbps throughput with 4th Gen Intel Xeon Scalable processor and Intel DSA with DSA memif vs. software memif at MTU 9000 and 2400B frame size for CNI ,Calico-VPP.
4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on Intel M50FCP2SBSTD with GB (16 slots/ 32GB/ DDR5 4800) total memory, ucode 0x9000051, HT on, Turbo on, Ubuntu 22.04 LTS, 5.15.0-48-generic, 1x 894.3G Micron_​5300_​MTFD, 3x Ethernet Controller E810-C for QSFP, 2x Ethernet interface, Calico VPP Version 3.23.0, VPP Version 22.02, gcc 8.5.0, DPDK Version 21.11.0, Docker Version 20.10.18, Kubernetes Version 1.23.12, ISIA Traffic Generator 9.20.2112.6, NIC firmware 3.20 0x8000d83e 1.3146.0, ice 5.18.19-051819-generic, Calico VPP Core Number: 1/2/3/4/5/6, VPP L3FWD Core Number: 1/2/3/4/5/6, Protocol: TCP, DSA: 1 instance, 4 engines, 4 work queues, test by Intel on 10/26/2022. Calico VPP with DSA Test by Intel on November 3, 2022.
[W8] 4th Gen Intel Xeon Scalable processor with built-in Intel DSA brings up to 1.95x vSwitch throughput packet sizes above ~800B for 200Gbps bi-directional switching compared to existing software only implementation. 4th Generation Intel® Xeon® Platinum processor 8480+: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8480+ on ArcherCity with GB (16 slots/ 32GB/ DDR5 4800) total memory, ucode 0x2b0000a1, HT on, Turbo off, Ubuntu 22.04 LTS, 5.15.0-48-generic, 1x 745.2G INTEL_​SSDSC2BB800G7, 2x Intel® Ethernet Network Adapter E810-2CQDA2 (total 400Gbps), OVS DPDK, gcc 11.2.0, GLIBC ldd 2.35, DPDK version 22.11-rc0 (+patches, external branch: dma-tracking -7bdd32de18), OVS version 3.0.0 +patches, external branch: dpdk-dma-tracking -52aa5d7d9e), QEMU version 7.1.0, Virtual Machine OS (Kernel) Ubuntu 20.04.3 LTS (5.4.0-99-generic), Application running in VM DPDK (21.11.0) testpmd with Test case: MAC,CSUM forwarding, test by Intel on 10/20/2022. OVS with DSA Test by Intel on October 20, 2022.
[W9] 4th Gen Intel Xeon Platinum 8490H can process up to 1.47x higher transactions per seconds vs. prior generation platform on WordPress TLS 1.3 with CryptoNI.

4th Gen Intel Xeon Platinum 8490H can process up to 1.54x higher transactions per seconds vs. prior generation platform on WordPress TLS 1.3.

4th Gen Intel Xeon Platinum 8490H can process up to 2.5x higher transactions per seconds vs. 5-year refresh platform on WordPress TLS 1.3 with CryptoNI.

4th Gen Intel Xeon Platinum 8490H can process up to 2.75x higher transactions per seconds vs. 5-year refresh platform on WordPress TLS 1.3.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, 2x Intel® Xeon® Platinum 8490H on ArcherCity with GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 1.5T INTEL SSDPF21Q016TB, 1x Ethernet Controller I225-LM, Wordpress 5.6, PHP 8.0.18 (fpm-fcgi), mysqld Ver 10.3.37-MariaDB for Linux on x86_​64, test by Intel on 9/23/2022. . 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with 512 GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000375, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 3.5T INTEL SSDPF2KX038TZ, 2x Ethernet Controller X710 for 10GBASE-T, Wordpress 5.6, PHP 8.0.18 (fpm-fcgi), mysqld Ver 10.3.37-MariaDB for Linux on x86_​64, test by Intel on 9/20/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on PURLEY with 384 GB (12 slots/ 32GB/ DDR4 2666) total memory, ucode 0x2006e05, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, 1x 3.6T INTEL SSDPE2KX040T7, 2x Ethernet Connection X722 for 10GBASE-T, Wordpress 5.6, PHP 8.0.18 (fpm-fcgi), mysqld Ver 10.3.37-MariaDB for Linux on x86_​64, test by Intel on 9/20/2022. WordPress Test by Intel on September 20, 2022.

Test by Intel on September 23, 2022.

[W10] Up to 1.58x higher average media transcode performance on 4th Gen Intel Xeon Platinum 8490H vs. prior generation.

Up to 2.62x higher average media transcode performance on 4th Gen Intel Xeon Platinum 8490H vs. 5-year refresh platform.
4th Generation Intel® Xeon® Platinum processor Geomean of AVC-1080p-fast-avx2, AVC-1080p-medium-avx2, AVC-1080p-veryslow-avx2, SVT-AV1-1080p-12-avx2, SVT-AV1-1080p-12-avx3, SVT-AV1-1080p-5-avx2, SVT-AV1-1080p-5-avx3, SVT-AV1-1080p-8-avx2, SVT-AV1-1080p-8-avx3, SVT-AV1-4k-12-avx2, SVT-AV1-4k-12-avx3, SVT-AV1-4k-8-avx2, SVT-AV1-4k-8-avx3, SVT-HEVC-1080p-preset1-avx2, SVT-HEVC-1080p-preset5-avx2, SVT-HEVC-1080p-preset5-avx3, SVT-HEVC-1080p-preset9-avx2, SVT-HEVC-4k-preset1-avx3, SVT-HEVC-4k-preset5-avx3, SVT-HEVC-4k-preset9-avx2, SVT-HEVC-4k-preset9-avx3, x265-1080p-medium-avx2, x265-1080p-medium-avx3, x265-4k-veryslow-avx2 8490H:1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on ArcherCity with 1024 GB (16 slots/ 64GB/ DDR5 4800) total memory, ucode 0x2b000070, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, FFMPEG Media Transcode, x264 Version 0.164.x, x265 Version Release_​3.5, SVT-HEVC Version v1.3.0, SVT-AV1 Version v0.8.7, test by Intel on 9/19/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with 512 GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0x0D000375, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, FFMPEG Media Transcode, x264 Version 0.164.x, x265 Version Release_​3.5, SVT-HEVC Version v1.3.0, SVT-AV1 Version v0.8.7, test by Intel on 9/20/2022. 8180: 1-node, 2x Intel® Xeon® Platinum 8180 on Purley with 384 GB (12 slots/ 32GB/ DDR4 2666) total memory, ucode 0x2006e05, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-47-generic, FFMPEG Media Transcode, x264 Version 0.164.x, x265 Version Release_​3.5, SVT-HEVC Version v1.3.0, SVT-AV1 Version v0.8.7, test by Intel on 9/20/2022. Media Transcode Test by Intel on September 20, 2022.

Test by Intel on September 19, 2022.

[W11] Up to 1.4x higher virtualization performance with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. prior generation platform.

Up to 2.78x higher virtualization performance with the 4th Gen Intel® Xeon® Platinum 8490H processor vs. 5-year refresh platform.
4th Generation Intel® Xeon® Platinum processor 8490H: 1-node, pre-production platform with 2x Intel® Xeon® Platinum 8490H on ArcherCity with 2048 GB (32 slots/ 64GB/ DDR4 4800[4400]) total memory, ucode 0x2b0000a1, HT on, Turbo on, RHEL8.6, 4.18.0-372.9.1.el8.x86_​64, 6xP5800 3.2T, 2x E810, virtualization workload, qemu-kvm-6.2.0-12, test by Intel on 10/26/2022. 8380: 1-node, 2x Intel® Xeon® Platinum 8380 on Whitley with 512 GB (16 slots/ 32GB/ DDR4 3200) total memory, ucode 0xd000363, HT on, Turbo on, Ubuntu 22.04.1 LTS, 5.15.0-46-generic, 1x 1.5T INTEL SSDPF21Q016TB, 2x Ethernet Controller X710 for 10GBASE-T, virtualization workload, qemu-kvm-6.2.0-11, test by Intel on 8/28/2022. 8180:1-node, 2x Intel® Xeon® Platinum 8180 on S2600WFD with 1536 GB (24 slots/ 64GB/ DDR4 2666) total memory, ucode 0x2006e05, HT on, Turbo on, Red Hat Enterprise Linux 8.6 (Ootpa), 4.18.0-372.9.1.el8.x86_​64, 1x 372.6G INTEL SSDSC2BA40, 1x 745.2G INTEL SSDSC2BA80, 3x 3.7T INTEL SSDPEDKX040T7, 4x Ethernet Controller XL710 for 40GbE QSFP+, 114x Ethernet Virtual Function 700 Series, 2x Intel® Ethernet Controller X710 (for 10GBASE-T), virtualization workload, qemu-kvm-6.2.0-11, test by Intel on 10/26/2022. Virtualization Test by Intel on October 26, 2022.

Test by Intel on August 28, 2022.

[E1] 2.9x average performance per watt efficiency improvement for targeted workloads utilizing built-in accelerators compared to the previous generation.

2.01x average performance per watt efficiency improvement for RocksDB (IAA vs. ZTD)

1.12x average performance per watt efficiency improvement for ClickHouse (IAA vs. LZ4)

1.26x average performance per watt efficiency improvement for ClickHouse (IAA vs. ZTD)

1.92x average performance per watt efficiency improvement for SPDK database request proxies (DSA vs. out of box)

3.18x average performance per watt efficiency improvement for SPDK large media (DSA vs. out of box)

8x average performance per watt efficiency improvement for Real Time Image Recognition ResNet-50 (AMX vs. FP32)

9.76x average performance per watt efficiency improvement for Batch Image Recognition ResNet-50 (AMX vs. FP32)

14.21x average performance per watt efficiency improvement for Real Time Object Detection SSD-ResNet-34 (AMX vs. FP32)

13.53x average performance per watt efficiency improvement for Batch Object Detection SSD-ResNet-34 (AMX vs. FP32)

1.22x average performance per watt efficiency improvement for NGINX (QAT vs. out of box)

28.85x average performance per watt efficiency improvement for QATzip (QAT vs. zlib)

1.61x average performance per watt efficiency improvement for Linpack (AVX-512 vs. AVX2)

4th Generation Intel® Xeon® Platinum processor Geomean of following workloads: RocksDB (IAA vs. ZTD), ClickHouse (IAA vs. ZTD), SPDK large media and database request proxies (DSA vs. out of box), Image Classification ResNet-50 (AMX vs. VNNI), Object Detection SSD-ResNet-34 (AMX vs. VNNI), QATzip (QAT vs. zlib)

RocksDB

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.2.1,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022.

ClickHouse

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), Number of IAA device utilized=8(2 sockets active), on pre-production Intel platform and software, HT On, Turbo On, SNC off, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21, accel-config-v3.4.6.4, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel November 2022.

SPDK

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), HT On, Turbo On, SNC Off, microcode 0xd000375, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022.

ResNet-50

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), on pre-production SuperMicro SYS-221H-TNR with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000c0, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=Resnet 50 v1_​5, best scores achieved: BS1 FP32 8 cores/instance (max. 15ms SLA), BS1 INT8 2 cores/instance (max. 15ms SLA), BS1 AMX 1 core/instance (max. 15ms SLA), BS16 FP32 5 cores/instance, BS16 INT8 5 cores/instance, BS16 AMX 5 cores/instance, using physical cores, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=Resnet 50 v1_​5, best scores achieved: BS1 FP32 8 cores/instance (max. 15ms SLA), BS1 INT8 2 cores/instance (max. 15ms SLA), BS16 FP32 5 cores/instance, BS16 INT8 5 cores/instance, using physical cores, tested by Intel November 2022.

SSD-ResNet-34

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), Intel platform with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=SSD-ResNet34, best scores achieved: BS1 FP32 60 cores/instance (max. 100ms SLA), BS1 INT8 4 cores/instance (max. 100ms SLA), BS1 AMX 4 core/instance (max. 100ms SLA), BS8 FP32 8 cores/instance, BS2 INT8 1 cores/instance, BS2 AMX 1 cores/instance, using physical cores, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model= SSD-ResNet34, best scores achieved: BS1 FP32 40 cores/instance (max. 100ms SLA), BS1 INT8 10 cores/instance (max. 100ms SLA), BS16 FP32 4 cores/instance, BS16 INT8 1 cores/instance, using physical cores, tested by Intel November 2022.

QAT.zip

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon Scalable Processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), QAT device utilized=8(2 sockets active), on pre-production Intel platform and software with DDR5 memory Total 1024GB (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, QAT v20.l.0.9.1, QATzip v1.0.9, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable Processors( 40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, QAT v20.l.0.9.1, QATzip v1.0.9, tested by Intel November 2022.

Intel distribution for Linpack

New Configuration AVX512, Baseline AVX2: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core), on pre-production Supermicro SYS-221H-TNR and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000c0, HT On, Turbo On, SNC 4, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, 10GbE x540-AT2, One API BaseKit 2022.2.0.262, One API HPC 2022.2.0.191, Linpack ver 2.3, tested by Intel November 2022.

Energy Efficiency: Perf/watt Tested by Intel as of November 2022.
[E2] 3.9x average performance per watt efficiency improvement for targeted workloads utilizing built-in accelerators. 4th Generation Intel® Xeon® Platinum processor Geomean of following workloads: RocksDB (IAA vs. ZTD), ClickHouse (IAA vs. ZTD/LZ4), Linpack (AVX-512 vs. AVX2), SPDK large media and database request proxies (DSA vs. out of box), Image Classification ResNet-50 (AMX vs. VNNI), Object Detection SSD-ResNet-34 (AMX vs. VNNI), NGINX TLS Handshake (QAT vs. out of box), QATzip (QAT vs. zlib) RocksDB 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.2.1,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022. ClickHouse 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), Number of IAA device utilized=8(2 sockets active), on pre-production Intel platform and software, HT On, Turbo On, SNC off, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.1.21, accel-config-v3.4.6.4, gcc 11.2, Clickhouse 21.12, Star Schema Benchmark, tested by Intel November 2022. LINPACK 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core), on pre-production SuperMicro SYS-221H-TNR and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000c0, HT On, Turbo On, SNC 4, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, One API BaseKit 2022.2.0.262, One API HPC 2022.2.0.191, Linpack ver 2.3, tested by Intel November 2022. SPDK 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022. ResNet-50 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), on pre-production SuperMicro SYS-221H-TNR with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000c0, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=Resnet 50 v1_​5, best scores achieved: BS1 FP32 8 cores/instance (max. 15ms SLA), BS1 INT8 2 cores/instance (max. 15ms SLA), BS1 AMX 1 core/instance (max. 15ms SLA), BS16 FP32 5 cores/instance, BS16 INT8 5 cores/instance, BS16 AMX 5 cores/instance, using physical cores, tested by Intel November 2022. SSD-ResNet-34 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), Intel platform with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=SSD-ResNet34, best scores achieved: BS1 FP32 60 cores/instance (max. 100ms SLA), BS1 INT8 4 cores/instance (max. 100ms SLA), BS1 AMX 4 core/instance (max. 100ms SLA), BS8 FP32 8 cores/instance, BS2 INT8 1 cores/instance, BS2 AMX 1 cores/instance, using physical cores, tested by Intel November 2022. QAT.zip 1-node, 2x pre-production 4th Gen Intel® Xeon Scalable Processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), QAT device utilized=8(2 sockets active), on pre-production Intel platform and software with DDR5 memory Total 1024GB (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, QAT v20.l.0.9.1, QATzip v1.0.9, tested by Intel November 2022. NGINX TLS Handshake QAT Accelerator: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel QuickAssist Accelerator (Intel QAT), Number of QAT device utilized=4(1 socket active), on pre-production Intel platform and software with DDR5 memory total 1024GB (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel® Ethernet Network Adapter E810-2CQDA2 (100Gbps), QAT engine v0.6.14, QAT v20.l.0.9.1, NGINX 1.20.1, OpenSSL 1.1.1l, IPP crypto v2021_​5, IPSec v1.1, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, 65K CPS target SLA, tested by Intel November 2022. Out of box configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel QuickAssist Accelerator (Intel QAT), Number of QAT device utilized=0, on pre-production Intel platform and software with DDR5 memory total 1024GB (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel® Ethernet Network Adapter E810-2CQDA2 (100Gbps), NGINX 1.20.1, OpenSSL 1.1.1l, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, 65K CPS target SLA, tested by Intel November 2022. Energy Efficiency: Perf/watt Tested by Intel as of November 2022.
[E3] 9.6% wall power savings and 11% higher performance per watt on 4th Gen Intel Xeon SP with negligible impact from Optimized Power Mode vs. default BIOS configuration on IPSec with an SLA of 200Gbps. 4th Generation Intel® Xeon® Platinum processor 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel QuickAssist Accelerator (Intel QAT), QAT device utilized= 4(1 socket active), on pre-production Intel ® platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel® Ethernet Network Adapter E810-2CQDA2 (200Gbps), QAT v20.l.0.9.1, DPDK v21.11, IPsec v1.1, VPP 22.02, nasm v2.14.02, AES 128 GCM, VAES instructions, 200Gbps target SLA, tested by Intel November 2022. Energy Efficiency: Perf/watt Tested by Intel in November 2022.
[E4] 19% wall power savings and 22% higher performance per watt on 4th Gen Intel Xeon SP with 2% performance impact from Optimized Power Mode vs. default BIOS configuration on NGINX TLS Key Exchange with a SLA of 65K connections per second. 4th Generation Intel® Xeon® Platinum processor 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable Processor (60 cores) with integrated Intel QuickAssist Accelerator (Intel QAT), Number of QAT device utilized=4(1 socket active), on pre-production Intel platform and software with DDR5 memory total 1024GB (16x64 GB), microcode 0x2b0000a1, HT On, Turbo Off, SNC Off, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel® Ethernet Network Adapter E810-2CQDA2 (100Gbps), QAT engine v0.6.14, QAT v20.l.0.9.1, NGINX 1.20.1, OpenSSL 1.1.1l, IPP crypto v2021_​5, IPSec v1.1, TLS 1.3 AES_​128_​GCM_​SHA256, ECDHE-X25519-RSA2K, 65K CPS target SLA, tested by Intel November 2022. Energy Efficiency: Perf/watt Tested by Intel in November 2022.
[E5] 9.3% wall power savings and 9% higher performance per watt on 4th Gen Intel Xeon SP with 1% performance impact from Optimized Power Mode vs. default BIOS configuration on SPDK NVMe TCP 128K block size with QD=64. 4th Generation Intel® Xeon® Platinum processor 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, Intel® Ethernet Network Adapter E810-2CQDA2 (200 Gbps), FIO v3.30, SPDK 22.05, tested by Intel November 2022. Energy Efficiency: Perf/watt Tested by Intel in November 2022.
[E6] With new Optimized Power Mode in the platform BIOS, you can achieve up to 20% socket power savings with a less than 5% performance impact for selected workloads. SHORT Version: Saves up to 20% CPU power at less than 5% performance impact for selected workloads.

Saves power where customers tend to run (~30-40% utilization) up to 70W per socket at low utilization.
4th Generation Intel® Xeon® Platinum processor 1-node, Intel Reference Validation Platform, pre-production platform with 2x Intel® Xeon 8480+ (56C, 2GHz, 350W TDP), HT On, Turbo ON, Total Memory: 1 TB (16 slots/ 64GB/ 4800 MHz), 1x P4510 3.84TB NVMe PCIe Gen4 drive, BIOS: 0091.D05, (ucode:0x2b0000c0), CentOS Stream 8, 5.15.0-spr.bkc.pc.10.4.11.x86_​64, Java Perf/Watt w/ openjdk-11+28_​linux-x64_​bin, 112 instances, 1550MB Initial/Max heap size, Tested by Intel as of Oct 2022. Energy Efficiency: Perf/watt Tested by Intel as of October 2022.
[E7] AI - 55% lower TCO by deploying fewer 4th Gen Intel® Xeon® processor-based servers to meet the same performance requirement. 4th Generation Intel® Xeon® Platinum processor A More Cost-Efficient Server Architecture using ResNet50 Image Classification New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable 8490H processor (60 core) with Intel® Advanced Matrix Extensions (Intel AMX), on pre-production SuperMicro SYS-221H-TNR with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000c0, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=Resnet 50 v1_​5, best scores achieved: BS1 AMX 1 core/instance (max. 15ms SLA), using physical cores, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable 8380 Processor ( 40 cores) on SuperMicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), microcode 0xd000375, HT On, Turbo On, SNC Off, CentOS Stream 8, 5.19.16-301.fc37.x86_​64, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Intel TF 2.10, AI Model=Resnet 50 v1_​5, best scores achieved: BS1 INT8 2 cores/instance (max. 15ms SLA), using physical cores, tested by Intel November 2022. For a 50 server fleet of 3rd Gen Xeon 8380 (RN50 w/DLBoost), estimated as of November 2022: CapEx costs: $1.64M OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $739.9K Energy use in kWh (4 year, per server): 44627, PUE 1.6 Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394 For a 17 server fleet of 4th Gen Xeon 8490H (RN50 w/AMX), estimated as of November 2022: CapEx costs: $799.4K OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $275.3K Energy use in kWh (4 year, per server): 58581, PUE 1.6 Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394 TCO Tested by Intel in November 2022.
[E8] Database - 52% lower TCO by deploying fewer 4th Gen Intel® Xeon® processor-based servers to meet the same performance requirement. 4th Generation Intel® Xeon® Platinum processor

A More Cost-Efficient Server Architecture using RocksDB

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® Scalable 8490H Processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800),  microcode 0x2b0000a1, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.2.1,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel® Xeon® Scalable 8380 Processor ( 40 cores) on SuperMicro SYS-220U-TNR, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1x3.84TB P5510 NVMe, Intel® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic,  ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel  November 2022.

For a 50 server fleet of 3rd Gen Xeon 8380 (RocksDB), estimated as of November 2022:

CapEx costs: $1.64M

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $677.7K

Energy use in kWh (4 year, per server): 32181, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

 

For a 18 server fleet of 4th Gen Xeon 8490H (RockDB w/IAA), estimated as of November 2022:

CapEx costs: $846.4K

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $260.6K

Energy use in kWh (4 year, per server): 41444, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

TCO Tested by Intel  November 2022.
[E9] HPC - 66% lower TCO by deploying fewer Intel® Xeon® CPU Max processor-based servers to meet the same performance requirement 4th Generation Intel® Xeon® Platinum processor

A More Cost-Efficient Server Architecture using OpenFOAM

New Configuration: 1-node, 2x pre-production 4th Gen Intel® Xeon® CPU Max Series (56 cores) on pre-production Intel platform and software, HT On, Turbo On, SNC4 mode, Total Memory 128 GB (8x16GB HBM2 3200MT/s), microcode 0x2c000020, 1x3.5TB INTEL SSDPF2KX038TZ NVMe, CentOS Stream 8, 5.19.0-rc6.0712.intel_​next.1.x86_​64+server, OpenFOAM 8, Motorbike 20M @ 250 iterations, Motorbike 42M @ 250 iterations, Tools: ifort:2021.6.0, icc:2021.6.0, impi:2021.6.0, tested by Intel December 2022. Baseline:  1-node, 2x production 3rd Gen Intel® Xeon® Scalable 8380 Processor ( 40 cores) on SuperMicro SYS-220U-TNR, HT On, Turbo On, 512GB (16x32GB DDR4 3200 MT/s), microcode 0xd000375, 1x2.9TB INTEL SSDPE2KE032T8 NVMe, CentOS Stream 8, 4.18.0-408.el8.x86_​64, OpenFOAM 8, Motorbike 20M @ 250 iterations, Motorbike 42M @ 250 iterations, Tools: ifort:2021.6.0, icc:2021.6.0, impi:2021.6.0, tested by Intel December 2022

 

For a 50 server fleet of 3rd Gen Xeon 8380 (OpenFOAM), estimated as of December 2022:

CapEx costs: $1.50M

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $780.3K

Energy use in kWh (4 year, per server): 52700, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

 

For a 16 server fleet of Intel® Xeon® CPU Max Series 56 core, estimated as of December 2022:

CapEx costs: $507.2K

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $274.9K

Energy use in kWh (4 year, per server): 74621, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

TCO Tested by Intel December 2022
[E10] Storage – 60% lower TCO by deploying fewer 4th Gen Intel® Xeon® processor-based servers to meet the same performance requirement. 4th Generation Intel® Xeon® Platinum processor

New Configuration: 1-node, 2x pre-production 4th Gen Intel Xeon Scalable processor (60 core) with integrated Intel Data Streaming Accelerator (Intel DSA), DSA device utilized=1(1 active socket), on pre-production Intel platform and software with 1024GB DDR5 memory (16x64 GB), microcode 0x2b0000a1, 10GbE x540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel® SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel® Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel Xeon Scalable Processors( 40 cores) on Supermicro SYS-220U-TNR, DDR4 memory total 1024GB (16x64 GB), HT On, Turbo On, SNC Off, microcode 0xd000375, 10GbE x540-AT2, Ubuntu 22.04.1 LTS, 5.15.0-52-generic, 1x 1.92TB Intel SSDSC2KG01, 4x 1.92TB Samsung PM1733, 1x Intel Ethernet Network Adapter E810-2CQDA2, 2x100GbE, FIO v3.30, SPDK 22.05, tested by Intel November 2022.

For a 50 server fleet of 3rd Gen Xeon 8380 (SPDK), estimated as of November 2022:

CapEx costs: $1.77M

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $630.6K

Energy use in kWh (4 year, per server): 22762, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

For a 15 server fleet of 4th Gen Xeon 8490H (SPDK w/DSA), estimated as of November 2022:

CapEx costs: $743.8K

OpEx costs (4 year, includes power and cooling utility costs, infrastructure and hardware maintenance costs): $220.1K

Energy use in kWh (4 year, per server): 43387, PUE 1.6

Other assumptions: utility cost $0.1/kWh, kWh to kg CO2 factor 0.42394

TCO Tested by Intel in November 2022.
[E11] Up to 5:1 consolidation with 75% TCO reduction with 4th Gen Intel Xeon processors  4th Generation Intel® Xeon® Platinum processor

Calculations as of March 28, 2023 based on the Intel® Node TCO & Power Calculator using default cost, power and TCO assumptions over a 5 year TCO horizon comparing replacing 50 older servers with Intel Xeon 4110 processors  with new servers using new Intel Xeon 5420+ processors.  Results may vary. 

Performance measurements based on published SPECrate®2017_​int_​base on spec.org as of March 28, 2023

4110: https://www.spec.org/cpu2017/results/res2020q4/cpu2017-20201015-24218.html

5420+: https://www.spec.org/cpu2017/results/res2023q1/cpu2017-20230130-33925.html]

TCO Calculations as of March 28, 2023