System 2: Intel® Core™ Ultra 9 185H 64GB Ubuntu 24.04.3 LTS

System 2: ingestion rate: 148 records/s

System 1: Latency: 0.75ms

System 2: Latency: 1.28ms

System 2: Intel® Core™ Ultra 7 155H 32GB Ubuntu 22.04 MVTec OpenVino AI2 Plugin

System 2: DeepOCR model: 468.96ms

System 1: GC-AD model: 5.26ms

System 2: GC-AD model: 24.08ms

System 2: Name: orin, CPU: 12-core Arm Cortex-A78AE v8.2 64-bit CPU 3MB L2 + 6MB L3 @ 2.2GHz, GPU: NVIDIA Ampere architecture with 2048 NVIDIA® CUDA® cores and 64 Tensor cores @ 1.3GHz, Memory: 64GB 256-bit LPDDR5 @ 204.8 GB/s, Storage: 64GB eMMC 5.1, Accelerator: 2× NVDLA v2.0, PVA v2.0, Accelerator: APE, NVENC, NVDEC, NVJPG, NVJPG1, VIC, SE, OS: Ubuntu 22.04.5 LTS, Kernel: 5.15.148-tegra, Jetpack: 6.2.1, Deepstream: 7.0, CUDA: 12.6, TensorRT: v100300, Jetson clocks: Default, NVP modes: 0 - MAXN.

System 2: DeepSeek Ilama-8B first token latency measurement: 543

System 2: Name: Orin AGX 64GB, CPU: 12-core Arm Cortex-A78AE v8.2 64-bit CPU 3MB L2 + 6MB L3 @ 2.2GHz, GPU: NVIDIA Ampere architecture with 2048 NVIDIA® CUDA® cores and 64 Tensor cores @ 1.3GHz, Memory: 64GB 256-bit LPDDR5 @ 204.8 GB/s, Storage: 64GB eMMC 5.1, Accelerator: 2× NVDLA v2.0, PVA v2.0, Accelerator: APE, NVENC, NVDEC, NVJPG, NVJPG1, OFA, VIC, SE, OS: Ubuntu 22.04.5 LTS, Kernel: 5.15.148-tegra, Jetpack: 6.2.1, CUDA: 12.6, Jetson clocks: Default, NVP modes: 0 - MAXN.

System 2: 854 inference/sec

As estimated by Action Chunking Transformer (ACT) Bimanual insertion model (using four cameras - 640x480 with FP32)

System 2: Intel Core i5-10210U NVIDIA GeForce RTX 4060 (8GB VRAM) 32G DDR4 Ubuntu 24.04.3 LTS

System 2: 2234.11ms

System 2: 60W (GPU only)

Electricity: $0.15 per kWh. U.S. national average commercial electricity rate provided by US Energy Info Admin, which was approximately $0.1255/kWh in 2022, choosing $0.15/kWh as a conservative estimate that rounds upward to reflect potential regional variability and rate increases. Heavy use: 24×7 = 8,760 h/yr × 5 yrs Power draw: System 2 = 60 W; System 1 = 22.6 W

System 1: $999 (estimate based on prior gen Intel Core Ultra 5 or ARL H-based Asus NUC 15 Pro [US$799, source: https://shop.asus.com/us/asus-nuc-15-pro-cnf.html plus 15% premium.)

System 2: $1499 (estimate based on https://www.newegg.com/p/1VK-01YR-00218?item=9SIACP1K370559)

Hardware savings: $1499 - $999 = $500

5-Year Energy Cost:

System 1: 0.0226 × 8760 × 0.15 × 5 = $148.40

System 2: 0.06 × 8760 ×0.15 × 5 = $394.2

Energy savings = $245.80

Total TCO:

System 1: $1,147.40

System 2: $1,893.20

Total savings = $745.80 or 39.4%

System 2: Intel Core i5-10210U NVIDIA GeForce RTX 4060 (8GB VRAM) 32G DDR4 Ubuntu 24.04.3 LTS

System 2: 2234.11ms

System 2: 60W (GPU only)

Electricity: $0.15 per kWh. U.S. national average commercial electricity rate provided by US Energy Info Admin, which was approximately $0.1255/kWh in 2022, choosing $0.15/kWh as a conservative estimate that rounds upward to reflect potential regional variability and rate increases.

Heavy use: 24×7 = 8,760 h/yr × 5 yrs

Power draw: System 2 = 60 W; System 1 = 22.3 W

System 1: $1150 (estimate based on prior gen Intel Core Ultra 7 or ARL H-based Asus NUC 15 Pro [US$999, source: https://shop.asus.com/us/asus-nuc-15-pro-cnf.html plus 15% premium.)

System 2: $1499 (estimate based on https://www.newegg.com/p/1VK-01YR-00218?item=9SIACP1K370559)

Hardware savings: $1499 - $1150 = $349

5-Year Energy Cost:

System 1: 0.0223 × 8760 × 0.15 × 5 = $146.70

System 2: 0.06 × 8760 × 0.15 × 5 = $394.2

Energy savings = $247.50

Total TCO:

System 1: $1296.70

System 2: $1,893.20

Total savings = $596.50 or 31.5%

System 2: Nvidia Jetson AGX Orin Ampere GPU 32GB Ubuntu 22.04

System 2: 1.94 RTF

System 2: 3.03 token/sec

System 2: 1.95 FPS

System 2: 17W

System 2: $1599 (estimate based on https://www.seeedstudio.com/AGX-Orin-32GB-H01-Kit-p-5569.html) Hardware savings: $1,599 - $1,150 = $449 or 26%

System 2: Nvidia Jetson AGX Orin Ampere GPU 32GB Ubuntu 22.04

System 2: 4.62 RTF

System 2: 12.77 token/sec

System 2: 2.72 FPS

System 2: 55W

Electricity: $0.15 per kWh. U.S. national average commercial electricity rate provided by US Energy Info Admin, which was approximately $0.1255/kWh in 2022, choosing $0.15/kWh as a conservative estimate that rounds upward to reflect potential regional variability and rate increases.

Use: 12 h/day

Power draw: System 2 = 55 W; System 1 = 25 W

System 1: $1150 (estimate based on prior gen Intel Core Ultra 7 or ARL H-based Asus NUC 15 Pro [US$999, source: https://shop.asus.com/us/asus-nuc-15-pro-cnf.html plus 15% premium.)

System 2: $3499 (estimate based on https://www.seeedstudio.com/NVIDIA-Jetson-AGX-Thor-Developer-Kit-p-9965.html)

Hardware savings: $3499 - $1150 = $2349

5 Year Energy Cost:

System 1: 0.025 kW × 21,900 h = 547.5 kWh × 0.15 = $82.125

System 2: 0.055 kW × 21,900 h = 1,204.5 kWh x 0.15= $180.675

Total TCO:

System 1: $1,150 + $82.13 = $1,232.13

System 2: $3,499 + $180.68 = $3,679.68

Total savings = $3,679.68−$1,232.13 = $2,447.55 or 66.5%

System 2: c2-standard-4 (4 vCPUs) Intel Cascade Lake (Google Cloud instance) No 16G GCP VM Debian 11 + Ubuntu24.04 Docker container

System 2: 305ms

System 2: 53.5ms

System 2: 139.5ms

System 2: 77 sec

System 2: Quad-core ARM® Cortex®-A57 MPCore processor 128-core NVIDIA Maxwell™ architecture GPU 4GB 64-bit LPDDR4 Ubuntu 22.04 LTS

System 2: 0.4 images/sec

System 2: Nvidia Jetson AGX Orin Ampere GPU 64GB Ubuntu 22.04

lerobot/pi0_​base Inference latency System 1: 1057.87 ms System 2: 8804.47 ms

lerobot/pi05_​base Inference latency System 1: 1283.32 ms System 2: 10977.88 ms

System 2: $1999 (estimate based on https://www.seeedstudio.com/NVIDIArJetson-AGX-Orintm-64GB-Developer-Kit-p-5641.html)

Hardware savings: $1,999 - $1,150 = $849 or 42%

System 2: Nvidia Jetson AGX Orin Ampere GPU 64GB Ubuntu 22.04

Cartken Panoptic Model Power Efficiency System 1: 25.15 System 2: 38 Efficiency gain: 33.82%

Cartken Panoptic Model Inference/Watt System 1: 393.2 / 25.15 = 15.63 System 2: 330 / 38 = 8.68 Improvement gain: 80%

Cartken Segmentation Model Inference Throughput System 1: 474.2 System 2: 380 Improvement gain: 24.79%

Cartken Segmentation Model Power Efficiency System 1: 25.18 System 2: 38 Efficiency gain: 33.74%

Cartken Segmentation Model Inference/Watt System 1: 474.2 / 25.18 = 18.83 System 2: 380 / 38 = 10 Improvement gain: 88%

Electricity: $0.15 per kWh. U.S. national average commercial electricity rate provided by US Energy Info Admin, which was approximately $0.1255/kWh in 2022, choosing $0.15/kWh as a conservative estimate that rounds upward to reflect potential regional variability and rate increases.

Operating hours: 16 h/day × 365 = 5,840 h/year. 16 hours/day operation, representative of two-shift industrial or campus robotic deployments.

Power draw: System 2 = 38W; System 1 = 25.15W

System 1: $1150 (estimate based on prior gen Intel Core Ultra 7 or ARL H-based Asus NUC 15 Pro [US$999, source: https://shop.asus.com/us/asus-nuc-15-pro-cnf.html plus 15% premium.)

System 2: $1999 (estimate based on https://www.seeedstudio.com/NVIDIArJetson-AGX-Orintm-64GB-Developer-Kit-p-5641.html)

Hardware savings: $1,999 - $1,150 = $849 or 42%

Annual Energy Cost: System 1: 25.15 W × 5,840 h ÷ 1000 = 147 kWh × 0.15 = $22 System 2: 38 W × 5,840 h ÷ 1000 = 222 kWh × 0.15= $33

5 year Energy Cost: System 1: $22 × 5 = $110 System 2: $33 × 5 = $165

Total TCO:

System 1: $1,150 + $110 = $1,260

System 2: $1,999 + $165 = $2,164

Total savings = $2,164−$1,260 = $904 or 42%

System 2: Intel® Core™ i7-14700 Processor NVIDIA GeForce RTX 3090Ti 192 GB DDR5 5600 Ubuntu 22.04 LTS

System 2: 4.22 Watt/Token/Sec

Efficiency gain = 1.97X

System 2 (consist of 2 systems): Intel® Core™ i7-14700 Processor NVIDIA GeForce RTX 3090Ti 192 GB DDR5 5600 Ubuntu 22.04 LTS Intel® Core™ Ultra 7 Processor 255H Integrated Intel Arc 140T GPU 64GB DDR5 5600 Ubuntu 22.04 LTS

System 2: $4700 + $1850 = 6550

TCO savings = $6550 - $3150 = $3400 / 51% lower

System 2: Intel 13600KF NVDIA RTX 5080 64GB DDR4 Windows 11 24H2

Mistral 7B TTFT (128 tokens): System 1: 143 ms System 2: 290 ms Improvement gain: 2X

Power Efficiency System 1: 22W System 2: 86W Efficiency gain: 4X

System 2: Intel® Core™ i9-12900K NVIDIA GeForce RTX 3090 24GB (Dedicated) GDDR6X Ubuntu 24.04.3 LTS

System 2: 63.7s

System 2: Intel® Core™ i9-12900K NVIDIA GeForce RTX 3090 24GB (Dedicated) GDDR6X Ubuntu 24.04.3 LTS System 3: 12-core Arm® Cortex®-A78AE v8.2 64-bit CPU 2048-core NVIDIA Ampere architecture GPU with 64 Tensor Cores 64GB (Shared) LPDDR5 Ubuntu 24.04.3 LTS

System 2: $4700

System 3: $1999 Delta = ($4700+$1999) - $1150 = $5549 $6699/$1150 = 5.82X $1150/$6699 x 100% = 17%

System 2: Name: orin, CPU: 12-core Arm Cortex-A78AE v8.2 64-bit CPU 3MB L2 + 6MB L3 @ 2.2GHz, GPU: NVIDIA Ampere architecture with 2048 NVIDIA® CUDA® cores and 64 Tensor cores @ 1.3GHz, Memory: 64GB 256-bit LPDDR5 @ 204.8 GB/s, Storage: 64GB eMMC 5.1, Accelerator: 2× NVDLA v2.0, PVA v2.0, Accelerator: APE, NVENC, NVDEC, NVJPG, NVJPG1, VIC, SE, OS: Ubuntu 22.04.5 LTS, Kernel: 5.15.148-tegra, Jetpack: 6.2.1, Deepstream: 7.0, CUDA: 12.6, TensorRT: v100300, Jetson clocks: Default, NVP modes: 0 - MAXN.

System 2: DeepSeek Ilama-8B first token latency measurement: 543

System 2 MobileNet BS1 FPS measurement: 2114

System 2 measurement: 4E-04