Nvidia’s Tesla A100 has a whopping 6,912 CUDA cores – Specs Detailed

Right now, we know that Nvidia’s Tesla A100 features 6,912 CUDA cores, which feature the ability to calculate FP64 calculations at half-rate.

With 7nm, Nvidia has delivered a greater than 2x increase in transistor count over the company’s Tesla V100 core design, a feat which allows Nvidia to deliver some incredible performance increases for its Tesla A100. Nvidia has also combined this core design with 40GB of HBM memory by using five 8GB HBM memory modules. The image below shows that Nvidia’s Tesla A100 can feature up to six of these chips to deliver 48GB of VRAM. This reduction in VRAM capacity was likely a design decision which will help Nvidia increase the production yields of such a large graphics card. 

Nvidia’s Tesla A100 chips use Nvidia’s SMX3 form factor and support PCIe 4.0 and third-generation NVLink. With 3rd Generation NVLink, Nvidia can support up to 4.8 TB per second in bi-direction bandwidth and 600 GB per second in GPU-to-GPU bandwidth. This means that Nvidia can connect up to eight Tesla A100 graphics cards together with 600 GB per second of bandwidth. Nvidia has also combined its offerings with 200Gbps Mellanox interconnects to increase platform scalability further. 

With the company’s new 3rd generation Tensor cores, Nvidia’s Tesla A100 is said to deliver a 20x increase in eight-bit integer math (INT8) performance when compared to the company’s older Tesla V100. Nvidia has also reported a 2.5X increase in double-precision floating-point, FP64, performance. While there are fewer Tensor cores on the Tesla A100, these redesigned Tensor cores are much more powerful than before, more than making up for this decrease Tensor core count. 

Nvidia is expected to reveal more information about its Ampere architecture later today.