SR
← Course contents · Part 1

· Visual lesson · Session 9 of 56

What do specialized math units do?

Some GPU units specialize in particular math. NVIDIA's Tensor Cores accelerate supported matrix multiply-and-accumulate operations.

Picture the idea

One result from a row and a column

  1. Select the values

    Row [2, 3] and column [4, 5]

  2. Multiply pairs

    2 × 4 = 8 and 3 × 5 = 15

  3. Add the products

    8 + 15 = 23

Matrix multiplication repeats these row-and-column combinations. Specialized hardware accelerates supported forms of this work.

What is a matrix

A matrix is a rectangle of numbers. Multiplication combines rows from one matrix with columns from another.

For example, combining [2, 3] with [4, 5] gives:

2 × 4 + 3 × 5 = 23

Neural networks do lots of related arithmetic, which makes specialized hardware useful.

Software must use it

Tensor Cores do not speed up every instruction. The operation, number format, and software path must be supported. Reading files or waiting for a network does not become faster just because these units exist.

A headline throughput number also needs context: which precision, dense or sparse data, and measured performance or theoretical peak?

Try it yourself

Multiply A = [1 2; 3 4] by B = [5 6; 7 8]. Start with the top-left result.

Check your answer

The top-left result is 1 × 5 + 2 × 7 = 19. The full result is [19 22; 43 50]. This is row-by-column multiplication, not multiplying matching positions.

Sources and further reading

← All sessions