broadcasting Theano vs. Numpy

broadcasting Theano vs. Numpy

broadcast mechanism allows a scalar may be added to a matrix, a vector to a matrix or a scalar to a vecotor.

Examples

T and F stands for True and False respectively, denoting which dimension can be broadcasted.

Diference

1. numpy broadcast dynamically;
2. theano needs to knows, for any operations which supports broadcasting, which dimensions will need to be broadcasted.

Numpy Broadcasting

broadcasting describe how numpy treats arrays with difference shapes during arithmetic operations:

the smaller array is broadcast across the larger array so that they have compatible shapes

Simple Case

in this case, the two arrays must have exactly the same shape:

a=np.array([1.0,2.0,3.0])
b=np.array([2.0,2.0,2.0])
print a*b
>>> array([2., 4., 6.])

numpy broadcast mechanism relaxes this constraint when the arrays' shape meet certain constraints:

1. they are equal, or
2. one of them is 1

a=np.array([1.0,2.0,3.0])
b=2
print a*b
>>> array([2., 4., 6.])

rules

Image  (3d array): 256 x 256 x 3
Scale  (1d array):             3
Result (3d array): 256 x 256 x 3

A      (4d array):  8 x 1 x 6 x 1
B      (3d array):      7 x 1 x 5
Result (4d array):  8 x 7 x 6 x 5

more examples

A      (2d array):  5 x 4
B      (1d array):      1
Result (2d array):  5 x 4

A      (2d array):  5 x 4
B      (1d array):      4
Result (2d array):  5 x 4

A      (3d array):  15 x 3 x 5
B      (3d array):  15 x 1 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 5
Result (3d array):  15 x 3 x 5

A      (3d array):  15 x 3 x 5
B      (2d array):       3 x 1
Result (3d array):  15 x 3 x 5