ANNS-05

What is the difference between a traditional (shallow) and a deep neural network?

• DNNs have more hidden layers

• DNNs have more neurons within layers

• DNNs are expensive to train

• DNNs require less data

Training of Deep networks is [] than shallow ones

• easier

• harder

• faster

DNNs training is cumbersome due to

• random initialization

• GPU memory constraints

• vanishing gradient effect

• difficulty of implementation

DNNs can be pretained by [] method

• DBN

• RBM

• Discriminative

• ADM

Rectifier activation:

• z^2-1

• min(z,0)

• avg(z,0)

• max(z,0)