Deep Learning Interview Questions

Question 1

How does Spatial Dropout conceptually differ from standard Dropout in a Convolutional Neural Network?

Accepted Answer

Spatial Dropout drops entire 2D feature maps instead of dropping individual random pixels within the maps

Question 2

How does padding affect a convolutional layer?

Accepted Answer

It allows the spatial dimensions of the input to be preserved in the output feature map

Question 3

What is the specific limitation of the standard Self-Attention mechanism regarding computational complexity with respect to sequence length N?

Accepted Answer

It scales quadratically with N squared

Question 4

Why does the Leaky ReLU activation function help mitigate the dying ReLU problem?

Accepted Answer

It sets a small non-zero slope for negative inputs ensuring the gradient never becomes exactly zero

Question 5

How does Adaptive Instance Normalization inject style into the generated images of a StyleGAN?

Accepted Answer

By aligning the mean and variance of the content feature maps to match the mean and variance of the style feature maps

Question 6

What is the most common visible symptom of the exploding gradient problem during the training of a neural network?

Accepted Answer

The training loss suddenly evaluates to NaN or Infinity and the model weights become completely corrupted

Question 7

In the context of generative models and regularized autoencoders what is the reparameterization trick used for in a Variational Autoencoder?

Accepted Answer

To allow backpropagation through a stochastic sampling process by separating the random noise from the learned parameters

Question 8

What is a 1x1 convolution commonly used for in deep learning architectures like Inception?

Accepted Answer

To reduce the number of channels/depth in feature maps decreasing computational cost

Question 9

In a Recurrent Neural Network how does Truncated Backpropagation Through Time mitigate computational issues?

Accepted Answer

It unrolls the network for a fixed number of timesteps rather than the entire sequence to compute gradients

Question 10

When comparing GRU to LSTM what is a true statement about their architectures?

Accepted Answer

GRU lacks a separate cell state and merges the forget and input gates into a single update gate

Question 11

What is the main architectural innovation of the Transformer model compared to traditional RNNs?

Accepted Answer

It relies entirely on self-attention mechanisms allowing parallel processing of sequences

Question 12

How does global average pooling differ from a standard fully connected layer at the end of a CNN?

Accepted Answer

It takes the average of each feature map directly outputting a single value per channel drastically reducing parameter count

Question 13

What happens if the stride size in a convolutional layer is larger than the filter size?

Accepted Answer

The filter will skip over some input pixels completely ignoring parts of the image

Question 14

Which paradigm focuses on training models to recognize new classes given only a tiny number of examples per class?

Accepted Answer

Few-shot learning

Question 15

What is the difference between Categorical Cross-Entropy and Sparse Categorical Cross-Entropy?

Accepted Answer

Categorical expects one-hot encoded labels while Sparse expects integer-based labels

Question 16

In a sequence-to-sequence Transformer model what is the purpose of the masked multi-head attention in the decoder?

Accepted Answer

To prevent the decoder from looking at future tokens when predicting the next token

Question 17

Which technique is specifically used to fix the exploding gradient problem in Recurrent Neural Networks?

Accepted Answer

Gradient Clipping

Question 18

What does catastrophic forgetting refer to in neural networks?

Accepted Answer

The tendency of a model to completely lose previously learned information when it is trained on a new distinct task

Question 19

When using Dropout during the inference phase how is it applied?

Accepted Answer

Dropout is turned off and the weights are scaled down by the dropout probability

Question 20

What is the distinct practical effect of using L1 regularization compared to L2 regularization?

Accepted Answer

L1 drives many weights exactly to zero resulting in a sparse model while L2 shrinks weights evenly without making them strictly zero

Question 21

What is the mathematical formulation of the Gaussian Error Linear Unit activation function?

Accepted Answer

It multiplies the input by the cumulative distribution function of the standard normal distribution

Question 22

What is the purpose of positional encoding in a Transformer model?

Accepted Answer

To inject information about the relative or absolute position of tokens in the sequence

Question 23

Why does the standard Transformer architecture rely on Layer Normalization instead of Batch Normalization?

Accepted Answer

Layer Normalization performs better on sequence data where batch sizes are small or sequences have highly variable lengths

Question 24

Which of the following best describes internal covariate shift in the context of deep learning?

Accepted Answer

The change in the distribution of network activations due to the change in network parameters during training

Question 25

In NLP what does the Out-Of-Vocabulary problem refer to?

Accepted Answer

When the model encounters a word during inference that was not present in its predefined training dictionary

Deep Learning Interview Questions

Ready to test yourself?

Want to study the theory first?

Practice with 8,400+ similar questions