What does machine learning algorithms improve through experience?
Their performance on a task.
What kind of data does regression use to predict continuous numeric variables?
Labeled data with a numeric target for each observation.
What kind of variables does classification predict?
Categorical variables, such as classes or class probabilities.
What does clustering find in an unsupervised learning task?
Natural groupings based on the dataset’s inherent structure.
How does deep learning learn useful representations directly from data?
By composing nonlinear transformations of the data.
How do deep learning feature extractors differ from traditional machine-learning feature extractors?
Deep learning feature extractors are trainable and learned from data.
What does an end-to-end recognition system map raw pixels to?
Labels.
How does an end-to-end recognition system map raw pixels to labels?
By composing simple nonlinear functions with learnable parameters.
What is a neural network?
A set of connected neurons with randomly initialized weights and nonlinear activation functions, optimized on training data to minimize prediction error.
What formula gives a single-layer neuron's pre-activation value ?
What formula gives a single-layer neuron's output ?
What sequence does neural-network training follow for each training example?
Initialize weights, compute outputs left to right, measure loss, propagate error backward, and adjust weights with gradient descent.
What does backpropagation calculate in a multilayer feedforward neural network?
Each weight’s contribution to prediction error.
What is the gradient descent update formula for weight ?
What does represent in the gradient descent update formula?
The learning rate.
What does a convolutional neural network extract from images?
Topological properties.
How does image filtering apply a convolution kernel?
It slides a window across the image and approximates the result from neighboring pixels.
Which Sobel kernels detect orientations in an image?
Horizontal and vertical gradient kernels.
Which reference is associated with the HOG method?
Dalal, 2005.
What two quantities are calculated in a HOG descriptor?
Gradient norm and gradient direction.
How does HOG detect points of interest?
It applies a threshold to the gradient norm.
Why do CNNs reuse the same kernel across image locations?
Features such as edges can occur anywhere in the image.
What can a CNN learn from raw pixels in an early image-classification layer?
Edges.
What two properties let CNNs detect the same feature at different image positions?
Local connections and shared weights.
What does pooling do to each convolutional feature map’s spatial resolution?
It reduces the spatial resolution.
Which invariances can pooling provide?
Some shift and distortion invariance.
What is overfitting?
Fitting training data, including noise or outliers, while failing to generalize to new test examples.
How does dropout encourage robust features during training?
It randomly removes units and their connections, reducing complex co-adaptations and overfitting, especially in dense layers.
What is a pre-trained network?
A saved network previously trained on a large dataset.
How does feature extraction differ from fine-tuning?
Feature extraction trains a new classifier from scratch; fine-tuning jointly trains selected unfrozen top layers with the new model part.
What features do earlier convolutional layers extract?
Local, generic features such as edges, colors, and textures.
Along how many dimensions does a Conv2D kernel slide?
Two dimensions.
What kind of data is Conv2D generally used on?
Image data.
Along how many dimensions do Conv3D kernels slide?
Three dimensions.
What data types are Conv3D mainly used with?
3D images and video sequences.
How many classes and labeled photos does ImageNet contain?
1,000 classes and 1.2 million labeled photos.
What is the approximate human top-5 error on ImageNet?
Approximately 5%.
Which three outcomes are examples of regression?
Real-estate prices, stock price movements, and student test scores.
Which three outcomes are examples of classification?
Email spam, financial fraud, and student letter grades.
Which three applications are examples of clustering?
Customer segmentation, grouping similar e-commerce items, and social network analysis.
What can traditional feature extraction require?
Expert knowledge, time-consuming hand-tuning, and problem-specific choices.
What two learning settings can deep learning methods use?
Supervised or unsupervised learning.
What is a multilayer perceptron?
A non-convolutional neural-network architecture built by composing layers of simple models.
Which machine-learning paradigms can neural networks address?
Supervised, semi-supervised, and unsupervised learning.
How does a neural network map input to output ?
What happens to sigmoid and tanh gradients near extreme output values?
They saturate and become close to zero.
What problem does Leaky ReLU help address when its input is negative?
The dead-neuron problem of ReLU.
Across what kinds of instances does training repeat weight adjustments?
Randomly selected training instances.
How many times may training repeat weight adjustments?
Thousands or possibly millions of times.
What input-output mappings can backpropagation theoretically perform?
Any input-output mapping.
What class of problems can backpropagation learn?
Linearly inseparable problems.
How are convolutional neural networks trained?
With a version of backpropagation.
How does convolution process a pixel?
Using information from neighboring pixels.
What is multiplication equivalent to in the spatial domain?
Convolution.
What are typical convolution-kernel dimensions?
3×3, 5×5, or 7×7.
What can describe detected orientations using a histogram?
A histogram of directions.
What does CNN topology restrict connections to?
Local image regions.
What is a feature map?
The output produced by convolving one feature.
How are filters applied to the channels of an RGB input?
A separate filter is applied to each channel.
Which layer types does a CNN architecture combine?
Feature-extraction, convolution, pooling or subsampling, and fully connected layers.
Which pooling types are common, and what filter size and stride are illustrated?
Max and average pooling use 2×2 filters with stride 2.
When is dropout applied?
During training, not testing.
How can data augmentation reduce overfitting?
By applying small image transformations or adding noise.
Which image changes can data augmentation use to reduce overfitting?
Small rotations, translations, scaling, flipping, brightness changes, and noise.
How does each node in a fully connected layer combine its inputs?
It computes a weighted sum of all input nodes.
What does a fully connected layer use to classify an input pattern?
High-level features from previous layers.
What did course experiments report about deeper CNNs?
Deeper replication of convolution and pooling layers produced better results.
When a new dataset differs greatly from the original training data, what may be preferable for feature extraction?
Using only the first few layers.
How can an ImageNet-pretrained CNN be adapted to new data?
Use it as an initialization, then fine-tune its weights on the new data.
What do the three values in Conv2D input_shape (128, 128, 3) represent?
Height, width, and depth.
What does Conv2D kernel_size (3, 3) specify?
Kernel height and width.
What context do Conv3D activation maps preserve?
Temporal or volumetric context.
In Conv3D, what does kernel_size (3, 3, 3) specify?
Kernel height, width, and depth.
What accuracy did LeNet-5 achieve on MNIST handwritten-character recognition?
99.2%.
How long was AlexNet trained, and on how many Nvidia GeForce GTX 580 GPUs?
Six days on two GPUs.
What approach did VGG-16 use to improve CNN efficiency?
Stacking more layers.
Which work attributes Inception-v1 (GoogLeNet) to Christian Szegedy et al.?
“Going Deeper with Convolutions” (2014).
Which work attributes Inception-v3 to Christian Szegedy et al.?
“Rethinking the Inception Architecture for Computer Vision” (2015).
Which work attributes ResNet-50 to Kaiming et al.?
“Deep Residual Learning for Image Recognition” (2015).
Which authors are cited for the 1995 account of convolutional neural networks trained by backpropagation?
Yann LeCun and Yoshua Bengio.
What response results from applying kernel [2, 3, 1] to sequence [1, 2, 3, 4, 5, 6, 7]?
[11, 17, 23, 29, 35].
How many weight parameters do 100 shared 10×10 filters require for a single-channel image?
10,000, excluding biases.
Answer 24 questions on Convolutional Neural Networks.
1. Which statements correctly describe machine learning and regression?
2. A project predicts whether each email is spam and estimates house prices; which statements classify these tasks correctly?
Go over the full study guide for Convolutional Neural Networks.
Read the study guide →Upload your class notes and Revizly writes your flashcards in about a minute.
Flashcard maker