What does machine learning study?
Systems trained from data rather than explicitly programmed.
What relation does supervised learning seek to predict outputs for new inputs?
An underlying input-output relation.
In what space do inputs belong, and how is it bounded?
.
What output space does regression use?
.
What output labels does binary classification use?
.
What output labels does multiclass classification use?
.
How are data assumed to be sampled from the unknown distribution?
Independently and identically from a fixed but unknown distribution .
What uncertainty does the conditional distribution model?
Output uncertainty for a given input.
What uncertainty does the marginal distribution model?
Uncertainty in input sampling.
What is the regression data model?
What does represent in the regression data model?
A fixed, unknown regression function.
What does a loss function map actual and predicted outputs to?
A nonnegative point-wise error.
How is a loss function’s domain and codomain written?
What formula defines a predictor’s expected risk?
What is the target function given a loss and data distribution?
The function that minimizes expected risk.
What is the target function’s domain and codomain?
Why is the target function generally estimated from data rather than computed directly?
The distribution is unknown.
What does a learning algorithm map a training set to?
An estimator .
When does an estimator generalize?
When it mimics the target function, corresponding to a small excess expected risk.
How is expected excess risk over training sets expressed?
What condition defines consistency of a learning algorithm?
What two qualities must good learning balance?
Good data fit and stability under slight data changes.
When does an algorithm overfit?
When its estimator fits the data but is unstable.
What do regularization parameters control?
The trade-off between data fitting and stability.
How is a training set of n input-output examples written?
What is the data space formed from input and output spaces?
.
How does the joint distribution factorize?
.
What is the binary classification relationship between and ?
What values can take in noiseless classification for every ?
or .
Which data does expected risk measure prediction error on?
Past and future data.
How is excess expected risk defined?
What information does a learning rate provide?
Finite-sample information by bounding expected excess risk for a specified accuracy when the sample size is sufficiently large.
What does sample complexity specify?
A sample-size threshold above which expected excess risk is at most .
When does an algorithm oversmooth?
When its estimator is stable but disregards the data.
What Gaussian distribution may regression noise follow?
Answer 18 questions on Statistical Learning Basic Concepts.
1. A software system improves its performance by learning patterns from examples rather than receiving explicit instructions for each task. Which field studies this approach?
2. A researcher has input-output examples and wants a model that predicts an output for a previously unseen input. What is the goal of this supervised-learning task?
Go over the full study guide for Statistical Learning Basic Concepts.
Read the study guide →Upload your class notes and Revizly writes your flashcards in about a minute.
Flashcard maker