Statistical Learning Basic Concepts Flashcards (35 cards)

All cards

1Question

What does machine learning study?

Answer

Systems trained from data rather than explicitly programmed.

2Question

What relation does supervised learning seek to predict outputs for new inputs?

Answer

An underlying input-output relation.

3Question

In what space do inputs belong, and how is it bounded?

Answer

X⊆RDX \subseteq \mathbb{R}^D.

4Question

What output space does regression use?

Answer

Y⊆RY \subseteq \mathbb{R}.

5Question

What output labels does binary classification use?

Answer

Y={−1,1}Y=\{-1,1\}.

6Question

What output labels does multiclass classification use?

Answer

Y={1,2,…,T}Y=\{1,2,\ldots,T\}.

7Question

How are data assumed to be sampled from the unknown distribution?

Answer

Independently and identically from a fixed but unknown distribution p(x,y)p(x,y).

8Question

What uncertainty does the conditional distribution p(y∣x)p(y\mid x) model?

Answer

Output uncertainty for a given input.

9Question

What uncertainty does the marginal distribution pX(x)p_X(x) model?

Answer

Uncertainty in input sampling.

10Question

What is the regression data model?

Answer

y=f∗(x)+εy=f^*(x)+\varepsilon

11Question

What does f∗f^* represent in the regression data model?

Answer

A fixed, unknown regression function.

12Question

What does a loss function map actual and predicted outputs to?

Answer

A nonnegative point-wise error.

13Question

How is a loss function’s domain and codomain written?

Answer

ℓ:Y×Y→[0,∞)\ell:Y\times Y\to[0,\infty)

14Question

What formula defines a predictor’s expected risk?

Answer

E(f)=E[ℓ(y,f(x))]=∫p(x,y)ℓ(y,f(x)) dx dyE(f)=\mathbb{E}[\ell(y,f(x))]=\int p(x,y)\ell(y,f(x))\,dx\,dy

15Question

What is the target function given a loss and data distribution?

Answer

The function that minimizes expected risk.

16Question

What is the target function’s domain and codomain?

Answer

f∗:X→Yf^*:X\to Y

17Question

Why is the target function generally estimated from data rather than computed directly?

Answer

The distribution pp is unknown.

18Question

What does a learning algorithm map a training set SS to?

Answer

An estimator fSf_S.

19Question

When does an estimator generalize?

Answer

When it mimics the target function, corresponding to a small excess expected risk.

20Question

How is expected excess risk over training sets expressed?

Answer

ES[E(fS)−E(f∗)]\mathbb{E}_S[E(f_S)-E(f^*)]

21Question

What condition defines consistency of a learning algorithm?

Answer

lim⁡n→∞ES[E(fS)−E(f∗)]=0\lim_{n\to\infty}\mathbb{E}_S[E(f_S)-E(f^*)]=0

22Question

What two qualities must good learning balance?

Answer

Good data fit and stability under slight data changes.

23Question

When does an algorithm overfit?

Answer

When its estimator fits the data but is unstable.

24Question

What do regularization parameters control?

Answer

The trade-off between data fitting and stability.

25Question

How is a training set of n input-output examples written?

Answer

S={(x1,y1),…,(xn,yn)}S = \{(x_1,y_1),\ldots,(x_n,y_n)\}

26Question

What is the data space formed from input and output spaces?

Answer

X×YX \times Y.

27Question

How does the joint distribution factorize?

Answer

p(x,y)=pX(x)p(y∣x)p(x,y)=p_X(x)p(y\mid x).

28Question

What is the binary classification relationship between p(1∣x)p(1\mid x) and p(−1∣x)p(-1\mid x)?

Answer

p(1∣x)=1−p(−1∣x)p(1\mid x)=1-p(-1\mid x)

29Question

What values can p(1∣x)p(1\mid x) take in noiseless classification for every x∈Xx\in X?

Answer

00 or 11.

30Question

Which data does expected risk measure prediction error on?

Answer

Past and future data.

31Question

How is excess expected risk defined?

Answer

E(fS)−E(f∗)E(f_S)-E(f^*)

32Question

What information does a learning rate provide?

Answer

Finite-sample information by bounding expected excess risk for a specified accuracy when the sample size is sufficiently large.

33Question

What does sample complexity n(ε)n(\varepsilon) specify?

Answer

A sample-size threshold above which expected excess risk is at most ε\varepsilon.

34Question

When does an algorithm oversmooth?

Answer

When its estimator is stable but disregards the data.

35Question

What Gaussian distribution may regression noise follow?

Answer

N(0,σI)\mathcal{N}(0,\sigma I)

Test yourself with the quiz

Answer 18 questions on Statistical Learning Basic Concepts.

1. A software system improves its performance by learning patterns from examples rather than receiving explicit instructions for each task. Which field studies this approach?

2. A researcher has input-output examples and wants a model that predicts an output for a previously unseen input. What is the goal of this supervised-learning task?

Take the quiz →

Read the study guide

Go over the full study guide for Statistical Learning Basic Concepts.

Read the study guide →

Related classes

Make your own flashcards

Upload your class notes and Revizly writes your flashcards in about a minute.

Flashcard maker