Principal Component Analysis (PCA): A Comprehensive Guide for Statistics Students

Question 1

In PCA, an eigenvector of the covariance matrix is a vector that:

Accepted Answer

Maximizes the variance of the data projected onto it

Answer

Is perpendicular to all other eigenvectors

Answer

Preserves the original Euclidean distance relationships between data points

Answer

Has a zero eigenvalue

Question 2

Centering the data before performing PCA is crucial because it:

Accepted Answer

Removes the mean from each feature, ensuring that the data is symmetric around zero

Answer

Makes the data more normally distributed, improving the accuracy of PCA

Answer

Reduces the influence of outliers, making the principal components more robust

Answer

Ensures that all features have equal variance, allowing for more balanced dimensionality reduction

Question 3

In PCA, loadings and scores differ in the following way:

Accepted Answer

Loadings represent the weights of the original variables in each principal component, while scores represent the coordinates of the data points in the principal component subspace

Answer

Scores represent the weights of the original variables in each principal component, while loadings represent the coordinates of the data points in the principal component subspace

Answer

Loadings and scores are essentially the same concept

Answer

There is no distinction between loadings and scores

Question 4

Regarding the classification of PCA, which statement is true?

Accepted Answer

PCA is an unsupervised learning technique

Answer

PCA is a supervised learning technique

Question 5

The primary goal of Principal Component Analysis (PCA) is to:

Accepted Answer

Reduce data dimensionality while maximizing variance

Answer

Increase data dimensionality

Answer

Identify outliers in data

Answer

Perform regression analysis

Question 6

In PCA, the eigenvectors of the covariance matrix represent:

Accepted Answer

Axes of maximum variance in the data

Answer

Directions of least variance in the data

Answer

Eigenvalues of the covariance matrix

Answer

Number of principal components to extract

Question 7

The scree plot in PCA is used to:

Accepted Answer

Determine the optimal number of principal components to extract

Answer

Visualize the distribution of eigenvalues

Answer

Identify outliers in the data

Answer

Check the assumptions of PCA

Question 8

Which is NOT a type of PCA algorithm?

Accepted Answer

K-Means Clustering

Answer

Singular Value Decomposition (SVD)

Answer

Eigenvalue Decomposition (EVD)

Answer

Power Iteration

Question 9

The cumulative variance explained by the principal components is typically plotted on the:

Accepted Answer

Scree plot

Answer

Loading plot

Answer

Biplot

Answer

Residual plot

Question 10

The purpose of mean centering before PCA is to:

Accepted Answer

Remove the influence of scale on the analysis

Answer

Transform the data to a normal distribution

Answer

Reduce the number of variables in the analysis

Answer

Improve the interpretability of the results

Question 11

Compare and contrast PCA with Linear Discriminant Analysis (LDA) and t-SNE. Discuss their key differences, strengths, and weaknesses.

Accepted Answer

PCA and t-SNE are unsupervised dimensionality reduction techniques, while LDA is supervised.

Accepted Answer

PCA assumes linear relationships between features, while t-SNE can capture non-linear relationships.

Answer

LDA is particularly suitable for datasets with a low number of features and a relatively high number of samples.

Question 12

Which method(s) can be employed to determine the optimal number of principal components to retain?

Accepted Answer

Any combination of the above.

Answer

Examining the scree plot.

Answer

Applying the Kaiser-Guttman criterion.

Answer

Considering the cumulative percentage of explained variance.

Question 13

Revised: A technique used to improve the interpretability of PCA results is:

Accepted Answer

Varimax rotation

Answer

K-means clustering

Answer

Logistic regression

Answer

Neural networks

Question 14

Revised: The relationship between PCA and discriminant analysis is:

Accepted Answer

PCA is unsupervised, while discriminant analysis is supervised

Answer

PCA is a supervised technique, while discriminant analysis is unsupervised

Answer

PCA and discriminant analysis are both supervised techniques

Answer

PCA and discriminant analysis are both unsupervised techniques

Question 15

Select the option that accurately describes an application of PCA in data visualization.

Accepted Answer

Reducing data dimensionality for convenient visualization

Answer

Identifying clusters and anomalies in high-dimensional data

Answer

Extracting features for predictive modeling

Answer

Detecting fraudulent activities in financial transactions

Question 16

In PCA, what is the significance of the eigenvectors of the covariance matrix?

Accepted Answer

They represent the directions of the principal components.

Answer

They indicate the variance of each principal component.

Answer

They determine the number of principal components that can be extracted.

Question 17

Which mathematical concept forms the basis of Principal Component Analysis (PCA)?

Accepted Answer

Eigenvalues and eigenvectors

Answer

Linear regression

Answer

Clustering

Question 18

Clarity and Precision: Which of the following is NOT a core concept of Principal Component Analysis (PCA)?

Accepted Answer

Hypothesis testing

Answer

Dimensionality reduction

Answer

Data visualization

Answer

Feature transformation

Question 19

Correctness and Precision of the Answer: In the context of PCA, what does the term 'orthogonal' refer to?

Accepted Answer

The principal components are perpendicular to each other.

Answer

The principal components are uncorrelated.

Answer

The principal components are parallel to each other.

Answer

The principal components are linearly dependent.

Question 20

Which of the following is NOT a primary objective of Principal Component Analysis (PCA)?

Accepted Answer

Variable selection

Answer

Dimensionality reduction

Answer

Data exploration

Answer

Data transformation

Question 21

Data visualization is a key application of PCA. Which of the following is NOT a common method for visualizing principal components?

Accepted Answer

Bar charts

Answer

Scatter plots

Answer

Heatmaps

Answer

Line graphs

Question 22

In Principal Component Analysis (PCA), what do the eigenvectors of the covariance matrix represent geometrically?

Accepted Answer

Directions of maximum variance in the data.

Answer

Orthogonal directions to the principal components.

Answer

Identical directions to the principal components.

Answer

No geometric interpretation.

Question 23

In PCA, what is the geometrical interpretation of the eigenvectors of the covariance matrix in the original feature space?

Accepted Answer

They define a new set of orthogonal axes that capture the directions of maximum variance in the data.

Answer

They are the axes of the ellipsoid that represents the spread of the data.

Answer

They are the features with the highest variance.

Question 24

How does data normalization, such as centering and scaling, affect the interpretability of Principal Component Analysis (PCA) results?

Accepted Answer

Normalization enhances the clarity and meaningfulness of PCA results.

Answer

Normalization has no impact on PCA results.

Answer

Normalization reduces the variance of the data.

Answer

Normalization is essential for PCA to function properly.

Question 25

Which of the following is an advantage of Principal Component Analysis (PCA)?

Accepted Answer

Improved interpretability

Answer

Loss of information

Answer

Increased computational complexity

Answer

Vulnerability to outliers

Question 26

A primary objective of Principal Component Analysis (PCA) is to:

Accepted Answer

Reduce dimensionality and extract meaningful features.

Answer

Predict future events.

Answer

Estimate population parameters.

Question 27

Which mathematical concept is central to PCA?

Accepted Answer

Eigenvalues and eigenvectors.

Answer

Correlation and covariance.

Answer

Mean and standard deviation.

Answer

Hypothesis testing and significance.

Question 28

A common application of PCA in data science is:

Accepted Answer

Feature selection and dimensionality reduction for machine learning models.

Answer

Hypothesis testing and statistical inference.

Answer

Time series forecasting and trend analysis.

Question 29

In PCA, the covariance matrix plays a crucial role by:

Accepted Answer

Capturing the correlation and variance of the original variables.

Answer

Computing the eigenvalues and eigenvectors.

Answer

Defining the number of principal components to extract.

Question 30

To determine the optimal number of principal components in PCA, you can:

Accepted Answer

Use a scree plot or consider the cumulative variance explained.

Answer

Set the number of components equal to the number of original variables.

Answer

Use a statistical test to determine the significance of the components.

Question 31

A key difference between PCA and Linear Discriminant Analysis (LDA) is that:

Accepted Answer

LDA incorporates class labels while PCA does not.

Answer

LDA extracts linear combinations while PCA extracts nonlinear combinations.

Answer

PCA is parametric while LDA is non-parametric.

Answer

PCA is supervised while LDA is unsupervised.

Question 32

PCA and factor analysis are related techniques. Their main difference is that:

Accepted Answer

PCA focuses on variance while factor analysis focuses on explaining observed variables.

Answer

PCA is a supervised technique while factor analysis is unsupervised.

Question 33

Which of the following is NOT an advantage of utilizing Principal Component Analysis (PCA)?

Accepted Answer

Increased data intricacy

Answer

Diminished noise

Answer

Enhanced data visualization

Answer

Reduced dimensionality

Question 34

What is the overarching objective of PCA?

Accepted Answer

To determine a fresh set of variables that are orthogonal and maximize variance.

Answer

To forecast potential results.

Answer

To detect clusters of data points.

Question 35

What is the interpretation of an eigenvalue in PCA?

Accepted Answer

It signifies the variance accounted for by the corresponding principal component.

Answer

It determines the number of principal components to extract.

Answer

It denotes the direction of the corresponding principal component.

Question 36

Which technique can be employed to determine the optimum number of principal components to extract?

Accepted Answer

Scree plot

Answer

Linear regression

Answer

K-means clustering

Answer

Decision tree

Question 37

What is the purpose of using PCA in data visualization?

Accepted Answer

To decrease the number of dimensions for simpler graphical representation.

Answer

To estimate missing values.

Answer

To identify outliers in the data.

Question 38

Explain the distinction between PCA and Singular Value Decomposition (SVD).

Accepted Answer

SVD can handle matrices that are not square, whereas PCA is restricted to square matrices.

Answer

PCA is a more computationally efficient algorithm than SVD.

Answer

SVD produces different results than PCA on the same data.

Question 39

Which of the following is a key assumption of Principal Component Analysis (PCA)?

Accepted Answer

The variables in the data are linearly related.

Answer

The data is normally distributed.

Answer

All variables have equal variances.

Answer

The data contains no outliers.

Question 40

Which of the following is NOT a benefit of using PCA?

Accepted Answer

Increased computational complexity for large datasets.

Answer

Feature extraction.

Answer

Dimensionality reduction.

Answer

Improved data visualization.

Question 41

What do the eigenvectors represent in PCA?

Accepted Answer

The directions of maximum variance in the data.

Answer

The eigenvalues of the covariance matrix.

Answer

The reduced dimensions of the data.

Question 42

Which statistical measure indicates the proportion of variance explained by each principal component?

Accepted Answer

Eigenvalue.

Answer

Standard deviation.

Answer

Correlation.

Answer

Variance.