iui-group-l-name-zensiert/0-pilot-project/Process.md

Using kNN

Finding optimal k-Value

Through testing on the original dataset (split 80:20) we found, that the optimal k-value is 3.

Running the kNN on the dataset without any preprocessing results in:

weighted avg 0.97 0.97 0.97 56000

Dataset optimization

Standardization

Standard

It seemed like StandardScalar on the MNIST dataset wouldn't change the outcome, so we ommitted standardization. Reason for that is probably, that the MNIST Dataset was already optimized for processing.

MinMax

Needs to be updated.

Feature selection

To be tested

Feature reduction

PCA

Testing with PCA and plotting component vs. variance we found that a 98.64% variance could be archived with only 300 components ¹.

Testing further the a variance of 99.99999999999992% was archived at 709 components, which was also the same for 784 components (the original amount of components), which means, that no/minimal variance/information is lost when using 709 components in comparison to 784 components².

For now we will simply go with n_components of 709.

LDA

To be tested

TODO

• Look up point of Covariance Matrix and how it works
  • https://www.youtube.com/watch?v=152tSYtiQbw
  • Probably part of PCA
• Reference for standardization not changing results of classifier
• Reference for MNIST already been standardized
• Test standardization method other than StandardScalar
• Test feature reduction method other than PCA (i.e. LDA(Linear Discriminant Analysis))
  • https://en.wikipedia.org/wiki/Dimensionality_reduction
  • https://towardsdatascience.com/is-lda-a-dimensionality-reduction-technique-or-a-classifier-algorithm-eeed4de9953a
  • https://medium.com/machine-learning-researcher/dimensionality-reduction-pca-and-lda-6be91734f567
  • https://towardsdatascience.com/dimensionality-reduction-does-pca-really-improve-classification-outcome-6e9ba21f0a32
• Add feature selection process
  • https://scikit-learn.org/stable/modules/feature_selection.html

---

1. https://medium.com/@miat1015/mnist-using-pca-for-dimension-reduction-and-also-t-sne-and-also-3d-visualization-55084e0320b5 ↩︎

2. Could be due to rounding in python ↩︎