Instructor: Murat Saraçlar (http://busim.ee.boun.edu.tr/~murat)

Catalog Description: This course is designed for senior year undergraduate and graduate students. The students will be exposed to contemporary pattern recognition and machine learning approaches. The emphasis of the course is not only the theory but also practical applications.

Textbook:
Duda, Hart, Stork, Pattern Classification, 2nd Ed., Wiley-Interscience, 2004
Reference Texts:
Hastie, Tibshirani, Friedman, The Elements of Statistical Learning, Springer, 2001.
Ethem Alpaydin, Introduction to Machine Learning, 2004.
Bishop, Pattern Recognition and Machine Learning, 2006.

Prerequisites by topic: Probability

Topics:

1. Overview of Learning (DHS ch1, EA ch1, HTF ch2)
2. Statistical Decision Theory (DHS ch2, EA ch3)
3. Parameter Estimation: ML, Bayesian (DHS ch3)
4. Linear Regression: Least Squares, Subset Selection and Shrinkage (HTF ch3)
5. Linear Methods for Classification: LDA, Perceptron, SVM (DHS ch5, EA ch10)
6. Dimensionality Reduction: PCA, Reduced Rank LDA (DHS ch3.8, EA ch6)
7. Kernel Methods: Support Vector Machines and Support Vector Regression
8. Nonparametric Methods: Parzen Windows, Nearest Neighbors (DHS ch4)
9. Model Assessment and Selection: BIC, MDL, VC-Dimension, Cross Validation (DHS ch9, HTF ch7, EA ch14)
10. Tree-Based Methods (DHS ch8.2-8.4, HTF ch9.2, EA ch9)
11. Ensemble Methods: Voting, Bagging, Bayesian Model Combination, Stacking, Boosting (HTF ch8.7-8.8, ch10, EA ch15)
12. Unsupervised Learning: Clustering, Mixture Models (DHS ch10, EA ch7)

Course Structure: Three lectures per week, assignments, survey, one midterm, and a final exam.

Grading:
Assignments: problems and computer implementation, every other week (30%)
Survey/Project (30%)
Midterm (15%)
Final Exam (25%)

Prepared by: Murat Saraçlar

Last revised: 06/10/08