Week 04:
	-
	-

---

Topic: Supervised Learning III & Unsupervised Learning

---

Objectives:

• Define learning in the context of artificial intelligence
• Define supervised and unsupervised learning and identify when supervised and unsupervised learning should be used
• Define and give examples of regression and classification in supervised learning and identify when regression or classification should be used
• Describe and use various supervised and unsupervised learning algorithms and specify the model characteristics in those algorithms:
  • Global versus Local
  • Hierarchical versus Non-Hierarchical
  • Parametric versus Non-Parametric

---

1. Study the following parts of the Open Source Computer Vision (OpenCV) Version 2.4.4 Application Programming Interface (API):
   Note: OpenCV version 2.4.4 is only located on linprog4.cs.fsu.edu. In order to compile, an OpenCV 2.4.4 C++11 program in main.cpp using g++ version 4.7.2, use the following command on linprog4.cs.fsu.edu:
   g++47 -o main.exe main.cpp -std=c++11 -O3 -Wall -Wextra -Werror -I. -I/usr/local/include/ -I/usr/local/include/boost_1.53.0/ -L/usr/local/lib64/ -lopencv_calib3d -lopencv_contrib -lopencv_core -lopencv_features2d -lopencv_flann -lopencv_gpu -lopencv_highgui -lopencv_imgproc -lopencv_legacy -lopencv_ml -lopencv_nonfree -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_ts -lopencv_video -lopencv_videostab -Wl,-rpath,/usr/local/lib64/ && ./main.exe
   • Basic Structures
     
   • Operations on Arrays
   • MLData
   • K-Nearest Neighbors
   
   
2. Watch the "Week Two: Decision Tree Induction" series of videos in Machine Learning with Pedro Domingos course from Coursera:
   1. Sign In to Coursera.
   2. Click the Explore Courses link at the top of the page.
   3. Search for Machine Learning using the search text box.
   4. Select the Machine Learning with Pedro Domingos.
   5. Press the "Preview" button.
   6. Watch the following videos in the "Week Two: Decision Tree Induction" section (2 hr 31 min 35 sec):
      
      Note: Click on the slide in the lower-right corner to make the slide larger and the video smaller.
      
      
      • Decision Trees (11:31)
      • What Can a Decision Tree Represent? (7:53)
      • Growing a Decision Tree (7:52)
      • Accuracy and Information Gain (23:49)
      • Learning with Non-Boolean Features (7:17)
      • The Parity Problem (5:42)
      • Learning with Many-Valued Attributes (6:06)
      • Learning with Missing Values (10:25)
      • The Overfitting Problem (13:54)
      • Decision Tree Pruning (22:28)
      • Post-Pruning Trees to Rules (26:25)
      • Scaling Up Decision Tree Learning (8:13)
   
   Reference: Coursera's Machine Learning course by Pedro Domingos.
   
   
3. Complete the "6. Unsupervised Learning" unit in the Introduction to Artificial Intelligence course from Udacity:
   1. Sign In to Udacity
   2. Select Introduction to Artificial Intelligence from the Course Catalog
   3. Press the "Take the Class" button
   4. Watch the videos and take the quizzes in the "6. Unsupervised Learning" unit.
   
   Alternatively, download the following ZIP file containing Flash Video files:
   • 6. Unsupervised Learning
   
   
4. Watch the "Hierarchical Clustering" video from the "Week Ten: Clustering and Dimensionality Reduction" series of videos in Machine Learning with Pedro Domingos course from Coursera:
   1. Sign In to Coursera.
   2. Click the Explore Courses link at the top of the page.
   3. Search for Machine Learning using the search text box.
   4. Select the Machine Learning with Pedro Domingos.
   5. Press the "Preview" button.
   6. Watch the following video in the "Week Ten: Clustering and Dimensionality Reduction" section (5 min 49 sec):
      
      Note: Click on the slide in the lower-right corner to make the slide larger and the video smaller.
      
      
      • Hierarchical Clustering (5:49)
   
   Reference: Coursera's Machine Learning course by Pedro Domingos.
   
   
5. Using the following texts:
   • Artificial Intelligence: A Modern Approach, Third Edition (cited as AIMA)
   • The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (cited as ESLII)
   
   Read the following pages:
   • Required: 50 Pages

     Order	Topic	Book	Section	Page Numbers	Pages	Presentations
     1.	Learning Decision Trees	AIMA	Section 18.3:	Pages 697–707	(11 Pages)	Learning from Observations
     2.	Ensemble Learning	AIMA	Section 18.10:	Pages 748–753	(6 Pages)
     3.	Practical Machine Learning	AIMA	Section 18.11:	Pages 753–757	(5 Pages)	Machine Learning
     4.	Chapter 18 Summary	AIMA	Section 18.12:	Pages 757–758	(2 Pages)
     5.	K-Means	ESLII	Section 14.3.6:	Pages 509-511	(3 Pages)	Machine Learning
     6.	Learning with Hidden Variables: The EM Algorithm	AIMA	Section 20.3 Intro:	Pages 816–817	(2 Pages)	Machine Learning
     7.	Unsupervised Clustering: Learning Mixtures of Gaussians	AIMA	Section 20.3.1:	Pages 817–820	(4 Pages)	Machine Learning
     8.	Learning Bayesian Networks with Hidden Variables	AIMA	Section 20.3.2:	Pages 820–822	(3 Pages)	Statistical Learning and Machine Learning
     9.	Chapter 20 Summary	AIMA	Section 20.4:	Page 825	(1 Page)
     10.	Hierarchical Clustering	ESLII	Section 14.3.12:	Page 520-528	(9 Pages)	Hierarchical Clustering
     11.	Curse of Dimensionality	AIMA	Section 18.8.1 & Figure 18.27:	Page 739-740	(2 Pages)
     12.	Curse of Dimensionality	lacking capital(s) by Boris Babenko	harry potter and the curse of dimensionality:	Blog Post	(1 Page)	Curse of Dimensionality
     13.	Curse of Dimensionality	on Topology by John Moeller	Reasoning in Higher Dimensions: Measure:	Blog Post	(1 Page)	Curse of Dimensionality

     
     
   • Optional: 6 Pages

     Order	Topic	Book	Section	Page Numbers	Pages	Presentations
     1.	Local Methods in High Dimensions	ESLII	Section 2.5:	Page 22-27	(6 Pages)	Curse of Dimensionality

     
     
   Reference: University of California, Berkeley's CS 188: Introduction to Artificial Intelligence (Fall 2005) by Stuart Russell
   Reference: Stanford University's CS221: Artificial Intelligence: Principles and Techniques (Autumn 2012-2013) by Percy Liang
   
   
6. Do Assignment 03.