Machine Learning in Computer Vision

Winter 2018


In recent years, Deep Learning has become a dominant Machine Learning tool for a wide variety of domains. One of its biggest successes has been in Computer Vision where the performance in problems such object and action recognition has been improved dramatically. In this course, we will be reading up on various Computer Vision problems, the state-of-the-art techniques involving different neural architectures and brainstorming about promising new directions.

Please sign up here in the beginning of class.

Course overview

This class is a graduate seminar course in computer vision. The class will cover a diverse set of topics in Computer Vision and various machine learning approaches. It will be an interactive course where we will discuss interesting topics on demand and latest research buzz. The goal of the class is to learn about different domains of vision, understand, identify and analyze the main challenges, what works and what doesn't, as well as to identify interesting new directions for future research.

Prerequisites: Courses in computer vision and/or machine learning (e.g., CSC320, CSC420, CSC411) are highly recommended (otherwise you will need some additional reading), and basic programming skills are required for projects.

back to top

Course Information

When emailing me, please put CSC2548 in the subject line.

Forum

This class uses piazza. On this webpage, we will post announcements and assignments. The students will also be able to post questions and discussions in a forum style manner, either to their instructors or to their peers.

back to top

Requirements

Each student will need to write two paper reviews each week, present once in class (depending on enrollment), participate in class discussions, and complete a project (done individually or in pairs).


The final grade will consist of the following
Participation (attendance, participation in discussions, reviews) 15%
Presentation (presentation of papers in class)25%
Project (proposal, final report)60%

Paper reviewing

Every week (except for the first two) we will read 3 to 4 papers. The success of the discussion in class will thus be due to how prepared the students come to class. Each student is expected to read all the papers that will be discussed and write two detailed reviews about the selected two papers. Depending on enrollment, each student will need to also present a paper in class. When you present, you do not need to hand in the review.

Deadline: The reviews will be due one day before the class.

Structure of the review
Short summary of the paper
Main contributions
Positive and negatives points
How strong is the evaluation?
Possible directions for future work

Presentation

Depending on enrollment, each student will need to present a few papers in class. The presentation should be clear and practiced and the student should read the assigned paper and related work in enough detail to be able to lead a discussion and answer questions. Extra credit will be given to students who also prepare a simple experimental demo highlighting how the method works in practice.

A presentation should be roughly 20 minutes long (please time it beforehand so that you do not go overtime). Typically this is about 15 to 20 slides. You are allowed to take some material from presentations on the web as long as you cite the source fairly. In the presentation, also provide the citation to the paper you present and to any other related work you reference.

Deadline: The presentation should be handed in one day before the class (or before if you want feedback).

Structure of presentation:
High-level overview with contributions
Main motivation
Clear statement of the problem
Overview of the technical approach
Strengths/weaknesses of the approach
Overview of the experimental evaluation
Strengths/weaknesses of evaluation
Discussion: future direction, links to other work

Project

Each student will need to write a short project proposal in the beginning of the class (in January). The projects will be research oriented. In the middle of semester course you will need to hand in a progress report. One week prior to the end of the class the final project report will need to be handed in and presented in the last lecture of the class (April). This will be a short, roughly 15-20 min, presentation.

The students can work on projects individually or in pairs. The project can be an interesting topic that the student comes up with himself/herself or with the help of the instructor. The grade will depend on the ideas, how well you present them in the report, how well you position your work in the related literature, how thorough are your experiments and how thoughtful are your conclusions.

close Detailed Requirements

back to top

Syllabus

The first class will present a short overview of various machine learning techniques, however, the details will be covered when reading on particular topics. Readings will touch on a diverse set of topics in Computer Vision. The course will be interactive -- we will add interesting topics on demand and latest research buzz.


Machine Learning
convolutional neural networks
recurrent neural networks
neural networks on graphs
generative models (GAN, variational autoencoders)
reinforcement learning
graphical models
Computer Vision
object detection
semantic, instance segmentation
action recognition
stereo / flow
tracking
captioning, VQA, retrieval
3D scene understanding
image/video generation, style transfer
close Tentative Schedule

back to top

Schedule (tentative)

DateTopicReading / MaterialSpeakerSlides
Jan 10Admin & Introduction  Sanja Fidleradmin
Convolutional Neural Networks
Jan 10Convolutional Neural Nets (tutorial)Resources: Stanford's cs231 class, VGG's Practical CNN Tutorial
Code: CNN Tutorial for TensorFlow, Tutorial for caffe, CNN Tutorial for Theano		  Amlan Kar, Chaoqi Wang
[pdf]
Jan 17CNNs, DetectionDynamic Routing Between Capsules   [PDF]
Sara Sabour, Nicholas Frosst, Geoffrey E Hinton		 Sara Sabour, Nicholas Frosst (invited)
Overview of Object Detection  Bin Yang (invited)
Jan 24CNNsDeformable Convolutional Networks  [PDF]
Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei

Robin Swanson[pdf]
DetectionYOLO9000: Better, Faster, Stronger  [PDF]
Joseph Redmon, Ali Farhadi		Haris Khan[pdf]
Segmentation[pdf]
Jan 31CNNs, SegmentationMulti-Scale Context Aggregation by Dilated Convolutions  [PDF]
Fisher Yu, Vladlen Koltun
Najmus Ibrahim[pdf]
Instance SegmentationDeep Watershed Transform for Instance Segmentation  [PDF]
Min Bai, Raquel Urtasun		Min Bai (invited)
Feb 7Instance SegmentationMask R-CNN  [PDF]
Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick		Aditya Sanghi[pdf]
Memory efficient DL The Reversible Residual Network: Backpropagation Without Storing Activations  [PDF]
Aidan N. Gomez, Mengye Ren, Raquel Urtasun, Roger B. Grosse

In-Place Activated BatchNorm for Memory-Optimized Training of DNNs  [PDF]
Samuel Rota Bulo, Lorenzo Porzi, Peter Kontschieder

Harris Chan[pdf]
StereoEfficient Deep Learning for Stereo Matching  [PDF]
Wenjie Luo, Alexander G. Schwing, Raquel Urtasun

End-to-End Learning of Geometry and Context for Deep Stereo Regression  [PDF]
Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, Adam Bry

Dominic Cheng[pdf]

back to top

Resources

Tutorials, related courses:


Software:


Popular datasets:


Online demos:


Main conferences:

back to top