ECE 594: Convex Optimization (Fall 2019)

Lectures

Tue/Thu, 5:00-6:15pm, SH 319 (Stevenson Hall)

Instructor

Shuo Han (hanshuo@uic.edu)
Office hour: Mon, 3:00-5:00pm, 1110 SEO

Teaching Assistant

TBA
Office hour: N/A

Course Description

This graduate-level course covers three main aspects of convex optimization: theory, algorithms, and applications (e.g., machine learning, signal/image processing, controls). After taking the course, students should be able to recognize convexity and use convex optimization to model and solve problems that arise in engineering applications. Students will also gain a basic understanding of how convex optimization problems are solved algorithmically so as to determine whether a given problem can be solved using off-the-shelf solvers.

Prerequisites

Good knowledge of linear algebra (e.g., as in ECE 550 or ECE 531). Exposure to probability (e.g., ECE 341). Familiarity with MATLAB.

Topics

• Introduction (including a review of linear algebra)
• Theory
  • Convex sets
  • Convex functions
  • Convex optimization problems
  • Duality
• Applications
  • Geometric problems: Projection, distance, covering
  • Machine learning: Regression and classification
  • Data processing: Estimation, reconstruction, denoising
  • Decision making under uncertainty: chance-constrained optimization, robust optimization
• Algorithms
  • First-order methods: Gradient methods, proximal methods
  • Second-order methods: Newton's method, interior-point methods
• Advanced topics (time permitting)
  • Algorithms for large-scale optimization
  • Convex relaxation of nonconvex problems

Grading

• Homework (30%): Homework sets are issued every Thursday and are due the following Thursday in class.
  • There will be about 10 homework sets in total.
  • Some homework sets will be designated as "half assignments."
  • The lowest one will be dropped. Namely, if there are X homework sets ("half assignments" will count as 0.5) in total, then the best X-1 sets will be used to compute your homework total grade.

• Midterm (30%): In-class, closed-book, and closed-notes. The date will be announced at a later time. You may use a single letter-sized double-sided "cheat sheet" during the exam. The cheat sheet may contain formulas, facts, definitions, and theorems. However, your cheat sheet may not contain worked examples. You must hand in your cheat sheet along with your completed exam. No calculators/computers are allowed in the exam.

• Final project (40%): It will consist of a project proposal, a midterm report, a final report, and an in-class presentation. The due dates and instructions of the report will be announced later. Students can work in groups of up to 3, but the report must include a statement describing the contributions by each group member. The grade will be determined based on the final report, presentation, peer evaluation, and level of contribution within group. See project guidelines for details.

• Guaranteed cutoff grades: A-85%, B-75%, C-65%.

Course Policy

• Check the Piazza page regularly for latest announcements and course material. The UIC Blackboard page will only be used for posting grades and lecture recordings.

• Homework
  • Collaboration on homework assignments is encouraged. You may consult the lecture notes, the textbook listed below, references mentioned in class, other students, the TA, or the instructor. You may also consult outside references not listed on the course webpage, provided that you cite those references in your homework solutions.
    
  • You cannot consult homework solutions from prior years or solution manuals. All solutions that are handed in should be written up individually and should reflect your own understanding of the subject matter at the time of writing.
  • MATLAB scripts and plots are considered part of your writeup and should be done individually (you can share ideas, but not code).
    
  • Show all your work to receive credits. However, we will deduct points from long needlessly complex solutions, even if they are correct. If you find that your solution to a problem goes on and on for many pages, you should try to come up with a simpler one. One useful way to shorten your solutions is to cite results derived in class or from the textbooks.
    
  • Make a copy of your homework prior to submission. You will be asked to provide your copy for any claims of missing homework.
  • Late homework: Late homework will not be accepted without proper supporting documents (e.g., a note from doctor or the Dean).
• Academic Integrity
  • We treat academic dishonesty seriously. All offenses are reportable to the University.
  • Identical solutions will receive no credits. This penalty applies to all parties involved.
  • For first-time offense in homework, the final letter grade of the offender will be lowered by one tier, e.g., from A to B.
  • For second-time offense in homework or cheating/plagiarism in exam or project, the offender will receive a letter grade of F.

• You should keep the course material (lecture notes, audio recordings, and homework problems and solutions) for personal use only and not post them on public websites (e.g., Course Hero).

• Students who wish to observe their religious holidays should notify instructor by the tenth day of the semester of the date when they will be absent unless the religious holiday is observed on or before the tenth day of the semester. In such cases, the students should notify the instructor at least five days in advance of the date when he/she will be absent. Every reasonable effort will be made to honor the request.

Course Text and References

The required textbooks for the course are:

• [BV04] S. P. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004. Online access
• [BN01] A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, SIAM, 2001. Online access