STATS 606
Syllabus
Course description
STATS 606 is an introduction to mathematical optimization with emphasis on theory and algorithms relevant to statistical practice. The goal of the course is proficiency with common mathematical optimization techniques in statistics and data science. At a high-level, the course consists of 4 parts:
- convex calculus: This part of the course covers ways of formulating (statistical) problems as (convex) optimization problems. Most of it is based on chapters 2 and 3 of the textbook Convex Optimization, but there is some additional material on subgradients that we shall provide additional references for.
- modeling: This part of the course covers ways of formulating (statistical) problems as (convex) optimization problems. It is based on chapters 4 to 8 of the textbook. If there is time, we shall cover (some) approximation algorithms, robust optimization, stochastic control, and variational inference.
- algorithms: This part covers first-order and decomposition/splitting algorithms for (convex) optimization problems. We picked these algorithms because they are the ones that you are most likely to encounter in (statistical) practice.
- theory: This (small) part on optimization theory culminates in a study of perturbation of optimization problems.
The course is roughly split 20/30/40/10 between the parts respectively.
Prerequisites: You should have a good grasp of vector calculus (at the level of MATH 215), linear algebra (at the level of MATH 217) and intermediate (non-measure theoretic) probability (at the level of STATS 510). We shall review relevant concepts as they arise, but this should not be the first time you see them.
References and textbooks
The first and second parts of the course (on convex optimization)are based on the first 8 chapters of Boyd and Vandenberghe’s book on convex optimization. There is no textbook for the other parts of the course; see the schedule for relevant references.
Course staff
Grading
Your grade is determined by your overall score:
- 50% problem sets,
- 50% project.
Students who obtain an average of at least 90%, 80%, and 70% will receive grades of at least A-, B-, and C- respectively. We may lower the cutoffs at the end of the semester, but we will not raise them.
Problem sets
Problem sets are assigned weekly on Fridays and due at noon ET the following Friday. If you need an extension on a problem set, you must contact the course staff at least 24 hours before the due date. We want you to complete the problem sets because they are an integral part of the course, so we are generous with extensions. On the other hand, we do NOT accept late problem sets.
You must typeset your solutions with LaTeX. If you submit handwritten solutions to a problem set, 1 pt will be deducted from your score on the problem set. If you are new to LaTeX, we recommend editing LaTeX documents on Overleaf. The Overleaf documentation is also a great place to learn LaTeX. In fact, when you search for LaTeX related searches, the Overleaf documentation is often one of the first results that come up!
Grading: Problem sets are graded on a scale of 1 to 4. Each problem in a problem set is graded from 1 to 4: 4 pts = essentially correct, 3 pts = minor mistakes (solution is qualitatively correct), 2 pts = on the right track, 1 pt = FUBAR. Your grade on a problem set is the average of your grades on the problems. You are encouraged to collaborate on problem sets with classmates, but the final write-up (including any code) must be your own.
Collaboration: You are encouraged to collaborate on problem sets with classmates, but the final write-up (including any code) must be your own.
Keeping up with the course online
We strongly suggest you take the course in-person, but the course is set up so that you can keep up online if necessary. Most course material is available on Canvas and the course website, so please check it regularly for updates. You can also
- view lecture and lab recordings on Canvas/YouTube,
- communicate with the course staff and other students on Slack,
- submit assignments on Canvas.
Academic misconduct
The College of LSA prohibits all forms of academic dishonesty and misconduct. Minor infractions usually result in a zero on the assignment and a one letter grade reduction; more serious or repeated infractions will result in a failing grade and additional sanctions imposed by the Office of the Assistant Dean. For more information, including examples of behaviors that are considered academic misconduct and potential sanctions, please see LSA’s Community Standards of Academic Integrity.
Accommodations for students with disabilities
We work with Office of Services for Students with Disabilities to determine appropriate accommodations on an individual basis. Please follow the instructions on their website to request accommodations.