Assignments

The main goals of this class are to help you design, critique, code, and run rigorous, valid, and feasible evaluations of public sector programs. Each type of assignment in this class is designed to help you achieve one or more of these goals.

Weekly check-in

Each week you’ll submit a list of three most interesting and three most unclear things from the readings. (See the complete instructions and details here).

Problem sets

To practice writing R code, running inferential models, and thinking about causation, you will complete a series of problem sets.

You need to show that you made a good faith effort to work each question. I will not grade these in detail. The problem sets will be graded using a check system:

  • ✔+: (33 points (110%) in gradebook) Assignment is 100% completed. Every question was attempted and answered, and most answers are correct. Document is clean and easy to follow. Work is exceptional. I will not assign these often.
  • ✔: (30 points (100%) in gradebook) Assignment is 70–99% complete and most answers are correct. This is the expected level of performance.
  • ✔−: (15 points (50%) in gradebook) Assignment is less than 70% complete and/or most answers are incorrect. This indicates that you need to improve next time. I will hopefully not assign these often.

You may (and should!) work together on the problem sets, but you must turn in your own answers. You cannot work in groups of more than four people, and you must note who participated in the group in your assignment.

Evaluation assignments

For your final project, you will conduct a pre-registered evaluation of a social program using synthetic data. To (1) give you practice with the principles of program evaluation, research design, measurement, and causal diagrams, and (2) help you with the foundation of your final project, you will complete a set of four evaluation-related assignments.

Ideally these will become major sections of your final project. However, there is no requirement that the programs you use in these assignments must be the same as the final project. If, through these assignments, you discover that your initially chosen program is too simple, too complex, too boring, etc., you can change at any time.

These assignments will be graded using a check system:

  • ✔+: (33 points (110%) in gradebook) Assignment is 100% completed. Every question was attempted and answered, and most answers are correct. Document is clean and easy to follow. Work is exceptional. I will not assign these often.
  • ✔: (30 points (100%) in gradebook) Assignment is 70–99% complete and most answers are correct. This is the expected level of performance.
  • ✔−: (15 points (50%) in gradebook) Assignment is less than 70% complete and/or most answers are incorrect. This indicates that you need to improve next time. I will hopefully not assign these often.

Exams

There will be two exams covering (1) program evaluation, research design, and the mechanics of causation, and (2) the core statistical tools of program evaluation and causal inference.

These exams will occur in-person during class. They will be done collectively by all of you together.

I used to offer these online on iCollege and they were open book, open notes, and open internet, but LLMs have made that method of exams completely useless. Even if I say that AI use is banned, Google’s search AI appears at the top of every search result, so it’s impossible to avoid LLM output. I hate it all so much.

The rationale for allowing these exams to be open book/note/internet is that in the real world, you’ll consult with other resources when working on statistical things, so you might as well do that here too. The point of these exams is not to trip you up on little definitional things (i.e. “What did the 4th paragraph of the 2nd section of chapter 3 say about controlling for mediators?”), but to give you a chance to think more closely about the class materials and practice working with them in different ways.

This group exam approach is a new idea I’m experimenting with this semester—you’re the first class I’ve done this with. It will likely be open note and open book, but not open other-sites-on-the-internet (and definitely not open LLM).

There are study guides for each of the exams:

#TidyTuesday

At some point before the end of the semester, you’ll need to (1) take a dataset, (2) do something neat with it, and (3) share it with the public somehow. (See the complete instructions and details here).

Final project

At the end of the course, you will demonstrate your knowledge of program evaluation and causal inference by completing a final project.

Complete details for the final project are here.

There is no final exam. This project is your final exam.