top of page

Evaluation of Indiana University's Ph.D. Program in Instructional Systems Technology

Context

In R561, a course on evaluation that I took in the spring of 2022, students were asked conduct each step of an evaluation project focused on the evaluation of Indiana University's Ph.D. program in Instructional Systems Technology. The project was simulated, but the scenario and data were based on an evaluation that actually happened. All students worked on the same project in groups or individually. I chose to work alone. 

Summary

The first step of this project involved responding to a request for proposal from the Department of Instructional Systems Technology at Indiana University—Bloomington. This required summarizing the evaluation plan, process, and budget.

​

The second step was to create the data collection instruments that had been proposed in the initial proposal. I drafted surveys that would be sent to the program's students, faculty, and alumni.

​

The final step of the project was to analyze a set of data that was hypothetically returned by the data collection instruments and write a report summarizing the evaluation and resulting recommendations. We received three sets of data from students, faculty, and alumni.

Reflection

Evaluation, like analysis, is a critical tool in any leader's decision making toolkit. We must be able to follow directions like those in an RFP meticulously, home in on clear goals, establish a well thought through process, collect data, analyze it, and make decisions on how to move forward. This project was eye-opening for me. It helped me realize how much work goes into a thorough, systematic evaluation. It helped me understand how short seven months can be when you're putting together an evaluation schedule and how quickly you can spend $30,000. It gave me hands on experience with trying to craft smart, triangulating data collection instruments and with processing and deriving insights from big messy data sets. In my current role I oversee a multi-faceted curricular program for our members and I will be using many of the skills I sharpened with this project to evaluate and improve it. 

Key Takeaways

  • Evaluation: This project has given me a framework for methodical evaluation of a program involving multiple stakeholders.

  • Data collection and analysis: This project gave me practice creating effective data collection instruments and generating insights from data.

  • Report writing: This project gave me a structure for writing a long form evaluation report and practice on representing data within.

bottom of page