Since the university’s inception, CalSouthern has worked hard to continuously improve its academic programs, as well as the resources and services it provides its learners. Courses and degree programs are regularly reviewed, and learners are surveyed to help the university gauge the effectiveness of its faculty, curriculum and the institution as a whole.
For several years now, much of this work has been the responsibility of CalSouthern’s Institutional Research and Assessment Department (IRAD). If you are not yet familiar with IRAD and the work it does, that will soon change. Assessment has become an even more critical priority at CalSouthern. Over the past two years, the university has devoted increasing resources to IRAD, restructuring the department and its processes to make CalSouthern’s assessment methodology more systematic and data-driven.
As part of the increased emphasis on assessment, CalSouthern’s director of educational compliance, Nicole Lesher, has been named IRAD’s director. In the interview that follows, Nicole outlines IRAD’s mission, details how the assessment process works (including the ways current learners and alumni will participate and how the program will benefit them) and explain her vision for how IRAD’s work will evolve in the future.
What is IRAD’s mission?
Simply put, IRAD’s mission is to guide institutional assessment at all levels to improve the programs and services we offer our learners. It’s just good business to use data to make decisions that drive planning, budgeting and program development.
In addition to me, the department includes an assessment coordinator and an assessment analyst. Together, we lead and support all assessment activities at the university. However, that may be a bit deceiving. Academic and administrative staff as well as faculty (and learners and alumni, for that matter) are actively involved, too, depending on the program or service that is being assessed.
Don’t all organizations—on some level—assess processes and results, and then implement programs for improvement? How is IRAD different and what makes its methodology more effective?
Probably the most critical difference is that our assessment—and the resulting improvements we implement—are all data-driven. As your question shows, most people would assume that organizations typically use data to inform their decision-making. However, in many cases this just isn’t true.
In fact, that’s been the case in higher education. According to a 2009 survey titled, “More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education,” 20 percent of America’s 1,500 regionally accredited universities have no assessment staff whatsoever. You will see these numbers shift, though, as assessment becomes an increasingly higher priority in post-secondary education and as universities are asked to become more accountable to outcomes.
What makes us different and hopefully more effective is that we’re not only proficient at collecting data, we actually use the data that we collect on a daily basis, whether it’s to improve our academic programs or to enhance or broaden our student services. This has been a point of tremendous emphasis over the past 18 months or so and it will continue to be. And I think that’s what distinguishes us from many other organizations and institutions.
How has your background prepared you for this new role? Have you worked in assessment before?
I have worked in accreditation for the past 25 years and, of course, assessment and continuous improvement are attributes that any respected accrediting body is looking for. Assessment is something I have always been drawn to and passionate about, and I am thrilled to be taking a more active role. It is a core priority for CalSouthern and will do nothing but strengthen our university.
In addition, I have just completed the Western Association of Schools and Colleges (WASC) Assessment Leadership Academy. It’s a prestigious academy that turns away 50 percent of its applicants, and I am very honored and excited to have had the opportunity. It’s been an extraordinarily helpful program; it’s directly applicable to what we’re now accomplishing at CalSouthern. Every course I have taken has provided me with knowledge and best practices that I have been able to apply here at the university.
It’s been a lot of fun, too.
What are some of the ways in which assessment will benefit CalSouthern learners?
The most obvious way is that we are taking the data we gather and using it to improve our curriculum, as well as the services we offer our learners. And it’s important for learners to understand that the improvements won’t solely be for the benefit of future generations of students. We’re quite nimble and able to make many improvements very quickly.
For example, in October we began a course assessment of Psychology 86502. We collected embedded signature assignments and other feedback for that course and had it analyzed by two trained faculty mentors to determine whether our learners were achieving the desired learning outcomes for that course. Improvements were recommended and are now implemented. This example shows that ours is a process that allows us to revise and improve our courses—based on a systematic data analysis—in a matter of just a few months.
And the benefits to students will extend beyond this. We want to give learners more tools that enable them to understand where they stand in terms of achieving learning outcomes, and then provide them with opportunities to improve in any areas of weakness—all while they continue to work through the program. This is a very important, but perhaps overlooked benefit of the assessment process.
Will students be actively involved in assessment?
Yes. In fact, they already are. In some courses, they are asked to upload certain assignments into LiveText, which is the system that facilitates the collection and analysis of data related to student achievement. Students—and alumni, for that matter—are also invited to participate in program reviews. In addition, we frequently solicit learners to participate in focus groups and, of course, they provide feedback through student surveys. These surveys provide extremely valuable feedback, especially via the commentary sections.
Last year was a very busy one for your department. What were some of the programs and services you assessed in 2014?
The goal for 2014 was really to reinforce a culture of assessment. I wanted the entire CalSouthern faculty and staff to understand it, to get used to thinking about it, and to “live it” as a matter of course in their daily work. And I want us to love it! When you begin to truly live it on a day-to-day basis, see the data-driven improvements and measure the positive results, it becomes very rewarding and exciting. Everyone enjoys knowing that their work results in meaningful change.
As far as specific assessment initiatives, we assessed our written communication institutional learning outcome for all of our graduate programs. We also assessed four of our academic programs and two co-curricular activities. In addition, we analyzed internal processes within the registrar’s office, and assessed student retention university-wide.
What will you be looking at in 2015?
We will be assessing another institutional learning outcome—ethics—as well as several academic programs within the schools of business and psychology. We will also be assessing the Doctoral Project Boot Camp, as well as the law school’s Baby Bar Workshop.
What are some of your department’s broader goals?
We will be focusing on education and ease of use. For example, I want to make participating in the process easier for students through the development of a single sign-on into LiveText through our learning management system. I also want to educate our faculty regarding how to do assessment even more effectively. Another goal is to share assessment results with the entire CalSouthern community in a meaningful way, clearly communicating the benefits of the process to our learners.