When I talk with colleagues new to assessment or work with institutions, I am often asked about what resources are available. I have aggregated the list below to answer some of those questions. Each resource serves a different purpose so I encourage you to review each item on the list to see how it may be helpful to you.
Continue reading “Essential Assessment Resources (Updated May 2018)”
This post was originally published in the New England Journal of Higher Education
In many ways, higher education has not changed in the nearly 1,000 years since the first university was founded in Bologna, Italy in 1088. Many courses still have professors or “masters” lecturing in front of students, with exams being reproduction of facts learned in lectures. But in other ways, higher education changes daily. A brief perusal of headlines from the Chronicle of Higher Education or Inside Higher Ed demonstrates the changing landscape of colleges and universities. Given this perpetual state of chaos, colleges and universities not only need to withstand and manage change, they need to leverage that environmental change, and sometimes even foster change to better meet the needs of diverse stakeholders. To achieve their goals, colleges and universities need to become “learning organizations.”
Continue reading “Organized Anarchies: 13 Steps to Building a Learning Organization”
NOTE: This was originally posted on here.
About 16 months ago our institution started an institutional assessment steering committee (IASC) as a vehicle to improve assessment on campus. Composed of representatives from administrative and academic units across campus, we have been engaging in a number of activities to help increase confidence and capacity in assessment and also begin to build a culture of assessment.
While there are variations, the basic assessment cycle comprises four steps: identify outcomes, develop strategies to achieve the outcomes, gather and interpret data related to outcome achievement, and close the loop. A great deal of emphasis is placed on the first three steps, but less on the final and perhaps most important step: closing the loop. Assessment is meaningless without putting the findings into action.
(NOTE: This blog posted was also posted on http://www.cas.edu/blog_home.asp?display=73)
Recently, I attended the Assessment Institute in Indianapolis. During a discussion with a friend she introduced me to the concept of improvement science. I had no idea there was such a field of study, so I immediately search Google to learn more. After reading a few online articles, I was intrigued enough to purchase for my flight how a copy of New Directions for Evaluation (2017), number 153 as the issue was dedicated to improvement science.
To those implementing assessment, the concept is familiar. According to Moen, Nolan, and Provost (as cited in Christie, Lemire, & Inkelas, 2017, p. 11), “improvement science is an approach to increasing knowledge that leads to an improvement of a product, process, or system. That certainly sounds like the continuous improvement purpose of assessment. What is a little different is that between improvement science explicitly focuses on systems thinking. Because of this systems approach, a central principle of this approach is change management, as it is critical for improvement (Christie, Lemire, & Inkelas, 2017).
The journal issue goes into detail regarding the components of improvement science, an operational model, and then case studies to illustrate what the authors are describing. But, there was one discussion point that jumped out to me. A cycle of testing and learning is foundational to improvement science (Lemire, Christie, & Inkelas, 2017). This cycle includes four steps: Plan, Do, Study, Act (PDSA).
In this cycle, improvement occurs in small steps to see how effective implementation is and to better understand concomitant issues. The PDSA cycle is critical to change management as there are many issues that impact implementation of an improvement including user buy-in, resources, and sometimes even politics.
The PDSA cycle can easily be applied to assessment in higher education. Once the assessment is complete and recommendations for change are made, those changes should be implemented in small steps starting first with planning out the execution. After the planning takes place, a small-scale version of the improvement is implemented. Third, there would be assessment of that small-scale improvement. With this information, the improvement is scaled up.
Here is a basic example of how this might look in assessment practice. The orientation office at a small college completed a CAS self-study and learned that their program regarding sexual assault prevention did not achieve the learning outcomes intended. A recommendation in their self-study report is to contract with a professional organization that has developed a program called “Consent4U.” However, the program costs almost half of the entire orientation budget. While the benefits could be great, the orientation director wants to test the program first before making the significance financial investment. The director ran a pilot of the program with one of the residence hall learning communities. To understand change in learning, the director developed a pre- and post-test regarding sexual assault. In the first step of the cycle, “plan,” the director collaborated with the residence hall director to schedule a time and partnered with a faculty member in sociology to create the pre- and post-test. The program was implemented as part step 2, “do.” After the program was done, the director administered the post-test and “studied” the results. Based on the data, the director determined that given the amount of learning students obtained from this program, they were going to implement the Consent4U program and they requested additional funding from the provost for the program.
Some of the tenets of improvement science mirror those of assessment. However, the Plan, Do, Study, Act model may provide a way to manage the change that comes with making an improvement.
References
Christie, C., Lemire, S., & Inkelas, M. (2017). Understanding the similarities and distinctions between improvement science and evaluation. New Directions for Evaluation, 153, 11-22.
Lemire, S., Christie, C., & Inkelas, M. (2017). The methods and tools of improvement science. New Directions for Evaluation, 153, 23-34.
NOTE: This blog post is inspired by Margaret Leary. During the ACPA16 Convention she and I discussed ideas she had regarding how assessment can foster organizational change. That conversation me to learn more about organizational change and how assessment might be related to it. This blog post is a result of that research and contemplation.
Continue reading “Assessment as a Tool for Organizational Change”
Assessment isn’t an activity. It’s a state of mind.
The statement above has been my mantra for the past five years. Too often, assessment is seen as an afterthought rather than an integral part of the program or service planning and implementation process.
Interestingly, this statement didn’t occur to me when I was in the midst of an assessment project, teaching class, facilitating a workshop, or consulting with a division of student affairs. It occurred to me in a grocery store. Yes, in the deli aisle waiting on maple and brown sugar ham, to be exact. I won’t bore you with the details here, but you can read more regarding this “evaluative epiphany” here: http://bit.ly/1MSqT2x.
In this post, I don’t want to talk about the statement itself, but what it represents. Recently, I echoed this refrain during a webinar sponsored by ACPA’s Commission for Assessment and Evaluation. The topic was “Assessing Cultures of Assessment.” (You can access the webinar here by entering your name and email) At the outset, I suggested that this statement, Assessment isn’t an activity. It’s a state of mind was a definitive sign that a culture of assessment existed in an organization. When assessment is a state of mind, it is infused into every aspect of individual or organizational practice including planning, implementation, and – of course – evaluation. At this stage, assessment becomes an unconscious, embedded element of everyday work.
So, how does one arrive at this destination where assessment is a state of mind?
I think the four stages of competence, originally described as the four stages of learning, and outlined by Linda Adams (2011), can be a helpful guide. These stages are
In the unconscious incompetence stage, people are not engaged in assessment and don’t know why it’s important. They are unable articulate its value or purpose. This stage described most of our field 10-15 years ago. The prevailing attitude was that assessment was simply a fad and we just needed to “wait it out.”
Once awakened to the both the necessity and benefit of engaging in assessment, people realize they need to it, but are not sure how to do it. This is conscious incompetence. The first step is to identify the actual skills and knowledge needed to perform assessment. The 2nd Edition of the ACPA/NASPA Professional Competencies (released in August 2015) provides a framework for skill and knowledge development regarding assessment, evaluation, and research. A great way to develop these skills is to seek out books, workshops, and other resources to improve skill and knowledge. Fortunately, more and more resources are available. There are four new student affairs assessment books coming out this academic year. How exciting! One centers on leading assessment for student success and the other focuses on coordinating divisional assessment. Two more covering more general assessment practice will come out in early March. Workshops and conferences are another way to build assessment competence. Each summer, ACPA holds its Student Affairs Assessment Institute and NASPA sponsors its Assessment and Persistence Conference. Both associations have special interest groups regarding assessment for networking and professional development. (ACPA’s Commission for Assessment and Evaluation and NASPA’s Assessment, Evaluation, and Research Knowledge Community). In addition, many master’s level preparation programs have assessment courses to help students move from this stage of competence to the next.
As people gain skill and knowledge, they become conscious competent. They value the assessment process and are continually attentive when performing it. Assessment takes effort and concentration at this stage but continues to become more comfortable and more frequent.
At the apex of this competence hierarchy is unconsciously competence. At this stage, assessment has been performed so often that it becomes habit or second nature and is integrated into daily practice and processes. It is important to note that learning should still take place. Even in assessment, lifelong learning is important as new scholarship is constantly being created.
As I mentioned at the beginning of this post, the statement Assessment isn’t an activity. It’s a state of mind came to me in a grocery store. In the deli aisle, I recognized that I was shopping in a way that mirrored the assessment process. My goal was to be both effective and efficient and those goals shaped by actions from planning to execution. Assessment had become so unconscious that it was integrated into other parts of my life. While, you may not be as much of a geek as me and want assessment to infiltrate your personal life, I do hope you develop your assessment skill and knowledge to the point that you are unconsciously competent.
At what stage are you? What can you do to get to the next level?
References
Adams, L. (2011). Learning a new skill is easier said than done. [Blog]. Retrieved from http://www.gordontraining.com/free-workplace-articles/learning-a-new-skill-is-easier-said-than-done/
NOTE: This was originally posted to ACPA Developments Volume 13, Summer 2015 Issue (June 2015)
Please visit ACPA Video On Demand where I discuss this concept in a video interview.
On a Friday night in late April in 1987, during the spring of my sophomore year, I was attending a movie with friends in Brody Hall at Michigan State University. Back then, the Residence Hall Association screened movies in select lecture halls across campus. We didn’t have Netflix back then. We didn’t even have cable.
After the movie an event happened that changed my life forever.
On my way out of the lecture hall I ran into a friend of mine, Stacy Huffman, from my hometown of Saginaw, MI, who had also attending the movie. As friends who hadn’t seen each other in a while (it was a 21 acre campus of 30,000+ students), we caught each other up on our lives. As the semester was nearing an end, we discussed our summer plans in Saginaw. Stacy said that she was going to be staying on campus working for the Academic Orientation Program. She continued that they were looking for one more male orientation leader and encouraged me to apply. The interviews were the next day and there was an opening at 8am.
Wiping sleep from my eyes I got early, ate my Wheaties, and headed to the interview. A few days later I was notified that I had been selected as an orientation leader. Serendipity opened the door to my first student affairs job and my career – although I didn’t comprehend it at the time.
Fast forward to June of 1993. Frustration and anxiety was setting in because I had recently graduated with my master’s degree from MSU and, unlike my classmates, was still job searching. And searching. And searching. There had been a few phone interviews and even a couple of campus interviews, but nothing panned out. Self-doubt became all-consuming as I wondered why no one wanted to hire me despite what I thought were excellent grades and extensive experience.
When hope was waning, serendipity struck again.
I received a call from the Department of Residence Life at the University of New Hampshire for an on-campus interview. A hall director job at UNH was my “perfect” job from the start of my search. Unfortunately for me, a few weeks before, shortly after the phone interview, I was told that they had hired other individuals for their open spaces. However, a residence hall director had decided to leave UNH in June, opening up a position. I jumped at the chance for a campus interview. Two months later I was packing up a U-Haul to make the trip Durham, New Hampshire for my “perfect” job. This position created the foundation of who I am as an educator today. Plus, UNH was where I met my current partner. Call it destiny. Call it kismet. Or maybe it was just chance. But, it this result was certainly not intentional.
The orientation leader position began my career in student affairs but working for the Department of Residence Life at the University of New Hampshire changed how I approached my job. While I learned many things during my six years in residence life, one word has stuck with me and was the concept that compelled me into assessment work. That word is “intentionality.” During numerous staff and supervision meetings hall directors discussed how we were being intentional in our outreach to students and in our programming. Intentionality became a mantra for my work then and the driving force for the assessment work I have been undertaking the past 15 years.
Synonyms for serendipity include chance and accident while synonyms for intentional include designed, deliberate, and planned. While not antonyms of one another the concepts of serendipity and intentional are opposed to each other. I think serendipity vs. intentionality is a tension we continue to struggle with today in regard to learning in higher education. All too often we assume or think that learning is happening outside of the classroom and we aren’t doing as much as we can to intentionally foster it. We can no longer rely on serendipity to ensure student learning and success.
External demands for accountability are increasing the need for intentionality. The completion agenda dominates the national discourse of higher education. Students, parents, and legislatures are questioning the return on investment of a college education and want to known what students are learning after paying exorbitant amounts of money. College administrators are questioning the value of student affairs in an era of service provision where students are customers and clients. During a program session with college presidents at ACPA15, when asked what the priority should be for student affairs, all panelists stated that college student educators need to be able to demonstrate how they and their work positively contributes to student learning and retention. The completion agenda at the federal and state level is a major thrust behind the current accountability movement in higher education. This emphasis on retention is not simply because of the individual benefits for students who graduate but because of the financial impact of tuition revenue and state appropriations for colleges and universities.
More important than external calls for accountability are the internal calls for accountability that originate from inside each one of us as college student educators. We chose this profession for our careers because of the want and need to positively impact lives of college students. Thus, we strive to do the best job we can to assist students. As a field, we need to make intentionality an interwoven thread in the fabric of everyday practice to ensure student success, both academically and personally.
Intentionally isn’t rocket science. It can be explained in a four-step process outlined by Linda Suskie (2009). The first step is to begin with what you want students to achieve (aka outcomes which an be learning, operational, or program). Once outcomes are identified, existing literature and other evidence are used to identify strategies to foster those outcomes in step 2. Step 3 is to collect and analyze data to determine if the outcomes are achieved and how outcome achievement can be improved. The final step is the most important – closing the loop by making improvements. Intentionally is a process, not a destination.
Figure 1. Assessment cycle by Suskie, L. (2009). Assessing student learning: A common sense guide (2nd Ed.). San Francisco: Jossey-Bass.
How Can ACPA Help You Be More Intentional
ACPA’s focus is student learning. As stated in our mission “ACPA supports and fosters college student learning through the generation and dissemination of knowledge, which informs policies, practices and programs for student affairs professionals and the higher education community” (ACPA, 2015). There are many ways ACPA can help you foster student learning, development, and success.
Individuals can leverage ACPA to help foster and support student learning by accessing the scholarship that is generated through various outlets. ACPA’s signature publication is the Journal of College Student Development. There are articles in each issue pertinent to faculty and practitioners alike. Previous issues include scholarship regarding experiences of Asian American and Latino/a students at an HBCU, the academic performance of Black emerging adults, a method to increase the grade point averages of fraternity members, and others.
About Campus is a scholarly publication directed towards practitioners. This bi-monthly magazine provides insights to improve practice in higher education. Past issues include articles addressing positive psychology, long-term success in work and life, as well as high impact practices.
In addition to these publications, and Developments which you are reading now, ACPA also sponsors books and monographs. This past year ACPA published Job One 2.0: Understanding the Next Generation of Student Affairs Professionals which focuses on the first jobs of college student educators as well as Working With Students in Community Colleges: Contemporary Strategies for Bridging Theory, Research, and Practice which provides approaches to help community college students be successful. Additional publications can be found here. ACPA books and monographs coupled with our other publications provide faculty and practitioners a library of research and scholarship to inform further research and practice.
Another major way that ACPA supports student learning, development, and success is helping college student educators across the spectrum of higher education bridge theory to practice. Some of this work is done through our acclaimed professional development events. Most of our activities are driven by curricula rooted in research. Upcoming events include
You can find additional professional development events here.
Theory to practice is also addressed in other venues including ACPA On Demand and Student Affairs Live, sponsored by ACPA. ACPA On Demand is collection of pre-recorded videos covering a variety of topics relevant to college student educators. Student Affairs Live is a weekly talk show viewed via GoogleHangouts covering critical emergent issues in higher education. Recorded versions of the shows are available the Student Affairs Live website and podcasts are available in iTunes.
As college student educators, we need to be much more intentional in how we cultivate student learning and development. While learning may happen by serendipity, we can’t rely on that. Our students – our future – are too important to rely on chance. ACPA is your go-to source for research, scholarship, and proven practices for fostering student success. Tap into the resources now!
References
ACPA (2015). Mission, vision, and values. Retrieved on April 8, 2015 from http://www.myacpa.org/values
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd Ed). San Francisco: Jossey-Bass.
NOTE: This was posted as part of the ACPA President’s Blog for May 2015
Please visit ACPA On Demand for an interview of me discussing this topic.
Regardless of the news source one turns to, the completion agenda is in the headlines of the higher education section. It’s also the hot topic on the radar screens of policymakers’ and college presidents. While many higher education officials are concerned with the impact this federal policy has on higher education at the national, state, and local level, we as college student educators can actualize the opportunity the completion agenda presents. This policy provides a chance to demonstrate the impact we have on the success of college students.
In February of 2009, President Obama outlined his completion agenda with the goal of attaining the world’s highest proportion of college graduates by 2020 (Kanter, Ochoa, Nassif, & Chong, 2011). According to 2012 data from the Organization for Economic Cooperation and Development (OECD), the United States ranks 19th out of 28 countries studied (Weston, 2014). There is some work to be done on this front.
President Obama presented three reasons supporting this policy. The first was that a college degree is required for 60% of jobs. In addition to being a requirement for the majority of jobs, an individual with a college degree earns 40% more over a lifetime than those without. This is a significant financial benefit. Finally, President Obama believed that an educated citizenry was needed for an engaged democracy (Kanter, Ochoa, Nassif, & Chong, 2011).
Future of Higher Education Funding
This policy position has influenced conversations regarding state funding for higher education with a new focus on program-based budgeting. With this budgeting model state allocations are based on results, in this case completion rates, rather than the number of students enrolled, which has been the traditional model. As of January 2015, 34 states had some form of program-based budgeting for higher education funding although the percentage of overall funding based on graduation percentages does vary (National Conference of State Legislatures, 2015).
We can debate the pros and cons of this form of program-based budgeting, but it appears to be the future higher education funding model. With continued financial issues impacting colleges and universities including Louisiana State University (O’Donoghue, 2015), student affairs budgets are in jeopardy requiring vice presidents to not only advocate for more resources but also defend current resources.
At ACPA16 in Tampa, ACPA sponsored a panel of five college presidents who came up through the ranks of student affairs. These presidents included Joe Bertolino from Lyndon State College, Marybeth Cooper from Springfield College, Tom Jackson from Blackhills State University, Steve Tyrell from North Country Community College, and Karen Whitney from Clarion University. During this panel each president emphatically stated the need for college student educators to demonstrate their impact on retention and graduation. The continued that support of retention and graduation is the key to demonstrating value within a college or university.
Turning Crisis Into Opportunity
Some may view the issue of declining resources and centralized focus on retention as a crisis facing college student educators – a crisis that shifts to a financial bottom line, not students. However, a focus on retention actually centers on students and how educators can help each individual graduate. When one takes a moment to reviews the literature regarding retention and graduation, college student educators play a key role many of those success factors.
John Braxton, William Doyle, Harold Hartley, Amy Hirschy, Willis Jones, and Michael McLendon published Rethinking College Student Retention in November of 2013. (For a review of the book in the Journal of College Student Development 55(6), visit https://muse.jhu.edu/journals/journal_of_college_student_development/v055/55.6.henning.html) In this text, the authors discuss their theories of retention for residential colleges and commuter institutions explaining how they tested each. The factors influencing retention in residential colleges include:
The components in the theory of retention at commuter colleges include the following:
Upon reviewing these two theories, it’s clear that college student educators play a major role in retention. Here are just some of the roles we play:
What Can We Do and How Can ACPA Help
First, college student educators must become familiar with retention theory. As professionals it is our responsibility to bring theory into practice. When we use theory, we build and hone our practice on research rather relying on anecdotes and ideas of what “might work.” Being familiar with theory also permits college student educators to better tell the story of our impact on student success. And, in language that faculty are familiar – research and scholarship. And perhaps more importantly, we will be able to describe the outcomes of our work in the words of university leaders and educational policy makers – retention and graduation.
The focus on completion presents college student educators with an opportunity to be more intentional in our work but to also demonstrate our impact beyond housing students, feeding students, and helping them have fun.
ACPA is here to help you leverage this opportunity. Review the research and scholarship that ACPA generates and disseminates in the Journal of College Student Development, About Campus, Developments, as well as ACPA sponsored books and monographs. Bridge theory to practice by attending a professional development institute such as the Student Affairs Assessment Institute or the Residential Curriculum Institute. Connect with colleagues doing similar work. ACPA’s state chapters, coalitions (formerly standing committees), commissions, and newly created communities of practice provide these connection opportunities.
We have an opportunity in front of us. Rather than seeing it as a crisis or obstacle, we should leverage it to demonstrate our impact, but more importantly to help students succeed. ACPA can assist you in your journey. Tap into the resources now!
References
Donoghue, J. (2015). LSU drafting ‘academic bankruptcy’ plan in response to budget crisis. The Times-Picayne (2015, April 22). Retrieved from http://www.nola.com/politics/index.ssf/2015/04/lsu_academic_bankruptcy.html
Kanter, M., Ochoa, E., Nassif, R., & Chong, F. (2011). Meeting President Obama’s 2020 college completion goal. Retrieved from http://www.ed.gov/news/speeches/meeting-president-obamas-2020-college-completion-goal.
National Conference of State Legislatures (2015, January 13). Performance-Based Funding for Higher Education. Retrieved from http://www.ncsl.org/research/education/performance-funding.aspx
Weston, L. (2014). OECD: The US has fallen behind other countries in college completion. BusinessInsider (2014, September 9). Retrieved from http://www.businessinsider.com/r-us-falls-behind-in-college-competition-oecd-2014-9 .
I recently wrote a post for Student Affairs Feature entitled “Considering Assessment as Investment.” What would you get on your investment in assessment?
Gavin W. Henning 