Marilee Bresciani Visits UNC
Answers for Additional Questions
Workshop participants were invited to share any 
questions they still had at the end of the day. While we aren’t able to answer all of the questions without more information, we have responded to the more general questions here. If you need additional information or clarification, please contact the assessment staff.
- Will we be able to do this effectively?
- Who are our on-campus resources to continue to mentor us and what are the administration’s expectations around this?
- How do you decide where to start?
- Shouldn’t we identify needs before learning objectives?
- How are we supposed to implement this?
- When do you know when you reach the top?
- How have other schools implemented this information?
- For an average sized department, how many outcome-based assessments are reasonable to do per year to determine effectiveness?
- How to best identify the appropriate evaluation method for certain outcomes and programs?
- Will UNC put together a tool to use for developing outcome based assessment for consistency campus wide?
- How do we get access to TracDat – cost, implementation, etc., and where can we get help to set up the plan online?
- How will the University use the information?
- How effective is visual documentation in evaluation?
- How can we better assess the reflective learning that occurs after the program/intervention and before graduation?
- How can the various needs be identified in a response-based office?
- What will this look like in my office as far as collecting evidence for our main objectives?
- Need more information? Contact the Office of Assessment.
Will we be able to do this effectively?
Absolutely! You probably already have many of the elements of a successful assessment plan in place. Whether you are just starting out or are just looking for ways to enhance current practice, the assessment staff are here to support you and ensure that your assessment program is effective.
Who are our on-campus resources to continue to mentor us and what are the administration’s expectations around this?
Programs are expected to engage in meaningful and ongoing assessment practices to ensure program quality and continuous improvement. UNC has established the Office of Assessment to provide ongoing support for programs’ assessment needs. Professional journals and participation in conferences and workshops are also good sources of support. There are also a variety of listserves and other resources available on the World Wide Web. North Carolina State University has a particularly comprehensive list of resources.
How do you decide where to start?
A good place to start is by contacting the assessment staff to set up a consultation. They can help you evaluate where you are now and provide suggestions for the best place to begin. The staff will tailor their service to your needs, ranging from helping you refine and/or develop goals and outcomes to providing suggestions for assessment methods and tools.
Shouldn’t we identify needs before learning objectives?
Identifying needs is actually the part of the outcomes assessment process where we ask, “What do students need in order to attain the desired outcomes resulting from participation in our program, service, or other learning activity?” Needs assessment helps us understand where the gaps exist between the intended outcomes and the actual outcomes. It can also help us understand the unique needs of an individual or group of students in relation to the outcomes we intend. Needs assessment can be an effective part of most programs’ assessment processes.
“Assessing student needs is the process of determining the presence or absence of the factors and conditions, resources, services, and learning opportunities that students need in order to meet their education goals and objectives within the context of an institution’s mission” (Upcraft and Schuh, 1996).
How are we supposed to implement this?
The assessment staff is available to help you develop an assessment program appropriate to your area of service. In the process, we will help you set realistic and attainable processes to implement your assessment plan and provide continuing support.
When do you know when you reach the top?
Assessment is less a race to the top than a process for continuous improvement. Effective assessment provides programs with the opportunity to reflect on what is working well and what could be better. Based on your assessment data, you may decide to make few changes to a program; however, you should continue to assess the program and periodically review the student learning outcomes it is designed to produce. For example, what worked well with students in 1997 may not be as effective for students enrolled today. Also, what students need to know and be able to do can change significantly over time. Effective assessment helps programs remain current with changes occurring within higher education and the larger society.
How have other schools implemented this information?
- California State University, Sacramento– provides Word and PDF versions of comprehensive assessment plan
- DePaul – provides examples of assessment reports
- George Mason University – provides information on process, model report templates, and examples
- Ohio State University – provides PowerPoint presentations describing specific assessment initiatives
- Oregon State University – Student Affairs Assessment Handbook (pdf file)
- Texas A & M – Examples of outcomes, rubrics, and presentations
- West Texas A & M – Examples of how assessment results are being used
For an average sized department, how many outcome-based assessments are reasonable to do per year to determine effectiveness?
Typically, most programs will have five to eight major student learning outcomes. You should plan to assess at least one of your major learning outcomes each year. Depending on what you are assessing, you will likely continue to collect data that you will use later in the assessment cycle. For example, you may conduct exit interviews with a sample of students each year but only analyze these every third year or so.
If you are just starting your assessment program, you will want to establish baseline data for all of your learning outcomes to help define your plan and to serve as a comparison for future evaluations.
How to best identify the appropriate evaluation method for certain outcomes and programs?
There is a wealth of information in the professional literature and on the World Wide Web about effective assessment methods. The American College Personnel Association and the National Association of Student Personnel Administrators have drafted Principles of Good Practice for Student Affairs that include maintaining knowledge about the professional literature and applying it to practice.
In addition, your colleagues at UNC and professionals working at other institutions are a great source of information about best practices in assessment. Finally, UNC’s assessment staff are available to provide suggestions or to research solutions for your assessment needs.
Will UNC put together a tool to use for developing outcome based assessment for consistency campus wide?
UNC has selected a web-based assessment management software system, TracDat, to help programs manage their assessment activities.
How do we get access to TracDat – cost, implementation, etc., and where can we get help to set up the plan online?
There is no cost to UNC programs for using TracDat. To get started, contact Angie Koponen at (970) 351-1690.
How will the University use the information?
Assessment data are used for a variety of purposes. Programs use the information to support program-level improvements, identify resource needs, and to document program quality. Administrators use these data to make strategic decisions about program development, resource allocation, and organizational structure.
How effective is visual documentation in evaluation?
Visual documentation in the form of graphs, tables, and charts can provide an effective way of reporting assessment results for multiple audiences. You can use visual displays for written reports and also for posting to your program’s website to communicate important information about your program to students, administrators, faculty, and members of the larger community. Mississippi State University has an excellent description of various types of graphs and charts, what they are used for, and samples.
How can we better assess the reflective learning that occurs after the program/intervention and before graduation?
One way to assess outcomes that occur after the student has left your program is to work collaboratively with other academic and student services programs to share information and assessment data. For example, if your program has established a learning outcome related to students’ ability to appreciate other cultures, you might work with instructors in a senior capstone course to incorporate questions that ask students to reflect on how various curricular and co-curricular experiences have contributed to this outcome. Results from these types of course-embedded assessments could be shared between programs.
Another strategy might be to conduct a focus group with students no longer actively served by your program or to use quantitative or qualitative surveys with random samples.
How can the various needs be identified in a response-based office?
You will want to start by thinking about what services your program is intended to provide. Student learning outcomes can be supported directly or indirectly, but the basic principle of assessing whether you have accomplished the results you intend holds true regardless of the type of program or service. If you start with the results you are looking to achieve, your assessment can help you uncover service gaps and other issues that either promote or interfere with successful outcomes.
What will this look like in my office as far as collecting evidence for our main objectives?
It is likely that you are already collecting data that could be used for assessment. For example, advisor log notes could be one form of data that is already collected. Even if these notes required a little tweaking regarding what advisors are asked to record, they could provide a rich source of data for identifying training needs, common obstacles or challenges faced by students, and the extent to which students are applying advisors’ suggestions.
The most effective assessment occurs in environments where the entire staff is engaged in the process (although it helps to have one person assigned to coordinate and manage the assessment program). The goal should be to integrate assessment into your day-to-day activities rather than adding another set of responsibilities. Staff from the Office of Assessment can help you evaluate the data you are already collecting and also identify additional data sources that could be easily incorporated into your regular operations.
Have more questions or need additional information? Contact us!