Planning and reporting are two critical aspects of the assessment cycle.
Planning helps you connect your office or department’s mission to services, explicitly stating what your area does for students in terms of learning outcomes and how you will measure those outcomes. Reporting helps tell the story of what you found, what it means, why it matters, and what you plan to do about it.
And there’s one thing that’s crucial to both planning and reporting: setting targets for success.
What is a target?
When I talk about targets in assessment, I’m referring to the desired or expected level of performance for student learning. Ultimately, they are an indicator of whether or not an outcome was achieved. These go by a number of names – including targets, goals, success criteria – but I’ll use target for clarity and consistency here.
Targets are useful because they help ease reporting and interpretation of results to determine achievement or success in relation to student learning outcomes and operational objectives. Without a clear target, determining success or achievement can be subjective. After all, a workshop resulting in 75% of participants being able to articulate their leadership philosophy may sound like a “good” result at face value. But it’s much less impressive if last year’s workshop yielded a 90% rate. Having a set value to compare your results against makes reporting and interpretation easier.
Types of Targets
You might set different forms of targets, including:
- an expectation target – there is no further action or effort needed to achieve the target since the results are what you’d expect based on historical data or performance (for example: a target of achieving a mean score of NSSE results to meet the National Average score).
- an aspirational target – you may need to focus and make some effort in order to achieve this target based on inconsistent or slightly lower past performance (for example: a target of achieving a mean score that’s higher than the NSSE National Average in at least four areas)
- a stretch target – you need to exert a major focus or efforts to work with students to achieve this target (for example: a target of achieving a mean score higher than the NSSE National Average in at least 10 areas)
Targets can be both qualitative or quantitative. You can set qualitative targets focused on the number of focus groups, interviews, or listening sessions conducted, as well as specifying more or less of certain themes being present in the qualitative and narrative data. Quantitative targets could take the form of averages/means, medians, percent of the total, or proportions of students.
Below is some sample text for quantitative targets:
- Mean score
- At least 80% of students will indicate their grades improved as a result of receiving tutoring
- The average student will have 90% or more correct responses for the student org officer renewal process
- Rubric Scores
- Overall score of 23 or higher for the rubric
- Average score of 3.0 or higher on rubric criteria number 3
- Proportion of students
- 90% or more of students perform at the basic or proficient level
- Less than 10% of students perform at the beginner level
You may be wondering where to start in setting your targets. The good news is that your department, institution, or facilitators of individual programs may already have some targets established. Ask around and see if KPIs (key performance indicators) exist or if leadership expects a certain level of output or results.
If established targets or metrics don’t exist, then you likely find some good starting points by looking at your past data for internal benchmarks. If you participate in any externally benchmarked or comparable efforts, you can use your data, aggregate average results, or results from specific cohort groups or institutions to set targets.
If no established targets exist and you don’t have past data (or access to it), it’s ok to create your own starting point! Believe it or not, there is nothing wrong with using your gut.
You can probably imagine results that would be low or disappointing, those that would be too high or unrealistic, and an average that wouldn’t be surprising. Lean into those feelings to hypothesize and set your targets. And remember: though targets are important, don’t worry too much about getting them exactly “right”. You can always explain your target-setting process when interpreting results in your report and – with data in hand – take action to be data-informed in setting your target for the next assessment cycle.
When setting your own targets, consider also going beyond your own intuition, using the following methods:
- Engage and involve students. Talk to students about what they might consider reasonable or expected results or demonstrated learning given the intervention. If you have instruments, like surveys or rubrics, have students test them so you can get some mock data to inform your target setting.
- Gather input from colleagues. Check with fellow staff and faculty for targets they may have for related data or even just amounts set for their own targets. Faculty and staff can also test or review any instruments you have, perhaps offering you additional perspectives to help you set your targets.
- Compare with institutional information. Put your learning outcomes and methods alongside institutional data, outcomes, and levels of learning. Is what you are doing at the same level as divisional or institutional expectations? Are there areas, outcomes, or interventions aligned to the same institutional framework elements? Similar to the last point, use people and documentation to inform your target-setting approach.
You should probably use at least one of the methods above in addition to going with your gut. And remember to triangulate the responses. Compare and contrast the information and perspectives as you review your outcomes and approach, then set targets accordingly.
Now that you know the general information, I want to offer a few reminders for target setting.
First and foremost, language matters. Make sure to have multiple people review your target to share what they think it’s saying.
For example, consider the target of “students will be able to articulate career goals.” To me, this reads as “all students,” which leads to questions like: would the target be fair to be the same for first-year students versus fourth years or undergraduate versus graduate students? Also, does it matter how many career goals they can articulate? Is one enough? Should they be able to articulate more based on the intervention or circumstance?
Here are two possible enhancements for a clearer target:
- After attending at least one academic advising appointment, 90% or more of first-year students will be able to articulate at least one academic goal by the end of their first year.
- Less than 10% of graduate students will be unable to articulate at least two career goals.
Details also matter in setting your target in relation to the data you’ll collect. You should be reporting in relation to your target, so set clear success criteria based on your planned data reporting. Don’t, for example, set a target that specifies average scores per rubric dimension but then report overall rubric scores.
Consequently, when clarifying level language (like beginner, developing, proficient) or numerical scores, be clear if you are talking about select individual dimensions or all dimensions. (By the way, I apologize for all the rubric jargon.)
And for all the surveying being done, don’t set vague targets like “students will agree” or “knowledge will be increased”. How many students will agree? All of them? 80% of them?
Whether talking about agreement or increase in a given outcome, how much does the target expect? Anything above “strongly disagree” or “no increase at all”? Be specific, like this example: “80% or more of students will indicate that their familiarity with student organization policy increased ‘a great deal’ or ‘considerably’.”
Lastly, spending time reflecting on targets matters — even if you happen to be one of those top-performing areas that always meet its targets. For starters, look to raise your targets to aspirational or stretch targets whenever appropriate. As that may not always be possible given your measurement or method, disaggregate your data for equity concerns and to see if student performance across demographics meets your target.
I would also encourage you to measure your learning outcomes in different ways and at different points in the student experience. Don’t just focus on summative assessments. Get innovative and brave in digging deeper or piloting new efforts. Likewise, you can look to compare results with external benchmarks or partners willing to share data. There’s plenty more to get into here, as I can almost guarantee that no area is serving all of its students with perfect, 100% learning achievement across all learning outcomes and across all interventions.
I hope this post has been helpful in reiterating the importance of setting targets, as well as offering a few tips and tricks to do so. If you’ve set targets, what words of wisdom do you have to offer? Share with us @HelloPresence and @JoeBooksLevy