Most Youth Programs Are Collecting Too Much Data
As a self-proclaimed data nerd, I say this with some hesitation—but in many cases, it’s true: We’re often collecting more data than we can realistically use to support learning and improvement. In many youth-serving organizations, lack of data isn’t the problem. There’s usually no shortage of surveys, tracking systems, or reporting requirements. If anything, the challenge can be the opposite: Teams are collecting more than they have the time or capacity to meaningfully use.
In practice, this can look like:
Staff feeling stretched thin across multiple priorities
Youth wondering what ever happened with that survey they were asked to complete
Data getting reviewed occasionally but not always in ways that inform day-to-day decisions
Over time, data collection can start to feel more like an obligation than something that meaningfully supports the work. This isn’t because organizations don’t care about learning or impact—they do. In my experience, it’s often the result of data collection evolving over time. A new funding requirement gets added. A new question emerges. A new tool is introduced with good intentions.
But there isn’t always space to step back and ask: What do we really need to know and why?
The Cost of More Data
When data collection isn’t intentional, it creates tradeoffs:
For youth: More surveys, but not always more meaningful opportunites to share their experiences
For staff: More data to collect, but less time to reflect on what it means
For organizations: Information that gets reported, but not always used
Over time, this can pull evaluation aways from its core purpose: Supporting learning, improvement, and better outcomes for young people.
Sometimes the Answer is Do Less
This can feel counterintuitive, but one of the most effective ways to improve data collection is often to scale back and refocus. Instead of asking: What else should we measure?
In can be useful to ask:
What decisions are we trying to inform?
What outcomes matter most right now?
What do we already know from our existing data, relationships, and experiences?
What is realistic to collect given staff time and youth experience?
Aligning Data with What Matters
One place where teams often get stuck is trying to track many things at once, without all of them clearly connecting back to what the program is trying to achieve. When data collection is grounded in clear program goals, intended outcomes, and a shared understanding of how change is expected to happen, it becomes easier to prioritize what to measure, let go of what isn’t essential, and focus on data that supports both learning and decision-making.
Designing for Real-World Use
Even when the “right” data is identified, how it’s collected matters.
This often means:
Using short, more focused surveys
Integrating data collection into existing program activities
Combining methods (e.g., surveys & conversations & staff observations)
Involving youth in shaping what is asked and how
The goal isn’t to lower rigor—it’s to design approaches that are useful and sustainable in real program settings.
From Data Collection to Reflection
Data becomes valuable not just when it is collected, but when it is used. That means creating space for teams to review what they’re seeing, interpret it together, and connect it to decisions and changes in practice. Without that step, even well-designed data collection can fall flat.
A Practical Tool to Support This Work
To make this more concrete, I created a simple, team-based reflection tool to help organizations step back and think through their approach to data collection. It’s designed to support conversations about: Purpose, alignment with program goals, prioritization, feasibility, and data use.
The goal isn’t to prescribe a single approach but to support teams in making more intentional, data decisions.
Good evaluation isn’t about collecting more data. It’s about collecting data that is meaningful, manageable, and actually used—data that respects the time and experiences of both youth and staff.
