How Can I Make Sense of My Data to Drive Innovation?

Administrative data has a great capacity to help public and private systems understand the outcomes of the people that they serve. While the best evidence is often gathered as part of formal research, in human services, they are desperate to understand what works in actual practice. The challenge is to understand what works in less controlled, public systems. This requires using data in real time to track performance and outcomes. Data that is collected for one purpose, like case management, and used for another, like trying to understand outcomes, requires careful thought to get the data to fit the problem. Administrative data is often made up of case records, one record for everybody served. If your system is like most, it was designed to count program participants for the purpose of payment. But administrators need to answer complicated questions, like “are children safe in placement or home?” or “why does it take so long to reunify families?” Getting the most out of data requires asking the right question, and then carefully structuring reports, files, dashboards, and measures to isolate the information that’s needed. Too many leaders rely on static, point-in-time counts delivered monthly to try and understand complex systems. 

I mostly work with public agencies with outdated reporting structures that are somewhere in the process of modernizing. There’s little flexibility around underlying data and how it is structured. Leaders are further confined by mandated measures and reports from litigation from federal mandates; data teams build and generate mandated reports but have little time left for more proactive analysis that are focused on improving services, programs, or outcomes. But effective reports and dashboards communicate leaders’ top priorities and monitor progress towards those goals over time. This keeps the focus on where it needs to be, and it prevents leaders from developing a reactive stance towards their data. For too many agencies, a data report is a list of client names that fit a certain criteria that requires certain action. Reports should display aggregate information, that’s necessary to improve the quality of services, and reports that aren’t useful should be retired. An agency’s priorities should be able to be detected by looking at their data reports. A data dashboard is an opportunity to communicate what is most important, and where improvement is needed. Baselines and targets help employees understand what is important, and to tap into their own values and motivation for doing the work. 

Data is at the heart of any systemic equity analysis. The analysis begins with gathering and analyzing program process and outcome by race. This includes designing a formal process for regularly monitoring and analyzing disaggregated data. Most systems have plenty of data, but they need more effective ways of sorting out the patterns. Systems are governed by a series of data snapshots, instead of seeing progress, or lack of, over time. Outcomes vary a great deal across racial groups, which is a good reason for tracking important measures over time and across groups. Quantitative data needs to be examined alongside active listening to those who work in every part of your agency, including clients. Systematic biases hidden in well-intentioned policies and practices, something as complicated as race equity, need to be approached quantitatively for objectivity, and qualitatively for validity and truth. Using data to guide these conversations adds a neutral focus point to a difficult conversation. And siloed data needs to be supplemented with shared outcomes, like community well-being indicators, that cross all human services organizations.