Banner 3

Flawed Study Fails to Measure CASA’s Impact

November 4, 2019

By Kathleen LaValle, president and CEO of Dallas CASA

Dallas CASA volunteers, who are trained and supervised by our professional program staff, gather information for judges about the well-being and circumstances of children who have been removed from homes due to abuse or severe neglect. A CASA volunteer frequently is the only consistent, reliable adult during a frightening, uncertain time when every child needs to know they have not been lost or forgotten. CASA makes an immeasurable difference by advocating for services for children to help them heal from trauma, but we don’t— and don’t claim to — make decisions about how or when a child leaves the child welfare system. A judge’s decision about where a child can live safely is a complex determination dependent on parental cooperation, as well as inputs from CPS and other parties to the case, attorneys, experts, and fact witnesses.

 

In a recent journal article, University of Texas researcher Cynthia Osborne focuses critically on the impact she claims CASA (Court Appointed Special Advocates) volunteers have on permanency outcomes in Texas child welfare cases. Permanency outcomes are the alternative ways a child exits the child welfare system, including reunification with parents, adoption by relatives or non-relatives, legal guardianship to kin, or aging out of the system. Osborne’s analysis glosses over the reality that CASA volunteers do not actually determine the final disposition of Child Protective Services (CPS) cases. Ask any of our judges and I’m confident they would agree. While claiming to fill a gap “regarding the impact of CASA appointment on permanency decisions and final case outcomes,” flaws in the underlying study methodology and generalizations based on small data differences render Osborne’s findings unreliable and unremarkable. As I’ll share at the end, the state-wide study results Osborne cites also are not representative of outcomes in Dallas County CPS cases.

Osborne acknowledges that “CASA may also have additional impacts on children that are not measurable in our data.” Anyone familiar with the role of a CASA volunteer will recognize this as an understatement. The article references a more recent focus on well-being and social supports and sources of consistency and normalcy, but cites those as “outside the scope of the present study and not measurable using administrative data.” Being a consistent support for a child during a frightening and uncertain time and providing detailed reports to judges on the current circumstances of each child are the most frequent claims CASA programs make regarding the impact of our advocacy. Osborne acknowledges that the work CASA does to ensure the well-being of children is not captured in a database. The difficulty in measuring that significance does not justify or excuse focusing on permanency outcomes that a CASA program does not — and does not claim to — determine. I routinely remind new volunteers at swearing-in ceremonies that the important decisions are made by our judges in the courtroom and the primary and significant impact our volunteers have happens outside the courtroom.

According to Osborne, she focuses solely on permanency outcomes because “[a]chieving safe permanency is the ultimate focus of the child welfare system.” Because CASA volunteers do not actually determine permanency outcomes for children in the legal custody of the state, the article contributes nothing to any meaningful dialogue about CASA’s impact or effectiveness. Judges, jurors, and parties to a mediated settlement agreement decide permanency outcomes and the inputs to the final decisions come not only from CASA, but from many other stakeholders.

Osborne describes as “relatively small” the distinctions in the data she relies upon to make broad, sweeping generalizations. Of those children in the study who were reunified, Osborne reports that the group with a CASA was <2.5% smaller than the group without a CASA. She also reports that of those children not reunified, the number who did not achieve permanency — either because the youth aged out or ran away — was 1% higher in the group with a CASA assigned. In addition to relying on relatively small distinctions, the article includes no correlation to CASA’s actual recommendations for permanency, making the findings even less relevant and less reliable. Even if CASA recommends a child be returned home, that may not be the outcome the court orders. Acknowledging that reunification is within the court’s discretion and depends on a parent’s completion of service plans, the article nonetheless insists it is describing the impact CASA has on outcomes, again without reference to the outcomes CASA actually recommended.

Osborne reports that “CASA children are also significantly more likely to be adopted” and “significantly less likely to be placed in permanent kin guardianship at the end of their case.” Yet, the article draws no distinction between children adopted by relatives versus non-relatives. If adoption rates are higher when CASA is appointed, it may also mean that more relatives commit to adopt, rather than assume legal guardianship. The article creates the misimpression that a relative assuming legal guardianship is a superior outcome to adoption — regardless of whether the adoptive parent is a relative. Particularly when parental rights have been terminated, adoption by a relative clearly is a more positive outcome than legal guardianship because it brings a greater degree of permanency for a child.

The article’s statement that children with a CASA are “more likely not to experience legal permanency than children with no CASA” is especially problematic because children whose cases were ongoing at the end of Osborne’s study period were inaccurately and arbitrarily coded and counted as children with no permanency outcome. Children who were in the legal custody of the state and had not aged out should have been classified as “permanency outcome unknown” or this group of children should have been excluded from the study. This group includes children, for example, whose relatives are preparing to adopt or to assume legal guardianship. Not surprisingly, the Texas Department of Family and Protective Services’ annual report on permanency outcomes includes only children who exited the system during the reporting time period.

According to Osborne, in many counties where a CASA program lacks capacity to serve all children, judges assign CASA only to the more complex cases. While claiming to have controlled for the difference in the difficulty of cases assigned a CASA, the reported characteristics of children included in the comparison groups are dramatically different. Looking at the differences in the samples, the CASA cases plainly are more complex. The children are older, with more prior investigations and more prior removals, and with a higher rate of domestic violence and other caregiver risk factors that reduce the likelihood a judge will order reunification. While claiming to apply a “quasi-experimental” methodology to account for preexisting differences between “with a CASA” and “with no CASA” groups, the article acknowledges that “it is likely that there remain unobserved differences between the groups in our sample.”

As I mentioned at the start, the state-wide results cited in the Osborne article do not represent outcomes for children Dallas CASA has served. Looking at outcomes for more than 2,100 children whose cases closed from January 1, 2018 through the end of October, 2019, a child was significantly more likely to be reunified than either groups (with a CASA and with no CASA) included in the state-wide study. For children Dallas CASA served, 34.8% were reunified compared to 29.12% of the study group with a CASA and 31.6% of the study group without a CASA. If children returned home as a result of CPS nonsuiting are included, the reunification rate for children Dallas CASA served increases to 40.4%.

Children in cases assigned to Dallas CASA also were significantly more likely to result in reunification or legal guardianship to a relative rather than adoption. Of the children Dallas CASA served, 20.4% were adopted versus 31.06% in the state-wide group with a CASA and 28.38% in the group without a CASA. Also, the number of children in the Dallas CASA data who were adopted by a relative was nearly 50% higher than the number adopted by non-relatives. Differences also appear when comparing the percent of children whose cases closed with legal guardianship to relatives. Of the children Dallas CASA served, 34% had relatives assume legal guardianship compared to lower percentages in both state-wide study groups (30.51% with a CASA and 31.8% with no CASA). The percentage of children who aged out or otherwise failed to achieve permanency also was significantly lower for children Dallas CASA served (4.6%) compared to the state-wide “with a CASA” (9.32%) and “with no CASA” (8.21%) study groups.

Dallas CASA does not take credit for these better-than-reported permanency outcomes applying standard outcome reporting methods, but we also strongly reject being associated with Osborne’s faulty generalizations. The flaws in the underlying study methodology leave open the strong possibility that Dallas CASA is not alone among CASA programs whose actual case outcomes may be significantly different than those state-wide results Osborne reports.

Even recognizing the desire of researchers to publish, the positive contributions from Osborne’s article are a null set and the negative repercussions are real and immediate. One anti-child welfare system national blogger quickly disseminated a rant incorporating quotes from the article and resurrecting claims that CASA programs are a haven for white supremacists. Despite the distractions and misunderstandings Osborne’s article creates, Dallas CASA volunteers and staff will continue to advocate for the best interest of children we are privileged to serve and we will continue to offer fact-based reports and recommendations to our judges. We are grateful to Dallas County judges for the opportunity to now be serving more than 9 out of 10 of our community’s child victims of abuse and neglect who are currently in protective care. We also are grateful to the more than 1,400 trained and professionally supervised Dallas CASA volunteers who have been assigned to Dallas child victims of abuse and neglect so far this year.

If you would like to broaden your sense of community and see how you can make a positive difference for vulnerable children who will be our future and our legacy, we invite you to register for one of our weekly information sessions at dallascasa.org. We promise we will never tell you that you can determine permanency outcomes for children. Instead, we will encourage you to walk humbly while recognizing the many profound and immeasurable ways you can support and make a difference for our community’s child victims of abuse and neglect. Please come see if you were born to do this work.


Dallas CASA logo_color_small

Thank you for registering for a Dallas CASA information session.

You will receive an email confirmation with additional information within 24 business hours.

CASA ACCW_blue text