Arnold Ventures, formerly the Laura and John Arnold Foundation, is actively seeking applications to fund evaluations of child welfare-related services, with an emphasis on programs that could be funded under the Family First Prevention Services Act.
“We are very interested in more proposals in this area,” said Jon Baron, Vice President of Evidence-Based Policy. “We are especially interested in programs that could be funded through Family First because the law provides a potential pathway to scale.”
Family First provides matching funds to states for services that prevent child maltreatment and meet its evidence requirements. The law established the Title IV-E Prevention Services Clearinghouse to rate eligible programs. The clearinghouse has reviewed a dozen programs so far, with half receiving its highest evidence rating. Another 16 reviews are in the pipeline for release in the coming months.
The clearinghouse has been criticized for the slow pace of its approvals, but the primary barrier is a shortage of evidence-based programs. Of the 482 interventions catalogued by the California Evidence-Based Clearinghouse for Child Welfare, which has similar evidence standards, just 35 meet its criteria for being well supported by research.
Congress anticipated this shortfall when it enacted Family First. Under the law, states will not be able to access federal matching funds for evidence-based prevention programs until they have submitted and received approval of a five-year prevention plan. These plans must include evaluations for any services that do not meet the law’s highest evidence standard.
The Children’s Bureau announced in October that it will cover half the cost of these evaluations, but the remaining expense may be a barrier for some states. The District of Columbia is the only jurisdiction that has received federal approval of its plan so far, but at least eleven states are working their way through the process. It is not clear how they are handling the cost of evaluations.
Baron said that Arnold Ventures would be willing to help cover these expenses if the studies are sufficiently rigorous. “We have funded randomized controlled trials that are also partly funded by the federal government,” he said, citing previous projects in K-12 education. “We could do so in child welfare.”
The foundation is considering new submissions on a rolling basis. “The bottleneck is the number of strong proposals,” Baron said.
So far, Arnold has invested about $50 million in over 75 studies intended to build evidence across a range of social services. It announced early results for four of the projects last month. According to Baron, about 30-40 percent of the studies are on track to produce positive findings on important outcomes, a figure that is considerably higher than the 10-15 percent that is more typical for randomized trials.
The higher success rate is due to several criteria in the foundation’s Request for Proposals, said David Anderson, Director of Evidence-Based Policy. “One funding criterion is it can’t just be an idea or chasing a theory,” said Anderson. “It needs to be an intervention that has been fielded somewhere, at least on a small scale which helps avoid implementation failures. There also should be promising evidence from a prior study or a pilot so a meaningful effect is plausible.”
Grantees must have experience implementing the program, with existing services funded by other governmental or philanthropic sources, and they must be willing to conduct a randomized controlled trial (RCT) and be working with an experienced evaluator.
Arnold has already funded five projects that are child welfare-related, with grants ranging from about $260,000 to $2.5 million (see accompanying article). The largest grant is for a replication study of Child First, a home visiting program that was already backed by a well-conducted RCT.
The new replication study, if successful, would boost its evidence rating. “This study is designed to get the highest rating in the clearinghouse,” said Meghan McCormick, the principal investigator on the project for MDRC, a New York-based evaluation firm.
How can other projects make it into this pipeline? “We typically work with organizations to raise funding to support research and evaluation,” said McCormick.
Child First was first developed in the early 2000s, with initial grants from the Connecticut Health Foundation and SAMHSA, which funded its initial study. The results produced a positive rating on the MIECHV home visiting evidence registry, which drew attention from other states.
“If you do high-quality work and pay attention to what they are asking in their grants. funders are willing to support you based on the level of rigor that you have shown,” said Child First’s CEO, Darcy Lowell. The program is now operating in 23 sites across Connecticut, Florida, and North Carolina.
Lowell is seeking more funding for an implementation study. “There is so much more that we could learn that would to contribute to the field,” she said.
“We do not have the full array of evidence-based practices or programs that we need,” agreed Heather Taussig, principal investigator for the Fostering Healthy Futures for Teens project. Taussig urged researchers to think contextually and consider which services may work best for which populations.
“Subgroup effects are really important. Not every intervention will be equally effective across every demographic or risk profile,” she said. She also advised researchers to think beyond their immediate study and consider the practical realities of implementing and scaling a program, including logistics, costs, and reimbursement mechanisms.
Earlier-stage programs may want to be cautious, however, before they make the leap to a randomized trial, warned Elysia Clemens, Deputy Director of the Colorado Evaluation and Action Lab, which is overseeing two Arnold-funded evaluations. “Family First is an incredible opportunity to up our game on evidence,” she said, “but it can incentivize researchers to jump ahead to a randomized trial too early.”
“What is the theory of change? Is the intervention well documented? Does it have a manual? Is it replicable? Has it been implemented with fidelity? Does it have a preliminary pre-post study? If all of those answers are yes, then it’s a great candidate for Arnold Ventures or another funder that supports randomized trials,” she said.
“If we jump to rigorous causal studies too fast, they are likely to fail, and we may lose the momentum and practitioner interest in building the evidence,” she said.
Arnold’s interest in Family First is consistent with a broader shift in its strategy, which is now focused on achieving system-level change. Founded as the Laura and John Arnold Foundation in 2008, it made several early grants to evidence and science-focused initiatives, such as the KIPP charter school network and the Center for Open Science. (My organization, the Social Innovation Research Center, has also previously received funding from the foundation).
Baron, who previously led the Coalition for Evidence-Based Policy, joined Arnold in 2015, a move that allowed his organization to become a major funder of randomized controlled trials. Earlier this year, the foundation announced that it would be restructuring itself as a limited-liability company called Arnold Ventures. The new organization encompasses the foundation, a donor-advised fund, and the Action Now Initiative, a 501(c)(4) advocacy organization.