Pay for Success Feasibility Studies: Catalytic Investments, Moving the Needle

This blog was originally posted on NFF's Pay for Success Learning Hub. For more information on the latest developments in PFS, visit www.payforsuccess.org.

The California Pay for Success Initiative is encouraging the use of creative funding approaches to find new ways to deliver vital services that achieve results. The Initiative is funded by the James Irvine Foundation and coordinated by the Nonprofit Finance Fund. Launched in 2014, work supported by the Initiative is already taking off. In 2015, Santa Clara County announced that it would launch a Pay for Success project to reduce chronic homelessness.

But the value of the approach is paying off in other ways, as a recent meeting of leaders from across California who are exploring Pay for Success projects has revealed. In October 2015, the Nonprofit Finance Fund convened a group of nine leaders from across the state to discuss how simply exploring Pay for Success projects is strengthening their organizations and helping them better serve their communities.

All the organizations that participated in the meeting had expressed interest in exploring Pay for Success, and as part of their inquiry they conducted what are known as feasibility analyses to determine whether their projects would be good candidates for Pay for Success agreements.

A feasibility analysis determines whether it’s possible to gather the right kind of data that will demonstrate whether a particular project is achieving its intended outcomes. For example, an education project might measure the reading scores of third grade students as part of an effort to help prepare students for middle school. The data that the project collects would then be used to design Pay for Success contracts in which payments for services would be linked to outcomes.

The bottom line is that each organization that conducted a feasibility analysis reported that this process alone has been tremendously valuable, regardless of whether they are ready to embark on a Pay for Success or other outcomes-related contracting arrangement.

The benefits of feasibility include:

Organizations discovered that they had not been investing in the infrastructure necessary to track data and measure the outcomes of social programs. More importantly, they began to prioritize making these investments.
The process helped to shift cultural attitudes around measuring whether service programs were truly making a difference in people’s lives.

Improving Data Collection to Assess Programs and Outcomes

Delivering services to people in need can involve what appear to be very difficult tradeoffs. Service providers—and more specifically, the funders who support them—understandably want to put as much money as possible toward serving their clients, and as a result it can be difficult to find or justify spending money on infrastructure or evaluation. Feasibility assessments have helped service providers make the case for prioritizing investments in improving how they track and measure success.

Here are some examples:

The Children and Families Commission of Orange County explored the feasibility of expanding the Bridges Maternal Child Health Network, a service referral and home visitation program for new moms in Orange County, and observed: “While the Commission has invested in technology and data infrastructure to assist its grantees in collecting and reporting data, we realized we weren’t necessarily collecting the best data to measure some of the outcomes that potential payers care about most. Pay for Success has been the catalyst to invest in what is really a foundational asset for attracting new payers and better informing our own investments.” As a result of their feasibility assessment, the Children and Families Commission intends to invest more in its evaluation systems to better demonstrate how maternal and child health services can improve the health of families and young children.
Similarly, while assessing the feasibility of using Pay for Success to address homelessness, the City and County of San Francisco learned that its existing systems made it difficult to track how clients fared after using their services. They lacked the necessary infrastructure and staff capacity to collect sufficient data or conduct effective evaluations to learn whether families served by San Francisco’s shelters were later able to secure stable housing. While the results of feasibility analysis led San Francisco not to pursue a Pay for Success contract at this time, it did inform the City and County’s decision to invest considerable funding into services for homeless families. San Francisco is also updating and adding systems, processes, and capacity to collect and use more robust outcomes data.
Through the feasibility assessment work done to develop Project Welcome Home, the first Pay for Success project in California, Santa Clara County realized that it could improve its ability to leverage its data systems, as well as certain service delivery practices. This included improving the client referral and intake process to allow Santa Clara to track each individual’s engagement with County services across multiple agencies with sometimes incompatible records systems. Such improvements to how it tracks information are helping the County to gather better data on their services and do a better job of delivering them. In fact, the process of developing Project Welcome Home has produced what it considers to be “lasting benefits that extend beyond the Pay for Success project itself,” and position the County to make significant progress in helping homeless individuals and families.
Through its feasibility assessment, First 5 LA found it needed to track outcomes most relevant to potential sustainability partners like hospitals and health plans before it could pursue an outcomes-oriented funding agreement such as Pay for Success. This realization has affected not just the evaluation of Welcome Baby, a program to help new mothers and their babies in Los Angeles County, but reinforced First 5 LA’s broader approach to program design and grant making. “The process helped staff and the Commission understand additional outcomes and sustainability efforts we need to consider as we develop and invest in new programs and interventions. Staff has a much clearer understanding of the value of identifying targeted outcomes before we design, develop and invest in new programs…and our service providers need to consider how much time and what type of staff capacity it will take to do the data gathering, inputting and reporting that is needed for our upcoming evaluation studies that could enable a Pay for Success contract in the future.”

Many service providers and governments appreciate the value of being able to measure the outcomes of their services. This more nuanced understanding of the data issues and capacity needs required to collect information about outcomes that can be used to evaluate a program’s effectiveness represents a significant shift for many social service providers and funders. In the past, many have focused on serving as many clients as possible, but it has been hard to know whether the projects were successful or not. Feasibility assessments are providing one way by which governments and service providers are learning how to put systems in place to monitor whether their programs are working and find new ways to improve them.

Changing Culture: Collaborating to Achieve Outcomes

As important as feasibility assessments are in changing how governments and nonprofit service providers are now measuring progress, they are also helping to change these organizations’ culture. The conversation is starting to shift toward measuring outcomes, and making sure that the money and people are in place to do it. It also acknowledges the importance of collaborating with governments, outside funders, and service providers to achieve shared goals.

The City and County of San Francisco’s recent feasibility analysis helped transform the way it plans to serve homeless people in the city. It recently announced plans to create a new city department to oversee all of San Francisco’s homeless services, aligning and coordinating its efforts for the first time. Mayor Ed Lee defined the ultimate goal as “housing every homeless person we can with the help that they need to live healthy lives.” The City and County also reported that the Pay for Success feasibility analysis has “been an important driver in looking at how we can invest our and partner dollars in a more coordinated and outcomes-driven way.”
In developing the new homelessness department, the City and County will draw on examples such as the HOPE SF partnership, which aligns the efforts of governments and foundations to redevelop public housing in San Francisco. On a related note, HOPE SF is also exploring a Pay for Success initiative to expand economic opportunity for young adults living in public housing. This project would coordinate public funding with support from The Super Bowl 50 Fund, the Annie E. Casey Foundation, and others.
Santa Clara County is utilizing its first Pay for Success project as an opportunity to partner with a private foundation that funds the program evaluation of the County’s selected service provider. This approach offers the government and service provider additional resources that may not have otherwise been available through traditional government procurement to grow capacity or evaluate programs.
The Center for Employment Opportunities’ feasibility assessments of its jobs programs for formerly incarcerated people in New York and San Diego allowed the organization to collaborate with other counties across California to explore expanding their program by focusing on outcomes and using new data. The Center for Employment Opportunities noted: “While we have a longstanding commitment to being focused on outcomes, the feasibility analysis in San Diego was immensely helpful in establishing a track record for us in California and serves as a symbol of our commitment to evaluation and evidence. The promising results gave us a great deal of confidence that we could reasonably expect to see a similar impact in other California jurisdictions where we were serving the same population.”

The Inherent Value of Feasibility Assessment

Whether or not these efforts immediately lead to Pay for Success projects being launched, feasibility assessments are changing the way California service providers, governments, and philanthropic partners approach their goals.

Feasibility assessments bring to light chronic, systemic issues in our social sector. The simple fact is that we need to invest more in evaluation and data collection, and we need to better integrate the efforts of government, philanthropy, and service providers to serve the most vulnerable members of our community.

The good news is that transitions such as these suggest, we may well be on our way. Pay for Success feasibility analysis offers an opportunity for leaders to shift their organizations’ culture and begin to allocate resources more effectively to address some of society’s most critical issues.