Quality assessment of ~120,000 public schools

The client is a UK based non-profit agency working with a state education department in central India. They had designed a program to assess the quality of ~120,000 public schools based on specific parameters. They were looking for a technology driven solution that would allow them to manage the flow of information from distant, rural areas to the state headquarters in a seamless and near real-time basis. They also wanted generation of insights and reports to be automated.

Technology to the rescue

The non-profit was grappling with a lot of challenges owing to the scale and the landscape of the program.

Offline data entry for usage of the app at locations with no internet connectivity

Bi-lingual platform for a higher uptake by school staff in rural areas

Shift from paper based entry, collection, reporting enabling real-time insights

There were couple of other challenges – designing a web application intuitive enough for first time users, no password (but secure) login based system

Large program with challenging infrastructure

The non-profit had signed up an MoU with the state Education department in central India to assess all public schools on quality, and to provide recommendations from cluster level to state level – to improve the school quality. They had developed a survey that was focused on eight different parameters. They had planned for a paper based survey, but soon realised the logistical challenges and the cost implications of the entire exercise.

There were multiple challenges to even execute the program through tech enabled solution. Internet connectivity in remote and rural areas was a challenge. Then there was the issue of English language competency. The biggest challenge of all was that most users were not familiar with using a laptop or a smart phone. As a consequence, a standard password based login was not effective as well.

Program Intelligence at scale

The school quality assessment program was a program intelligence solution requirement that challenges the limits of technology.

Once functional, numerous benefits were driven through the application:

a. Allowed results to be reflected in real-time

b. This data was then collated and insights were generated automatically through the system at the desired administrative level

c. Allowed for fast, click-of-few-buttons based comparison of one admin level against other

d. Served as a repository for all surveys completed across regions – without the need to preserve paper based records.


Compliance risk scorecard for the partners of an INGO

The UK based INGO works in 90+ countries across the world. In India, they engage with local partners for poverty alleviation, education, livelihood, education and few other thematic areas through local non-profit partners. This allows them to leverage their reach and local knowledge. As non-profits are required to be compliant on source of funds, accounting and management of funds – the INGO wanted an automated tool to monitor monthly compliance.

What gets measured gets managed

The non-profit needed a risk measurement tool that would allow for easy scoring and measurement of risk at monthly level.

Developed a matrix to categorize non-profits based on risk profiles

Highlighted common gaps and identify trainings needs for the partners

Developed a compliance progression graph to monitor progress over time

The INGO was able to measure the risk, thereby identifying gaps and issues and improving the financial and operational reporting of the partner non-profits.

Paper based compliance highly inefficient

The INGO was headquartered in New Delhi, India and had a hard time monitoring the risk profile of 100+ partners spread across the country. With the compliance focus on foreign funding, proper accounting and better management of funds disbursed, it was getting tricky for the INGO to ensure compliance and timely reporting. Paper based and excel based reports were making life more difficult – with multiple versions and the slow turn around.

There were specific challenges. For one, majority of the partner non-profits did not have trained staff complete the compliance requirement correctly, without assistance. The second issue was with the right infrastructure and System in place to generate the numbers at short notice. The issues compelled the INGO to device a partner risk Assessment matrix to ensure compliance.

Better risk monitoring and assessment of progress

Partner Risk assessment allows the INGO to monitor non-complying partners, ensure the best practices are being followed and government reporting needs are fulfilled.

The tool eliminates the need for multiple follow ups over email and phone. Since all the risk and compliance reporting needs to be done within the tool, the INGO has a single version of the truth.

The tool also eliminates the slow paper based reporting, that needs to pass from one administrative level to the other before it reaches the National HQ. This enables the INGO to compile and monitor compliance at a faster pace, meeting government deadlines with ease.

And finally, the tool enables the INGO to monitor progress of partners over time.


A non-profit used web based platform for immunization program monitoring

The largest health focused non-profit in the world was running a fund to achieve immunization SDG goals in India. The nature of the program required them to hold immunization activities in remote locations across the country. Since the staff was spread across India, it was important for them to develop a web based system of program monitoring that could be assessed anywhere from India and abroad. We developed a program monitoring solution for them.

Program Monitoring at scale

The non-profit wanted a solution that was mostly visual, had an excel based data upload system and easy to follow.

A web based data upload system with excel based files

Visual dashboard, with intuitive colour coding makes it easier to understand

District level interactive graphs for indicator based monitoring

The system provided sharing of reports & dashboards from within the tool. This enabled the non-profit to collaborate with colleagues spread across the country.

Scattered Data. Scattered Team

The non-profit was focused on two immunization programs – Lymphatic Filariasis (LF) and Routine Immunization (RI). The program monitoring system had been designed for offline reporting. The non-profit was facing multiple challenges with the monitoring of the platform. They wanted to monitor both the programs in real-time, as soon as some program data was available. As the decision makers used to be located across offices, a web based application was the need of the hour.

One of the major challenges to building the system was the infrequent data. Sometimes, data would be available within days, and not even for months at other times. Besides, the team used to travel a lot – and sharing and working on multiple versions was becoming cumbersome – for program monitoring and to track the progress.

Intuitive, quick and visual

One of the challenges in developing a web based application for non-profits is the simple, intuitive and visual requirement.

BY the nature of the program, a major part of the work is done in remote locations having internet connectivity issues. People who are responsible for data entry may not always understand the importance of accurate and quality data. Data validation within these tools therefore become most important – as, if not with proper checks, the data quality and decisions made on the basis of such data may lead to undesired outcomes.

We therefore placed emphasis on seamless data upload through excel, a colour coded, interactive set of visualizations and robust data validation checks. The team is using the platform for program monitoring and progress towards immunization goal.


SRoI: Measure Impact of a Non-Profit/Social Program

Non-profits work on environmental or social value problems that cannot be measured through conventional financial accounting methods. Social Return on Investment (SRoI) is a way to measure the impact of a non-profit’s work in terms of financial ‘proxy’ values.
SRoI is similar to the concept of Return on Investment (RoI), however it includes social and environmental outcomes along with the economic outcomes. SRoI is about value, rather than money. Having said that, money is the common unit that is used, simply because it’s the widely accepted and understood way of conveying value. According to The New Economics Foundation, SRoI “captures social value by translating outcomes into financial values.”

Early adopters of SRoI in the non-profit sector have realized it to be one of the best ways to gauge the impact an organization is making – be it an NGO, a government body or a social enterprise.
SRoI is not just a number, it is the whole framework and the process of how you arrive at that number. It’s not right to base your decisions purely on economic returns when social and environmental factors and outcomes have such a big role to play. SRoI tells the story about how change is being created. The framework provides you the factors to base your decisions on and includes qualitative, quantitative and financial information. SRoI can take different forms – can be done for a specific program or for the whole organization.
There are two types of SRoI measurement – Forecast and Evaluative.
Forecast SRoI predicts the social value assuming activities meet their social outcomes. It is extremely helpful in the initial planning stages since it tells you what outcome data need to be collected. The team is therefore well prepared and knows how to collect them before the start of the program.
Evaluative SRoI is done using actual outcomes that have already taken place. Good outcome data is the key to perform this kind of SRoI assessment.
Best results are achieved through Forecast SRoI so that you have the proper data collection systems in place and you have everything to execute Evaluative SRoI with the real outcome data at the end of the program.
SRoI has developed from social accounting and cost-benefit analysis and is based on seven principles:
1. Involve stakeholders
2. Understand what changes
3. Value the things that matter
4. Only include what is material
5. Do not over-claim
6. Be transparent
7. Verify the result
A lot about SRoI methodology is about practitioner’s judgment. There will be situations while performing the analysis when one needs to assume certain aspects and use judgment. Misrepresenting any information may impact the calculations in a big way.
Taking the example of “Providing education to underprivileged adolescents in Africa” program – the intended outcome of the program are:
1. The participants know how to read and write
2. They are healthier since the program involves some physical sports
3. Decreased teen pregnancy cases
4. Decreased crime rates in the community due to better awareness
The trick for an effective SRoI study is to find the indicators for the outcomes and a suitable financial proxy for each of them to be able to convert all of them to a dollar value. For the outcome – healthier participants, the financial proxy can be the cost saved on number of hospital visits over the year. For decrease in teen pregnancies, some of the economic benefits are realized immediately while the others unfold during later life of those teens. A particular research has shown that reduction in teen pregnancies lead to increased high school graduation and reduced use of public assistance by the mother. Conversely, children born to teen mothers have lower high school graduation rates, are more likely to repeat grades, are more likely to commit crimes, and are more likely to experience child abuse or neglect.
The size of these effects need to be multiplied by the value of each outcome to produce an estimate of benefits. For example, the reduction in the probability of a teenager dropping out of school should be multiplied by the value of completing high school to yield an estimate of the value of the benefit.
The formula for SRoI expressed simply is:

SRoI = (Tangible + Intangible value to community) / Total resource investment
In cases, where the cost or benefits span multiple years, the concept of time value of money is used and we bring all monetary values to a base year value (using Net Present value).
How can a non-profit use SRoI?
As complicated as calculating the SRoI seems, the benefits lie in easily comprehended value a non-profit program is generating – to grant makers, funders, own staff and the community

  • A non-profit can put a statement in the annual report – “Every $1 invested with us generates $2.75 of community benefits”
  • From a grant’s perspective, a non-profit can infer that “Program X is more sustainable/has a greater impact than Program Y because it has the best evaluative SRoI value”
  • As a non-profit, use the SRoI for fund raising and grant proposals. You could use this messaging in reports, direct mail campaigns, media campaigns, and social media.
  • SRoI helps non-profits make strategic decisions and maximize the social value an activity generates. It is a great way to initiate a formal dialogue with the stakeholders.

Non-profits should use SRoI as a more communicable way to measure the value their programs make and reap the advantages an SRoI study can provide.


Beneficiary management system to track program beneficiaries

The pan India non-profit had more than 2 million members participating in different programs related to skill development, livelihood, poverty alleviation and vocational training. The programs were designed based on local needs, and often overlapped in scope and coverage. The biggest issue with the non-profit was that they were not aware of how many different families they were reaching, and if the same person was getting enrolled across different centres.

Mapping beneficiaries to social programs

The non-profit wanted a better understanding of whom they were serving and how many unique households were being served through the social programs

The unique id mapped beneficiaries with 99.8% correctness (pilot of 10,000)

Family trees could be generated using the unique ids

A web based interface with digital solution to register new beneficiaries

The solution was deployed through a cloud based server and could be accessed through the web interface from anywhere. Even at locations with no internet connectivity.

Multiple centres. High duplication

The non-profit had multiple centres across localities where people could walk-in, enquire about the programs and register themselves or their family members. Each centre was supposed to enter the details, including family details, home address and phone numbers. Each of the beneficiaries was provided a registration number, that had a high potential of getting misplaced or being used by someone else in the absence of verifiable records.

The biggest problem was the lack of understanding of why the beneficiary data was important and the need to maintain them. They were looking at the solution merely as a tool to measure the true reach of their programs and the number of households impacted. We presented how the beneficiary management system was a prelude to number of improvements that could be enabled – from program implementation to return on investments.

Not just a data entry tool

Beneficiary Management System solved the primary requirement of our client to remove duplicates, identify the unique number of beneficiaries their programs were supporting. It also allowed mapping a family tree to measure the number of households they were serving. This helped them communicate better with the funders and improve program outcomes.

Apart from above, the non-profit is now able to measure the impact of their program in a better way, through social return of investment (SRoI) measure. It has also allowed the non-profit to have a database of all individuals they have worked with, allowing them to create a vibrant community of beneficiaries. They have used the information to reach out to corporate partners for skill based job fulfilments. And lastly, the non-profit has now defined KPIs to measure their operational efficiencies & performance each month.