Objective

To evaluate the impact and effectiveness of the RHD Action Small Grants Programme.

Award selection process

Requests for proposals were issued through RHD Action, PASCAR and WHF email distribution lists and cross-promoted on websites. A review panel with representation from Reach, WHF and demonstration projects used structured criteria for scoring based on clear, feasible, measurable objectives. Preference was given to proposals with dissemination plans using local publicity and social platforms, and for garnering support from local MOH officials. Final selections were approved by Medtronic Foundation. Funding increased from $2,000 to $2,500 after the first round.

Study evaluation

This was a mixed method study that involved both quantitative and qualitative surveys through the following channels;

1. Online survey

An invitation to complete an online survey, using a Google Forms format, was issued in Dec 2019 to Small Grant Project Directors and Co-Directors for projects funded between 2017 and 2019. The online survey requested basic project information using tick boxes, Likert scales, and short answer open-ended questions about successes and challenges faced by recipients.

The questionnaire addressed recipients’ experiences with the RHD Action Small Grants process – applying for the grant, nature and quality of support received to carry out the project, the reporting process, and any media coverage generated.

Respondents were asked to indicate if they would agree to be contacted for a phone interview and if so, to provide best contact details. Completing the online survey required 15-25 minutes for completion. Recipients were given a four-week response window that occurred over the holiday period. Three follow-up attempts were made to any non-responders. The respondent identity was concealed, and a random anonymous number was used for identification.

2. Phone interviews

Responses to the short-answer questions were used as the basis for follow-up phone interviews. These discussions were recorded, transcribed and thematically analyzed for new and recurring themes emerging from the in-depth discussions. The phone interview took about 30 minutes to be completed.

Ethics approval

The activity was framed as an “Impact Assessment/Evaluation” activity for a single programme. Therefore, ethics approval is not required.

Characteristics of the grant recipient

The RHD Action Small Grants Programme was initiated in 2017, 21 proposals have been funded from a pool of over 60 submissions. Recipient countries include Zambia, Uganda (2), Namibia, Kenya, Malawi (2), Egypt, Ethiopia, Nigeria (3), Rwanda (2), Mozambique, and Cameroon (2) as well as Fiji (2), the Philippines and Nepal. Five recipients were funded in 2017, eight in 2018 and eight in 2019. Project directors are primarily junior doctors and project managers supervised by senior mentors. For most of the beneficiaries, The RHD Action Small Grants Programme is their first funding award.

Among the funding beneficiaries, 14 responded to the qualitative questionnaires and 16 participants responded to the quantitative questionnaire. Table 1 shows a summary of the grant beneficiaries who responded to the quantitative survey aggregated based on the year of the grant. For both quantitative survey and qualitative interviews there was a 76% response rate with most of the respondents being grant recipients from 2018 and 2019, Figure 1 shows the distribution of respondents below.

Grant recipient project completion

Of the recipients who responded, half of them completed the project and the other half are in progress. This is due to majority of recipients being awarded grants in 2019, with funds disbursed in 2020 (Table 1).

Grant application process and award giving

The grant application was shared through RHD Actions digital channels and cross promoted by partner organizations. Most recipients received the opportunity via emails and the website (Figure 2).

Overall, on a Likert scale of 1–5 (1 being not helpful and 5 being very helpful) all the respondents explained that they found the RFP and Guidelines very helpful for the application process. The qualitative interviews further revealed that the application process was straight forward.

‘The application was extremely straight forward and easy. The guideline was helpful and made the application process easy’.

In addition, on a likelihood scale of 1–5 (1 being not efficient and 5 being very efficient) all the respondents found the timeline between application to the awarding of the grant to be efficient (50%) and very efficient (50%) with quick turnaround.

Overall, the support provided by RHD Action to carry out the project was very responsive and helpful with all the respondents reporting a helpful and responsive support (25% helpful/responsive and 68% very helpful/responsive). In addition, all the respondents confirmed that the reporting process required by RHD Action was doable and very reasonable.

Over and above, one of the beneficiaries responded that the grant has gone way beyond to benefit the recipients.

And actually, the money is a lot of money. It doesn’t seem like a lot when you look at in numbers, but how much we were able to do with it has been impressive (Respondent 13).

And we still are using a lot of that money for our world heart day, printing T-Shirts that type of thing and whenever someone has a bigger idea, it’s also easy because of the initially boost we had, it’s easier to get other sponsors to assist us on something (Respondent 6).

Impact on Healthcare Workers

a) Impact of grants among various stakeholders:

Training was one of the platforms to enhance health care worker’s awareness and knowledge. Most of the trainings were conducted among health care workers at different career levels as shown in Figure 3 below.

The health care worker training explored all domains of RHD but mainly emphasized how to recognize and treat the condition (Figure 4).

The quantitative findings were investigated further using a qualitative survey where the health care workers explained the area of training that they benefited from.

previously on diagnostic skills on RHD, we didn’t know how to diagnose it, but with this project it has taken us very far. Currently, I am able to know if it is RHD or Rheumatic fever and compare. So, it has helped the diagnosis skills (Respondent 9).

when I got the RHD grant, facility X was struggling, most of the nurses were refusing to give penicillin because they had some bad outcomes as a result of the grant we were able to provide some pretty intense education and workshops to hospital staff to alleviate some of that fear. The health care workers after the workshops were more comfortable giving penicillin and improved their practice and care for kids with RHD (Respondent 10).

b) Impact on the participant/patient and community

Through education and awareness, the grant opportunity had a significant impact on the community. One of the respondents explained the impact of education on both patient (children) and parents:

it involved a lot of educating of not only the children who were the patients, but their parents as well, so they knew exactly what was happening and they knew what RHD, so they knew what they were dealing with? (Respondent 9).

The training and awareness targeted both health care facilities and non-health care facilities i.e. schools. Teachers were also involved to relay information to the community on management of RHD. The health care workers always engaged the parents when the children were involved.

We had an awareness and training campaign where we had targeted school-going children; some parents and teachers. The teachers were used as patrons for the children as a youth plan was formed, to assist to relay the information to the community (Respondent 9).

RHD Action provided digital media coverage of the work being done by the grantees (Figure 5). Among the grant recipients, 88% confirmed that they promoted their project using digital media and 69% using non-digital platform like posters/flyers. Facebook and twitter were the digital media most used among the beneficiaries.

c) Impact on the organization

The qualitative interviews discussed how the recipient felt the grant impacted their organization. Many recipients spoke in terms of their organization’s ability to carry out their work and sustain their mission. This was framed in terms of how patient and community awareness allowed them to be much more effective. The grant opportunity enhanced adherence and completion of treatment as advised by the doctor. During the patient visit, the organization created and enhanced awareness through education to the patient on the management of RHD.

So, now they are able to know quickly if someone was to have a sore throat. The community were educated on the importance of following the prescription and taking medication according to the doctor’s advice (Respondent 9).

The grant enabled us to help engage more with the community and stakeholders, so as a result staff and volunteers now have much more experience than we had before in community engagement (Respondent 3).

It was evident that before the grant the patients were non-adherent to treatment and follow up.

Many would take medication for five days and not finish the medication. So, the emphasis was taking the medication according to the doctor’s advice (Respondent 7).

The grant also helped to increase the feasibility of the organization and build more collaborations.

The grant has increased the feasibility on the organization. Both to the community and the healthcare workers on the facility (Respondent 8).

…. we could get more collaboration with the hospitals following the RHD grant, we actually trained more people and created more awareness, so I believe it was a bonus to move forward the RHD initiatives (Respondent 11).

The RHD grant did a very amazing impact on my organization, as it spread the impact of hours in the field and we just tried to get connected to other stakeholders all around the county. Therefore, the grant assisted us to get connected to other stakeholders who are interested in the field (Respondent 6).

d) Impact on project director

Most of the respondents expressed the relevance of the grant to themselves and the beneficiaries generally. Most of the project directors mentioned that the project would not have gone ahead if the grant was not given (Figure 6).

The qualitative survey alluded to the quantitative findings where the directors expressed that their ‘dream had come true’ through the grant opportunity. One respondent expressed that the opportunity of having grant has had such a huge impact on the RHD clinic.

There is a lot more awareness from the patients. There is a lot more patient support, the nurses, the healthcare professionals that work with our patients and even their family members because of the ambassador program and trying to get people involved in things (Respondent 14).

The directors of the project were motivated to work with the grant recipients/health care workers to make the best opportunity out of the programme. In one of the countries the project is working on empowering the patients and giving them a voice and their own obstacles.

Right now, they are working on making film, because we always said we would make a film, but because of the distances between us and the different patients due to COVID-19, and I mean I was quite surprised by, everyone wants to be in it. But you can’t have all six hundred people in a film (Respondent 14).

It is important to note that the grant boosted skills and confidence in many of the project directors. For some recipients it was the first grant they had received.

But for me personally, how can I say? the grant was like, a boost, so, a powerful way to encourage me (Respondent 1).

After receiving the grant, I think it opened up an opportunity to reach more people and I was able to meet people in the community who were at a higher level than I was. Hence, it opened connections and I was also able to be invited to conferences speak about it, meet more people and get more ideas. Generally, it broadened my network and I was able to learn more about the disease and how better I could help people (Respondent 4).

my care towards the patients also improved. I became more conscious and more, you know sensitive, looking out for people. Also, my professional profile improved in terms of the communication between other hospitals (Respondent 8).

Collaboration was one of the keys the directors of the projects used to ensure project continuity even after the lapse of the grant. A majority (93%) of the directors worked in partnership with various stakeholders, that is, organizations/institutions/government officials in the delivery of their project (Figure 7).

Communication platforms

Among the communication platforms, face book was one of the most used digital media among the beneficiaries as well as physical posters/fliers (Figure 8).

The grant recipients were creative to create a communication platform where the patients and the health care providers engaged to have a healthy discussion.

The grant has led to openness where it facilitated us having groups and community being more openness. The organization have created WhatsApp group where we have both patient and health care staff, I have actually never had that before, where I am on a WhatsApp group with my patients and I’ve said before you know, it’s not a medical group. It’s just a WhatsApp group, there is no ward, there is no penicillin, there’s no this, people can know, and we can all help each other at the hospital (Respondent 14).

Government Engagement

A key criterion for selecting a grant was that the applications demonstrated how the grantee would engage the government in their work. Despite well thought out plans, grant recipients faced various challenges engaging their government.

The challenge is to bring the entire health sector and educational sector together, because I think if we can reach the entire health sector and educational sector, we can bring them together and reach the entire community (Respondent 12).

The challenges actually really were getting administrative approval, but actually we got some support from the military district in country X, they facilitated everything for us to access the region and to organize with the school because actually this time was exams and children were engaged in their exams (Respondent 2).

Yes, there were some challenges well some were political and in that we were having a national election at the time. So, we didn’t engage with one of the communities we had originally planned to do, because it was elections and it became quite intense and then eventually, we could not engage with that community. Yes, so political challenges (Respondent 3).

Case studies on experiences on relevance of the grant to the recipient.

Permission was granted to publish the case study extract 1 above.

Permission was granted to publish the case study extract 2 above.

Future opportunity and recommendations

The grant recipients were invited to share their reflection on future opportunities that could open from the Small Grant Programme and to make recommendations on how the program could be improved. Below is a selection of quotes from the recipients:

What we want to do is aim to conduct training for these community health care workers because they are at the grass root level. In fact, community are our ears and eyes and they are our contact to all our RHD patients out there in the communities.” (Respondent 12)

“The grant created opportunity for a forum where people who are interested can come together and share ideas on how their experience was and what worked and how we can apply it in our country. That will probably create a difference (Respondent 4).

recent years some of my mentees have wanted to apply and have had great ideas, but I think it has become if I remember correctly a little narrower in the scope. I forget what the actually scope was last year, but you know he had some great ideas, but it didn’t really fit with what the grant wanted that year, so I would suggest broadening the scope as future recommendation (Respondent 10).

But to improve, maybe in the more future, if the amount of money would be more, so the larger population could benefit but then generally I liked the grant because the response has been very good (Respondent 9).