Blanket Supplementary Feeding Programs

Measuring the impact of a MAM treatment SFP is easy (perhaps "easy" is the wrong word ... "feasible" is perhaps a better word in this context). It is pragmatic efficacy (i.e. the "cure rate") multiplied by program coverage.

It is difficult to measure the impact of a BSFP. One approach is repeated cross-sectional surveys at the times when you expect MAM incidence to be high. If (e.g) in the month prior to harvest in the year without BSFP prevalence was estimated to be 14% (95% CI = 11% - 17%) and in the month prior to harvest with BSFP prevalence is estimated to be 8% (95% CI = 6% - 10%) the reduction is taken to be the impact.

The non-overlapping confidence intervals are taken to indicate a statistically significant impact (there are slightly more complicated methods than looking for lack of overlap in the confidence intervals which perform better ... I can find appropriate references if required). There are significant problems with this approach. The principal one in emergency BSFP is "regression to the mean". You intervene because the first survey and food security and agriculture data indicated it. This was probably a year with untypical high prevalence. Whatever happened the prevalence in the following year was likely to be close to historical norms. This means that the observed effect may have had little to do with the BSFP. You might also be measuring a secular trend or the effects of companion programs.

Another approach is to do some sort of comparison trial. Here you might survey two areas at the same time (one with the intervention and the other with ... see below ... nothing?) and compare prevalences. The problem here is ethics. You want the control area to be very similar to the intervention area in terms of factors associated with malnutrition. You have already decided that the intervention area needs a BSFP. If the intervention and control areas are similar enough for a comparative trial then you really cannot do nothing in the control area. I suppose you could compare BSFP in the intervention area and (e.g.) a MAM treatment SFP in the control area. You'd want to be wary of a self-fulfilling prophecy in which the idea is to validate the BSFP and efforts, even non-conscious efforts, are made to make the intervention appear better then the control. This is one reason we blind clinical trials - We want our drugs to work so we don't let the patient or the clinician or, in some cases, the statistician know who got the new drug and who got the placebo (or the old drug). I worry about such a simple comparative trial as above. A community randomised trial might be better but these can be difficult to get right even when done by experienced academics. I think a community randomised trial is likely to provide the best evidence of impact but such trials are probably too complicated to be used for routine monitoring and evaluation purposes.

Last time I looked the "draft MRP for SFPs" was (in my opinion) weak on coverage in terms of standards and methods for ascertaining coverage. Mostly, I think, these apply to MAM treatment SFPs.

Your part (c) has two parts ... quality could be measured as you would with a general food distribution (e.g. basket monitoring). You'd want to be sure, if you were giving a fortified cereal-legume blend, that the stuff was actually fortified. You'd also want some process indicators. Coverage is a big issue. I'd start with something like SQUEAC as this is relatively low cost and is suited to ongoing audit / total quality management efforts. SQUEAC is good at identifying barriers and suggesting solutions. Things like "significant access problems" might, if I get your meaning correct, have simple local solutions. I have seen (e.g.) programs that give transport tokens to allow carers to take a ferry free of charge (the ferryman collects the tokens which the program purchases back), programs that arrange protective escorts when security is low and there is a risk of abduction or rape, programs that use rough terrain vehicles (e.g. 6x6 and half-track trucks, 4x4 pickup trucks, or mules) to deliver a mobile distribution local to the beneficiary population, and programs that decentralise to village-based workers such as using GMP workers or CHWs to hold stock and manage distribution locally.

I hope this is of some help.

Two points ...

(a) I am confused. BSFP is a universal program with a considerable logistics overhead. A MAM treatment SFP is targeted and has a much lower logistics overhead (e.g. if SAM prevalence is 10% then the logistics overhead is 10 times lower than BSFP). So, in the situations you describe the MAM treatment SFP is likely to be more feasible than a BSFP. The issue is, perhaps, the cost of targeting. If it costs a lot to screen then the BSFP might be the better option. Screening is usually only costly if you use an inappropriate case-definition. W/H makes screening very costly and usually has a negative impact on coverage. Better to follow the CTC lead and decentralise case-finding to community-based volunteers using a MUAC case-definition. That will allow low cost and high coverage of screening.

We have to be very careful about how we talk about SFP coverage. I always start with a program deficit model (this is a SQUEAC basic). With this model we say that coverage is low because the program does not fit the context. To improve coverage we must make the program a better fit to the context. The alternative approach is to blame the population, security, geography, &c. for poor coverage. The problem with this approach is that all of the "causes" of poor coverage are outside of your control. So, if you subscribe to this model you just carry on running an inappropriately designed program rather than reforming your program to fit the context better.

It is a SQUEAC basic that it is easier to change the program then it is to change the geography. This is encapsulated nicely as "If the mountain won't come to Muhammad then Muhammad must go to the mountain".

(b) You are right ... it is very difficult to measure impacts of these types of program for the reasons I wrote about previously.

We can, however, separate the general from the specific. If we need evidence that BSFPs can have an impact then a study could be designed to discover this. This would probably use a community randomised design. That would give us evidence in the general sense just as we use a drug based on the findings of clinical trials. If we cannot find evidence that BSFPs work then we shoudl not do them. I think we would need some further evidence since we would have "study efficacy" rather than "pragmatic efficacy" from the study ... we'd have to design the study carefully to avoid overly optimistic outcomes. Once we knew that BSFP worked we could then use process indicators to ensure a good program was being delivered. That would probably be good enough.