Keywords
antimicrobial, antibiotic resistance, drug, infection, surveillance
antimicrobial, antibiotic resistance, drug, infection, surveillance
Since the introduction of antibiotics into medicine in the 1940s, they have provided dependable treatment for many infectious diseases. The emergence of antimicrobial resistance (AMR) and related drug resistant infections (DRIs) challenges this1. One of the early priorities identified to tackle the emergence of AMR and DRIs is to strengthen surveillance of AMR in low and middle income countries (LMICs)2.
Despite LMICs having the highest incidence of infection (and associated mortality and morbidity) there are significant gaps in our understanding of the aetiology and incidence of infectious diseases in these contexts3,4. This constrains national governments and international organisations in their efforts to detect evolving trends and emerging threats.
Data on the true prevalence of AMR in LMICs are limited. In 2014, the World Health Organization (WHO) summarized the most recent information on AMR surveillance for a selected set of nine bacteria–antibacterial drug combinations of public health importance from 129 Member States5. Among WHO regions, the greatest volume of country-level data were obtained from the European Region and the Region of the Americas, where longstanding regional surveillance and collaboration exist. In contrast, data from LMIC were underrepresented, and in some countries, lacking. However, there is no evidence to suggest that the burden of AMR for many common pathogens in LMICs is not increasing in line with the rest of the world5, and can be the place of emergence6. Understanding current capacity for AMR surveillance in LMICs is important, as well as considering how to develop capacity in different settings and contexts, and their requisite costs.
Modelled estimates put the number of AMR attributable deaths per year in Europe at 25,000 with €1.5 billion in associated health care costs and productivity losses7; in the United States there are 23,000 deaths per year attributable to AMR at a direct cost of US$20 billion and an indirect cost of US$35 billion in productivity losses8,9. There are no reliable estimates available on the economic losses due to AMR available from LMICs, but they are likely to be considerable. Despite the economic losses identified in high income countries, funding to tackle AMR has, until recently, been limited. This is in part because costs associated with establishing and running AMR surveillance were considered to be high. With a view to informing future capacity building for AMR surveillance in LMIC settings, this work aimed to characterise the requirements for achieving capacity to detect the emergence of AMR at national levels. The specific objectives were:
i) To map and compare existing models for laboratory system strengthening (for AMR surveillance), including the identification of specific contexts and geographical settings where certain models have been more or less successful, analysis of challenges and barriers in different geographical and sociodemographic contexts, consideration of costs and sustainability and assessment of governance issues, including the ability to share data.
ii) To map and compare existing AMR surveillance systems in at least three countries, identifying different approaches for monitoring emergence and spread of resistance in different countries or regional settings, including the range of baseline data gathered, the best models and mechanisms for surveillance, capacity strengthening and training in the different country or regional settings.
A fully formatted report is included as Supplementary File 1.
A team with expertise in health systems, health economics, anthropology, microbiology, laboratory capacity, surveillance, and epidemiology was brought together. This group discussed and developed an initial outline of a conceptual model for AMR surveillance on which to frame the desk based analysis, case studies and costing of AMR (Figure 1).

At a national level, there is a need for leadership to support training, and quality, with a reference laboratory function. At a site an integrated model includes development of laboratory capacity, clinical surveillance, health care utilisation surveys and census or enumeration data to determine the population catchment. This provides the framework for local, national and international public health action.
A desk-based analysis of capacity for AMR surveillance in Sub-Saharan Africa and South-east Asia was done, first by reviewing the WHO’s AMR report and then looking at those countries that had been able to contribute data, and identifying which components of the conceptual model they had put in place (or other) to do this. The challenges faced in these systems were also considered. Where there were few or no data in WHO sub-regions, examples of other surveillance systems that had been established, outside of the context of AMR surveillance were reviewed.
Site visits were made to Malawi, Ethiopia and Vietnam, and capacity in these countries reviewed according to the components included in the conceptual model and the functions that would be needed to improve/develop AMR surveillance, as well as the strengths of the systems and models in place. This work involved extensive review of relevant guidelines and health system and policy literature, as well as discussions with in-country personnel, including laboratory staff, policy-makers, clinicians, microbiologists, veterinarians and epidemiologists.
We identified the functions, sub-functions and activities needed for AMR surveillance at a national and site level, and defined four levels of AMR surveillance capacity involving increasing complexity. Capacity at Level 1 represents the most limited system, with incomplete clinical data, minimal clinical investigation and no laboratory testing. At level 2 there is some laboratory testing but no standardization of processes in place. Level 3 represents a core surveillance system with standardized processes, focusing on invasive disease in children under five years and the identification and antimicrobial susceptibility testing of WHO priority pathogens (Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, Streptococcus pneumoniae, Acinetobacter baumannii, Salmonella spp., Shigella spp., Neisseria gonorrhoeae). Level 4 represents extended AMR surveillance capacity, with capacity to include AMR surveillance from all patients and all sample types. The four functions at the sentinel site level were 1) clinical admission assessment & investigation; 2) isolate identification and susceptibility testing; 3) isolate storage (local) & referral to reference laboratory; and 4) data system and review. At the national level these functions were 1) leadership; 2) training; 3) quality assurance; and 4) national reference laboratory. Each function was further broken down into sub-functions; for example “clinical admission assessment & investigation” has four sub-functions, namely i) clinical admission assessment; ii) clinical data; iii) clinical investigation; and iv) training and quality assurance, all describing the different activities relevant for clinical surveillance. Components are detailed in Table 1 and Table 2.
In each sub-function, the interventions, activities and resources needed to move from one level to the next were defined. The costs of establishing and implementing (start-up costs), as well as running (recurrent costs) of the described activities were included and recorded following an ingredients approach, where unit costs and quantities were required to calculate total costs for each input. This allowed the cost estimates to be adapted more easily to different country settings, by applying local unit costs. The cost results were presented by year, ranging from year 1 to year 5. Due to the different capacities and models used in the different sites that were visited, as well as logistical constraints, costs were assessed in a single highly functioning site in Kenya (KEMRI-Wellcome Trust Research Programme), where clinical, laboratory and demographic surveillance systems are integrated. We used site specific costs, for personnel, for subsequent specific country estimates. Data collection for the costing involved interviews with key personnel, review of key documents and observation of a functioning clinical, laboratory and demographic surveillance system. The approach to the costing was pragmatic, prioritising the most accurate recording of high cost items (with most influence on estimates of total cost). All information was collected and analysed in Excel 2016.
Capacity for AMR surveillance varies between sub-Saharan Africa and South-east Asia, and within countries in these regions. In the WHO African region (AFRO), a recent review reported only 2 of the 47 AFRO member states (Ethiopia and South Africa) had national AMR plans in place5,10, and 7/47 members (Ethiopia, Ghana, Kenya, Lesotho, South Africa, Tanzania and Zimbabwe) had overarching national infection prevention and control (IPC) policies11. Establishment of AMR surveillance is limited, but capacity is being developed, for example in 24 hospital laboratories in Ghana12. In contrast, South-East Asian countries have, are developing, or are initiating AMR surveillance systems, for example, Thailand, Vietnam, and Cambodia13, respectively. This is in part supported by the more advanced development of networks and collaborations for AMR surveillance. Networks and examples of AMR surveillance and stewardship programmes are described in detail in Supplementary File 1, as well as surveillance of other infectious diseases. AMR surveillance strategies could also learn from and benefit from recent laboratory strengthening efforts as an important element of the work against the Ebola epidemic in West Africa since 2014 (Supplementary File 1: Box 3) and for specific diseases, for example rotavirus (Supplementary File 1: Box 4)14.
Emerging themes in strengthening laboratory capacity for AMR surveillance included:
1. Leadership
Leadership, and the national policy addressing AMR, as well as international collaboration, are necessary to strengthen AMR surveillance in LMIC countries, to ensure an enabling policy environment. However expertise in microbiology is limited in these settings, and, whilst in the longer term this capacity is important to develop, in the interim leaders may need to be drawn from clinical infectious diseases and public health. Leaders are important for advocacy/championing, and for local credibility of the surveillance effort, as well as providing the focal point for activities.
2. Training
Where health systems are weak and still developing, there is a need to develop human resources across cadres of staff: clinical, laboratory and data management. Well-trained staff are likely to move elsewhere for better salaries; it is important that there is sufficient budget to ensure staff can be recruited and retained.
3. Laboratory quality assurance
In terms of strengthening laboratory capacity, it is clear that inadequate laboratory infrastructure limits the quality and the ability to reliably detect pathogens and conduct antimicrobial susceptibility testing.
There are different options for models of AMR surveillance (outlined in Supplementary File 1 and summarised in Supplementary File 1: Table 1). Overall, a model with active, continuous, comprehensive integrated population and laboratory disease surveillance would provide the most robust, comprehensive data. However, within the constraints of resources, an appropriately designed sentinel site system would also be appropriate. Whilst it would be feasible to assess AMR outside of a health care context, for example through cross-sectional studies of population colonisation (as part of a population based survey), these would not include surveillance of the most serious drug resistant infections, and fall outside the remit of public health surveillance as observational research studies requiring appropriate ethical approvals and individual informed consent.
Laboratory based surveillance identifies “a resistant organism”, but without clinical or population data cannot be fully interpreted and inform policy. To achieve the aim of AMR surveillance, and inform interventions, data are needed on:
• the proportion of the particular bacterial species identified (e.g. Klebsiella pneumoniae) that are resistant
• whether resistance is developing in a particular patient group (e.g. neonatal), or particular presentations (e.g. pneumonia)
• whether this is localised to a setting (e.g. ward, or geographic area)
• who seeks health care
• who lives in the population catchment area
Within the different models of surveillance, the functions are broadly consistent, and can be split into those provided at a national level and those provided at one or more sites, depending on capacity and the model in place, and including the themes identified above. At a national level, there is a need for leadership to support training, and quality across the system. A national AMR laboratory can support development of site laboratories and conduct more extensive testing (thus focussing resources, Supplementary File 1: Table 3, Figure 1). At a site level an integrated model includes development of laboratory capacity (isolate identification and susceptibility testing), clinical surveillance (systematic investigation of patients according to diagnosis), health care utilisation surveys (to assess representativeness of the health facility data) and census or enumeration data to determine the population catchment (Supplementary File 1: Table 2, Figure 1). These functions were used to cost AMR surveillance (Supplementary File 1: Tables 2 and 3) and consider how they were being achieved in the different models and contexts of the country case studies.
Full site reports are included in Supplementary File 1, the findings for each country are summarised here.
Malawi. Malawi faces substantial challenges due to extremely limited country resources, which reduces health system capacity. However, whilst it does not have the capacity for a comprehensive AMR surveillance system, it has been developing institutional infrastructure and policy for AMR surveillance. It also has considerable capacity through international collaboration and an academic centre of excellence (Malawi-Liverpool Wellcome Trust (MLW)) undertaking microbiological surveillance as part of a research programme. In order to take advantage of available research infrastructure i.e. sentinel surveillance capacity beyond Queen Elizabeth Central Hospital (QECH), the following should be considered for new surveillance sites in Malawi:
1. Whether current capacity is representative of its area
The current AMR surveillance is conducted at the referral hospital (QECH), which limits the representativeness of the sampled patient population in relation to Blantyre district as a whole. It is therefore important to consider supporting surveillance capacity development in other sites, using, if possible, the expertise available in country.
2. What the needs for investment/enhancement are in this setting
Clinical surveillance and sampling is being strengthened with triage nurses to support systematic investigation in QECH. Microbiological surveillance is very strong, but electronic linking of clinical and microbiological databases is needed at this site and extension sites.
3. Whether the hospital is connected to national institutions, and regional networks
There is growing collaboration between MLW, QECH and the Ministry of Health in Lilongwe, which would be needed to support extension of AMR surveillance sites.
Ethiopia. Ethiopia is developing a system for AMR surveillance that includes a national reference laboratory and sentinel regional laboratories across the country reporting data (continuous, passive surveillance). At present, whilst capacity is in development, there are some hospitals where the standard of clinical examination is adequate for surveillance, but there is limited microbiology laboratory capacity for processing specimens for culture, particularly blood cultures. Record keeping is not formalized or electronic, and there is potential for improved communication between the ward and the laboratory to link microbiology data to patients. In strengthening capacity for AMR surveillance, the following should be considered:
1. Determine whether the hospital(s) are epidemiologically representative
It is important to include a range of health care settings for AMR surveillance; district hospitals, outside of the capital are more generalizable to the Ethiopian population as a whole than very urban settings.
2. Establish surveillance within the hospital
Systems for development would be national guidelines supporting standardised clinical examination and systematic investigation. Laboratory capacity needs strengthening in terms of automated blood cultures to support AMR surveillance to a core level. Clinical and laboratory data should be linked, this could be through a double sided form with data collection for both.
3. Connect the site to a national and regional network
The Ethiopian Public Health Institute is an arm of the Federal Ministry of Health and combines public health policy making with laboratory capacity in the same institution. Stronger regional links could support capacity development as well as continued partnership with international organizations.
Vietnam. Vietnam has established national infrastructure for AMR and has worked with regional networks and international collaborators to develop a surveillance system, including a national reference laboratory and twenty laboratories attached to district hospitals. The hospitals are public and broadly representative, there are functioning microbiology laboratories and the physical, legal and ethical capacity to link these records for anonymised aggregation at individual case level. In strengthening capacity for AMR surveillance, the following should be considered:
1. Consider the hospitals in the network
The hospitals in the network are mid points in the hierarchical system in Vietnam and broadly representative. However, there would be considerable benefit from understanding health care utilisation patterns and population catchments to support laboratory data interpretation. In addition, given the ready accessibility and high volume use of antibiotics in Vietnam, surveillance may be needed at lower levels of health care.
2. Establish surveillance within hospital
It is important that clinical examination and investigations are strengthened in the AMR surveillance system, to have the benefits of an integrated model, as well as the strengthening of laboratory microbiology services.
3. Build a network within country (and on to WHO)
Many of this activities are in place in Vietnam, supported by strong national leadership and collaboration, and a hierarchical public health care system where national strategies can be implemented into care.
We developed a flexible model, based on an assumption that each country would have a number of sentinel sites with clinical and laboratory surveillance taking place in each site. We made an assumption that a country would aim for one surveillance facility per five million population, with a minimum of five surveillance facilities per country. However, cost results are presented for varying number of sites per country (2, 5 and 10), rather than for the target number of sites, with a focus on costing inputs required to achieve surveillance at level 3 (a core level) and level 4 (an extended level).
For a sentinel site (Table 1), each sub-function was broken down into all input activities and resources required to achieve the different levels of surveillance sophistication (core and extended) and summarized under “interventions” below each sub-function in clinical surveillance, laboratory surveillance, isolate storage and transport, data system and review (Supplementary File 1: Tables 4A–D). A number of inputs were costed independently of the sub-function framework, for example personnel costs required for laboratory surveillance or the cost of setting-up a laboratory including the cost of building and equipment. They are presented directly as inputs. The additional demographic component to surveillance was costed as an initial and final population enumeration round in year 1 and 5. Inputs of activities and resources required at the national surveillance level are shown in Table 2. The inputs (Supplementary File 1: Tables 4A–D and 5) were used to cost the surveillance components by determining quantities and unit costs at different levels of surveillance sophistication.
Costs depend on existing capacity, number of sentinel sites proposed for surveillance, and, across countries, variability in costs of personnel. Illustrations are given for Kenya and Vietnam according to different numbers of sites (Supplementary File 1: Figures 7 and 8) across clinical, laboratory, national and demographic surveillance, as well as data system & review functions.
The largest contributor to the total cost of AMR surveillance is laboratory surveillance at the sentinel site (including isolate storage and transport to national reference laboratory), together with the national reference laboratory cost. These laboratory costs range, depending on number of sites and country, between 67% and 77% of the total cost at level 3 and between 78% and 85% at level 4. Within the laboratory cost the largest contributors to the total costs are the initial set up cost of the laboratory, and personnel. The increase in cost from core laboratory functions to extended laboratory functions is substantial (Supplementary File 1: Figures 6 and 7) and is driven by the increase in equipment and staff to support the increase in samples being processed (all sample types vs blood cultures), and isolates tested (all isolates vs WHO priority pathogens (excluding Neisseria gonorrhoeae, omitted as frequently tested for in outpatient settings with swabs)).
Within the overall envelope costed for AMR surveillance, the costs for clinical surveillance, data system and demographic surveillance are a much smaller proportion of total cost than laboratory surveillance. This is partly because for the clinical elements the costs of clinical staff providing care were not included, as in-post, except where additional clinical staff would be needed to undertake specific activities needed to support surveillance. In contrast, laboratory personnel costs were all included in the costing. However, this likely reflects the real-world situation, in terms of the need for recruitment of laboratory staff.
We aimed to map and compare existing models for laboratory system strengthening for AMR surveillance in LMICs. The work was based on desk review and three case studies on Ethiopia, Malawi and Vietnam, as well as costing based on a site visit at KEMRI Wellcome Trust Research Programme, Kilifi, Kenya. In the settings described in sub-Saharan Africa and South-east Asia, the AMR surveillance models are continuous, largely passive, laboratory based, using sentinel sites in country. An integrated model would be preferable for AMR surveillance so that data can be interpreted in a known clinical and demographic context, and case study countries were examined with this in mind.
It was apparent that the three countries visited were very different in terms of institutional architecture, health systems and AMR surveillance capacity, even on the short site visits undertaken for this work. For example, in Ethiopia there was early development of laboratory capacity in some broadly representative public hospitals, but no linked clinical data system, or population data; in Malawi there was very limited laboratory capacity in public hospitals, but a highly functioning research centre conducting surveillance in Blantyre, with high quality laboratory data, basic clinical data and a population denominator for urban and rural Blantyre; in Vietnam there was a broadly functioning laboratory capacity with a network of representative public hospitals across the country, supported by a consortium of international collaborators and with major investment but conducted at a level of the referral hierarchy (District hospitals), which is strongly susceptible to biases attributable to antibiotic access lower down in the referral hierarchy.
There were factors that were consistent in supporting and driving success in AMR capacity (Supplementary File 1: Figure 9). These include political commitment and collaboration amongst and between government and international stakeholders to utilise local and international expertise in supporting AMR surveillance development, as well as national policies on AMR surveillance developed within legal and ethical frameworks. Although an assessment of interventions in relation to AMR was beyond the scope of this work, political commitment and collaboration amongst stakeholders will also be crucial when considering and implementing these interventions (e.g. stewardship and antibiotic use), and initiating policy change.
It was clear that a functioning health system can support development of an integrated sentinel surveillance site system, progressing perhaps to comprehensive systems, more readily than a weaker health system. The integration of AMR surveillance into a health system requires an assessment of the form, state and functioning of a country’s health system. This is a central issue as the degree to which AMR surveillance will benefit national and local concerns (as opposed creating further burdens to staff) will largely be determined by the strength of its country’s health systems.
An important component, and common limitation of health systems in LMICs are the human resources to do work, across all cadres of staff, including clinical, laboratory, managerial, policy-making, data analysis, project management groups. Microbiological expertise is particularly limited. There were examples of support for human capacity development in the country case studies through Universities (national and international), research institutions (national and international) and interest in developing technical postgraduate programmes. However, where the health systems need strengthening there is insufficient training available in country to support the development of technical capacity to the level required. In these situations, AMR surveillance will be more likely to succeed if it is built up more slowly, using the expertise available from the most appropriate institutions. To ensure success at a “core level”, strengthening of surveillance may require a staged process with capacity built up over time. Here one site would be supported (which could be through a national reference laboratory, external partner, or research laboratory) to build capacity. An example of this would be utilising the expertise available in the Malawi Liverpool - Wellcome Trust Research Programme to support AMR surveillance in a district hospital. With development of capacity to a core level at a sentinel surveillance site, this site could subsequently serve as a training and support centre to develop capacity at another site, thus overcoming barriers to high quality surveillance when capacity is thinly spread across a weak health system. From this point, over time, a network of sites could be developed, as they are being developed in Vietnam, with the potential for sub-regional reference laboratories as the network expands.
Laboratory infrastructure with the capacity for standardised, quality diagnostics, are important. However, they have to be supported by appropriate logistical support (procurement, budget management), otherwise even the most highly equipped laboratories will not be able to function. With this is mind, securing a core set of standard activities, which can be maintained and result in useful, interpretable data is more important in the early stages of capacity development than the use of more advanced techniques. It also requires considerably less investment, and supporting five sentinel sites to a core level will likely produce more generalizable data than two sites, who have an extended capacity, for a similar investment.
Countries who have fragile and/or transitioning health systems are least able to respond to the present burden of infectious disease. Surveillance capacity depends on the existing health system and considerable investment is needed to develop laboratory infrastructure aligned to this. High level expertise in-country to provide advocacy, championing and trusted guidance to the Ministry of Health is important. Engagement with both the Ministry of Health and other national stakeholders is essential, combined with the expertise of in-country partners such as the public health organizations or academic institutions. Establishing whether these infections are treatable will inform treatment and facilitate a response; contributing to reducing the inequities in understanding disease burden worldwide.
A full report on the data collected are included as Supplementary File 1. A costings tool in Excel 2016 can be made available through the corresponding author on request. This allows costs of AMR surveillance to be estimated based on the target number of sites in a country.
The team involved in this work included Anna C Seale (LSHTM), Coll Hutchison (LSHTM), Silke Fernandes (LSHTM), Helen Kelly (LSHTM), Nicole Stoesser (University of Oxford), Paul Turner (University of Oxford), Kara Hanson (LSHTM), Clare Chandler (LSHTM), Catherine Goodman (LSHTM), Richard Stabler (LSHTM), Brett Lowe (University of Oxford) and J Anthony G Scott (LSHTM). The report was first drafted by Coll Hutchison, Silke Fernandes, Helen Kelly, Nicole Stoesser and Anna Seale. The site visits were done by Anna Seale, Coll Hutchison, and Nicole Stoesser and (for economics) Silke Fernandes. The costing analysis was done by Silke Fernandes.
This is an independent study commissioned by the Wellcome Trust (202959) and funded by Department of Health, UK. Anthony Scott and Anna Seale are funded by The Wellcome Trust (098532, 205184, respectively). Nicole Stoesser is funded through a University of Oxford/Public Health England Clinical Lectureship.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
In Ethiopia we would particularly like to thank the following institutions and personnel: Ethiopian Public Health Institute, Addis Ababa - Dr Adugna Woyessa, American Society for Microbiology, Laboratory Capacity Development Consultant - Dr Lourens Robberts; Centers for Communicable Disease Control, Ethiopia - Dr Yared Tedla, Laboratory Officer for the PEPFAR programme; Armauer Hansen Research Institute, Addis Ababa - Dr Abraham Aseffa, Scientific Director and Dr Taye Tolera Balcha (Director General); Haramaya University College of Health and Medical Sciences - Dr Yadeta Dessie, Dr Nega Assefa, Mr Melkamu Dedefo; Harar Regional Laboratory, Harar; Harar Hospital Laboratory, Haramaya University, Hiwot Fana Specialized Hospital, Harar. We would like to thank our collaborators in Malawi: particularly the Malawi-Liverpool-Wellcome Trust Research Unit at Queen’s Hospital, Blantyre including Prof Stephen Gordon, Dr Dean Everett, Brigitte Denis, Dr Rebecca Lester, Dr Jennifer Cornick, Dr Jamilah Meghji, Dr Melita Gordon and Dr Nicholas Feasey. In Vietnam, we would like to thank the staff of the Oxford University Clinical Research Unit (OUCRU), including Guy Thwaites, Rogier Van Doorn, and Bui Huyen Trang, as well as Dr Nguyen Van Vinh Chau (Director, Hospital for Tropical Diseases) in Ho Chi Minh City, and Prof Nguyen Van Kinh, Prof Nguyen Vu Trung and Dr Le Thi Hoi from the National Hospital of Tropical Diseases; Prof Luong Ngoc Khue, Ngo Thi Bich Ha, Dr Nguyen Trong Khoa, Mr Cao Hung Thai from the Ministry of Health, Medical Services Administration (MSA). In Kenya, we would like to thank the KEMRI-Wellcome Trust Research Programme including Neema Mturi, Shebe Mohammed, Mary Muthomi, Brett Lowe, Salim Mwarumba, Oscar Kai, Moses Mosobo, Horace Gumba, Anthony Etyang, Eduard Sanders, Elinah Kaulu, Dorothy Tole, Lucian Mshote, Barnes Kitsao, Godfried Otieno. We would also like to thank others who gave their time and advice, including Prof Rob Heyderman, Dr Chris Parry, Fatima Serhan and Adam L. Cohen.
Supplementary File 1: A fully formatted report with all Tables and Figures entitled “Supporting surveillance capacity for antimicrobial resistance: Laboratory Capacity Strengthening for Drug Resistant Infections in Low and Middle Income Countries”.
| Views | Downloads | |
|---|---|---|
| Wellcome Open Research | - | - |
Data from PMC are received and updated monthly. | - | - |
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Partly
Are sufficient details of methods and analysis provided to allow replication by others?
No
If applicable, is the statistical analysis and its interpretation appropriate?
I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health systems
Alongside their report, reviewers assign a status to the article:
| Invited Reviewers | |||
|---|---|---|---|
| 1 | 2 | 3 | |
| Version 1 26 Sep 17 | read | read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with Wellcome Open Research
Already registered? Sign in
If you are a previous or current Wellcome grant holder, sign up for information about developments, publishing and publications from Wellcome Open Research.
We'll keep you updated on any major new updates to Wellcome Open Research
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
We have sent an email to , please follow the instructions to reset your password.
If you don't receive this email, please check your spam filters and/or contact .
Comments on this article Comments (0)