A mixed methods evaluation of Advanced Life Support in Obstetrics (ALSO) and Basic Life Support in Obstetrics (BLSO) in a resource-limited setting on the Thailand-Myanmar border

Background: Short emergency obstetric care (EmOC) courses have demonstrated improved provider confidence, knowledge and skills but impact on indicators such as maternal mortality and stillbirth is less substantial. This manuscript evaluates Advanced Life Support in Obstetrics (ALSO) and Basic Life Support (BLSO) as an adult education tool, in a protracted, post-conflict and resource-limited setting. Methods: A mixed methods evaluation was used. Basic characteristics of ALSO and BLSO participants and their course results were summarized. Kirkpatrick’s framework for assessment of education effectiveness included: qualitative data on participants’ reactions to training (level 1); and quantitative health indicator data on change in the availability and quality of EmOC and in maternal and/or neonatal health outcomes (level 4), by evaluation of the post-partum haemorrhage (PPH) related maternal mortality ratio (MMR) and stillbirth rate in the eight years prior and following implementation of ALSO and BLSO. Results: 561 Thailand-Myanmar border health workers participated in ALSO (n=355) and BLSO (n=206) courses 2008-2020. Pass rates on skills exceeded 90% for both courses while 50% passed the written ALSO test. Perceived confidence significantly improved for all items assessed. In the eight-year block preceding the implementation of ALSO and BLSO (2000-07) the PPH related MMR per 100,000 live births was 57.0 (95%CI 30.06-108.3)(9/15797) compared to 25.4 (95%CI 11.6-55.4)(6/23620) eight years following (2009-16), p=0.109. After adjustment, PPH related maternal mortality was associated with birth before ALSO/BLSO implementation aOR 3.825 (95%CI 1.1233-11.870), migrant (not refugee) status aOR 3.814 (95%CI 1.241-11.718) and attending ≤four antenatal consultations aOR 3.648 (95%CI 1.189-11.191). Stillbirth rate per 1,000 total births was 18.2 (95%CI 16.2-20.4)(291/16016) before the courses, and 11.1 (95%CI 9.8-12.5)(264/23884) after, p=0.038. Birth before ALSO/ BLSO implementation was associated with stillbirth aoR 1.235 (95%CI 1.018-1.500). Conclusions: This evaluation suggests ALSO and BLSO are sustainable, beneficial, EmOC trainings for adult education in protracted, post-conflict, resource-limited settings.


Introduction
Skilled attendance at birth providing quality emergency obstetric and newborn care is an essential element for reducing the high burden of maternal and neonatal morbidity and mortality in resource-limited settings (RLS) 1,2 .
A number of practical skills or "hands-on" short course courses are aimed at improving emergency obstetric care (EmOC) 3 . The Advanced Life Support in Obstetrics (ALSO) course developed by the American Academy of Family Physicians is one of the oldest commencing in 1991, is standardized and evidenced-based, and available in multiple languages 4,5 . ALSO promotes inter-professional and multidisciplinary training to equip the maternity team (nurse midwives, registered nurses, physicians, residents and other members) to improve patient safety and positively impact maternal and newborn outcomes in obstetric emergencies. Positive ALSO evaluations from high-income countries 6,7 and low-income and RLS (Colombia, Guatemala, Honduras, and Tanzania) report decreased in-hospital maternal mortality, episiotomy use, and postpartum haemorrhage (PPH); and active management of third stage of labour and vacuum-assisted vaginal delivery increased in frequency after ALSO training 8 . In 2010 a systematic review in low-resource environments suggested there was a lack of strong evidence of the effectiveness of emergency obstetric trainings 9 . In 2020 a global review reported improved provider knowledge/skills and change in clinical practice but evidence for a reduction in the number of cases of post-partum haemorrhage (PPH), case fatality rates, stillbirths and institutional maternal mortality were less strong 3 .
The ALSO approach to learning is adult based. Learning suits professionals working in government health systems where tertiary education and certification of health care training is the norm. Tertiary qualified staff may not be available in postconflict and politically unstable RLS with broken education and health infrastructure 10 . Evidence for the type of learning and teaching suitable for healthcare workers in politically unstable and post-conflict settings is limited 11 . ALSO requires a degree of health literacy that may challenge healthcare workers based in countries rebuilding education and health systems 12,13 .
Basic Life Support in Obstetrics (BLSO) is complementary to ALSO but structured for pre-hospital health care providers, first responders, and medical, nursing and physician assistant students 14 . The objective of BLSO is to improve the management of normal deliveries, as well as obstetric emergencies by standardizing the skills of frontline responders. BLSO is more difficult to compare internationally as there is more flexibility on topics included in the course, realizing the need for context appropriate training.
Reported success of training in emergency obstetric care in low-income countries within government health systems is commendable but are these replicable in post-conflict or politically unstable health care systems? 10,15-18 Shoklo Malaria Research Unit (SMRU), a humanitarian and research organization on the Thailand Myanmar border, has provided antenatal care and childbirth facilities for refugees since 1986 and migrants since 1998. Facility based ALSO and BLSO training for Karen and Burmese healthcare workers was introduced in 2008. This was the first time the ALSO course has been held in a refugee camp. The objective of this analysis is to evaluate ALSO and BLSO as an adult education tool in emergency obstetric care, from the first course in 2008 to the most recently completed course in May 2020, in a protracted, post-conflict, RLS.

Study design
This mixed methods study collates several data sources to evaluate ALSO and BLSO as an adult education tool over the time period of 2008 to 2020. Before each course participants provided information about themselves using a standard form. Change in perceived confidence was measured before and after the course, and a sub-cohort (of the 2008 course) had a 12-month evaluation. The course theory and skill results were summarized. Course strengths were evaluated through an openended question. Finally, obstetric indicators including PPH related maternal mortality and stillbirth in the eight year block preceding the implementation of ALSO and BLSO and in eight years after were sourced from routine surveillance data.

Setting
Maternal and child health care. Refugees Figure 1).

Amendments from Version 1
The major difference between version 1 and 2 includes minor clarifications: a few extra details in the methods, some formatting of the layout e.g. number of decimal places in OR reporting, reformatted Figure 3, correct numbering of the tables, addition of relevant references and the addition of uteronics available in this limited resource-limited setting.
Any further responses from the reviewers can be found at the end of the article REVISED Initially, P. falciparum infection accounted for the largest proportion of maternal mortality (estimated at 1,000 per 100,000 live births in 1985-1986 19 ) and was the focus of activity for the first two decades. In 1994 basic tools such as the World Health Organization (WHO) partogram in labour and essential medications such as magnesium sulphate for eclampsia and severe pre-eclampsia were introduced. To standardize interventions for common obstetric emergencies and medical complications SMRU produced a three language (Karen, Burmese and English) 'Manual of Obstetrics' in 1998, which has subsequently been revised three times (English only). As post-partum haemorrhage (PPH) 20 replaced malaria as the largest cause of maternal mortality, the need further training in obstetric emergencies was recognized.

Participants
Human resources. There are significant barriers to displaced persons in Thailand (refugees or migrants) obtaining formal training and certification as health workers in either Thailand or Myanmar. For more than three decades healthcare work by NGOs and humanitarian organizations on the border including SMRU and MTC have been provided by local Karen and Burmese from the community trained to work as medics, nurses, midwives, sonographers or counsellors as defined here: Medics: A 'medic' takes the role of doctor by doing clinical examinations, ordering laboratory tests, making the diagnoses and prescribing treatment. They have normally received a minimum of 6-12 months of theory training and a further 6-12 months of on the job practical training 12,13 .
Nurse: A 'nurse' works in the same capacity as nurses in developed countries and have normally received 3-6 months of theory and 3-6 months of practical training.
Midwife: A 'midwife' (or skilled birth attendant) provides antenatal and delivery care services. SMRU developed its first curriculum in 1994, with the current course taking a minimum of 15 months (30 deliveries) 21 .
Sonographer: SMRU trains local staff in ultrasound gestational age assessment. They have the hardest entrance test compared to the other health staff and the quality of their work is reported elsewhere 22 .
Counsellor: A 'counsellor' provides antenatal care counselling particularly in explaining screening for pregnant women (blood tests, gender-based violence, mental health, nutrition).
Myanmar doctors: SMRU employs doctors to support the health workers and some come from Myanmar.
All these health workers were invited to participate on courses as they were offered.

ALSO and BLSO
With support from ALSO Australasia, it was decided to launch the program at SMRU and MTC. The initial course was supported by four instructors from Australia. The ALSO course includes a syllabus to be studied in advance of the course, an intensive two-day course reviewing obstetric emergencies, and an examination of theory and skills ( Table 1). The BLSO course is conducted in one to two days, context adapted for front line responders (Table 1), with the emphasis on skills.
Instructors for ALSO and BLSO. ALSO and BLSO instructors must have successfully passed both ALSO and the ALSO Instructors course, detailed elsewhere but briefly a short intensive training of the ALSO teaching methods 4,5 . A range of languages are used in the clinics with most local staff speaking one or two dialects of Karen (Sgaw Karen or Pho Karen), Burmese and English as a 2 nd , 3 rd or 4 th language. Following the first course, local staff were selected as Instructors based on qualities such as being a good team player, performing well on the actual ALSO course and languages spoken 23 . Courses on the border retain the recommended participant to instructor ratio as 4:1.
ALSO for the Thailand-Myanmar border. SMRU course organizers reduced the volume and complexity of the English course material to reflect the baseline education of the group and to ensure the content was relevant to the local setting e.g. including malaria and sepsis as a cause of maternal mortality (Table 1). While the Thailand-Myanmar border commenced with ALSO, the Global ALSO manual has emerged as the most relevant tool to adapt the ALSO course to local circumstances, especially in resource limited settings 24 . ALSO course teaching was structured into two parts: the preparation phase and the standard two day intensive course. The preparation phase reviewed one to two chapters per week, mostly in English (60-120 minutes, led by a doctor). A study group for 2-3 hours later in the week, mostly in Karen and Burmese language led by a senior midwife or medic included practice of skills using donated mannequins. The total preparation phase took approximately 12-16 weeks. Perinatal loss/Birth crisis ^As maternal venous thrombosis mortality in the area is much lower than mortality from severe malaria and sepsis; it was de-emphasized while severe malaria and sepsis were included from 2008. ^^USA and Australasia deviate slightly on content in different editions of ALSO so the 9 th Edition USA does not include neonatal resuscitation but it is retained on the Thailand-Myanmar border as no alternative course is available.
The ALSO written exam was also modified for relevance and clarity, removing difficult English e.g. double negatives, and intrapartum (electronic) fetal surveillance, leaving 50 questions. The exams were available in Burmese starting in 2008 and Karen in 2013. Translation verification included back translation. Courses and exams were updated regularly, and numerical scores were given to participants.
BLSO for the Thailand-Myanmar border. BLSO was directed at frontline staff in border clinics who might encounter pregnant women or neonates in their work. English literacy was a recognized barrier to learning for these staff. Student midwives with little or no clinical experience were also encouraged to participate in BLSO before the end of the first year of training to promote confidence in the workplace and familiarity with some emergencies.
BLSO does not have a preparation phase for participants, the content and the assessment all took place in a two-day course. Material was prepared in English and taught in Burmese and Karen, with a 12-page handout (>50% pictures) provided.
The BLSO was assessed by 10 short answer questions and was available in four languages (Burmese, Karen, Thai and English). The practical component assessed emergencies that were likely to be seen in the community and scored as pass or fail.
Course fees and size. Sustainability in high-income countries comes at a cost to the participant. In this RLS both ALSO and BLSO were conducted by SMRU without cost to the participant. Doctors were asked to pay for the cost of copying the materials. Course size was limited by mannequins to a maximum of eight per group and three groups i.e. 24 participants at a single course.

Evaluation of adult education effectiveness
Kirkpatrick's framework for assessment of effectiveness of adult education includes four levels of data: (level 1) participants' reactions to training, (level 2) change in knowledge and/or skills, (level 3) change in behaviour or clinical practice and (level 4) change in the availability and quality of EmOC and in maternal and/or neonatal health outcomes 25 . In this analysis data on level 1 and level 4 were measured.
All level 1 data was voluntary following a verbal explanation to participants at course orientation. Two measures were included: perceived confidence and self-reported strengths of the course. Prospective repeated-measures surveys at three time periods on perceived confidence: pre course, immediate post course and in a sub-cohort from the original ALSO course in 2008, at 12 months post baseline. The survey tool was adapted with permission from ALSO courses in the USA 6,26 .
Self-reported strengths of the course were measured for ALSO participants by response to an open-ended survey question: "What are the strengths of the course?". Answers in native language were encouraged with responses in Karen and Burmese translated to English.
Level 4 data were extracted from the routine birth facility data curated by SMRU, and measured pre and post ALSO and BLSO in two equal year blocks: before (2000-2007) and after (2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016) implementation. The eight year time frame was chosen as all archived data were cleaned and data checked, while more recent data was not.

Variables
Participant characteristics included age, gender, profession, years of practice, workplace, high school and university attendance, and theory and skills results (as a percentage).
For qualitative analysis perceived confidence to manage the obstetric emergency situations covered in the ALSO or BLSO curriculum (detailed in Table 1) was measured using a fourpoint [1 terrified, 2 scared, 3 coping, 4 comfortable] rank-ordered response Likert scale.
For quantitative analysis extracted variables related to maternal mortality and stillbirth included: status (migrant/refugee), parity, smoking, underweight (<40 kg), four or less antenatal consultations, anaemia the first antenatal visit (haematocrit < 30%) and falciparum malaria confirmed by microscopy by active and frequent screening in pregnancy.

Statistical analysis
The analysis was conducted using SPSS 23.0 for Windows. Participant characteristics such as gender were summarized mostly by proportions (n, %) while age and scores on the course were summarized using the median (interquartile range). The Likert scale responses were summarized by median, interquartile range (IQR) or range (minimum-maximum (min-max)), or proportions, as appropriate. The 95% confidence intervals of proportions were calculated using Wilson's method 27 . The comparison of confidence scores was based on individual paired testing using the Wilcoxon signed rank test.
Qualitative data were analyzed using conventional content analysis to identify major themes 28 . Two authors independently and manually reviewed printed transcripts of the data and suggested possible codes. After discussion, and analysis of the relationship between proposed codes, three common themes were agreed on for the final analysis. Data review was repeated and content was summarized into these three themes. Each reviewer independently chose exemplary comments for each of the three themes and final inclusion was based on discussion and by mutual agreement.
A word cloud was developed using free online software Voyant Tools (https://voyant-tools.org/), to examine the most commonly used words from all the responses to the openended question.
The odds ratio (OR) and confidence interval (CI) for factors associated with the binary outcomes, PPH related maternal mortality (yes/no) and stillbirth (yes/no), were calculated by univariate analysis. The outcome of interest, birth before ALSO and BLSO implementation (yes/no) was retained in the models, as were factors with a p < 0.10 in univariate analysis.
These factors were entered in their respective logistic regression model, and were included in the relevant tables, along with adjusted odds ratios (aOR) and 95% CI.

Bias
The participants include all those who participated to the ALSO and BLSO courses reducing selection bias. The qualitative data was analyzed by two of the local instructors who support the course and therefore are at risk of 'positive' bias however they are not responsible for the numerous volunteered responses from the participants. The quantitative data is limited to what data was available and at risk of not being unable to account for factors that can affect maternal mortality and stillbirth. For example 'socio-economic status' was not available and may have contributed to improvements although refugees and migrants have remained marginalized populations over the entire period.

Ethical approval
As part of standard procedure in this setting the details of this project were presented, discussed and approved by the local Community Advisory Board (TCAB-311-12,016), Mae Sot Thailand. No ethical approval was sought for the education evaluation data which was fully anonymized and previously collected. Oxford University Ethics Committee, ethical approval reference code OxTREC 28-09 was obtained for extraction of PPH related mortality and stillbirth data from anonymized hospital records.  Figure 2).

First
Local participant characteristics included ( Table 2) young age (median <30 years), female predominance 84.8% (476/561) with a majority with five or less years of practice. There was a high uptake to the courses by midwives (none were male), and a variety of workplaces (all non-tertiary) were involved. Less than four years high school was reported by one in four participants and university attendance (not completion) in less than one in five participants The main written and spoken language of participants confirmed that no one language was available to all participants ( Table 3).    Missing data: n=7 ALSO, n=3 BLSO Overall, 88.9% (8/9) doctors passed both practical and theory but all attended tertiary education.

Changes in perceived confidence
Complete data was available for 97% (n=345/355) of ALSO participants for pre-and immediate post-course paired testing of perceived confidence. For all items there was a strong and significant signal for an increase of median perceived confidence, with p<0.001 for all comparisons (median scores not shown). The proportion of participants who were terrified, scared, coping or comfortable were summarized and breech, vacuum, shoulder dystocia and neonatal resuscitation showed the greatest improvement in the proportion 'comfortable' from baseline ( Figure 3, Panel A).
Of 36 participants from the first ALSO cohort in 2008, 32 had a 12-month post-course evaluation: perceived confidence remained significantly higher than baseline for 8 of 13 situations; and was significantly lower for 5 of 13 situations at 12 months compared to the immediate post ALSO course evaluation (Table 4).
There were 89.3% (184/206) of participants on BLSO who completed pre-and immediate post-course evaluation. BLSO perceived confidence at baseline was low except on universal precautions; and the median post evaluation scores were significantly higher for all items (data not shown) p<0.001. The proportion of participants who were terrified, scared, coping or comfortable were summarized (Figure 3, Panel B). Despite improvements most were still 'scared' of normal delivery at the end of the course.
Thematic analysis course strengths Almost all (97.7%; n=347/355) ALSO participants responded in their language of choice to the survey "What are the strengths of the course?": Karen (21%), Burmese (16%) and English (63%). The most common themes to emerge from the responses were: "Improved knowledge", "I can do it" and "Teamwork".   A summary of the most common terms from all the comments on the strength of the course from all participants were collated in a word cloud (Figure 4). The proportion of PPH related mortality and stillbirths with associated factors were summarized (Table 5). After adjustment, birth before ALSO and BLSO was implemented (aOR 3.825, 95% CI 1.1233-11.870), having migrant (not refugee) status (aOR 3.814, 95% CI 1.241-11.718) and attending four (or less) consultations (aOR 3.648, 95% CI 1.189-11.191) were all associated with PPH related mortality. Seven of the eight factors including birth before ALSO and BLSO was implemented (aOR 1.235, 95% CI 1.018-1.500) were associated with increased odds of stillbirth (Table 5).

Discussion
This evaluation of ALSO and BLSO as education tools for health care workers mostly without tertiary qualifications in a post-conflict RLS demonstrates that they can achieve excellent standards for practical components of emergency care in obstetrics. The increased perceived confidence on most items, including at 12 months post baseline, and self-declared benefits from the training in knowledge, skills and teamwork, is consistent with former evaluations in other settings 6-8 . The risk of PPH-related maternal mortality and stillbirth declined following the implementation of ALSO and BLSO in this setting, suggesting a meaningful impact on the quality of care.
On the Thailand-Myanmar border, an area of protracted conflict resulting in disrupted education systems, the written theory test has proved to be challenging for health staff 29 . Approximately one in six participants attended (not necessarily completed) tertiary education in Myanmar though none studied health related subjects. Despite low overall pass marks on theory results 'increased knowledge' emerged as one the strongest themes from the qualitative analysis. However, unlike the ALSO course administration in high income countries, this increased knowledge required months of guided study. Provision of the syllabus without facilitation was impossible given the baseline education and diverse language abilities among the staff.
BLSO on the Thailand-Myanmar border has been used as a confidence builder and tool for raising awareness in maternal and child health for staff working mostly outside the delivery room. While BLSO significantly increased perceived confidence, the short nature of this course is no replacement for comprehensive midwifery and nursing training. This is perhaps best reflected by the participants' self-assessment, where 50% reported being 'scared' of normal vaginal delivery after the BLSO course.
The use of historic comparison, before and after ALSO and BLSO were implemented, is a weakness of this evaluation, as changes in the time frame may have occurred that impact on PPH related maternal mortality and stillbirth, and have not been accounted. Nevertheless, the factors associated with PPH maternal mortality highlight known problems in the area related to access to antenatal care 30-32 , and delivery before or after implementation of the courses remained a significant factor after adjustment for known confounders. Reduction of MMR over time is not smooth or inevitable in this setting. The PPH-related MMR (25.4 per 100,000 live births) on the Thailand Myanmar border after ALSO and BLSO were implemented was higher than the all cause official maternal mortality ratio in Thailand 33 . Changes were unrelated to availability of uteronics -syntocinon, methergyn, and misoprostol have been available since before 2000 and tranexamic acid was not introduced until 2017. While researchers could not retrospectively assess if the team-based approach to management of PPH taught in ALSO and BLSO training used for women experiencing PPH in SMRU delivery rooms after 2008, the qualitative and quantitative results presented here suggest that the methods learned were implemented.
The factors significantly associated with stillbirth including grand multiparity, smoking, underweight, four or less antenatal consultations, anaemia and falciparum malaria, all typical of RLS also stress the need for improved antenatal care and may in ALSO sets a standard. In protracted conflict and post-conflict settings where education as well as health are neglected reaching standard levels of practice are critical 29,36 . Standards are the main reasons for the minimal modifications of ALSO on the Thailand-Myanmar border, limited to only those situations that are extremely rare or skills requiring equipment that is unavailable. This long and positive evaluation with improved outcomes while ALSO and BLSO have been the only emergency obstetric courses on offer in SMRU clinics and where the level 4 data was reviewed, supports their benefit 37-39 and sustainability in RLS 40 .

Limitations
This is an evaluation of a single emergency obstetric course, although many exist, and the comparison for PPH related maternal mortality and stillbirth is historical which is acknowledged as imperfect 41 . Deeper analysis into the factors associated with each case of PPH related mortality e.g. timeliness of uterotonics; and stillbirth e.g. pre or intrapartum; would also be helpful to understand if ALSO skills could have prevented the adverse outcome 42 . ALSO relies on mannequins for the practical component and is a burden for most RLS to purchase or maintain but invaluable in visualization of emergency obstetric skills. A complete cost analysis has not been conducted but the cost of replacing mannequins far exceeds other facility-based course costs as instructors work on a voluntary basis 43 .

Conclusion
The best obstetric emergency course for RLS is unknown, however ALSO and BLSO are clearly feasible, effective and sustainable for adult education. Visualization, writing, reviewing and editing: All authors.

3.
Open Peer Review study to do so in an area of protracted conflict.
The model for studying the syllabus as a group over time is creative and effective. It may be a helpful model in other resource limited settings.
The analysis in the context of Kirkpatrick's framework is helpful. The level 4 data on the course's impact of maternal mortality ratio (MMR) and stillbirth rates is especially interesting. While the MMR decreased from 57.0 to 25.4 pre to post-ALSO/BLSO, the change is not statistically significant since the confidence intervals overlap (30.06-108.3) and (23.1-62.8), respectively. The decrease in stillbirth rate was statistically significant (16.2-20.4) vs (9.8-12.5).
As the authors acknowledge, the study design (pre-post analysis) has limitations. Changes noted may be due to factors other than the ALSO/BLSO course. A future study with a control group could better analyze the impact of ALSO/BLSO alone.
For the Odds Ratio, I believe fewer digits would be appropriate. Could report to 100 th (2 decimal places) rather than 10,000 th (4 decimal places).
Page 4, last paragraph: Effect of malaria interesting Great to have in Karen, Burmese and English.
Page 6, table 1: Normal labor taught in BLSO; not in ALSO -would leave ALSO column blank on the row containing Normal labor in BLSO column.
ALSO teaches "Intrapartum fetal surveillance" BLSO has basics in "Emergent/urgent prenatal Assessment" chapter with optional "electronic fetal monitoring" section for medical students in the "Normal labor" chapter.
BLSO also teaches Pre-eclampsia/Eclampsia BLSO chapter is titled "Breech, Twin Gestation, Cord Prolapse, and Other Malpresentations"; Covers these topics, just in a more basic manner than in ALSO. Prehospital personnel may need to attend emergent breech delivery. ALSO main focus is also on emergent (not planned) vaginal birth Neonatal resuscitation is not included in the 9 th edition of ALSO; Most providers in the US take the neonatal resuscitation program course instead; Neonatal resuscitation is still in the BLSO course ALSO starts its course with Safety in Maternity Care; BLSO does not have safety in maternity care chapter; it has a short section in normal labor chapter recommending universal precautions 9 th edition of ALSO has a Maternal Sepsis chapter but there is little material on Malaria. There is a full chapter on Malaria in Global ALSO Manual.
Page 6 "ALSO for the Thailand-Myanmar border" Adaptations are appropriate; the Global ALSO manual is available to help with adapting ALSO courses to local circumstances, especially in resource limited settings Nice creative model of reviewing ALSO chapters over time in preparation for in person course especially given language and other barriers in study setting.
Page 10 Love the comments and the word cloud! Page 10 last paragraph Would cut "(95% CI)" on the 4 th line of the paragraph and add "95%" before "CI 23. Methods including participants, who were eligible for the ALSO and BLSO courses, the structure of the courses were reported and the assessment of the educational effectiveness qualitative, quantitative then chart review. It is unclear in the methods page 6/16, the first paragraph under Table 1 the ALSO course was standard 2 day course but the mentioned the preparation phase reviewed 1-2 chapters per week (60-120) minutes. (Question: How many days/weeks took for the preparation of the chapters) the ALSO course is designed as a 2 day course. How many questions were used for the ALSO written examination? you mentioned BLSO using 10 questions. Results: Results were clear in regard to passing rates for both ALSO and BLSO. the clinical assessment hands on were high rate passing >90%, but the written exam was 50% passing for ALSO evaluation. the reported self assessment of the participants was showing clear confidence in 8/13 situations which seems to persist after 12 months. The comparison of the outcome related to PPH caused mortally pre-course and post-course were positive results. although I agree with the limitation mentioned here that they did not look at the reasons/ etiology/risk factors for PPH if preventable or not, the availability of uterotonic in a resource limited area. And I think another limitation included: researchers did not have the chance to study if the participants in these PPH scenarios had received the ALSO course training? Did they implement what they learned and used a team based approach?
The stillbirth result was reported well with a good conclusion. The tables numbering are incorrect:  How many questions were used for the ALSO written examination? you mentioned BLSO using 10 questions. We do 50 -there are some questions e.g. on electronic fetal surveillance that are not applicable in this setting. The sentence has been amended as follows: "The ALSO written exam was also modified for relevance and clarity, removing difficult English e.g. double negatives, and electronic fetal surveillance, leaving 50 questions".
Results: Results were clear in regard to passing rates for both ALSO and BLSO. the clinical assessment hands on were high rate passing >90%, but the written exam was 50% passing for ALSO evaluation. the reported self assessment of the participants was showing clear confidence in 8/13 situations which seems to persist after 12 months.
The comparison of the outcome related to PPH caused mortally pre-course and post-course were positive results. although I agree with the limitation mentioned here that they did not look at the reasons/ etiology/risk factors for PPH if preventable or not, the availability of uterotonic in a resource limited area. The uterotonics available in this area at the time of this evaluation have been added in the discussion section for clarity: Changes were unrelated to availability of uteronics -syntocinon, methergyn, and misoprostol have been available since before 2000 and tranexamic acid was not introduced until 2017.
And I think another limitation included: researchers did not have the chance to study if the participants in these PPH scenarios had received the ALSO course training? Did they implement what they learned and used a team based approach? This important point has been included in the discussion.
The stillbirth result was reported well with a good conclusion.

Thank you
The tables numbering are incorrect: Table 3 is mentioned twice  Page 12/16, table 3 needs to change to table 4-correct link to this table on page 8/16  Table 4 on page 13/16 needs to change to Table 5 and the link to this tale needs to be corrected on page 11/16. Thank you for pointing this out. It is our error and has now been amended.
Thank you for allowing me to review this review. I enjoyed reading it. Shahla Namak, MD Associate Professor, Family and community Medicine at Wake Forest School of Medicine.