The utility of an AMR dictionary as an educational tool to improve public understanding of antimicrobial resistance [version 1; peer review: awaiting peer review]

Background: Communicating about antimicrobial resistance (AMR) to the public is challenging. Methods: We developed a dictionary of terms commonly used to communicate about AMR. For each term, we developed learning points to explain AMR and related concepts in plain language.  We conducted a pilot evaluation in 374 high school students in Ubon Ratchathani, Thailand. In three 50-minute sessions, students were asked to answer five true/false questions using a paper-based questionnaire. The first session assessed their understanding of AMR at baseline, the second after searching the internet, and the third after the provision of the printed AMR dictionary and its web address. Results: We developed the AMR dictionary as a web-based application (www.amrdictionary.net). The Thai version of the AMR dictionary included 35 terms and associated learning points, seven figures displaying posters promoting AMR awareness in Thailand, and 66 recommended online videos. In the pretest, the proportion of correct responses to each question ranged from 10% to 57%; 10% of the students correctly answered that antibiotics cannot kill viruses and 57% correctly answered that unnecessary use of antibiotics makes them ineffective. After the internet searches, the proportions of correct answers increased, ranging from 62% to 89% (all p<0.001). After providing the AMR dictionary, the proportions of correct answers increased further, ranging from 79% to 89% for three questions (p<0.001), and did not change for one question (p=0.15). Correct responses as to whether taking antibiotics often has side-effects such as diarrhoea reduced from 85% to 74% (p<0.001). The dictionary was revised based on the findings and comments received. Conclusions: Understanding of AMR among Thai high school students is limited. The AMR dictionary can be a useful supportive tool to increase awareness and improve understanding of AMR. Our findings support the need to evaluate the effectiveness of communication tools in the real-world setting.


Introduction
Improving awareness and understanding of antimicrobial resistance (AMR) through effective communication, education, and training is the first strategic objective of the Global Action Plan on AMR (GAP-AMR) 1 . Although AMR is a serious threat to modern medicine, limited public understanding of AMR is one of the main factors that hinders actions from tackling AMR 2,3 . AMR occurs when microorganisms change in ways that render the drug used to treat the infections they cause ineffective. The review on AMR by the economist Jim O'Neill estimated that around 700,000 deaths each year globally might be due to AMR bacterial infections 2 . In 2015, the World Health Organization (WHO) conducted a global Survey on Public Awareness called "Antibiotic resistance: Multi-country public awareness survey" including 9,772 respondents in 12 countries, and found that the general public has limited awareness and understanding of AMR 4 . For example, 14% of respondents had never heard of any of the terms related to AMR (including antibiotic resistance, drug resistance, antibiotic-resistant bacteria, superbugs and AMR), and 64% of respondents incorrectly believed that colds and flu could be treated with antibiotics 4 . The GAP-AMR encouraged each country to develop their own national action plans on AMR in line with the global strategy 1 . One of the potential measures for the first strategic objective is the inclusion of the correct use of antimicrobial agents to prevent AMR in school curricula to promote better awareness and understanding of AMR from an early age 1 .
Thailand had launched the Antibiotic Smart Use programme focusing on changing prescribing practices made by healthcare providers in 2007 5 and Thailand National Strategic Plan (NSP-AMR) on AMR in 2017 6 . However, the Thai general public still have a limited understanding of AMR 7 . The problem could partly be attributed to the challenges of expressing technical concepts and the lack of AMR materials in Thai language 8 . In 2017, the national health and welfare survey in Thailand surveyed 27,762 Thai adults nationwide and found that 80% of respondents incorrectly believed that colds and flu can be treated with antibiotics 7 . An observational study conducted in 1,158 rural villagers in Thailand found that 96% of respondents were aware of antibiotics, but 88% referred to antibiotics as 'anti-inflammatory drugs' and the actual anti-inflammatory drugs are usually referred to by their brand name 8 . In addition, there are many other Thai terms commonly used to describe AMR and related concepts such as drug-resistant infections and stewardship 4,9 , and the public might not immediately recognize and understand those terms accurately 6 . Based on our experience engaging with the public in Thai during Antibiotic Awareness Week 10 in Thailand, common questions received from the participants included, but were not limited to, "What is antimicrobial resistance?", "What is AMR?", and "What is the difference between antibiotics, antimicrobials, and anti-inflammatory drugs?".
In order to mitigate the presence of many confusing terms associated with AMR in different languages, we developed an 'AMR dictionary' as a public educational tool in English that was translated to 11 languages. Here, we report on the development of the Thai version of the AMR dictionary and the evaluation of its utility among high school students.

Development of the AMR dictionary
The AMR dictionary was first developed in English based on the terms commonly associated with AMR. Using the concept of learners' dictionaries 11,12 , we selected a list of AMRrelated terms, provided simple definitions for them, and showed examples of their usage. We selected terms used by the mass media, international organizations and researchers. In addition, we developed a learning point for each term to raise understanding of AMR. We aimed at lay readers to able to access to the useful information as possible concerning AMR. Where feasible, we tried to include and refer to open-access contents, posters, and videos from international organizations such as the World Health Organisation (WHO), the Food and Agriculture Organization of the United Nations (FAO), the World Organization for Animal Health (OIE), Public Health England, the European Centre for Disease Prevention and Control (ECDC), and the U.S. Centers for Disease Control and Prevention (CDC).
With contributors, we aimed to be inclusive rather than exclusive. We initially invited colleagues at the Mahidol-Oxford Tropical Medicine Research Unit (MORU) who are native English speakers to contribute to the content of the dictionary. Contributors did not have to be scientists or work on AMR. Within MORU, we regularly informed staff about the AMR dictionary in scientific meetings and public engagement activities about AMR. Subsequently, we invited contributions from people and institutions outside of MORU, including academic and non-academic institutions, e.g. Greenpeace, the Microbiology Society, and Health Security Partners. We acknowledged everyone who provided significant help with the revision and editing of the AMR dictionary. When permitted, we placed logos of institutions that supported the use of the AMR dictionary on the cover of the book. The AMR dictionary was released as a web-based application at https:// www.amrdictionary.net, and all content are freely downloadable as a single PDF file under a Creative Commons Attribution 4.0 International license (CC BY 4.0). The AMR dictionary is regularly revised to ensure that the content is easy-to-read, up-to-date, and accurate.
The AMR dictionary was then translated into Thai (and other languages) by contributors listed in the acknowledgement section of the dictionary, the terms and learning points were revised based on the Thai context (Extended data, AMR dictionary. Thai, version 1.1 13 ). We also collaborated with local organizations such as Drug System Monitoring and Development Center (DMDC), Greenpeace Thailand and Food and Drug Administration, Ministry of Public Health, Thailand for their input. We aimed to be consistent with on-going public engagement activities, the NSP-AMR 6 , and new AMR campaigns being delivered by policymakers in Thailand 14 . The Thai version of the AMR dictionary was released as a web-based application on the same platform at https://www.amrdictionary.net. The latest Thai version of the AMR dictionary was also printed as a hard copy, and used as a supporting educational tool during Antibiotic Awareness Week (AAW) in Thailand in 2018 and 2019.

Study cohort
The study cohort is from the Ubon Ratchathani province in Thailand. Ubon Ratchathani covers an area of 16,113 km 2 with a population of 1.8 million. It is one of the largest provinces in Northeast Thailand. Where the majority of population of northeast Thailand live in the area, where most works in the agriculture. It was also selected because of our long collaboration with local hospitals and administrative organizations. Of the total of 374 high school students tested, 280 (75%) were female. The median age was 17 years (range from 16 to 18 years). A total of 308 students (82%) majoring in science, while the remaining 66 (18%) were majoring in arts. A total of 155 students were from three high schools located at the Muang district (central district, and the remaining 219 students (60%) were from two other high schools located at Trakan Phuet Phon and Phibun Mangsahan districts in the province ( Figure 2). The study was approved by the Ethics Committee of the Faculty of Tropical Medicine, Mahidol University, Thailand (TMEC 19-055). Written informed consent was obtained from all participants and their parents before the study.
Pilot evaluation of the AMR dictionary A pilot evaluation of the Thai version of the AMR dictionary was conducted in 374 high school students in grade 11 and 12 in Ubon Ratchathani Province, Thailand in August 2019. Students from grades 11 and 12 were chosen as they were one of the target groups recommended by the GAP-AMR 1 . They were also familiar with internet searches, and could provide comments and feedback on the AMR dictionary compared with their searches of the internet. The students and parents were not informed that the topic for discussion was AMR or antibiotics; instead, they were told it was about infectious diseases. Our reason for doing this was to reduce the possibility that students would read or search for AMR or antibiotics before the study. Students who provided informed consent, together with their parental consent, participated in a 50-minute session. There were about 40-50 students from the same classroom per session. The study was conducted in a computer room or a room with WiFi signal at the participating schools.
The three-stage design was selected based on the simplicity for the pilot evaluation. The first step was to assess the pre-existing knowledge and awareness of AMR in high school students. The second step was to evaluate their understanding of AMR after searching the internet by themselves. The final step was to evaluate the utility of the AMR dictionary and whether it could improve understanding of AMR.
First, the group of students was asked to answer a paperbased questionnaire containing five true/false questions in Thai (Extended data, Questionnaire 15 ). The questions were derived from the national health and welfare survey in 2017 7 . The questions were: (1) "Antibiotics (often called anti-inflammatory drugs) can kill viruses" (the correct answer is false), (2) "Antibiotics (often called anti-inflammatory drugs) are effective against colds and flu" (the correct answer is false), (3) "Unnecessary use of antibiotics (often called antiinflammatory drugs) can make them become ineffective" (the correct answer is true) (4) "Taking antibiotics (often called anti-inflammatory drugs) can cause side-effects such as diarrhoea" (the correct answer is true), and (5) "Antibiotics (often called anti-inflammatory drugs) are used to treat inflammation. Their direct effects are to relieve pain and swelling." (the correct answer is false). The questionnaires were then collected from students.
Second, the same group of students was given 15 minutes internet access to answer the same set of questions. The searches could be done via computers in the computer room or their mobile phones. Students were asked not to talk or discuss the results of their searches with other students. The students were asked to provide web addresses used for answering each question, and a hard copy of their responses were collected.
Third, a printed hard-copy of the AMR dictionary (Thai version 1.1 updated in July 2019), and the written web address of the AMR dictionary, were provided to every student. Students were allocated another 15 minutes for reading or searching the AMR dictionary individually and answer the same set of questions. We determined their preference between internet searches or the AMR dictionary with five statements. The statements were: (1) information is easy to find, (2) information is easy to understand, (3) information is enough, (4) information is reliable, and (5) other points (please define). We also provided another two free text opinion boxes, including "I think that the AMR dictionary is …" and "I want the AMR dictionary to be improved by …"

Statistical analysis
Statistical analyses were performed using Stata version 14.2 (StataCorp LP, College Station, TX). Comparisons of categorical and continuous variables between groups were performed by using the Chi-square test and Kruskal-Wallis test, respectively. The McNemar Exact test was used to compare the proportion of correct answers between (a) baseline, (b) after their internet searches, and (c) after they were provided with the AMR dictionary. The sample size of the pilot study was estimated based on the previous national health and welfare survey among adults in Thailand 7 . We hypothesized that 30% of students would answer the questions correctly on the pretest. We assumed that the AMR dictionary could increase the proportion of correct answers by 20%. With 90% power and an alpha rate error of 0.01, we estimated that the required sample size was 360 students.

Results
The AMR dictionary We developed the AMR dictionary as a web-based application (www.amrdictionary.net) that can also be downloaded in PDF format. The web-based application is also compatible with mobile phones and handheld devices (Figure 1). The English version of the AMR dictionary (Extended data, AMR dictionary. English, version 1.2 16 ) included 35 terms and learning points, seven figures displaying posters promoting AMR awareness, and 54 recommended videos in English. We added the terms related to AMR (i.e. antibiotic resistance, drug resistance, antibiotic-resistant bacteria, superbugs, antimicrobial resistance [AMR] and drug-resistant infections [DRI]), microbes (i.e. bacteria and viruses), antibiotic use in humans (i.e. antibiotic overuse and over-the-counter), antibiotic use in agriculture (i.e. critically important antibiotics [CIA] and raised without antibiotics [RWA]), and other related terms (i.e. vaccine and sepsis). Examples of learning points include "Is AMR a global concern?" (for the term antimicrobial resistance) and "When antibiotics don't work anymore" (for the term antibiotic). As of July 2020, the www.amrdictionary.net was available in 11 languages; including Arabic, Chinese, English, French, Khmer, Laotian, Malayalam, Myanmar, Tamil, Thai and Vietnamese ( Figure 1A).
In the Thai version (Extended data, AMR dictionary. Thai, version 1.1 13 ), the dictionary included 35 terms, 35 learning points, seven figures (displaying posters promoting AMR awareness in Thailand), and 66 recommended online videos in Thai. The content in the Thai version was based on on-going public engagement activities, national action plans, research findings, and local context relevant to Thailand. The content was not translated word-by-word from the English version. For example, we repeatedly stated that antibiotics are commonly and inaccurately referred to as anti-inflammatory drugs in Thai throughout the Thai version of the AMR dictionary. We also rewrote the learning point "When antibiotics don't work anymore" to "Antibiotics are not anti-inflammatory drugs" (for the term antibiotic), and provided important information about that point. Two thousand copies of Thai version 1.0 were printed and used as part of the information package for Thailand AAW 2018; and 1,000 copies of version 1.1 were printed for Thailand AAW 2019.
Pilot evaluation of the AMR dictionary A total of 374 students were included in the pilot evaluation. In the pretest, understanding of AMR varied by topic ranging from 10% of the students answering correctly that antibiotics cannot kill viruses to 57% of the students answering correctly that unnecessary use of antibiotics makes them ineffective (Figure 3). A total of 79 (21%), 151 (40%), 101 (27%), 34 (9%), 9 (2%) and 0 (0%) gave correct answers to zero, one, two, three, four and five questions, respectively. There was no clear difference in the total number of correct answers by gender, grade, school, and science or art major 17 .   After allowing internet searches for 15 minutes and repeating the questionnaire, we found that the proportion of students answering each question correctly increased to a range from 62% to 89% (all p<0.001; Figure 3). While 62% of the students answered correctly that antibiotics cannot kill viruses, more than 85% answered correctly that taking antibiotics often has side-effects such as diarrhoea (85%) and that unnecessary use of antibiotics makes them become ineffective (89%) (Figure 3). A total of 4 (1%), 19 (5%), 57 (15%), 61 (16%), 111 (30%) and 122 (33%) gave correct answers to zero, one, two, three, four and five questions, respectively. The most common websites consulted by the students are in Thai. A total of 67 students (18%) found the AMR dictionary (www.amrdictionary.net) during their internet searches.
After providing the hard copy and web address of the AMR dictionary (version 1.1) and repeating the questionnaire after another 15 minutes, we found that proportions of students answering each question correctly increased further ranging from 79% to 89% for three questions (p<0.001; Figure 3). The proportion of students answering correctly that unnecessary use of antibiotics makes them become ineffective did not change significantly (from 89% to 92%, p=0.15). The proportion of students answering correctly that taking antibiotics often has side-effects such as diarrhoea reduced from 85% to 74% (p=<0.001). A total of 1 (<1%), 9 (3%), 20 (5%), 36 (10%), 119 (32%) and 189 (51%) students gave correct answers to zero, one, two, three, four and five questions, respectively.
Overall, students preferred the AMR dictionary to searching the internet on their own (Figure 4). More than two-thirds of the participants preferred the AMR dictionary because of the information being reliable (78%), enough (71%), and easy to understand (69%). Although 51% of the participants answered that the information was easier to find by using the AMR dictionary, 27% answered that performing internet searches were easier (Figure 4). There were no missing data in the multiple-choice questions. Of 374 students, 189 (51%) stated what they want the AMR to be improved in a free-text question. The most common comments obtained from the students were that the AMR dictionary should be printed in colour rather than in black-and-white (50%, 95/189), should be made easier to understand (8%; 16/189) and should have more information (7%; 14/189). The content of the Thai version of the AMR dictionary was revised based on the findings and comments received. Because of the low proportion of correct responses on the question "taking antibiotics often has side-effects such as diarrhoea", we revised the content related to the topic and added two new figures displaying posters promoting understanding of side-effect of antibiotics into the Thai version of the AMR dictionary (version 1.2) available at https://www.amrdictionary.net/.

Discussion
We have described the development and utility of an educational support tool to educate the general public on antimicrobial resistance, the AMR dictionary (www.amrdictionary.net), written in local languages and adapted to the local context of multiple countries.
During the pilot evaluation, only 2% of the students answered at least four of the five questions correctly, suggesting that baseline understanding of AMR in high school students in Thailand was limited. With access to internet, the percentage of participants correctly answering at least four questions rose to 63%. This finding suggests that there are a number of information sources about AMR available on the internet in the Thai language that students can access easily. We show the utility of the AMR dictionary by observing that the proportion of students answering correctly for at least four of the five questions rose further from 63% to 83% after providing the AMR dictionary. The questionnaires used were derived from the 2017 national questionnaires, and not streamlined to be answerable by the AMR dictionary. The students preferred the AMR dictionary over the internet searches for information. A significant proportion of students (26%) failed to recognize that taking antibiotics often has side-effects such as diarrhoea, even after being given the AMR dictionary. The lack of antibiotic literacy supports the need to evaluate all communication tools developed for AMR for clarity before their implementation 3 . Based on our experience, previous AMR engagement efforts in Thailand did not routinely evaluate whether the terms and explanations used were understandable by the target audience. We found that developing explanations of terms surrounding the topic of AMR in different languages needed to be adjusted based on the local context and through a wide range of partners. We also found that the AMR dictionary could complement other local AMR engagement efforts, such as press releases made by doctors or researchers and events to raise awareness of AMR in the public. This is because technical terms (such as AMR, antibiotics and antimicrobials), which are regularly used by doctors and researchers in different languages, are difficult for the public to understand.
The limited understanding of AMR among high school students in Thailand is of concern, and policy makers who are implementing national action plans on AMR need to enhance school curricula on the topic of AMR. The low level of understanding of AMR observed is consistent with the last global study conducted by the WHO 4 and the previous national survey in Thai adults 7 . This problem is worrying because there has been no improvement observed, even though the national action plan on AMR has been implemented in Thailand from 2017 6 . This low level of understanding of AMR finding is probably common in many countries worldwide in which the national curriculum does not cover the topic of AMR 18 , policy makers should call on educators to improve their national curriculum and carefully evaluate the level of awareness and understanding of AMR among students as guided by the global action plan on AMR 1 . This study has a number of strengths. First, the AMR dictionary incorporates readily available contents, campaign materials, and online videos from properly accredited sources rather than reinventing the wheel. This approach also improves the credibility of the content presented by the AMR dictionary, particularly by referring to the campaign materials from AAW, which are coordinated by WHO and organized by each country's public health authorities 19 . Second, the AMR dictionary was produced in collaboration with national public health authorities, to ensure that the content in the local language was in line with the national action plan on AMR, and was consistent with the way in which national authorities wanted to convey information to the public. This engagement with key stakeholders helped gain support for this pilot implementation and evaluation. Third, the AMR dictionary includes translations of difficult-to-understand terms in different languages, and this can support students, educators, campaigners, and users to cross check their understanding of each term in each language. The translation and educational functions of AMR dictionary are similar to those of e-BUG, an education resource for schools covering microbes, hygiene, antibiotics and prevention of infection, which has been translated into all European Union languages 20 . Fourth, this study provided an opportunity to evaluate the AMR dictionary, and to improve it to be clearer about possible side-effects of antibiotic use based on evaluation findings. This is supported by the recent work of the Wellcome Trust 3 , showing that it is important to test the message, frame AMR as undermining modern medicine, explain the fundamentals succinctly, and emphasize that it affects everyone. This would also complement efforts to implement the NAP, as the AMR dictionary is a web-based educational tool that could raise the awareness of the lay audience and encourage them to comply with the policy makers and healthcare providers to prevent misuse of antibiotics.
There are several limitations to this study. First, the AMR dictionary was developed as an advance learners' dictionary; therefore, its utility as a stand-alone educational tool is limited. Nonetheless, it has been found useful during AAW in Thailand, and it may be used as an educational tool in the countries for which translation in local languages are available in the AMR dictionary. Second, the sample size and study area of the pilot evaluation was small. Therefore, the generalizability of our findings is limited. Further studies to evaluate awareness and understanding of AMR in students, and to evaluate the utility of the AMR dictionary in different settings are recommended. Third, the utility of the AMR dictionary observed could be affected by the internet searches. Different study designs, such as a crossover study or a randomized controlled trial, could be used. Fourth, the content of the AMR dictionary may contain errors, and it can be revised and improved over time. The development team is open to comments and suggestions, and new collaboration to expand the translation of the AMR dictionary in as many languages as possible. The most up-to-date version will always be made available on the website (www.amrdictionary.net).

Conclusions
Awareness and understanding of AMR among high school students in Thailand is low, and this is likely to be mirrored in all low-to-middle-income countries (LMICs). The AMR dictionary, adapted to the local context and language, can be a useful tool to support measures to improve awareness and understanding of AMR among the general public, complementing countries' national policy efforts. We also strongly support the need to evaluate communication or educational tools developed for AMR in local settings to improve them prior to using them widely. Extended data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).