This is an analysis of government plans from the 2016 and 2021 Peruvian presidential elections. For this analysis we used NLP algorithms: Term Frequency – Inverse Document Frequency (TF-IDF) ¹⁶ , Latent Dirichlet Allocation (LDA) ¹⁷ , TextRank ¹⁸ , Keywords Bidirectional Encoder Representations from Transformers (KeyBERT) ¹⁹ and Rapid Automatic Keywords Extraction (Rake) ²⁰ .

In this analysis, we focused on the government plans regardless of the political party. That is, we do not make specific references or conclusions about the political parties; however, the names of the parties included in the study are presented in the tables and figures. We do not make specific arguments about the political parties to avoid introducing political bias or preferences. We aimed to make this a data-drive analysis to dissect the content of the government plans, and not a quality assessment of the government plans (or the political parties behind it).

According to the World Bank, Peru is an upper-middle income country in South America with a total population of 32.5 million people ²¹ . Peru is a unitary presidential republic, in which the president is elected in general elections every five years; the most recent presidential elections took place in 2016, and the next one will take place in April 2021. The campaign for the forthcoming presidential elections started in December 2020 with over 15 candidates (i.e., political parties). Each candidate must publish a government plan.

In this study, we analysed the government plans of the 2016 and 2021 presidential elections in Peru. These plans are open access and in the public domain, so that all citizens can read these and become informed of their proposals ^{22, 23} . For this work, which is health-oriented, we only selected the health chapters.

For the 2016 presidential elections, there were 19 government plans presented, although we analysed only 18 in this study as one was only available as a scanned image and could not be analysed as text. For the 2021 presidential elections, there were 20 government plans and we analysed just 19 as one was only available as a scanned image and could not be analysed as text. As we aimed to analyse the government plans, and not the political party, we analysed all government plans regardless of whether the party or the candidate was disqualified during the campaign period.

Within the health chapter of each government plan, we copied and pasted onto a spreadsheet each sentence in a row; in other words, for each government plan, we had as many rows as sentences. Furthermore, the dataset we generated had three columns: the name of the political party, a sentence indicator (e.g., political_party_A_sentence_1), and the sentences we extracted. All sentences were copied as they were in the original government plan. The government plans were translated into English using Google translate.

Data preparation. NLP is a branch of artificial intelligence which aims to process and understand the human language. Not only does NLP need to understand the text (words), but it also needs to make sense of the context to provide accurate meaning to the text. To begin with NLP, we need to put a text into a two-dimensional matrix. In the model Bag of Words ²⁴ , each text or document is represented by the group of words in it.

As a previous step to the construction of the data matrix, a vocabulary is elaborated with the union of all the terms that appear in a document. From this, a matrix is built in which each row represents a document, and each column (each characteristic) corresponds to a term.

For this analysis we conducted a simple pre-processing, which included deleting HTML labels and non alphabetic characters for all datasets. Moreover, an advanced preprocessing of the datasets depends on the algorithm used. Therefore, in the description of the algorithms, the data pre-processing that has been used will be described for each algorithm.

Algorithms

We used five NLP algorithms: TF-IDF, LDA, TextRank, KeyBERT, and Rake. We chose these algorithms because they are frequently used in the NLP literature, they are available through most libraries in Python and R facilitating the replication and outreach of our work, and they satisfied our goals (i.e., to describe the health chapters in government plans by summarising words and terms).

TF-IDF and TextRank reveals keywords; TF-IDF does so based on the relative frequency of each term whereas TextRank selects keywords based on a trained model. Compared to TF-IDF and TextRank, KeyBERT as well as Rake reveal keywords or short phrases (e.g., 3-5 words) based on trained models. LDA, on the other hand, groups texts into unknown clusters and the user labels such clusters (i.e., as in unsupervised machine learning with K-means). Altogether, these algorithms provide a framework to describe the health chapters in government plans going from unique terms (TF-IDF and TextRank), to short phrases (KeyBERT and Rake), and grouping into unknown clusters (LDA).

TF-IDF: This is the most basic NLP algorithm. Each position ( d, t) in the matrix has a value for the metric Term Frequency which is denoted as t f _{d , t} , and reflects the number of times the term t appears in the document d. There are variants to the Term Frequency metric, so that the matrix can include:

There are terms which appear several times in a document, but which not carry much information; these could include ’the’, ’a’, ’an’, and ’of’, among others. The metric Inverse Document Frequency penalises the terms in a magnitude equal to the frequency in which they appear in the text. The inverse frequency of a term is computed as:

i d f t = l o g ( # d o c u m e n t s # d o c u m e n t s t h a t i n c l u d e w o r d t )

Because both metrics ( Term Frequency and Inverse Document Frequency) provide information to the NLP algorithm to dissect the text, it is common practice to combine them as the TF-IDF metric, which can be expressed as:

t f − i d f d , t = t f d , t × i d f t

For TF-IDF, we used the dataset partitioned by phrases to analyse globally because a wide corpus of words is needed, so here the frequency of words in general is analysed (for all political parties).

Hence, we used the TfidfVectorizer class from the Scikit-Learn library with the parameter stop_words=’english’ ²⁵ ; this parameter reads all the text from each government plan and removes non-relevant terms from a text such as ’the’ and ’as’, among others. Moreover, lemmatisation was used with the class WordNetLemmatizer from nltk library ²⁶ . The following filler words were excluded: execute, force and change; although these are not traditional filler words, in the political context they are filler words.

LDA: LDA builds a generative model from a set of observations that belong to unobserved groups. Basically, it considers each group as a probability distribution over the characteristics, and each observation as generated from a mixture of the groups’ probability distributions.

This model is frequently used to categorise a document into subjects or topics. For each topic, this model estimates the probability of the terms, while for the document, this model estimates the relevance of each topic.

Therefore we used LDA to define and describe, through the most relevant terms, the categories in which each data entry can be assigned to. We trained the LDA model with a dataset in which each sentence was an instance, and the categorisation was then conducted for the government plan.

The input for training the LDA algorithm was all the government plans (separately for each year), having removed the stop words and lemmatized the terms. Then, with the cleaned (lemmatized and without stop words) text for each political party, the model assigns each party to a group defined during the LDA training. We used LatentDirichletAllocation class from the Scikit-Learn library ²⁷ with the parameter n_components=2; this parameter reads the number of topics to be created. For this work, we defined two groups (i.e., we specified that the LDA algorithm should create two groups following an unsupervised approach from the data). Two is the absolute minimum number of groups and we chose such option this being an exploratory work compounded with other NLP algorithms, and we did not focus solely on the LDA results. For a discussion on this limitation, please refer to the Strengths and Limitations section. Finally, for the data pre-processing, we used stop_words and WordNetLemmatizer classes from nltk library ²⁶ .

TextRank: This algorithm is a graph-based algorithm and it is based on PageRank ¹⁸ developed by Google, where PageRank is used to define the position of websites in the search engine through the extraction of key terms.

TextRank informs about the semantic sequence used in the text with unique keywords. We used the library Gensim ²⁸ .

For this analysis, we used the full government plans to extract the key words in each plan. Following the recommendations in the documentation for TextRank, we did not pre-proccess the data and thus used the original text as it was.

KeyBERT: KeyBERT ^{13, 29} is an adapted keyword extraction tool based on BERT ¹⁹ . In the last couple of years, algorithms based on BERT are pushing the boundaries of the state of the art technology in social and technical multiple aspects of the natural language processing tasks ³⁰ ?]. BERT is a pre-trained model on the Book-Corpus and English Wikipedia datasets ³¹ . It uses a multi-head attention mechanism, so it can create better representations for entities considering the context, and leveraging the classic word-based approach.

For this analysis we used the full government plans to extract the keywords in each one. Also following the documentation for the KeyBERT algorithm, the texts were not pre-processed and used as they were. A limitation of this algorithm is that it retrieves a maximum of six keywords; though this did now affect our work because we sought five keywords as it was the case for other algorithms.

Rake: Rake ¹⁵ is used to extract keywords and key phrases. To use the Rake algorithm, we followed these steps:

For this analysis we used the full government plans to extract the phrases in each one. Also following the documentation for the Rake algorithm, the texts were not pre-processed and used as they were. We used the class Rake from the rake_nltk library; for further details including the equation for the Rake method please refer to reference 15.

Ethics. The underlying data for this study is accessible in the public domain and did not include the names of individuals, but the name of the political parties which created each government plan; this information is also in the public domain. Therefore, we considered this work of minimal risk and did not seek approval by an ethics committee or institutional review board.

The original dataset with the 2016 government plans had 559 rows (i.e., sentences) in total, which represented 18 government plans; the shortest government plan contributed with four sentences, while the longest contributed with 96 sentences (See Table 1). The original dataset with the 2021 government plans had 1,586 rows (i.e., sentences) in total, which represented 19 government plans; the shortest government plan had 10 sentences, while the longest had 215 sentences (See Table 1).

The TF-IDF analysis showed differences between the plans in 2016 and 2021. When we set a threshold of 0.08 to select terms (i.e., terms that represented 8 per cent or more of all the words in the text), 26 terms met this criterion in 2016 and 27 terms met this criterion in 2021. These terms are shown in Figure 1 for the years 2016 and 2021.

In 2016, across all government plans, the term ’care’ had a TF-IDF frequency close to 0.40, while the terms ‘system, ‘service’, ‘level’, ‘public’, ‘population’ and ‘national’ had a frequency above 0.15 (see Figure 1(a)). In 2021, across all government plans, the term ’care’ had a TD-IDF frequency close to 0.35; the terms ‘system’, ‘public’ and ‘level’ had a frequency above 0.20 Figure 1(b).

The LDA algorithm was trained in a dataset with as many rows as sentences per government plan, to define two groups of ten words each (see Table 2); we did not define more groups, or more terms per group, because the dataset was small. For the prediction phase, we used the full text of each government plan (i.e., a dataset with as many rows as sentences per government plans), and analysed which group each government plan would belong to each group. We did this analysis for the years 2016 and 2021 separately.

Overall, in both 2016 and 2021, Group 0 appeared to cluster terms signaling things the population would receive (see Table 2). For example in 2016 and in 2021, Group 0 included ’service’, ’access’ and ’insurance’. Conversely, in both 2016 and 2021, Group 1 appeared to cluster terms that are related with the structure of the health system like ’care’, ’hospital’, and ’level’ (see Table 2).

After generating the two groups (Group 0 and Group 1), we then answered: to which group would each government plan belong?

For the year 2016, we observed that the government plans likely to belong to Group 0, were much less likely to belong to Group 1 (see Figure 2(a), Figure 2(b)). There were four government plans with a probability between 0.40 and 0.60 of belonging to either Group 0 or Group 1. There was strong evidence suggesting that seven (out of eighteen) government plans would belong, almost exclusively, to Group 1 (see Figure 2(a), Figure 2(b)).

The distinction in favour of Group 0 was less clear when analysing the 2021 government plans (see Figure 3(a), Figure 3(b)), when we observed one government plan very likely to belong to Group 0 (in 2016 there were four). Conversely, there were nine government plans with high probability of belonging to Group 1, yet very low probability of belonging to Group 0.

Figure 2 shows the LDA results for the government plans in 2016 and Figure 3 for the plans in 2021. While these figures show that political parties are more inclined to either Group 0 or Group 1, a direct comparison between 2016 and 2021 is troublesome because: i) the same exact political parties did not participate in the two elections; ii) the composition of the political parties in terms of roles, interests and priorities may have changed between 2016 and 2021; and iii) amidst the COVID-19 pandemic, the focus of the health-related government plans would be different between 2016 and 2021. Perhaps in response to the COVID-19 pandemic, the difference observed between Figure 2B and Figure 3B could signal that the 2021 plans focused more on terms related to the structure of the health system (i.e., Group 1 appeared to cluster terms that are related with the structure of the health system).

The TextRank algorithm shows the keywords in the text. The number of keywords depends on the size and coherence of the text; that is, longer texts and those with more complexity would have more keywords than small texts with poor context. For an informative representation, we chose the top six keywords per government plan. Keyword could have between one and three words.

In both 2016 (see Figure 4, Figure 5 and Figure 6) and 2021 (see Figure 7, Figure 8 and Figure 9), the term ’health’ was the most frequent keyword. In 2016, terms regarding ’universal health coverage’ were also frequent; in 2021 however, terms about ’universal health coverage’ were not present. In 2016, the terms were more general, and appeared to focus on improving the health system with words like ’hospital’, ’population’, ’public’ or ’region’.

In 2021, we found words relevant in the context of the COVID-19 pandemic, like ’national emergency’. We also found words that reflected the characteristics of the health system at the beginning of the COVID-19 pandemic, for example ’low investment’ and ’lacking’.

Bear in mind that although TextRank provides the most frequent keywords, these words alone do not provide insights about the context of the text. Therefore, it becomes relevant to study the context of the text, so we also used the KeyBERT (group of terms) and Rake (to study phrases) algorithms.

TextRank ¹⁸ and KeyBERT ³¹ provides keywords (between one and three words), but the latter uses a variation of the BERT algorithm to chose keywords based on the context of the text. Moreover, KeyBERT shows the main keywords with their score. For this algorithm, the five main keywords have been extracted using the same government plans, i.e., 2016 (see Figure 10, Figure 11 and Figure 12) and 2021 (see Figure 13, Figure 14 and Figure 15).

Consistent with what we observed using the TextRank algorithm, the 2016 (see Figure 10, Figure 11 and Figure 12) keywords appeared to be more general. The terms talked about, for example, ’health solidarity’, ’health crisis’, and ’health inefficient’. However, the KeyBERT algorithm did find other keywords which revealed more concise concepts: ’campaign vaccination’, ’equipped hospital’ and ’health education’. For one government plan (see Figure 12(c)), all keywords were about diseases: ’dyslipidaemias diabetes’, ’diabetes obesity’, ’mortality morbidity’, ’anemia vulnerable’, and ’malaria dengue’. Furthermore, the keywords retrieved with the KeyBERT algorithm also revealed underlying characteristics of the political party. For example (see Figure, 11(e)), the keywords of a left-wing political party were ’Peruvian richest’, ’subsidise poorest’ and ’discriminatory medical’.

As also pointed out with the TextRank algorithm, in 2021 (see Figure 13, Figure 14 and Figure 15) the keywords appeared to be more concrete (e.g., ’health networks’, ’Peruvians insured’ and ’hospitals needed’) and also covered topics related with the COVID-19 pandemic (e.g., ’health pandemic’, ’management pandemic’ and ’eaths pandemic’). In one government plan (see Figure 14(b)), four out of five keywords were about vaccination: ’vaccines reforming’, ’make vaccines’, ’vaccines Peru’, and ’Peru vaccination’. Similarly, other government plan had four keywords about diseases: ’malnutrition tuberculosis’, ’childhood malnutrition’, ’anemia chronic’ and ’morbidity mortality’.

For visualisation purposes, we chose the top five key phrases per government plan in 2016 (see Table 3) and 2021 (see Table 4).

Therefore, in 2016, a topic that was recurrent in four government plans was ’universal health coverage’. When the phrases addressed specific diseases/conditions, these were malnutrition, mental health, communicable diseases, and sexually transmitted diseases (see Table 3).

On the other hand, in 2021, ’universal health coverage’ was also a frequent topic found in phrases of four government plans. Mental health also appeared often (in three government plans). There were also phrases related to COVID-19, and the fact that the health system in Peru is fragmented (see Table 4).

We used standard and well-known NLP algorithms to study health chapters of the government plans of the political parties in the 2016 (18 plans) and 2021 (19 plans) general elections in Peru. We did not aim to develop new NLP algorithms nor to modify the available algorithms, rather, we aimed to apply available NLP algorithms in a novel topic: health policy/politics.

The TF-IDF algorithm revealed a similar number of terms between 2016 and 2021. In both years, the term ‘health’ had the highest frequency. In both years, all the other terms were not specific to a disease, a program or policy, or a new proposal. An analysis of frequency of terms may not be very informative, thus an in-depth analysis with other algorithms may be needed.

The LDA analysis defined two groups: one gathering words signalling things the population would receive (e.g., ’insurance’), and the other with terms about the health system (e.g., ’capacity’). The LDA analysis also assigned government plans to either group. This suggested whether a government plan would focus on delivering goods or services to the population, whereas others would focus on improving the health system.

The TextRank revealed some key words. Interestingly, the keyword phrase ’universal health coverage’ were frequent in 2016, but this did not appear in 2021, when keywords about the COVID-19 pandemic (e.g., ’national emergency’) and the limited capacities of the health system (e.g., ’low investment’) were frequent. This provides preliminary evidence on how the main focus of the government plans shifted between 2016 and 2021.

The KeyBERT analysis revealed keywords, but these were learnt based on the context of the text. Interestingly, these keywords provided more insight about the underlying profile of the political party. This suggests that it would be possible to identify characteristics of the writer (in this case of the political party) based on the text and its content.

The Rake analysis provided key phrases, instead of keywords, and the results provided more insight about the documents. For example, this algorithm revealed that phrases ’universal health coverage’ appeared both in 2016 and 2021. Similarly, phrases with “mental health” were found in both years. Phrases about COVID-19 were only found in 2021.

The first finding was that the dataset with the 2021 government plans was larger than the 2016 dataset. Similarly, the TF-IDF analysis also showed that the number of terms with a frequency of ≥0.10 was 4.8-fold higher in 2021 than in 2016. This could imply that health has gained more relevance over the last five years, particularly since 2020 when the COVID-19 pandemic revealed the limitations and deficiencies of the health systems in Peru.

The TF-IDF analysis showed that most words in 2016 were also found in 2021. This may suggest that the same subjects in 2016 are still valid issues or concerns in 2021. Two words in 2016 that were not repeated in 2021 were ‘private’ and ‘universal’. Probably, issues regarding access to private medical care, or lack of universal health coverage, were less concerning in 2021. On the other hand, words that were in 2021 but not in 2016 included ‘COVID’ and ‘capacity’. The former is self-explanatory, because ‘COVID’ was not a concern in 2016 at all. The term ‘capacity’ could represent the lack of resources, hospitalization beds (both overall and in intensive care units) and oxygen that Peru suffered during the 2021 COVID-19 waves. This subject –lack of capacity– may have driven the health policy agenda, thus gaining relevance in the 2021 TF-IDF analysis.

The LDA analysis provided two groups, and also showed to which group each government plan is most similar. Our hypothesis is that Group 0 included terms or things the population would receive or directly interact with, and Group 1 included terms or things that the healthcare provider would deliver. In 2016, the LDA analysis suggested that most of the government plans would belong to Group 1; this could imply that the 2016 government plans focused on how to improve the services they provide or deliver. In 2021, there were even more government plans highly likely to belong to Group 1, probably because the COVID-19 pandemic made them realise the urgent need for structural changes to improve the health system.

As pointed out before with the LDA algorithm, the TextRank analysis also showed that keywords in 2016 were more general than in 2021, when we also found keywords addressing the COVID-19 pandemic. Interestingly, in 2021 the TextRank analysis also revealed a critique view of the health system at the beginning of the COVID-19 pandemic. This suggests that some government plans were not only about proposing or offering new programs or interventions, but they also evaluated the health system beforehand, ideally to inform their proposals, making these as specific as possible to solve the main problems or limitations of the health system.

Even though TextRank and KeyBERT would deliver keywords, the keywords obtained with KeyBERT gave more context about the government plan or the political party because this algorithm learnt from the context in which the words were embedded. Government plans in which most of the keywords obtained with KeyBERT were about diseases, could suggest that they will focused on these illness, their risk factors and consequences; in other words, the proposals could be disease-specific, perhaps aiming to provide diagnosis and treatment. Similarly, government plans in which many of the keywords were about vaccination -in 2021 presumably regarding COVID-19-, could suggest that their priority would be to get vaccines, and deliver these to the best of their ability; it could also suggest that they will focus –mostly– on the pandemic, while other problems would be addressed in parallel or with less priority. Finally, in accordance with the fact that the KeyBERT algorithm finds keywords based on the context of the text, this algorithm gave insights about the underlying characteristics of the political party. This algorithm could be used to dissect the profile of the political party and how they may address a given issue, above and beyond the frequency of single words and learning from the context. This could be useful to understand the vision of the document and underlying priorities.

The Rake analysis provided key phrases that the unsupervised model could compose from the original texts. Both in 2016 and 2021, a recurrent topic was universal health coverage, signalling that this was a hot topic which, apparently, has not been solved since 2016. In this line, the 2021 phrases addressed the fragmentation of the healthcare system in Peru, where several systems coexist (e.g., private, public, social security, and military forces) ³² . This fragmentation has challenged, and sometimes limited, the response to the COVID-19 pandemic. Finally, mental health was addressed both in 2016 and 2021. This implies that the concern about the mental health of the population has risen since 2016. Also, this suggests that the improvement achieved so far still needs further work to secure optimal mental health for the population. Communicable diseases were also mentioned in both 2016 and 2021; however, non-communicable diseases were absent from these phrases, despite the large burden they impose on the population and health systems ³³ .

Finally, the contribution of this work lies in the fact that we expanded the data sources frequently used in NLP health research, which has mostly leveraged on texts from electronic health records and social media. Unfortunately, electronic health records may not be available in many low- and middle-income countries, where people may not engage as much in some social media outlets as their peers in high-income countries. Thus, our work may democratize the use of NLP in health research by leveraging on a new data source (e.g., materials distributed during national elections) that, arguably, is accessible to a much broader population. Future work, moving from electronic health records and social media, could focus on texts from newspapers and open-access documents published by authorities such as reports from government officers, summaries from government meetings, and summaries of debates at the congress; again, all these sources may be available to a large population above and beyond electronic health records and social media.

We used novel techniques to study the health chapters of the government plans of the 2016 and 2021 Peruvian presidential elections. In so doing, we provided novel insights about these documents to better inform practitioners, researchers, public health experts, and the general population in Peru. This exercise can be replicated in other countries hosting elections this year or soon. We also hope to have sparked interest in NLP techniques to strengthen public health and health policy research, which have lots of data sources which can benefit from NLP analysis.

There are, however, limitations we must acknowledge. First, we focused on the health chapter/section of each government plan which surely contained much of the information for the health sector. Nevertheless, we cannot completely rule out that other chapters could have included additional information about their health-related plans; for example, the chapter/section about economics could have addressed the budget for the health sector or plans of investments. We argued that the most relevant health information must have been included in the health chapters/sections herein analysed, so that arguments outside these chapters would not substantially change our findings or conclusions. Second, we used the government plans as they were, and translated them into English. To the best of our knowledge, the most used NLP algorithms are only available for texts in English. We acknowledge that some words/terms may not have had the ideal translation from Spanish to English through Google Translator; however, this potential limitation affecting single words may have not biased the overall findings and conclusions. We hope that this work sparks interest in the Spanish-speaking scientific community to use NLP into clinical medicine and public health research, while also developing NLP algorithms in Spanish. In addition to more algorithms in Spanish, the scientific community should also secure datasets in Spanish which currently lack to train available and new NLP models. Where possible, these datasets of text in Spanish (and other languages) should be tagged and untagged, as well as annotated or annotated. Third, the analyses and interpretations were mostly data-driven, and should be interpreted in that context. This work was not designed to be a comprehensive political (though NLP is also useful for studying political speech) ³⁴ , anthropological, or linguistic scrutiny of the 2016 and 2021 government plans in Peru, rather the application of novel artificial intelligence techniques to further expand the understanding of the health chapters in these government plans. Fourth, we did not compare the government plan of the same political party in 2016 and 2021. A prospective easement of the political parties was beyond the scope of this work, and such exercise would not be possible for all parties. Fifth, the text of the health chapters varied in length. The LDA analysis was trained in a dataset with all the government plans together. Government plans with longer text, hence with more words and information, could have driven the algorithm results; conversely, shorter government plans could have provided less information to the LDA algorithm. For transparency, we reported the length of each government plan and the results should be interpreted considering this potential limitation. Sixth, the LDA algorithm was applied with the absolute minimum number of groups (i.e., to identify two clusters). We could have tried to find 5, 10, 20 or more groups; however, dividing the texts in many thin groups would have not been ideal. First, we only worked with the health chapters which were, on average, one or two pages long; in other words, these chapter did not have much content to be dissected into many clusters. Had we analysed the full plans (not only the health chapters), finding more groups would have helped to separate all sections. Second, the government plans were concise straight-to-the-point documents (sometime bullet points); that is, they did not cover several topics nor in detail. This made it difficult to dissect the texts into many thin groups.