Cardiovascular Disease Prognostic Models in Latin America and the Caribbean

Background Cardiovascular prognostic models guide treatment allocation and support clinical decisions. Whether there are valid models for Latin American and Caribbean (LAC) populations is unknown. Objective This study sought to identify and critically appraise cardiovascular prognostic models developed, tested, or recalibrated in LAC populations. Methods The systematic review followed the CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) framework (PROSPERO [International Prospective Register of Systemic Reviews]: CRD42018096553). Reports were included if they followed a prospective design and presented a multivariable prognostic model; reports were excluded if they studied symptomatic individuals or patients. The following search engines were used: EMBASE, MEDLINE, Scopus, SciELO, and LILACS. Risk of bias assessment was conducted with PROBAST (Prediction model Risk Of Bias ASsessment Tool). No quantitative summary was conducted due to large heterogeneity. Results From 2,506 search results, 8 studies (N = 130,482 participants) were included for qualitative synthesis. We could not identify any cardiovascular prognostic model developed for LAC populations; reviewed reports evaluated available models or conducted a recalibration analysis. Only 1 study included a Caribbean population (Puerto Rico); 3 studies were retrieved from Chile; 2 from Argentina, Brazil, Colombia, and Uruguay; and 1 from Mexico. Four studies included population-based samples, and the other 4 included people affiliated to a health facility (e.g., prevention clinics). Most studied participants were older than 50 years, and there were more women in 5 reports. The Framingham model was assessed 6 times, and the American College of Cardiology/American Heart Association pooled equation was assessed twice. Across the prognostic models assessed, calibration varied widely from one population to another, showing great overestimation particularly in some subgroups (e.g., highest risk). Discrimination (e.g., C-statistic) was acceptable for most models; for Framingham it ranged from 0.66 to 0.76. The American College of Cardiology/American Heart Association pooled equation showed the best discrimination (0.78). That there were few outcome events was the most important methodological limitation of the identified studies. Conclusions No cardiovascular prognostic models have been developed in LAC, hampering key evidence to inform public health and clinical practice. Validation studies need to improve methodological issues.

There is some evidence that available cardiovascular risk prediction equations do not perform well in LAC [7], where cardiovascular key risk factors such as diabetes seem to have different strength of association with cardiovascular events [8]. Therefore, it becomes necessary to identify which available equations have undergone local scrutiny and whether new local tools have been developed. Even though there have been efforts to summarize cardiovascular prognostic models [9,10], they did not include studies written in Spanish or search engines with large LAC influence, hence reporting no results from LAC [10]. R Therefore, whether available cardiovascular prognostic models have been tested or a new model has been derived in LAC, remains largely unknown. Consequently, we conducted a systematic review to summarize and critically appraise studies evaluating or generating prognostic models for cardiovascular outcomes conducted in LAC. In so doing, we provide a comprehensive list of available prognostic models, their strengths and limitations, as well as recommendations and identification of research gaps to be addressed to improve cardiovascular prevention in LAC.

Study design
The protocol for this systematic review of the literature was registered at PROSPERO (International Prospective Register of Systematic Reviews) (CRD42018096553) [11]. This work adheres to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (see the PRISMA Checklist in the Online Appendix) and CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) framework [12][13][14]. Following the CHARMS framework, we sought prognostic models that predict the risk of having a cardiovascular (nonfatal and fatal) outcome in a pre-defined period of time to be used in the general population to guide prevention or treatment recommendations (Table 1). We exclusively focused on reports that included LAC populations.

Eligibility criteria
Inclusion criteria comprised the following: 1) The overall study design had to be prospective; this included studies that turned into prospective cohorts, even though they were not originally designed as such. For example, a cross-sectional survey/study in which participants were looked up in death/ hospital registries after some years. 2) The study reported a multivariable (i.e., at least 2 variables or predictors) model to predict the risk of developing a nonfatal and/or fatal cardiovascular event (e.g., stroke, myocardial infarction, and death) in an individual of the general population. On the other hand, exclusion criteria included the following: 1) the study population targeted symptomatic (e.g., emergency care) or only patients with specific pathologies (e.g., chronic kidney disease); and 2) the study population targeted people who, by the time of the baseline assessment, had already experienced a cardiovascular outcome (e.g., stroke).

Information sources
On July 15, 2018, a systematic search of publications was conducted using 5 search engines: Ovid (EMBASE and MEDLINE), Scopus, SciELO and LILACS; the latter 2 are LAC-specific. The search terms were based on a recent systematic review, which only included EMBASE, MED-LINE, and articles in English [10]. In addition, terms regarding LAC countries were included (i.e., country names). No additional filters (e.g., language or publication year) were set. The list of search terms is presented in the Online Appendix.

Search
Before titles and abstract screening, 2 reviewers (RMC-L and NP-B) agreed on a standard approach. Two random samples of 50 search results were selected for training purposes. Reviewers screened these titles and abstracts and the inter-rater agreement and kappa estimator were computed, aiming for an inter-rater agreement of at least 90% (see "Training of reviewers" in the Online Appendix). After this standardization process, the 2 reviewers (RMC-L and NP-B) screened all titles and abstracts following the pre-specified framework and selection criteria. Discrepancies were solved by a third reviewer independently (CA-F). After the title and abstract selection, full text of selected reports was sought and analyzed by 2 reviewers (RMC-L and CA-F) following the same above-mentioned selection criteria; discrepancies were solved by consensus between these reviewers. These selection processes-titles and abstracts as well as full texts-were conducted using the online tool Rayyan-a [15]. With the final list of studies to be included for qualitative synthesis, 2 reviewers (RMC-L and CA-F) extracted relevant information in a pre-specified form developed by the authors based on the CHARMS framework [13,14]. Meta-analysis was not conducted because of the large heterogeneity among studies. Results were summarized qualitatively and relevant point estimates (e.g., C-statistics) are presented.

Risk of bias
The PROBAST tool for risk of bias appraisal was used [16][17][18]. This tool was applied by 1 reviewer (RMC-L Tool) tool has been designed to assess risk of bias of multivariate prognostic models in 4 domains: 1) participants; 2) predictors; 3) outcome; and 4) analysis. The criteria within each domain have 5 possible answers: yes; probably yes; probably no; no; and no information. A positive answer suggests no risk of bias. The domains had 3 potential outcomes: low; high; or unclear risk of bias.

Study selection
The search retrieved 2,506 results. After removing duplicates, 2,420 titles and abstracts were screened, and of these 2,403 were excluded. Of the 17 reports studied in detail, 8 reports (N ¼ 130,482 participants) were selected for qualitative synthesis (see the "Extraction form" in the Online Appendix). There was the same number of reports and studies.

Study characteristics
Five of the 8 reports were published in Latin-American journals [19][20][21][22][23], with 3 of them written in Spanish [19,22,23]. The oldest report was published in 2001 [24] and the newest ones in 2018 [20,25]. Six reports were conducted in 1 country only [19,[21][22][23][24]26], and there were 2 multicountry efforts including Argentina, Chile, and Uruguay [20], as well as Argentina, Brazil, Chile, and Colombia ( Fig. 1) [25]. The country that has been mostly included in this review was Chile (3 of 8) [19,20,25]; Argentina, Brazil, Colombia, and Uruguay were studied in 2 reports [20,22,23,25,26]; Puerto Rico [24] and Mexico [21] were included in 1 report (Fig. 1) [19,20,25,26], 3 in the 1990s [21][22][23], and 1 between 1965 and 1968 [24]. One-half of the selected studies  [19], the outcome was cardiovascular mortality (did not include nonfatal events). Brown dots represent studies conducted with populations in 1 country alone, and orange dots are for multicountry studies. PROCAM is a prognostic model for men, the "PROCAM (sex)" indicates the adjusted model so that it can be used for men and women. Figure template from http://yourfreetemplates.com (see "Details for preparation of Figure 1" in the Online Appendix included population-based samples [20,21,24,25], whereas the other one-half included people that were somehow affiliated with health care facilities [19,22,23,26]. For example, Muñoz et al. [22] included individuals who attended a primary prevention clinic at the Central Military Hospital in Colombia. Likewise, Acevedo et al. [19] enrolled individuals who voluntarily attended a cardiology prevention program in Chile. Women accounted for more than one-half of the study population in 5 reports [20][21][22][23]25]; in 1 report the women-to-men ratio was 1 [26]; and in 2 reports, men accounted for a larger share of the study population [19,24]. Although it was not possible to extract baseline age information from 1 study [20], the others had either a mean of, or a larger proportion of, people !50 years old [19,[22][23][24][25][26]; in the report by Jimenez-Corona et al. [21], the mean age was 47 years. Across the studies summarized herein, the mean sample size was 16,310 (SD: 37,107.4). Sample size was over 1,000 people in 7 of the studied reports, and 1 report included 468 individuals [26]. The largest sample sizes were from multicountry endeavors [20,25]. The only study conducted in the Caribbean in the mid-1960s also had a large sample size (n ¼ 8,713) [24]. One study did not report follow-up time duration [25], and another reported 10 years of follow-up or censoring/event [22]; in general, follow-up time ranged from 12 AE 4 months [26] to 9.2 AE 2.1 years [23].
Most reports included the same outcomes as those of the original prognostic model being studied (e.g., Framingham model); however, although the original model included both nonfatal and fatal cardiovascular events, 1 report could only analyze the latter [19]. All reports included the same predictors of the original model, and these were assessed following standard procedures such as clinical examination or laboratory tests. Further details about the characteristics of the summarized reports, including methods for predictors and outcomes ascertainment, are shown in Table 2 and the Extraction Form in the Online Appendix.
Most reports studied prognostic models with few outcome events per predictor. One study had 352 outcome events [25], and in 6 reports this figure ranged from 15 [26] to 72 [23] (Table 2). In addition, none reported a formal sample size estimation or whether the number of outcome events was adequate given the number of predictors.
Missing data was handled by conducting a completecase analysis in 5 reports [20,21,23,25,26], only 1 conducted multiple imputation [22], and 2 did not provide information about this matter [19,24]. Only 1 of the reviewed reports conducted bootstrap analysis to compute the confidence intervals of the area under the receiver operator curve [19].
Performance. Most reports provided estimates of calibration and discrimination [20,[22][23][24][25]; 2 also reported classification metrics such as positive/negative likelihood ratio [20,26]. Calibration and discrimination estimates are presented in Table 3 and Figure 1 (Online Table 1). Although all discrimination estimates (e.g., C-statistics) were reported with wide confidence intervals, the highest discrimination metric achieved was for the American College of Cardiology/American Heart Association (ACC/AHA) pooled risk equation by Acevedo et al. [19]; however, they did not include all the outcomes of the original model so that this estimate should be interpreted cautiously.

Risk of bias
In the participants, predictors, and outcome domains of the risk of bias, all the reports were deemed to be of low risk; the outcomes were clearly defined and agreed with those of the original model except for 1 study that addressed the American College of Cardiology/American Heart Association Pooled Cohorts Equation [19]. The fourth criteria-analysis-was troublesome for the reviewed reports largely because the few number of outcome events, yielding a limited outcome-predictors ratio, and for conducting complete-case analysis rather than multiple imputation process. A summary of the risk of bias analysis is shown in Table 4, and details on each criterion across domains are provided in Risk of Bias (PRO-BAST) in the Online Appendix.

Summary of evidence
Although cardiovascular prognostic models have been summarized by global systematic reviews, none of them found models or efforts undertaken in, and for, LAC populations [9,10]. This work complements these reviews with evidence from LAC and clearly demarcates that there is scope for the improvement of cardiovascular risk prediction in the LAC region. Eight studies were selected for jg SCIENCE   Original NL-IHRS: C-statistic ¼ 0.72 (0.69e0.75); and so was for the recalibrated version. Original FC-IHRS: C-statistic ¼ 0.74 (0.71e0.77) and so was for the recalibrated version.
No details provided.

D'Agostino, 2001 [24]
Best chi-square using the Puerto Rico study's means on the risk factors and the Puerto Rico study's CHD incidence ¼ 7.2.
In Hispanic population, the best Cox (applying the Cox model developed on the Puerto Rico study's data): Cstatistic ¼ 0.72.
No details provided.
AUC, area under the curve; CHD, coronary heart disease; CI, confidence interval; LR, likelihood ratio; TIA, transient ischemic attack; other abbreviations as in Table 2. gSCIENCE j GLOBAL HEART, VOL. 14, NO. 1, 2019 qualitative synthesis: none developed a prognostic model, only 1 was conducted in the Caribbean [24], and many have major limitations with regards to sample size and analysis, for example, limited number of outcome events. The Framingham and the American College of Cardiology/ American Heart Association Pooled Cohorts Equation were the most studied tools. Calibration estimates changed substantially from one population to another, with serious overestimation in some cases, that is, individuals categorized as high risk when they were not. Discrimination was acceptable in many reports, particularly for the American College of Cardiology/American Heart Association Pooled Cohorts Equation. Based on these findings, and the heterogeneous health profile of LAC populations, it is premature to strongly advocate for 1 prognostic model in LAC.

Limitations at study level
Most studies analyzed few outcome events, which could account for wide confidence intervals, but most importantly for lack of power to make strong conclusions. Just recently, new approaches for estimating sample size or adequate number of events per predictors in the prognostic models have been proposed [28,29]. Future cardiovascular prognostic work could formally test these requirements. Joseph et al.
[25] conducted the work with the "largest" outcome events per predictor ratio (ratio ¼ 16). Nonetheless, and even though they had a model without laboratory variables, the model had over 20 predictors, which would make it troublesome to use in the field because it will require much information. In addition, some of the predictors were about diet profile [25], which could require further knowledge about local foods, hence making these questions difficult to ascertain. It seemed that the reviewed studies made a great effort to conduct follow-up rounds and to accurately capture the outcomes of interest; however, 1 study could not adjudicate nonfatal outcomes even though the model being evaluated needed them [19]. This highlights the necessity for national health registries, at least of major events of noncommunicable diseases. These could inform health authorities, as well as researchers who will ultimately provide evidence to advise public health and clinical practice.

Strengths and limitations at the review level
The research question and search strategy were defined following international guidelines for systematic reviews of prognostic models [13,14]. The search terms were based on previous systematic reviews and followed recommendations for finding prognostic studies [9,10,30]. In addition, risk of bias was formally assessed with a validated tool [16][17][18]. Nevertheless, this work is subject of several limitations. First, given the results about heterogeneity and the low number of reviewed reports, a quantitative summary (e.g., metaanalysis) was not be conducted. Second, we could have further reviewed gray reports, such as graduate programs dissertations of LAC universities. Although this could have retrieved more results, we doubt these would have been of greater quality than the published works herein analyzed. Therefore, we would have still not reached a strong recommendation for (or against) a given prognostic model.

Other relevant publications
To summarize the strongest evidence on prognostic research we focused on prospective studies. Nonetheless, it seemed fair to also acknowledge other endeavors in LAC that, despite following different study designs, have still provided relevant evidence.
For example, Icaza et al. [31] adapted the Framingham equation using population-based estimates on risk factors and incidence of cardiovascular events based on national registries in Chile. This work has informed clinical and research practice in Chile. More recently, a populationbased cohort was initiated in Maule, Central Chile. The MAUCO (Maule Cohort) study aims to enroll 10,000 people from 3 to 74 years old who will be followed for at  [32]. Similar efforts in Peru and Argentina may provide solid evidence for these countries [33,34], though a regional approach will still be missing. The systematic search yielded 1 additional work from Central America. This was a bold project that attempted to generate a new prognostic model for myocardial infarction using case-control data from Costa Rica [35]. Despite the design limitations, they included a key item among their predictors: socioeconomic status. This relevant health determinant has been systematically excluded from many prognostic models, though just recently it has been highlighted by a novel model developed for New Zealand [36]. LAC is a region with large socioeconomic inequalities, and their effect on cardiovascular outcomes may be important. Therefore, should a new cardiovascular prognostic model be developed for LAC, a marker of socioeconomic status should be at least tested among other potential predictors.
Although gray reports were beyond the scope of our search, a master's dissertation was also retrieved by our search strategy; the outcome was coronary artery disease and included 349 individuals in Brazil [37]. This work included genetic predictors [37], which although relevant and research on this field should be fostered, it is perhaps premature to have prognostic models with predictors that are not widely available.
Finally, the Globorisk investigators developed cardiovascular risk charts for 182 countries including those in LAC [38,39]. Even though this was an example of global collaboration and comprehensive research methods, the risk charts were developed using risk prediction associations (e.g., hazard ratios) from cohorts mostly in the United States. This may limit the applicability of this model to other world regions including LAC, calling for updates to this and other models with weights (i.e., hazard estimates) retrieved from LAC studies.

Research gaps
This review identified a lack of cardiovascular prognostic models developed in and for LAC populations, which needs urgent attention. Research funders, along with regional and local health agencies, should support new or ongoing prospective cohort studies to address this gap. This does not necessarily mean establishing a new multicountry or region-wide cohort but supporting further follow-up rounds of existing cohorts, particularly of population-based cohorts. In addition, regional and local health authorities could work to facilitate access to national health/death registries for research purposes. This way, large cross-sectional epidemiological studies and surveys could access data on cardiovascular nonfatal and fatal outcomes, in other words, entering the world of health big data in LAC. In the meantime, cohort data pooling may be a feasible solution.
Another relevant research gap observed by this review is the dearth of studies from the Caribbean and Central America regions. Besides a case-control study (Costa Rica) [35] and a prospective cohort, the baseline assessment of which was in the 1960s (Puerto Rico) [24], no additional results were retrieved for these regions. This calls for investigators to strengthen local research capacity and to generate, reuse, or update available cohorts in the Caribbean and Central America.
In terms of methods, this review found that 1 study followed multiple imputation to account for missing data; also, there were a low number of outcome events and some predictors were categorized when they could have been used as continuous variables [40,41]. These methodological approaches could hide a lack of training or expertise in validation (and development) of prognostic models. Despite abundant methodological evidence (e.g., statistical primers [42] and reviews [43][44][45][46]) on prognosis models, these are mostly in English, which could be a limitation for some researchers in LAC. In terms of capacity building, existing training programs in epidemiology and related fields should include prognostic research methods.
Reporting was also heterogeneous across studies. It would be impossible for studies published before 2015 to have adhered to the TRIPOD statement [47], but neither did the post-2015 papers. This again calls for investigators to improve prediction research capacity in LAC.

Implications for public health
Where poor-quality health care could be worse than any health care [48], the health system should secure adequate interventions to prevent diseases and complications. This ranges from highly complex treatments to prevention strategies. Among the latter, estimating the absolute risk of an event might guide the allocation of treatment and preventive measures in favor of high-risk individuals. However, if prediction tools are not accurate enough, people who do not need treatment would unnecessarily receive medication and people who need it would not be identified. Considering this systematic review, local and regional health authorities, researchers, and practitioners should understand the pitfalls of using available cardiovascular prognostic models in LAC and procure the development, comprehensive validation, or adequate recalibration of available models.

Implications for clinical practice
Several international societies and clinical guidelines have signaled the need to guide cardiovascular treatment and other prevention interventions based on absolute risk for which prognostic models are paramount [27,[49][50][51][52][53][54]. In particular, risk stratification based on these models contribute to allocation of health resources in a costeffective manner, allowing individuals with higher global risk to receive adequate treatment and avoiding overtreatment in individuals at low risk [27,[49][50][51][52][53][54].

gSCIENCE j
In Chile, the tables of cardiovascular risk stratification adapted by Icaza et al. [31] were adopted by the Ministry of Health and have been used in the public primary health system since 2009 [54]. In 2013, Kunstmann et al. [55] conducted a validation of the Chilean tables, confirming that they are useful in the prediction of coronary and cerebrovascular events. Therefore, the Chilean tables continue to be used to estimate the global risk of cardiovascular diseases at individual level in Chile [55].
The fact that no cardiovascular prognostic models have been developed in LAC, and the fact that attempts to test the accuracy of available prognostic models in LAC have faced several limitations, puts LAC in a disadvantaged position to successfully lessen the burden of cardiovascular diseases. This review could not strongly recommend any available prognostic model to be used across LAC settings. Notwithstanding, acknowledging their limitations, weighing the pros and cons, and reaching an informed decision with the patient, country-specific models could be used after adequate recalibration.

CONCLUSIONS
No cardiovascular prognostic model has been developed in or for the LAC region. The evaluation of available prognostic models signals to several limitations, and their prediction accuracy is questionable particularly regarding calibration, albeit discrimination was acceptable in most cases. Advancement of cardiovascular prognosis research might contribute to improve the allocation of scarce resources to people who need them the most, thus fostering the prevention of cardiovascular diseases in this world region and to achieve this goal, appropriate cardiovascular risk prediction is needed in the LAC region.