Developing Non-Laboratory Cardiovascular Risk Assessment Charts and Validating Laboratory and Non-Laboratory-Based Models

Razieh Hassannejad; Marjan Mansourian; Hamidreza Marateb; Mohammad Reza Mohebian; Thomas Andrew Gaziano; Rodney T Jackson; Emanuele Di Angelantonio; Nizal Sarrafzadegan

Original Research

Developing Non-Laboratory Cardiovascular Risk Assessment Charts and Validating Laboratory and Non-Laboratory-Based Models

Authors

Razieh Hassannejad
Marjan Mansourian
Hamidreza Marateb
Mohammad Reza Mohebian
Thomas Andrew Gaziano
Rodney T Jackson
Emanuele Di Angelantonio
Nizal Sarrafzadegan

Abstract

Background: Developing simplified risk assessment model based on non-laboratory risk factors that could determine cardiovascular risk as accurately as laboratory-based one can be valuable, particularly in developing countries where there are limited resources.

Objective: To develop a simplified non-laboratory cardiovascular disease risk assessment chart based on previously reported laboratory-based chart and evaluate internal and external validation, and recalibration of both risk models to assess the performance of risk scoring tools in other population.

Methods: A 10-year non-laboratory-based risk prediction chart was developed for fatal and non-fatal CVD using Cox Proportional Hazard regression. Data from the Isfahan Cohort Study (ICS), a population-based study among 6504 adults aged ≥ 35 years, followed-up for at least ten years was used for the non-laboratory-based model derivation. Participants were followed up until the occurrence of CVD events. Tehran Lipid and Glucose Study (TLGS) data was used to evaluate the external validity of both non-laboratory and laboratory risk assessment models in other populations rather than one used in the model derivation.

Results: The discrimination and calibration analysis of the non-laboratory model showed the following values of Harrell’s C: 0.73 (95% CI 0.71–0.74), and Nam-D’Agostino χ2:11.01 (p = 0.27), respectively. The non-laboratory model was in agreement and classified high risk and low risk patients as accurately as the laboratory one. Both non-laboratory and laboratory risk prediction models showed good discrimination in the external validation, with Harrell’s C of 0.77 (95% CI 0.75–0.78) and 0.78 (95% CI 0.76–0.79), respectively.

Conclusions: Our simplified risk assessment model based on non-laboratory risk factors could determine cardiovascular risk as accurately as laboratory-based one. This approach can provide simple risk assessment tool where laboratory testing is unavailable, inconvenient, and costly.

Keywords:

Year: 2021

Volume: 16 Issue: 1

Page/Article: 58

DOI: 10.5334/gh.890

Submitted on Jul 28, 2020

Accepted on Aug 20, 2021

Published on Sep 2, 2021

Peer Reviewed

CC BY 4.0

Introduction

Cardiovascular disease (CVD) is the most common preventable non-communicable diseases (NCD) worldwide, with an estimated 17.8 million deaths in 2017. It is predicted that CVD would be the cause of more than 23 million (about 30.5%) deaths by 2030 worldwide [, , ].

A reduction of CVD mortality rates has been reported in high income regions. However, 50% of CVD mortality and 80% of the CVD global burden occur in low and middle-income countries (LMICs), including the Eastern Mediterranean Region (EMR) [].

In the last two decades, the most common causes of death have been transited from infectious to NCDs specially CVD in Iran []. Global Burden of Disease(GBD) previous data in 2010 and 2015 reported that CVD was the first leading cause of mortality and DALYs that led to 46% of all deaths and 20–23% of the burden of diseases in Iran [].

Many global prevention and control guidelines recommend applying CVD risk assessment charts to identify people at high risk of developing CVD within a specified period of time, usually 10 years. Risk-based management helps to target specific intensive preventive and treatment interventions [, , , ]. Several risk assessment models have been reported previously [, , , , , , , ]. Although the widely used risk algorithms could be beneficial in developed countries, however, they are based on laboratory measures that are not affordable and available in some LMICs [, ].

Recently, significant efforts have been directed to develop non-laboratory based CV risk algorithms that can predict the disease as accurately as laboratory-based ones but are more feasible to use in clinical practice [, ].

Laboratory and non-laboratory based CVD risk charts were developed by WHO for different regions worldwide and validated in other cohorts []. However, one potential limitation is the use of HRs derived in a mainly western population in a LMIC. Additionally, the new WHO charts are recalibrated for region and might be improved by focusing on national data. Providing affordable approaches for the prediction of CVD risk based on national data is especially crucial in LMICs, where many primary care facilities are not available, and most individuals remain unaware of their underlying cardiovascular risk [, ].

In an attempt to simplify the Persian Atherosclerotic cardiovascular disease Risk Stratification (PARS) model that we reported previously and is a laboratory-based one [], we aim to develop a simplified Persian Atherosclerotic cardiovascular disease Risk Stratification (SPARS) model based on non-laboratory risk factors and to assess if it can predict CVD risk as accurately as the PARS laboratory-based one. Furthermore, we aim to validate PARS and SPARS models, both developed based on Isfahan Cohort study(ICS) and to validate them on the Tehran Lipid and Glucose Study (TLGS) population [].

Methods

Study population

The ICS is a population-based longitudinal study of 6504 Iranians adult, who were recruited in the year 2001 using multi-stage random cluster sampling and followed-up for at least ten years [].

Written, informed consents were obtained from all subjects and Ethical approval was obtained from the Isfahan Cardiovascular Research Center Ethics Committee, a WHO collaborating center in the Eastern Mediterranean Region (EMR), and Isfahan University of Medical Sciences and conformed to the Declaration of Helsinki.

Having Iranian nationality, aged ≥ 35 years, mentally competent, and not pregnant, were considered as inclusion criteria. Participants with known coronary heart disease, heart failure, stroke and ischemic heart attack (n = 181) and subjects without a single follow-up (n = 891) were excluded from the study. Among the original recruited sample, only 5432 were free of CVD at baseline and had at least one follow-up.

Phone call follow-up was done every two years to look for any report of CVD events including sudden death, unstable angina, fatal and non-fatal myocardial infarction, and fatal and non-fatal stroke. All medical records of participants who reported CVD events were collected and verbal autopsies were done for dead participants. A panel of cardiologists and neurologists made final decisions by reviewing on the patient’s medical records and validated and confirmed the events diagnosis [].

The loss to follow-up rate was 891 (14.1%) in the first phone call follow-ups and decrease to 104 (1.6%) in the fifth stage. The baseline characteristics and prevalence of CVD risk factors were not significantly different between subjects lost to follow-up and those studied [, ]. Participants without any event and loss to follow-up events were considered as censored. The detailed description of the design, methodology, follow-up, success rate of follow-up, risk factor measurements, and endpoints of ICS were previously reported [, , ].

External Cohort

The Tehran Lipid and Glucose Study (TLGS) was used for external validation because of the similarities of the research protocols including adequate follow-up duration, using similar CVD events definition, similar age range and systematic measurement of CVD risk factors.

The TLGS is a longitudinal study started in 1999 to identify risk factors of non-communicable diseases among population in district no.13 of Tehran, the capital of Iran. The study methods have been described elsewhere [, ].

Statistical analysis

Developing of Non-laboratory-based model

In our previously reported risk model based on laboratory risk factors, we considered age, sex, systolic blood pressure (SBP), total cholesterol (TC), diabetes (based on fasting blood test), smoking status, family history of CVD, and waist to hip ratio (WHR) []. In current study, we developed a new non-laboratory-based model and named it as simplified PARS (SPARS) by considering risk factors like: age, smoking status, SBP, self-reporting history of diabetes, and WHR. We used Cox proportional hazards regression to estimate the hazard ratios (HR). Deviation from the proportional hazards assumption was assessed graphically and by fitting an extended Cox model, including time-varying covariates. There was no significant deviation from the proportional hazards assumption. Then we constructed a 10-year simplified risk assessment chart of CVD incidence using the SPARS model. SBP was categorized into four classes based on Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7) []: <120, 120–139, 140–159, and ≥160 mm Hg. Waist-to-hip ratio (WHR) was categorized as <0.85, 0.85–0.90, 0.90–0.95 and >=0.95 in females and <1, 1–1.05, 1.05–1.10 and >=1.10 in males []. Smoking status was categorized as smoker and non-smokers.

The simplified risk chart displays CVD risk thresholds as <5% (low risk), 5% to <10% (intermediate risk), 10% to <15% (high risk) and >15% (very high risk) as a colored chart. The thresholds were chosen conceptually based on ROC curve and indices introduced by Song et al. []. Then they are confirmed by some cardiologists about usefulness of them in classifying individuals in low risk to high risk in target population.

The performance and predictive accuracy of models was assessed by discrimination and calibration. To avoid optimism that might result from assessing risk discrimination in the data from which the model was derived, resampling methods including 50000 random-sample bootstrapping and 10-fold cross-validation were used to assess discrimination using Harrell’s C []. The calibration of the risk models were assessed in the derivation dataset by Nam-D’Agostino chi-square test [].

Evaluation of External validation

External validation assesses whether the model can be used in a population other than the one in which the model was derived [].

The performance of the non-laboratory-based SPARS prediction function among the TLGS cohort as an external cohort was assessed using two evaluations. First, SPARS function was refitted using the same variables in the equation by applying multiple Cox (proportional hazards) regression model in the TLGS database. Second, recalibration of SPARS function were done using the method applied in the new revised WHO model []. The process involved the use of mean risk factor levels (from TLGS) and country-specific estimates of annual incidence of CVD within five-year age groups, from GBD 2017 (Methods.A in the Appendix).

Model performance was evaluated by three steps: comparison of regression coefficients (Methods.A in the Appendix), discrimination using Harrell’s C, and calibration by Nam-D’Agostino chi-square test.

Further, all external validation analysis were also repeated for previous reported laboratory-based model (PARS) to assess external validation of laboratory-based model.

SAS software, version 9.4 (SAS Institute Inc) was used for statistical modeling and analysis. Chart generation and model validations were performed using Matlab version 8.6 (The MathWorks Inc., Natick, MA, USA).

Results

Non-laboratory-based model development

During 49452.8 person-years of follow-up (range 0.1–12, median 10.9 years), there were a total of 705 events related to cardiovascular disease (564 IHDs, 141 strokes). Mean age of male and female was 51.2 ± 11.9 and 50.3 ±11.3 years, respectively. There was a higher prevalence of smoking in men than women (41.6 vs. 3.3), while a higher proportion of women had high WHR (94.6 vs. 39.2) and self-reported diabetes (8.6 vs. 5.8). SBP distribution was similar between sexes.

HRs for fatal and non-fatal CVD events for each risk predictor included in the non-laboratory-based model are provided in Table 1 and compared with the laboratory-based model.

Table 1

Non- Laboratory-based and laboratory-based models for predicting cardiovascular disease outcomes, Isfahan Cohort Study, 2001–2011.

Risk factors	Estimate	Hazard Ratio	95% CI*

Non-laboratory-based (SPARS)

Age at baseline per 10 years	0.38227	1.466	1.371–1.566
Male	0.32309	1.381	1.076–1.774
WHR
Female: <0.85, male: <1
Female: 0.85-0.90, male: 1–1.05	0.10478	1.11	0.867–1.422
Female: 0.90–0.95, male: 1.05–1.10	0.17986	1.197	0.898–1.595
Female: >=0.95, male: >=1.10	0.2944	1.342	1.012–1.780
SBP (mm Hg)
<120^a
120–139	0.47524	1.608	1.316–1.966
140–159	0.79045	2.204	1.743–2.788
>=160	1.13051	3.097	2.405–3.989
Self-reported Diabetes	0.73001	2.075	1.697–2.537
Smoking	0.2656	1.304	1.079–1.577
Harrell’s C (95% CI):0.73 (0.71–0.74) Nam-D’Agostino χ²:11.01 (p = 0.27).
Laboratory-based (PARS)

Age at baseline per 10 years	0.376	1.456	1.361–1.558
Male	0.28957	1.335	1.111–1.508
TC (mg/dl)
<150^a	–	–
150–200	0.20759	1.231	0.879–1.723
200–250	0.34201	1.408	1.013–1.957
250–300	0.45316	1.573	1.113–2.225
>300	0.54847	1.731	1.172–2.556
SBP (mm Hg)
<120^a	–	–
120–139	0.45643	1.578	1.291–1.929
140–159	0.73697	2.09	1.651–2.644
>=160	1.0467	2.848	2.207–3.676
Diabetes	0.63041	1.878	1.570–2.247
High WHR^b	0.26989	1.31	1.072–1.601
FH of CVD	0.40182	1.495	1.116–2.002
Smoking	0.28974	1.336	1.104–1.617
Harrell’s C (95% CI): 0.73 (0.71–0.75) Nam-D’Agostino χ²: 10.82 (p = 0.29)

Abbreviations: CI, confidence interval; CVD, cardiovascular Disease; TC, total cholesterol; SBP, systolic blood pressure; FH, family history; WHR, waist to hip ratio.

^a Reference category.

^b WHR ≥ 0.95 in men and ≥ 0.8 in women was considered as a high WHR.

^* 95% CI for HR.

PARS risk model, published previously by Sarrafzadegan et al. [].

The 10-fold cross-validation yielded a mean Harrell’s C: 0.73 (95% confidence interval [CI], 0.71–0.74). In bootstrap validation, the mean Harrell’s C was 0.73 (min-max; 0.70–0.77). The Nam-D’Agostino calibration χ² was 11.01 (p = 0.27). Thus, the proposed non-laboratory-based model could be internally validated and has good performance in terms of discrimination and calibration. Fitting new non-laboratory WHO model on our population resulted in C-index of 0.73 (95% [CI], 0.71–0.76) and χ² of 15.76 (p = 0.07) for female and C-index of 0.71 (95% [CI], 0.68–0.74) and χ² of 5.91 (p = 0.75) for male (Table A.1 in the Appendix). Thus, our proposed model performed better than new WHO model. After that, the risk chart based on the proposed non-laboratory-based model was created (Figure 1).

Figure 1

SPARS chart for prediction of 10-year risk of fatal and non-fatal cardiovascular disease in ICS population, 2001–2011.

We observed no difference in the Harrell C and a small one in the calibration χ² when we compared non-laboratory-based with the laboratory-based, reported previously (Harrell’s C: 0.73, Nam-D’Agostino χ²:10.82; p = 0.29). In addition, we found strong agreement between risk predictions based on laboratory and non-laboratory models. Of individuals at greater than 10% risk using the laboratory-based model, all were also identified as being at greater than 5% risk with the non-laboratory-based model. When using a 10% and 15% risk threshold with the non-laboratory-based model, about 93% and 71% of CVD events respectively were identified.

In spite of reporting C statistics as suitable criteria for the overall predictive discrimination of the risk models, since the most clinical policy regarding treatment are often considered a specific absolute level of risk, it is useful to illustrate if the risk models classified patients correctly at different levels of 10-year CVD risk, commonly used in guidelines []. One measure is the percentage of individuals correctly classified as the sum of the number of true negatives (those below the risk threshold without any CVD events) and the true positives (those who are above the risk threshold and ultimately did have an event) divided by the total number of individuals in the sample (Figure 2, Table 2). The laboratory-based and non-laboratory-based models classified individuals at the same rates across the commonly used risk levels in clinical guidelines. Both laboratory-based and non-laboratory-based models correctly classified over 80% of patients when the threshold for risk was set at 10% and 15%. This percentage was reduced as the risk threshold dropped, and correctly classified 77% and 68% of patients at 5% risk based on laboratory-based and non-laboratory-based models, respectively.

Figure 2

Patients correctly classified as a high or low risk at various cutoff levels of 10-year risk of cardiovascular disease.

Table 2

True positive and true negative based on PARS and SPARS models across different risk thresholds.

Thresholds	PARS		SPARS

	True positive	True negative	True positive	True negative

5%	50.1% (353/705)	81.3% (3844/4727)	66.2% (467/705)	68.1 (3217/4727)
10%	17.6% (124/705)	96.5% (4561/4727)	31.6% (223/705)	90.4% (4273/4727)
15%	5.2% (37/705)	98.9% (4676/4727)	13.9% (98/705)	96.7% (4573/4727)

External validation

A total of 824 CVD events occurred during 55982.8 person-years of follow-up in TLGS cohort. The 10-year CVD event rates were 1.4% in the ICS and 1.5% in the TLGS. Numbers of participants, person-years of follow-up, CVD events, and the risk factor levels at the baseline examination of the two cohorts are shown in Table 3.

Table 3

Baseline Risk Factors, Person-Years of Follow-up, and CVD events in the ICS and TLGS Cohorts.

	ICS*						TLGS

	Men (N = 2648)			Women (N = 2784)			Men (N = 2306)			Women (N = 2976)

	Participants WithRisk Factor, %	Person-years of follow-up	CVD events	Participants WithRisk Factor, %	Person-years of follow-up	CVD events	Participants WithRisk Factor, %	Person-years of follow-up	CVD events	Participants WithRisk Factor, %	Person-years of follow-up	CVD events

Total		23931	379		25522	326		23812.9	464		32169.9	360
TC (mg/dl)
<150	11.4	2826.4	27	7.5	1917	15	5.3	1294.2	17	3	991.4	2
150–200	35	8507.1	112	30.4	7938.5	71	34.4	8375.3	125	26.9	8999	39
200–250	34.1	8144.3	131	36.4	9206.3	123	42.2	10136.2	193	40.9	13156.3	148
250–300	14.2	3253.8	79	18.6	4710.25	77	15.1	3369.5	104	22.3	7036.2	112
>300	5.2	1198.9	30	7.2	1749.8	40	3	637.6	25	6.9	1986.9	59
SBP(mm/Hg)
<120	44.4	11132.3	96	44.9	11652	67	48	11988.3	141	45.3	15165	85
120–139	37.1	8741.5	148	33.7	8746.2	116	33.9	8038	162	33.5	10835.4	105
140–159	12	2683	76	13.4	3347.7	78	12.6	2699.7	102	14.5	4337.4	105
>=160	6.6	1374.1	59	8	1776.1	65	5.6	1086.9	59	6.7	1832.1	65
Diabetes
Laboratory-based	9.3	2059.8	82	12.6	3040.7	87	15.4	3009.7	142	17.8	5022.5	155
Self-reporting	5.8	1289.8	51	8.6	1984.4	69	10.7	2085.3	96	12.5	3480.3	123
Smoker	41.6	9809.1	159	3.3	805.3	18	27.6	6487.9	120	3.8	1188.6	14
FH of CVD	5	1205.7	22	5.7	1415.8	27	15.6	3618.3	83	19.9	6188.4	101
WHR
1	86.3	20719.7	299	13.8	3673.1	25	79.1	19236.2	306	39.8	13312.5	61
2	9.3	2236.6	49	16.4	4103.2	44	15.8	3557.3	107	21.6	7010.7	77
3	3.3	714.1	25	24.4	6272.7	71	4.2	841.3	41	20.8	6590.4	97
4	1.1	260.5	6	45.4	11472.9	186	0.9	178.1	10	17.8	5256.2	125

Abbreviations: ICS, Isfahan Cohort Study; TLGS, Tehran Lipid and Glucose Study; CVD, cardiovascular Disease; TC, total cholesterol; SBP, systolic blood pressure; FH, family history; WHR, waist to hip ratio.

* Data for the ICS risk factors are from Sarrafzadegan et al. [].

The HRs for major CVD risk factors based on non-laboratory-based risk function were obtained from Cox regression model for TLGS cohort and compared with original functions that were developed on ICS cohort (Table 4). Major risk factors showed a similar relation to CVD in both cohorts for non-laboratory model. For most risk factor categories, the magnitude of the HRs did not differ significantly. The results for the non-laboratory-based model showed few exceptions that reached statistical significance, the significant higher HR for men (P = 0.005) and WHR (P = 0.04, 0.03 for the 3^rd and 4^th category, respectively) in TLGS.

Table 4

Relative risk and performance comparisons of ICS and TLGS based on non-laboratory SPARS risk function.

Risk factors	ICS		TLGS

	Estimate	Hazard Ratio(95% CI)	Estimate	Hazard Ratio(95% CI)

Age	0.03823	1.039 (1.032–1.046)	0.04494	1.046 (1.039–1.053)
Male^†	0.32309	1.38 (1.076–1.774)	0.76677	2.153 (1.801–2.573)
WHR
1			–	–
2	0.10478	1.11 (0.867–1.422)	0.35549	1.427 (1.188–1.713)
3^†	0.17986	1.197 (0.898–1.595)	0.55846	1.748 (1.399–2.184)
4^†	0.2944	1.342 (1.012–1.780)	0.69904	2.012 (1.577–2.567)
SBP (mm/Hg)
<120^a			–	–
120-139	0.47524	1.608 (1.316–1.966)	0.21399	1.239 (1.029–1.490)
140-159	0.79045	2.204 (1.743–2.788)	0.70242	2.019 (1.643–2.481)
>=160	1.13051	3.097 (2.405–3.989)	0.84719	2.333 (1.825–2.982)
History of Diabetes	0.73001	2.075 (1.697–2.537)	0.88382	2.42 (2.064–2.837)
Smoking	0.2656	1.304 (1.079–1.577)	0.47038	1.601 (1.308–1.959)
	Harrell’s C (95% CI): 0.73 (0.71–0.74) Nam–D’Agostino χ²: 11.01 (p = 0.27)		Harrell’s C (95% CI): 0.77 (0.75–0.78) Nam–D’Agostino: 29.89 (p = 0.001)

Abbreviations: ICS, Isfahan Cohort Study; TLGS, Tehran Lipid and Glucose Study; CVD, cardiovascular Disease; SBP, systolic blood pressure; WHR, waist to hip ratio.

Hazard ratio in the ICS is significantly different from that in the TLGS (P-value < 0.05).

For the laboratory-based model, male gender was associated with a higher HR (P < 0.001), and SBP of 120 to 139 mm/Hg was associated with lower HR (P = 0.03) in TLGS. Other risk factors showed a similar relation to CVD in both cohorts (Table 5).

Table 5

Relative risk and performance comparisons of ICS and TLGS based on laboratory-based PARS risk function.

Risk factors	ICS^*		TLGS

	Estimate	Hazard Ratio(95% CI)	Estimate	Hazard Ratio(95% CI)

Age	0.03759	1.038 (1.031–1.045)	0.04592	1.047 (1.04–1.054)
Male	0.28957	1.335 (1.111–1.508)	0.71764	2.05 (1.748–2.403)
TC (mg/dl)
<150^a	–	–	–	–
150–200	0.20759	1.231 (0.879–1.723)	0.00956	1.01 (0.627–1.625)
200–250	0.34201	1.408 (1.013–1.957)	0.41855	1.52 (0.956–2.417)
250–300	0.45316	1.573 (1.113–2.225)	0.55737	1.746 (1.087–2.804)
>300	0.54847	1.731 (1.172–2.556)	0.95743	2.605 (1.571–4.321)
SBP (mm/Hg)
<120^a	–	–	–	–
120–139	0.45643	1.578 (1.291–1.929)	0.15221	1.164 (0.967–1.402)
140–159	0.73697	2.09 (1.651–2.644)	0.60783	1.836 (1.493–2.259)
>=160	1.0467	2.848 (2.207–3.676)	0.74208	2.1 (1.643–2.684)
Diabetes	0.63041	1.878 (1.570–2.247)	0.79142	2.207 (1.899–2.564)
High WHR	0.26989	1.31 (1.072–1.601)	0.31902	1.376 (1.154–1.640)
FH of CVD	0.40182	1.495 (1.116–2.002)	0.38066	1.463 (1.240–1.727)
Smoking	0.28974	1.336 (1.104–1.617)	0.49554	1.641 (1.341–2.010)
	Harrell’s C (95% CI): 0.73 (0.71–0.75) Nam–D’Agostino χ²: 10.82 (p = 0.29)		Harrell’s C (95% CI): 0.78 (0.76–0.79) Nam–D’Agostino χ²: 28.57 (p = 0.001)

* Data for the ICS risk factors are from Sarrafzadegan et al. [].

Hazard ratio in the ICS is significantly different from that in the TLGS (P-value < 0.05).

In the discrimination analysis, non-laboratory-based model separated individuals with CVD events from ones without CVD in the TLGS cohort nearly as well as in the ICS cohort. The Harrell’s C was 0.77 (95% CI, 0.75-0.78) for non-laboratory-based model in TLGS versus 0.73 (95% CI, 0.71–0.74) in ICS. However, with respect to calibration, non-laboratory-based model slightly overestimated the event rates observed in the TLGS cohort. The Nam-D’Agostino χ² was 29.89 (p = 0.001) in TLGS versus 11.01 (p = 0.27) in ICS (Figure 3). In the recalibrated non-laboratory-based function, the C statistic values was 0.73 (95% CI, 0.71–0.75). Thus, the model retained acceptable discrimination performance in the external validation cohort.

Figure 3

Actual and predicted cardiovascular disease events by deciles of risk for the laboratory and non-laboratory-based model in ICS and TLGS.

The same results were obtained for laboratory-based model (Harrell’s C: 0.78, 95% CI: 0.76–0.79 in TLGS versus 0.73, 95% CI: 0.71–0.75 in ICS; Nam-D’Agostino χ²: 28.57, p = 0.001 in TLGS versus 10.82, p = 0.29 in ICS; C statistic: 0.74, 95% CI, 0.72–0.76).

Discussion

We developed a ‘Simplified’ non-laboratory based CVD risk chart and named it SPARS, then evaluated and validated both PARS laboratory-based, published previously [], and SPARS risk prediction models with TLGS as an external cohort.

The proposed SPARS model does not require laboratory measurements such as serum lipids or glucose. This simplified non-laboratory-based model which is clinically convenient, user-friendly, and affordable, can be applicable in situations with little access to laboratory tests, particularly in primary health care systems (PHC) or in low-resource settings like health houses that cover the rural population in Iran []. Our study shows that the non-laboratory-based risk model can predict CVD outcomes with similar accuracy to a laboratory-based one. Our values of predictive discrimination of 0.73 for the non-laboratory-based model are not different from the corresponding values in the laboratory-based model. We also found strong agreement with the laboratory-based model with respect to the classification of patients to moderate or high risk group, similar to findings from previous reports [, , ]. Furthermore, our study showed that 90% of patients with diabetes, who were classified as being at greater than 10% risk of developing CVD in 10 years based on the PARS model, were also classified as being at greater than 10% risk with the SPARS model. Other studies reported poor performance among people with diabetes (e.g., 45% of men and 25% of women in WHO study) [, ]. It might be due to the inclusion of self-reported history of diabetes in our non-laboratory-based model that highlights the importance of including a diagnosis of diabetes in the risk score. Therefore, because of the ability of non-laboratory screening tools to correctly classify patients at the thresholds recommended by guidelines for initiating treatment, these tools are suitable to predict CVD risk in LMICs to reduce the cost regarding laboratory tests.

We developed SPARS in an analogous manner to the previous reported PARS model to provide user-friendly chart. Nevertheless, the color code has been revised to facilitate application compared with those of previous PARS model. Our chart differs from the new WHO chart, in that it includes waist to hip ratio instead of the body-mass index (BMI). In both PARS and SPARS models, WHR was a significant predictor of cardiovascular risk in our population.

Our user-friendly chart takes into account features of practicality, cost, and feasibility. Health workers can determine the risk value and treatment can be initiated in one clinic visit with minimum equipment, and less cost and time needed as there is no waiting time for laboratory results. Major global guidelines promote the use of multivariable risk models to guide treatment decisions. Decisions about whether to initiate pharmacological treatment in addition to lifestyle behaviors interventions as well as treatment intensity are guided by the level of the risk of these models. Considering risk assessment models in guidelines recommendations can be seen in many guidelines like hypertension or hyperlipidemia. Individuals at higher risk for CVD events require more intensive management. Conversely, low-risk individuals can be spared from the associated harms and high costs of overtreatment. The World Health Organization (WHO) has also recommended two packages of essential NCD interventions (WHO-PEN and HEARTS) with protocols that include simple and affordable tools such as CVD risk assessment charts for early detection and treatment [].

Moreover, in many LMICs, clinical guidelines are developed based on laboratory-based risk assessment models which are costly and guideline developers do not consider the availability of facilities or health-care workers to implement these laboratory-based screening or treatment guidelines [].

In the present study, PARS and SPARS prediction models were evaluated externally using data obtained from the TLGS study. HRs for major CVD risk factors were remarkably similar to those derived from the ICS, except, the HRs for males and for WHR were somewhat higher in TLGS. Both the original and the recalibrated PARS and SPARS functions discriminated well between individuals with CVD and without CVD in ICS and the TLGS. In the calibration analysis, slight overestimation was observed when the PARS and SPARS functions were applied directly to the TLGS. The calibration is generally poor, mainly owing to the differences in the relative risks associated with risk factors and the mean levels of the risk factors between the two cohorts []. To use a risk assessment tool optimally and to be acceptable for treatment guidelines, clinicians need to be confident that the absolute risk prediction functions can be generalized to other settings beyond where they were originally developed []. We have demonstrated that both PARS and SPARS prediction functions work reasonably well among the TLGS population.

To overcome over-estimation, we recalibrated both PARS and SPARS risk models based on national and more contemporary statistics from GBD applying new approach used in WHO models []. This approach involved few modeling steps [, , ]. Descriptive epidemiological data, including country-specific cardiovascular disease incidence to reflect changes in disease incidences and risk factor profiles can be readily incorporated to revise models [].

Recently, the WHO developed laboratory and non-laboratory risk charts, however they were only presented for regions and not for individual countries, although CVD risk differs between countries within some regions []. The risk chart developed in our study could be particularly useful for application in other countries in the EMR or other LMICs because many of these countries do not have locally developed risk scores based on their own cohort studies.

We also developed electronic and mobile application based on the SPARS risk assessment chart for easier use by general practitioners, physicians with related specialties like cardiologists and other health workers. All can rapidly implement a simple non-laboratory approach for initial screening, preventive, and treatment interventions without need for blood testing. Web-based program are freely accessible at http://www.prognosis.ir/Pars/index2.php. Other electronic and mobile application are under preparation to be used by the general public and provide them preventive recommendations.

Our study has several points of strength. First feature is the development of practical model merely based on non-laboratory measurement. Such simplified approaches could be used as part of stepwise approaches to help target laboratory testing in people most likely to benefit from the extra information and used even when values for some risk factors are unavailable for individuals in low-resource settings. A second is the recalibration approach we have used. It involves the use of GBD statistics. So, it can be easily developed and require fewer modelling steps. Third feature is that the risk model reported here can predict combined outcome of fatal and non-fatal events, thereby improving on risk calculators that predict fatal events alone. Predicting only fatal events in risk models significantly underestimates total CVD risk, particularly in populations with a low fatality rate [].

However, our study has some limitations. First, we did not include heart failure, vascular dementia, or peripheral vascular disease as outcomes, thus the overall cardiovascular risk may be underestimated, however, only few risk assessment models include these outcomes. Second, as in all cohort studies, some lost to follow-up participants was inevitable. The main reason for loss to follow-up in the ICS was related to phone number changes in some city blocks, which was done by the national government to develop and expand communication infrastructure. There were also some changes in address and only a few people were not willing to take part in multiple follow-ups. Therefore, loss to follow-up was likely to be largely random [, , ]. Finally, although the current study has broad coverage in comparison with other similar studies in Iran, recruitment was still limited to the central area of Iran.

In conclusion, we derived a non-laboratory-based cardiovascular risk prediction model using age, sex, smoking status, a self-reporting history of diabetes, blood pressure levels, and waist to hip ratio, all of which can be obtained in one outpatient visit. Such a user-friendly tool can help primary health care providers to determine the risk of cardiovascular disease (CVD) using an affordable screening tool. Remarkably, the simplified model was in close agreement with the laboratory-based model. We also validated and recalibrated both laboratory-based and non-laboratory-based models using mean risk factor levels based on the external cohort and country-specific estimates of annual incidence of CVD from GBD statistics, thereby enabling more accurate identification of individuals at high risk of cardiovascular disease in different settings.

Additional File

The additional file for this article can be found as follows:

Appendix

Methods A. DOI: https://doi.org/10.5334/gh.890.s1

Acknowledgements

This cohort study was conducted at the ICRC, affiliated with the Isfahan University of Medical Sciences (IUMS). We are grateful to the ICRC team, Isfahan Provincial Health Center, Najafabad Health Office and Arak University of Medical Sciences. The authors gratefully appreciate Dr. Davood Khalili and Prof. Fereidoun Azizi and the team of Research Institute for Endocrine Sciences for their valuable points and sharing Tehran Lipid and Glucose Study data for external validity. We also gratefully acknowledge the contribution of all participants of ICS and TLGS.

Funding Information

The baseline survey was supported by the undersecretary of research of the ministry of health and Isfahan cardiovascular research center, cardiovascular research institute affiliated to Isfahan University of Medical Sciences [grant number 31309304]. This work was also supported by the People Programme (Marie Curie Actions) of the European Union Seventh Framework Programme (FP7/2007–2013) [grant number 600388] and from the Agency for Business Competitiveness of the Government of Catalonia, ACCIO´.

Competing Interests

The authors have no competing interests to declare.

Author Contributions

RH and MM made a substantial contribution to the analysis and interpretation of data and drafted the article. HRM and MRM significantly contributed to analysis, acquisition, validation and preparing the risk chart and the corresponding web-page of the App. TAG and RTJ contributed to conception. EDA contributed to acquisition, analysis, and interpretation of data. NS made a substantial contribution to the conception and design. All authors critically revised manuscript for important intellectual content, provided final approval and agree to be accountable for all aspects of work ensuring integrity and accuracy and agreed to be named on the author list, and approved of the full author list.

Razieh Hassannejad and Marjan Mansourian contributed equally to this work.

References

Global health estimates: Deaths by cause, age, sex and country, 2000–2016. Geneva World Heal Organ; 2016.
Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics—2015 update: A report from the American Heart Association. Circulation. 2015; 131: e29–322. DOI: https://doi.org/10.1161/CIR.0000000000000152
Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet (London, England). 2012; 380: 2095–128. DOI: https://doi.org/10.1016/S0140-6736(12)61728-0
Roth GA, Johnson C, Abajobir A, et al. Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol. 2017; 70: 1–25. DOI: https://doi.org/10.1016/j.jacc.2017.04.052
Sarrafzadegan N, Mohammadifard N. Cardiovascular Disease in Iran in the Last 40 Years: Prevalence, Mortality, Morbidity, Challenges and Strategies for Cardiovascular Prevention. Arch Iran Med. 2019; 22.
Farshad Sharifi MD, Delavari A. The most prevalent causes of deaths, DALYs, and geriatric syndromes in Iranian elderly people between 1990 and 2010: Findings from the global burden of disease study 2010. Arch Iran Med. 2015; 18: 462.
World Health Organization (WHO). Technical package for cardiovascular disease management in primary health care: Evidence-based treatment protocols. Report; 2018.
Karmali KN, Persell SD, Perel P, Lloyd-Jones DM, Berendsen MA, Huffman MD. Risk scoring for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev; 2017. DOI: https://doi.org/10.1002/14651858.CD006887.pub4
Cooney MT, Dudina AL, Graham IM. Value and limitations of existing scores for the assessment of cardiovascular risk: A review for clinicians. J Am Coll Cardiol. 2009; 54: 1209–27. DOI: https://doi.org/10.1016/j.jacc.2009.07.020
Kaptoge S, Pennells L, De Bacquer D, et al. World Health Organization cardiovascular disease risk charts: Revised models to estimate risk in 21 global regions. Lancet Glob Heal. 2019; 7: e1332–45.
D’Agostino RB, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: The Framingham heart study. Circulation. 2008; 117: 743–53. DOI: https://doi.org/10.1161/CIRCULATIONAHA.107.699579
Conroy RM, Pyörälä K, Fitzgerald AP, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: The SCORE project. Eur Heart J. 2003; 24: 987–1003. DOI: https://doi.org/10.1016/S0195-668X(03)00114-3
Assmann G, Cullen P, Schulte H. Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the Prospective Cardiovascular Münster (PROCAM) study. Circulation. 2002; 105: 310–5. DOI: https://doi.org/10.1161/hc0302.102575
Goff DC, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014; 63: 2935–59. DOI: https://doi.org/10.1016/j.jacc.2013.11.005
Woodward M, Brindle P, Tunstall-Pedoe H. Adding social deprivation and family history to cardiovascular risk assessment: The ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC). Heart. 2007; 93: 172–6. DOI: https://doi.org/10.1136/hrt.2006.108167
Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: Prospective open cohort study. Br Med J. 2007; 335: 136–41. DOI: https://doi.org/10.1136/bmj.39261.471806.55
Hajifathalian K, Ueda P, Lu Y, et al. A novel risk score to predict cardiovascular disease risk in national populations (Globorisk): A pooled analysis of prospective cohorts and health examination surveys. Lancet Diabetes Endocrinol. 2015; 3: 339–55. DOI: https://doi.org/10.1016/S2213-8587(15)00081-9
Ueda P, Woodward M, Lu Y, et al. Laboratory-based and office-based risk scores and charts to predict 10-year risk of cardiovascular disease in 182 countries: A pooled analysis of prospective cohorts and health surveys. Lancet Diabetes Endocrinol. 2017; 5: 196–213. DOI: https://doi.org/10.1016/S2213-8587(17)30015-3
Beswick AD, Brindle P, Fahey T, Ebrahim S. A systematic review of risk scoring methods and clinical decision aids used in the primary prevention of coronary heart disease. R Coll Gen Pract [GRA]. 2008; 4: 105. DOI: https://doi.org/10.1002/14651858.CD006887
Gaziano TA, Young CR, Fitzmaurice G, Atwood S, Gaziano JM. Laboratory-based versus non-laboratory-based method for assessment of cardiovascular disease risk: The NHANES I Follow-up Study cohort. Lancet. 2008; 371: 923–31. DOI: https://doi.org/10.1016/S0140-6736(08)60418-3
Gaziano TA, Steyn K, Cohen DJ, Weinstein MC, Opie LH. Cost-effectiveness analysis of hypertension guidelines in South Africa: Absolute risk versus blood pressure level. Circulation. 2005; 112: 3569–76. DOI: https://doi.org/10.1161/CIRCULATIONAHA.105.535922
Sarrafzadegan N, Hassannejad R, Marateb HR, et al. PARS risk charts: A 10-year study of risk assessment for cardiovascular diseases in Eastern Mediterranean Region. PLoS One. 2017; 12: e0189389. DOI: https://doi.org/10.1371/journal.pone.0189389
Azizi F, Rahmani M, Emami H, et al. Cardiovascular risk factors in an Iranian urban population: Tehran lipid and glucose study (phase 1). Sozial-Und Präventivmedizin. 2002; 47: 408–26. DOI: https://doi.org/10.1007/s000380200008
Sarraf-Zadegan N, Sadri G, Malek-Afzali H, et al. Isfahan Healthy Heart Programme: A comprehensive integrated community-based programme for cardiovascular disease prevention and control. Design, methods and initial experience. Acta Cardiol. 2003; 58: 309–20. DOI: https://doi.org/10.2143/AC.58.4.2005288
Sarrafzadegan N, Talaei M, Sadeghi M, et al. The Isfahan cohort study: Rationale, methods and main findings. J Hum Hypertens. 2011; 25: 545–53. DOI: https://doi.org/10.1038/jhh.2010.99
Talaei M, Sarrafzadegan N, Sadeghi M, et al. Incidence of cardiovascular diseases in an Iranian population: The Isfahan cohort study. Arch Iran Med. 2013; 16: 138–44.
Sarrafzadegan N, Hassannejad R, Roohafza H, et al. A 10-year Isfahan cohort on cardiovascular disease as a master plan for a multi-generation non-communicable disease longitudinal study: Methodology and challenges. J Hum Hypertens; 2018. DOI: https://doi.org/10.1038/s41371-018-0126-2
Khalili D, Hadaegh F, Soori H, Steyerberg EW, Bozorgmanesh M, Azizi F. Clinical usefulness of the Framingham cardiovascular risk profile beyond its statistical performance: The Tehran Lipid and Glucose Study. Am J Epidemiol. 2012; 176: 177–86. DOI: https://doi.org/10.1093/aje/kws204
Chobanian AV, Bakris GL, Black HR, et al. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003; 42: 1206–52. DOI: https://doi.org/10.1161/01.HYP.0000107251.49515.c2
Abudukeremu N, Pan S, Ma Y, et al. The optimal cutoff point of waist-to-hip ratio for screening Uyghur population aged 35 years and over at high-risk of cardiovascular diseases in Xinjiang. Zhonghua Xin Xue Guan Bing Za Zhi. 2015; 43: 173–8.
Song B, Zhang G, Zhu W, Liang Z. ROC operating point selection for classification of imbalanced data with application to computer-aided polyp detection in CT colonography. Int J Comput Assist Radiol Surg. 2014; 9: 79–89. DOI: https://doi.org/10.1007/s11548-013-0913-8
Harrell FE, Jr, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15: 361–87. DOI: https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
D’agostino RB, Nam B-H. Evaluation of the performance of survival analysis models: Discrimination and calibration measures. Handb Stat. 2003; 23: 1–25. DOI: https://doi.org/10.1016/S0169-7161(03)23001-7
D’Agostino RB, Pencina MJ, Massaro JM, Coady S. Cardiovascular disease risk assessment: Insights from Framingham. Glob Heart. 2013; 8: 11–23. DOI: https://doi.org/10.1016/j.gheart.2013.01.001
Sepanlou SG, Malekzadeh R, Poustchi H, et al. The clinical performance of an office-based risk scoring system for fatal cardiovascular diseases in north-east of Iran. PLoS One. 2015; 10: 1–12. DOI: https://doi.org/10.1371/journal.pone.0126779
Zhao D, Liu J, Xie W, Qi Y. Cardiovascular risk assessment: A global perspective. Nat Rev Cardiol. 2015; 12: 301–11. DOI: https://doi.org/10.1038/nrcardio.2015.28
Liu J, Hong Y, D’Agostino RB, et al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-provincial Cohort Study. J Am Med Assoc. 2004; 291: 2591–9. DOI: https://doi.org/10.1001/jama.291.21.2591
Panagiotakos DB, Fitzgerald AP, Pitsavos C, Pipilis A, Graham I, Stefanadis C. Statistical modelling of 10-year fatal cardiovascular disease risk in Greece: The HellenicSCORE (a calibration of the ESC SCORE project). Hell J Cardiol. 2007; 48: 55–63.
World Health Organization (WHO). Global health estimates: Deaths by cause, age, sex and country, 2000–2015. Geneva World Heal Organ; 2016.

[B1] Global health estimates: Deaths by cause, age, sex and country, 2000–2016. Geneva World Heal Organ; 2016.

[B2] Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics—2015 update: A report from the American Heart Association. Circulation. 2015; 131: e29–322. DOI: https://doi.org/10.1161/CIR.0000000000000152

[B3] Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet (London, England). 2012; 380: 2095–128. DOI: https://doi.org/10.1016/S0140-6736(12)61728-0

[B4] Roth GA, Johnson C, Abajobir A, et al. Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J Am Coll Cardiol. 2017; 70: 1–25. DOI: https://doi.org/10.1016/j.jacc.2017.04.052

[B5] Sarrafzadegan N, Mohammadifard N. Cardiovascular Disease in Iran in the Last 40 Years: Prevalence, Mortality, Morbidity, Challenges and Strategies for Cardiovascular Prevention. Arch Iran Med. 2019; 22.

[B6] Farshad Sharifi MD, Delavari A. The most prevalent causes of deaths, DALYs, and geriatric syndromes in Iranian elderly people between 1990 and 2010: Findings from the global burden of disease study 2010. Arch Iran Med. 2015; 18: 462.

[B7] World Health Organization (WHO). Technical package for cardiovascular disease management in primary health care: Evidence-based treatment protocols. Report; 2018.

[B8] Karmali KN, Persell SD, Perel P, Lloyd-Jones DM, Berendsen MA, Huffman MD. Risk scoring for the primary prevention of cardiovascular disease. Cochrane Database Syst Rev; 2017. DOI: https://doi.org/10.1002/14651858.CD006887.pub4

[B9] Cooney MT, Dudina AL, Graham IM. Value and limitations of existing scores for the assessment of cardiovascular risk: A review for clinicians. J Am Coll Cardiol. 2009; 54: 1209–27. DOI: https://doi.org/10.1016/j.jacc.2009.07.020

[B10] Kaptoge S, Pennells L, De Bacquer D, et al. World Health Organization cardiovascular disease risk charts: Revised models to estimate risk in 21 global regions. Lancet Glob Heal. 2019; 7: e1332–45.

[B11] D’Agostino RB, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: The Framingham heart study. Circulation. 2008; 117: 743–53. DOI: https://doi.org/10.1161/CIRCULATIONAHA.107.699579

[B12] Conroy RM, Pyörälä K, Fitzgerald AP, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: The SCORE project. Eur Heart J. 2003; 24: 987–1003. DOI: https://doi.org/10.1016/S0195-668X(03)00114-3

[B13] Assmann G, Cullen P, Schulte H. Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the Prospective Cardiovascular Münster (PROCAM) study. Circulation. 2002; 105: 310–5. DOI: https://doi.org/10.1161/hc0302.102575

[B14] Goff DC, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014; 63: 2935–59. DOI: https://doi.org/10.1016/j.jacc.2013.11.005

[B15] Woodward M, Brindle P, Tunstall-Pedoe H. Adding social deprivation and family history to cardiovascular risk assessment: The ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC). Heart. 2007; 93: 172–6. DOI: https://doi.org/10.1136/hrt.2006.108167

[B16] Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P. Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: Prospective open cohort study. Br Med J. 2007; 335: 136–41. DOI: https://doi.org/10.1136/bmj.39261.471806.55

[B17] Hajifathalian K, Ueda P, Lu Y, et al. A novel risk score to predict cardiovascular disease risk in national populations (Globorisk): A pooled analysis of prospective cohorts and health examination surveys. Lancet Diabetes Endocrinol. 2015; 3: 339–55. DOI: https://doi.org/10.1016/S2213-8587(15)00081-9

[B18] Ueda P, Woodward M, Lu Y, et al. Laboratory-based and office-based risk scores and charts to predict 10-year risk of cardiovascular disease in 182 countries: A pooled analysis of prospective cohorts and health surveys. Lancet Diabetes Endocrinol. 2017; 5: 196–213. DOI: https://doi.org/10.1016/S2213-8587(17)30015-3

[B19] Beswick AD, Brindle P, Fahey T, Ebrahim S. A systematic review of risk scoring methods and clinical decision aids used in the primary prevention of coronary heart disease. R Coll Gen Pract [GRA]. 2008; 4: 105. DOI: https://doi.org/10.1002/14651858.CD006887

[B20] Gaziano TA, Young CR, Fitzmaurice G, Atwood S, Gaziano JM. Laboratory-based versus non-laboratory-based method for assessment of cardiovascular disease risk: The NHANES I Follow-up Study cohort. Lancet. 2008; 371: 923–31. DOI: https://doi.org/10.1016/S0140-6736(08)60418-3

[B21] Gaziano TA, Steyn K, Cohen DJ, Weinstein MC, Opie LH. Cost-effectiveness analysis of hypertension guidelines in South Africa: Absolute risk versus blood pressure level. Circulation. 2005; 112: 3569–76. DOI: https://doi.org/10.1161/CIRCULATIONAHA.105.535922

[B22] Sarrafzadegan N, Hassannejad R, Marateb HR, et al. PARS risk charts: A 10-year study of risk assessment for cardiovascular diseases in Eastern Mediterranean Region. PLoS One. 2017; 12: e0189389. DOI: https://doi.org/10.1371/journal.pone.0189389

[B23] Azizi F, Rahmani M, Emami H, et al. Cardiovascular risk factors in an Iranian urban population: Tehran lipid and glucose study (phase 1). Sozial-Und Präventivmedizin. 2002; 47: 408–26. DOI: https://doi.org/10.1007/s000380200008

[B24] Sarraf-Zadegan N, Sadri G, Malek-Afzali H, et al. Isfahan Healthy Heart Programme: A comprehensive integrated community-based programme for cardiovascular disease prevention and control. Design, methods and initial experience. Acta Cardiol. 2003; 58: 309–20. DOI: https://doi.org/10.2143/AC.58.4.2005288

[B25] Sarrafzadegan N, Talaei M, Sadeghi M, et al. The Isfahan cohort study: Rationale, methods and main findings. J Hum Hypertens. 2011; 25: 545–53. DOI: https://doi.org/10.1038/jhh.2010.99

[B26] Talaei M, Sarrafzadegan N, Sadeghi M, et al. Incidence of cardiovascular diseases in an Iranian population: The Isfahan cohort study. Arch Iran Med. 2013; 16: 138–44.

[B27] Sarrafzadegan N, Hassannejad R, Roohafza H, et al. A 10-year Isfahan cohort on cardiovascular disease as a master plan for a multi-generation non-communicable disease longitudinal study: Methodology and challenges. J Hum Hypertens; 2018. DOI: https://doi.org/10.1038/s41371-018-0126-2

[B28] Khalili D, Hadaegh F, Soori H, Steyerberg EW, Bozorgmanesh M, Azizi F. Clinical usefulness of the Framingham cardiovascular risk profile beyond its statistical performance: The Tehran Lipid and Glucose Study. Am J Epidemiol. 2012; 176: 177–86. DOI: https://doi.org/10.1093/aje/kws204

[B29] Chobanian AV, Bakris GL, Black HR, et al. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003; 42: 1206–52. DOI: https://doi.org/10.1161/01.HYP.0000107251.49515.c2

[B30] Abudukeremu N, Pan S, Ma Y, et al. The optimal cutoff point of waist-to-hip ratio for screening Uyghur population aged 35 years and over at high-risk of cardiovascular diseases in Xinjiang. Zhonghua Xin Xue Guan Bing Za Zhi. 2015; 43: 173–8.

[B31] Song B, Zhang G, Zhu W, Liang Z. ROC operating point selection for classification of imbalanced data with application to computer-aided polyp detection in CT colonography. Int J Comput Assist Radiol Surg. 2014; 9: 79–89. DOI: https://doi.org/10.1007/s11548-013-0913-8

[B32] Harrell FE, Jr, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15: 361–87. DOI: https://doi.org/10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4

[B33] D’agostino RB, Nam B-H. Evaluation of the performance of survival analysis models: Discrimination and calibration measures. Handb Stat. 2003; 23: 1–25. DOI: https://doi.org/10.1016/S0169-7161(03)23001-7

[B34] D’Agostino RB, Pencina MJ, Massaro JM, Coady S. Cardiovascular disease risk assessment: Insights from Framingham. Glob Heart. 2013; 8: 11–23. DOI: https://doi.org/10.1016/j.gheart.2013.01.001

[B35] Sepanlou SG, Malekzadeh R, Poustchi H, et al. The clinical performance of an office-based risk scoring system for fatal cardiovascular diseases in north-east of Iran. PLoS One. 2015; 10: 1–12. DOI: https://doi.org/10.1371/journal.pone.0126779

[B36] Zhao D, Liu J, Xie W, Qi Y. Cardiovascular risk assessment: A global perspective. Nat Rev Cardiol. 2015; 12: 301–11. DOI: https://doi.org/10.1038/nrcardio.2015.28

[B37] Liu J, Hong Y, D’Agostino RB, et al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-provincial Cohort Study. J Am Med Assoc. 2004; 291: 2591–9. DOI: https://doi.org/10.1001/jama.291.21.2591

[B38] Panagiotakos DB, Fitzgerald AP, Pitsavos C, Pipilis A, Graham I, Stefanadis C. Statistical modelling of 10-year fatal cardiovascular disease risk in Greece: The HellenicSCORE (a calibration of the ESC SCORE project). Hell J Cardiol. 2007; 48: 55–63.

[B39] World Health Organization (WHO). Global health estimates: Deaths by cause, age, sex and country, 2000–2015. Geneva World Heal Organ; 2016.