jbm > Volume 29(2); 2022 > Article
Mayer, Keevil, and Hansen: Concerns about Race and Ethnicity within the United States Fracture Risk Assessment Tool
Good predictive models are an asset in medicine. Ultimately, a model is a “simplification or approximation of reality”,[1,2] but by distilling complicated data into the chance of a given outcome occurring, such models support clinical and shared decision-making. Reliable model development and validation are crucial in creating a good predictive model, and this requires - among other things- formal statistical incorporation of each prognostically important variable into the model.[2-4]
We consider recent discussions surrounding the use of the race and/or ethnicity (R&E) in medicine to be of utmost importance.[5-7] We add to these discussions by noting the Fracture Risk Assessment Tool (FRAX®) is importantly problematic in its handling of R&E.
FRAX® formally incorporates many variables into a model aiming to predict a person’s 10-year risk of hip or major osteoporotic fracture (MOF). The variables include age, sex, body mass index, history of osteoporotic fracture, parental history of hip fracture, current smoking, current or previous glucocorticoid use for >3 months at a prednisolone (or equivalent) dose of ≥5 mg/day, rheumatoid arthritis, secondary osteoporosis, ≥3 servings of alcohol per day, and bone mineral density (BMD). In contrast to what FRAX® does for most countries, the FRAX® model for the USA (FRAX®-USA) offers different risk estimates based on R&E. Unfortunately, R&E was never formally evaluated as part of the modeling effort or incorporated into the FRAX®-USA model; thus, FRAX® failed to determine whether R&E have any independent predictive value when considered concurrently with all the other variables in FRAX®-USA. Instead, the base model was built using data from people who were predominantly White. If the user indicates the person is White, the base model risk is returned as is. For the options of Asian, Black, or Hispanic, FRAX®-USA applies to the base model post hoc “correction factors” (ranging from 0.43 to 0.64) that are de facto offsets that were derived outside the multivariable predictive model.[8-11] Such an approach is below established standards for modeling and results in R&E having extreme influence on predicted fracture risk.
For example, a female 65 years of age with a low-trauma wrist fracture who weighs 140 pounds, measures 65 inches tall, and has a femoral neck T-score of −2.5 is estimated to have a 21% risk of MOF in the next 10 years if she is White. Changing her race to Black reduces her risk by more than half (to 9.6%) due to the correction factor. For R&E to have such a profound impact raises serious skepticism. For example, the female who is Black would not reach the risk of a female who is White if one adds any other single dichotomous risk factor, increases her age to 85, or decreases her T-score to −3.5. Indeed, she would have to have a T-score of −4.0 instead of −2.5 for her MOF risk to be 21% (and recall this is someone who has already had a low-trauma fracture). Such an outsized impact compared with other variables seems implausible.
Although the authors of FRAX® have described much about their methods and results, they have never published or otherwise publicly shared precise details of the models themselves (e.g., coefficients). Such lack of transparency leaves users with an incomplete picture, meaning attempts to clarify the role of the correction factors (let alone the model as a whole) are limited to piecing together descriptions that appear in the literature and running test cases as just described. The opaque aspect of the FRAX® models stands in stark contrast both to expected practice in predictive modeling and to the tenet of transparency in science.[12]
Beyond use of post hoc correction factors, the studies used (Table 1) [8,13-20] to create the FRAX®-USA R&E correction factors heighten our concerns about use of R&E. Indeed, the studies do not even demonstrate there is a need for correction factors, as none demonstrate that R&E are independently associated with fracture after controlling for all the other variables FRAX® considers. For example, none of these studies had sufficient control for BMD. Instead, most studies only controlled for age and sex, or gave age- and/or sex-stratified estimates. In addition, some studies employed flawed methods of ascertaining R&E (e.g., by surname). The International Osteoporosis Foundation and International Society for Clinical Densitometry have also reviewed these correction factors.[10] However, their review - in large part, a comparison of the correction factors used by FRAX® with subsequent literature that examined ratios of fracture incidence in people who are Asian, Black, or Hispanic compared with people who are White - is insufficient and unconvincing to corroborate FRAX®’s use of correction factors. Even if there were evidence that R&E retained some association or predictive capacity after sufficient control for all other potentially mediating variables, R&E would still need to be formally incorporated - using accepted methods - into a multivariable modeling effort. Finally, even if R&E were independently associated with fracture after controlling for all other variables in FRAX®, such a finding could reflect important health disparities, but assuming R&E modifies fracture risk at the level of the individual is at serious risk of ecological fallacy and could perpetuate health disparities rather than help them.
We emphasize the increasing (and overdue) recognition that R&E are poor surrogates for individual health outcomes and that they have fundamentally different implications for population health analyses than they do for predictive models or clinical algorithms.[5-7] We have heard defenses of the R&E correction factors by invocation of various arguments, ranging from arguments that effectively depend on the existence of biologic differences that underpin fracture risk to suggesting the risk of falling (and therefore, by extension, fracture) varies by R&E. However, we consider these arguments either fundamentally flawed or demonstrative of poor insight into clinical epidemiology and best modeling practices. For example, even if fall risk is associated with R&E, why would one want to model R&E instead of fall risk? Surely, there would also be challenges with including fall risk as a variable, but this seems rather better than attempting to use R&E as a surrogate for fall risk.
We are not the first to question the FRAX®-USA model’s use of R&E,[5,6] and even while preparing this work, another group called into question the FRAX®-USA model.[21] We are also aware of the unconvincing response from the FRAX® authors [22] to some of these critiques [5] that resulted in no change to FRAX®-USA. We are encouraged by the creation of an American Society for Bone and Mineral Research task force to reassess the inclusion of R&E in FRAX®.[23] We hope the concepts added by our critique will support collegial steps toward productive remedy, akin to recent removal of R&E from the prediction of vaginal birth after cesarean delivery and estimation of glomerular filtration rate.[24,25] We also urge the authors of FRAX® to provide a full, transparent accounting for all critical components of each of its models.[12]
Until then, we call for clinicians to reject the FRAX®-USA model’s correction factors that lead to different fracture risk estimates based on R&E. The best interim solution, if using FRAX®-USA, may be to use the base model for all people, though we consider a dedicated effort sincerely considering the concepts delineated herein to be critical, including exploration of reparameterization in a more contemporary and diverse population. Likewise, although we focused on FRAX®-USA, we note FRAX® forwards different estimates for different groups in other geographic areas (e.g., South Africa and Singapore); thus, such models should receive similar scrutiny. In summary, we call for a moratorium on use of R&E within FRAX®-USA.

DECLARATIONS

Conflict of interest

No potential conflict of interest relevant to this article was reported. The authors submit this in their personal capacity, with the views and opinions expressed herein being those of the authors and not necessarily those of their employers or any related entity.

Table 1
Abridged summary of studies FRAX® cites in purported support of race and ethnicity “correction factors”
Reference Year Notes
Bauer [13] 1988 “Ethnicity” (“Hispanic, Black, non-Hispanic, White, or Asian”) determined by surname and data from admission; presents age- and sex-stratified estimates
Silverman and Madison [14] 1988 Presents age-adjusted estimates and age- and sex-stratified estimates
Jacobsen et al. [15] 1990 Presents age-adjusted estimates and age- and sex-stratified estimates
Griffin et al. [16] 1992 Controlled for age, sex, and nursing home residence
Baron et al. [17] 1994 Controlled for age and sex (with interaction term for sex and race); limited to people who were Black or White
Lauderdale et al. [18] 1997 “Cohorts of Whites and Blacks” formed by “race/ethincity codes for Whites and Blacks and excluding those with Spanish surnames”; inferred Asian ancestry by surname among those with “race codes Asian American and other”; presents age- and sex-stratified estimates; also present multivariable-adjusted estimates as part of assessing immigration year for those they categorized as Asian American (sex; age in 5-year increments; whether the person was categorized as Japanese, Chinese, or Korean based on their surname; and immigration year before 1965 or 1965 or later)
Lauderdale et al. [19] 1998 Inferred Hispanic ethnicity via surname “regardless of race/ethincity code” and inferred White and Black race by “excluding those with Spanish surnames”; presents age-adjusted estimates and age- and sex-stratified estimates
Zingmond et al. [20] 2004 Race and ethnicity determined by hospital discharge record; presents age-, sex-, and year-stratified estimates

Because all studies sought to examine fracture in different race and/or ethnicity (R&E) groups, the abridged summaries present information on how R&E were measured as part of that effort, but they do not reiterate the interest in examining R&E when discussing the studies’ results unless that is critical for clarity. For example, an abridged summary of a study that presents results stratified by age, sex, and R&E would note results were presented with stratification by age and sex.

REFERENCES

1. von Neumann J. The mathematician. In: Heywood RB, editors. The works of the mind. Chicago, IL: University of Chicago Press; 1947. p.180. -96.

2. Heinze G, Wallisch C, Dunkler D. Variable selection - A review and recommendations for the practicing statistician. Biom J 2018;60:431-49. https://doi.org/10.1002/bimj.201700067 .
crossref pmid pmc pdf
3. Wynants L, Collins GS, Van Calster B. Key steps and common pitfalls in developing and validating risk models. BJOG 2017;124:423-32. https://doi.org/10.1111/1471-0528.14170 .
crossref pmid pdf
4. Royston P, Moons KG, Altman DG, et al. Prognosis and prognostic research: Developing a prognostic model. BMJ 2009;338:b604. https://doi.org/10.1136/bmj.b604 .
crossref pmid
5. Vyas DA, Eisenstein LG, Jones DS. Hidden in plain sight - Reconsidering the use of race correction in clinical algorithms. N Engl J Med 2020;383:874-82. https://doi.org/10.1056/NEJMms2004740 .
crossref pmid
6. Cerdeña JP, Plaisime MV, Tsai J. From race-based to race-conscious medicine: how anti-racist uprisings call us to act. Lancet 2020;396:1125-8. https://doi.org/10.1016/s0140-6736(20)32076-6 .
crossref pmid pmc
7. Ioannidis JPA, Powe NR, Yancy C. Recalibrating the use of race in medical research. JAMA 2021;325:623-4. https://doi.org/10.1001/jama.2021.0003 .
crossref pmid
8. Kanis JA. on behalf of the World Health Organization Scientific Group. Assessment of osteoporosis at the primary health-care level. Technical Report. University of Sheffield, UK: World Health Organization Collaborating Centre for Metabolic Bone Diseases, University of Sheffield; 2007.

9. Kanis JA, Johansson H, Oden A, et al. The effects of a FRAX revision for the USA. Osteoporos Int 2010;21:35-40. https://doi.org/10.1007/s00198-009-1033-8 .
crossref pmid pdf
10. Cauley JA, El-Hajj Fuleihan G, Arabi A, et al. Official positions for FRAX® clinical regarding international differences from joint official positions development conference of the International Society for Clinical Densitometry and International Osteoporosis Foundation on FRAX®. J Clin Densitom 2011;14:240-62. https://doi.org/10.1016/j.jocd.2011.05.015 .
pmid
11. Leslie WD, Morin SN, Lix LM, et al. Fracture prediction from FRAX for Canadian ethnic groups: a registry-based cohort study. Osteoporos Int 2021;32:113-22. https://doi.org/10.1007/s00198-020-05594-8 .
crossref pmid pmc pdf
12. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55-63. https://doi.org/10.7326/m14-0697 .
crossref pmid
13. Bauer RL. Ethnic differences in hip fracture: a reduced incidence in Mexican Americans. Am J Epidemiol 1988;127:145-9. https://doi.org/10.1093/oxfordjournals.aje.a114774 .
crossref pmid
14. Silverman SL, Madison RE. Decreased incidence of hip fracture in Hispanics, Asians, and blacks: California Hospital Discharge Data. Am J Public Health 1988;78:1482-3. https://doi.org/10.2105/ajph.78.11.1482 .
crossref pmid pmc
15. Jacobsen SJ, Goldberg J, Miles TP, et al. Hip fracture incidence among the old and very old: a population-based study of 745,435 cases. Am J Public Health 1990;80:871-3. https://doi.org/10.2105/ajph.80.7.871 .
crossref pmid pmc
16. Griffin MR, Ray WA, Fought RL, et al. Black-white differences in fracture rates. Am J Epidemiol 1992;136:1378-85. https://doi.org/10.1093/oxfordjournals.aje.a116450 .
crossref pmid
17. Baron JA, Barrett J, Malenka D, et al. Racial differences in fracture risk. Epidemiology 1994;5:42-7. https://doi.org/10.1097/00001648-199401000-00008 .
crossref pmid
18. Lauderdale DS, Jacobsen SJ, Furner SE, et al. Hip fracture incidence among elderly Asian-American populations. Am J Epidemiol 1997;146:502-9. https://doi.org/10.1093/oxfordjournals.aje.a009304 .
crossref pmid
19. Lauderdale DS, Jacobsen SJ, Furner SE, et al. Hip fracture incidence among elderly Hispanics. Am J Public Health 1998;88:1245-7. https://doi.org/10.2105/ajph.88.8.1245 .
crossref pmid pmc
20. Zingmond DS, Melton LJ 3rd, Silverman SL. Increasing hip fracture incidence in California Hispanics, 1983 to 2000. Osteoporos Int 2004;15:603-10. https://doi.org/10.1007/s00198-004-1592-7 .
crossref pmid pdf
21. Reid HW, Selvan B, Batch BC, et al. The break in FRAX: Equity concerns in estimating fracture risk in racial and ethnic minorities. J Am Geriatr Soc 2021;69:2692-5. https://doi.org/10.1111/jgs.17316 .
crossref pmid pdf
22. Kanis JA, Cooper C, Dawson-Hughes B, et al. FRAX and ethnicity. Osteoporos Int 2020;31:2063-7. https://doi.org/10.1007/s00198-020-05631-6 .
crossref pmid pmc pdf
23. American Society for Bone and Mineral Research. ASBMR creates task force to reassess the inclusion of race in determining bone fracture risk in patients 2021 [cited by 2021 Jun 24]. Available from: https://www.asbmr.org/about/news-release-detail/asbmr-creates-task-force-to-reassess-inclusion-of- .

24. Grobman WA, Sandoval G, Rice MM, et al. Prediction of vaginal birth after cesarean delivery in term gestations: a calculator without race and ethnicity. Am J Obstet Gynecol 2021;225:664e1-7. https://doi.org/10.1016/j.ajog.2021.05.021 .
crossref pmid
25. Inker LA, Eneanya ND, Coresh J, et al. New creatinine- and cystatin C-based equations to estimate GFR without race. N Engl J Med 2021;385:1737-49. https://doi.org/10.1056/NEJMoa2102953 .
crossref pmid pmc
TOOLS
METRICS Graph View
  • 1 Crossref
  •  0 Scopus 
  • 2,084 View
  • 58 Download
ORCID iDs

Martin Mayer
https://orcid.org/0000-0003-3210-2274

Jon Keevil
https://orcid.org/0000-0002-9288-0097

Karen E. Hansen
https://orcid.org/0000-0003-4338-6073

Related articles


ABOUT
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
EDITORIAL POLICY
FOR CONTRIBUTORS
Editorial Office
#1001, Hyundai Kirim Officetel, 42 Seocho-daero 78-gil, Seocho-gu, Seoul 06626, Korea
Tel: +82-2-3473-2231    Fax: +82-70-4156-2230    E-mail: editors.jbm@gmail.com                

Copyright © 2024 by The Korean Society for Bone and Mineral Research.

Developed in M2PI

Close layer
prev next