If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author. Department of Orthopaedic Surgery, University Connecticut Health Center, 263 Farmington Avenue, Farmington, CT 06030, USA. Tel.: +1 860 679 2200.
The purpose of this study was to determine the optimal window for collection of patient-reported outcome measures (PROMs) after total joint arthroplasty (TJA).
Methods
Our prospectively collected institutional joint registry was queried for patients who underwent primary, unilateral TJAs. The primary outcomes were the net changes in WOMAC, SF-12 MCS, SF-12 PCS, OHS, KSCRS, and UCLA activity rating system at 6, 12, and 24 months postoperatively. Secondary outcomes were data acquisition costs and follow-up attrition rates.
Results
Eight hundred sixty-six procedures (450 total hip arthroplasties, 416 TKAs) were analyzed. A consistent plateau in all PROMs was noted by 6 months postoperatively–except for SF-12 MCS which showed no significant changes at any time interval. For TKA, the percentage of overall improvement achieved by 6 months was 88.7%, 84.5%, 100%, and 90.5% for the WOMAC, SF-12 PCS, UCLA, and KSCRS, respectively. For total hip arthroplasty, these values were 92.7%, 83.5%, 88.0%, and 89.8% for WOMAC, SF-12 PCS, UCLA, and OHS, respectively. There were marginal improvements from 6 to 12 months and no improvement from 12 to 24 months. Follow-up rates at 6, 12, and 24 months were 85%, 69%, and 40%, respectively. Our institutional costs for collecting a complete data set per patient were $128, $158, and $272 for 6, 12, and 24 months, respectively.
Conclusions
Most of the improvement in PROMs after primary TJA occurs within the first 6 months. In addition, limiting PROMs collection to 6 months appears to be cost-efficient owing to increased attrition rates beyond this time interval.
Total joint arthroplasty (TJA) is one of the most commonly performed orthopedic procedures and the utilization is projected to increase 4-fold by 2030 [
Patient-reported outcome measures in arthroplasty registries Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries Part II. Recommendations for selection, administration, and analysis.
]. This has significant administrative, research, clinical, and financial implications, which makes understanding PROMs behavior in the postoperative period essential [
Value-based payment arrangements that reward quality and efficiency of care are beneficial to both surgeons and patients. However, it is important that as physicians we lead the charge in defining precedents for proper use and implementation of PROMs, else this responsibility will fall to alternative stakeholders [
]. With the increase in PROMs utilization, it is prudent to develop a detailed understanding of their responsiveness to surgery such that implementation can be executed in a responsible, cost-effective, and mutually beneficial manner for all stakeholders. Although many validated PROMs exist, there remains no gold standard measure, alluding to the complexity of the task at hand and need for increased research and understanding [
Patient-reported outcome measures in arthroplasty registries Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries Part II. Recommendations for selection, administration, and analysis.
]. In addition, it remains unclear how long PROMs need to be collected after surgery. Specifically, at what time point following TJA do PROMs begin to reflect an accurate estimate of “success”? The answer to this question has several implications. First, it allows us to select an optimal time point in the postoperative period at which PROMs reasonably reflect the value added to patients. Second, this information has implications on research design and publications in journals that may require longer PROMs follow-up intervals, which may prove to be unnecessary [
]. Third, if performance and reimbursement are to be determined based on PROMs, it is crucial that the timing of PROMs collection is such that the results provided represent the most accurate estimate of value added [
The primary objective of this study is to present our 5-year joint registry experience collecting a panel of commonly used PROMs at a single tertiary care center in the United States. Focus was placed on when various PROMs peaked during the postoperative period and the associated registry maintenance costs, with the intention of better defining the optimal timing for PROMs collection following TJA. We hypothesized that significant improvements in PROMs occur early in the postoperative period, which may afford significant cost-saving opportunities.
Material and methods
Institutional Review Board approval was obtained. Our prospectively collected institutional joint registry was queried for patients who underwent primary hip and knee arthroplasty with a minimum of follow-up of 6 months. Patients undergoing nonelective, bilateral, unicompartmental, revision, or tumor-related procedures were excluded. In total, 866 procedures met these criteria and were included in the study: 450 total hip arthroplasties (THAs) and 416 total knee arthroplasties (TKAs). Procedures were performed between February 2012 and August 2017. Patient inclusion in the joint registry required consent and willingness to participate in regular surveys. All procedures were performed by 5 fellowship-trained arthroplasty surgeons. Patient characteristics collected in the study were age, sex, body mass index, American Society of Anesthesiologist physical classification system score, educational level, marital status, ethnicity, smoking status, and alcohol use. Patients had baseline Short Form-12 (SF-12) mental and physical component scores (MCS and PCS), Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), and University of California Los Angeles (UCLA) activity scores. Patients undergoing THA also had baseline Oxford Hip Score (OHS) and those undergoing TKA had baseline Knee Society Clinical Rating System (KSCRS) scores. All PROMs were then routinely collected at up to 6 months postoperatively and again at 12 and 24 months.
The WOMAC is a 24-item questionnaire used to assess the severity of osteoarthritis of the hip and knee [
Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee.
]. The SF-12, an abbreviated version of the original Short Form 36, is a measure of general health that can be used to generate physical and mental composite scores [
]. The UCLA activity rating is a single question measure of a patient’s overall activity level. The patient selects 1 of 10 distinct activity levels, 1 being the least and 10 the most active [
]. It is unique in that the Knee Score contains an objective physician-derived component, while the Function Score is derived purely from the patient. For the purpose of simplification, the combined score was reported in this study. The OHS is a 12-item questionnaire specific to assessing function and pain for patients undergoing THA [
]. Although no MCID has been defined for UCLA activity rating, it can be assumed that an increase of 1 level constitutes a meaningful clinical improvement given the structure of the questionnaire.
At our institution, data collection, entry, and management are performed by a dedicated research assistant. The research assistant personally administers the questionnaires to patients via pen and paper method during joint class before surgery and again when patients return to their postoperative visits. Patients who do not show up to their appointments receive reminder phone calls. Due to the number and length of questionnaires, these are not collected over phone. The registry data are housed and maintained by a data management vendor (Ortech, London, Canada). Costs collected in this study included the salary for the research assistant paid at 0.75 full-time equivalent (FTE) as well as licensing and data maintenance fees. The 0.75 FTE was determined based on projected time needed to collect 4 datasets per patient (baseline and again at 6, 12, and 24 months postoperatively) for an estimated 300 patients per year after the commencement year. For situations where the research assistant was not available, 2 support research personnel were available for coverage. To encourage patient participation in the registry, the purpose/benefit of collecting outcomes data along with time commitment for questionnaire completion was discussed with patients by their surgeons. Patients were encouraged to sign up at the time of their joint class. Enrollment was limited to patients who did not require interpretation services.
Data were summarized with mean and standard deviation for continuous variables and frequency and proportion for categorical variables. Changes in PROMs over time were analyzed with mixed effects linear regression. The alpha level for all analyses was set at 0.05.
Results
In total, 866 patients undergoing TJA (450 THAs and 416 TKAs) were included in the analysis. Table 1 summarizes the baseline demographic characteristics for the study group.
Table 1Baseline demographic characteristics of the study group.
Age (y)
60.2 (±12.2)
ASA
2.3 (±0.5)
BMI
31.5 (±5.5)
Sex
Female
443 (51%)
Male
423 (49%)
Procedure
Total hip
450 (52%)
Total knee
416 (48%)
Education
Secondary
406 (49%)
Higher
422 (51%)
Marital status
Married or living with significant other
378 (45%)
Not living with significant other
461 (55%)
Ethnicity
White
657 (76%)
Black/African American
126 (15%)
Hispanic/Latino
53 (6%)
Other
30 (3%)
Smoking
Yes
47 (12%)
No
357 (88%)
Alcohol
None/occasional
716 (86%)
Daily
117 (14%)
Back pain
No
373 (43%)
Yes
493 (57%)
ASA, American Society of Anesthesiologists; BMI, body mass index.
Values are given as mean and standard deviation or frequency and proportion.
The WOMAC improved by an average of 36.3 points at 6 months (P < .001). A clinically insignificant improvement of 4.1 points (P = .002) was observed between 6 and 12 months and no changes thereafter (P = .736). Similarly, the SF-12 PCS experienced its greatest increase, 14.3 points, by 6 months (P < .001). No significant changes were observed at 12 or 24 months (P = .058 and .237, respectively). The SF-12 MCS did not demonstrate improvement at any time interval (P = .912, .435, and .747 at 6, 12, and 24 months, respectively). The UCLA activity score increased by 1 point (P < .001) at 6 months and no changes were found beyond that point (P = .567 and .991 at 12 and 24 months, respectively). Finally, the KSCRS experienced the greatest improvement (62.2 points) by 6 months (P < .001). A clinically insignificant change of 9.3 points (P = .001) was observed between 6 and 12 months with no changes beyond 12 months (P = .501). The percentage of overall improvement achieved by 6 months was 88.7%, 84.5%, 100%, and 90.5% for the WOMAC, SF-12 PCS, UCLA, and KSCRS, respectively. Table 2 details the PROM scores over time for TKA. A subanalysis for implants used revealed minimal differences that were neither statistically nor clinically significant.
Table 2Patient-reported outcome scores over time for total knee arthroplasty.
Outcome measure/time interval
Mean
95% Confidence interval
P value
WOMAC
Baseline
55.6
53.7-57.6
6 mo
19.4
17.3-21.5
<.001
12 mo
15.3
13.0-17.7
.002
24 mo
14.7
11.4-18.1
.736
SF-12 PCS
Baseline
28.2
27.1-29.3
6 mo
42.5
41.4-43.7
<.001
12 mo
43.9
42.6-45.2
.058
24 mo
45.1
43.2-46.9
.237
SF-12 MCS
Baseline
56.4
55.3-57.4
6 mo
56.3
55.2-57.4
.912
12 mo
55.8
54.5-57.1
.435
24 mo
55.5
53.8-57.2
.747
UCLA
Baseline
4.3
4.1-4.5
6 mo
5.3
5.1-5.5
<.001
12 mo
5.3
5.1-5.5
.567
24 mo
5.3
5.0-5.6
.991
KSCRS
Baseline
87.4
82.7-92.0
6 mo
149.6
145.1-154.1
<.001
12 mo
158.9
153.8-164.1
.001
24 mo
156.1
148.5-163.8
.501
P values represent the statistical comparison of the indicated time interval compared to the previous time interval. Lower WOMAC scores indicate better health.
The WOMAC improved by an average of 46.4 points at 6 months (P < .001). A clinically insignificant improvement of 3.2 points (P = .017) was observed between 6 and 12 months and no changes thereafter (P = .788). Similarly, the SF-12 PCS experienced its greatest increase, 17.1 points, by 6 months (P < .001). A clinically insignificant increase of 2.1 points (P = .011) was observed between 6 and 12 months. The SF-12 MCS experienced significant improvement by 6 months (P < .001) although this change was not clinically significant. No improvements in SF-12 MCS were observed at 12 or 24 months (P = .578 and .122, respectively). The UCLA activity score experienced an average increase of 1.7 points (P < .001) at 6 months and no further changes at 12 or 24 months (P = .376 and .373, respectively). The OHS improved by an average of 22.7 points (P < .001) at 6 months. Improvements between 6 and 12 months were below not clinically significant (1.9 points, P = .001) and no changes were observed at 24 months (P = .419). The percentage of overall improvement achieved by 6 months was 92.7%, 83.5%, 88.0%, and 89.8% for WOMAC, SF-12 PCS, UCLA, and OHS, respectively. Table 3 summarizes the PROM scores over time for THA. Figure 1 details the changes in PROMs over time for TKA and THA. A subanalysis for surgical approach or implants used revealed minimal differences that were neither statistically nor clinically significant.
Table 3Patient-reported outcome scores over time for total hip arthroplasty.
Outcome measure/time interval
Mean
95% Confidence interval
P value
WOMAC
Baseline
62.9
61.0-64.8
6 mo
16.5
14.4-18.6
<.001
12 mo
13.3
11.0-15.7
.017
24 mo
12.8
9.5-16.2
.788
SF-12 PCS
Baseline
26.4
25.4-27.3
6 mo
43.5
42.4-44.6
<.001
12 mo
45.4
44.1-46.6
.011
24 mo
46.9
45.1-48.6
.130
SF-12 MCS
Baseline
52.7
51.6-53.9
6 mo
55.9
54.7-57.2
<.001
12 mo
55.5
54.1-56.9
.578
24 mo
53.8
51.9-55.8
.122
UCLA
Baseline
3.7
3.5-3.8
6 mo
5.4
5.2-5.6
<.001
12 mo
5.5
5.3-5.7
.376
24 mo
5.7
5.4-6.0
.373
OHS
Baseline
43.5
42.7-44.4
6 mo
20.8
19.9-21.7
<.001
12 mo
18.9
17.8-19.9
.001
24 mo
18.2
16.7-19.7
.419
P values represent the statistical comparison of the indicated time interval compared to the previous time interval. Lower WOMAC and OHS scores indicate better health.
Figure 1Changes in patient-reported outcomes plotted over time demonstrate a consistent plateau effect at 6 mo postoperatively. (a) Western Ontario and McMaster Universities Osteoarthritis Index; (b) Short Form-12 physical component summary; (c) Short Form-12 mental component summary; (d) University of California Los Angeles activity level rating; (e) Oxford hip score; (f) Knee Society clinical rating system. Lower WOMAC and OHS indicate better health.
Within the study period, there was a clear pattern that the MCIDs for all assessed PROMs mostly occurred within the first 6 months after surgery. Table 4, Table 5 describe the percentage of patients meeting the MCIDs over time for THA and TKA respectively.
Table 4Percentage of patients meeting the minimum clinically significant differences over time for total hip arthroplasty.
Follow-up rates and costs associated with PROMs collection
Our annual institutional costs for collecting PROMs data were $78,500. These included the salary for a dedicated research assistant paid at 0.75 FTE ($75,000) and licensing/data maintenance fees ($3500). Based on a target enrollment of 300 patients per year with 4 datasets/patients (1200 datasets per year), the allocated cost per dataset was $65.42. There was significant loss to follow-up at each successive time interval postoperatively. Follow-up rates were 85%, 69%, and 40% at 6, 12, and 24 months, respectively. Fifteen percent of patients opted out of the registry within the first 6 months after surgery. Loss to follow-up beyond 6 months was primarily due to patients not returning to their follow-up visits despite receiving reminder phone calls. As a result, the actual cost per dataset increased to $128, $158, and $272 at 6, 12, and 24 months, respectively. Table 6 summarizes our annual return on investment (allocated costs divided by number of completed datasets) for collecting PROMs, which decreased significantly over time.
Table 6Annual return on investment for collecting outcomes data in primary total joint arthroplasty.
The total allocated costs reflect the salary of a dedicated research assistant paid at 0.75 full-time equivalent ($75,000) and data maintenance/licensing fees ($3500).
Patient participation in the database is voluntary and requires willingness to participate in surveys during follow-up visits. Patients can opt out at any time if they wish.
Cost per dataset is derived by dividing the total costs for 3 time periods (0-6 mo, 0-12 mo, or 0-24 mo) by the number of complete datasets generated during each period.
Annual investment loss
Baseline
300
180
$65.42
$109
−$7850
6 mo
300
153
$65.42
$128
−$9616.74
12 mo
300
124
$65.42
$158
−$11,513.92
24 mo
300
72
$65.42
$272
−$14,915.76
Total
1200
529
$65.42
$148
−$43,896.82
a The total allocated costs reflect the salary of a dedicated research assistant paid at 0.75 full-time equivalent ($75,000) and data maintenance/licensing fees ($3500).
b Patient participation in the database is voluntary and requires willingness to participate in surveys during follow-up visits. Patients can opt out at any time if they wish.
c Actual number of datasets is based on actual follow-up rates of 100%, 85%, 69%, and 40% at baseline, 6 mo, 12 mo, and 24 mo, respectively.
d Cost per dataset is derived by dividing the total costs for 3 time periods (0-6 mo, 0-12 mo, or 0-24 mo) by the number of complete datasets generated during each period.
This study provided evidence that the vast majority of clinically relevant improvement in a wide range of PROMs following primary TJA occurred within the first 6 months postoperatively. With the exception of the SF-12 MCS, which underwent minimal change from baseline throughout the 2-year follow-up period, all other PROMs experienced significant improvements by 6 months after surgery with only marginal changes thereafter. In addition, there was significant increase in registry costs combined with high rate of follow-up loss beyond 6 months. These findings suggest that limiting PROMs collection to 6 months postoperatively appears to be a cost-efficient and clinically relevant mark to assess the quality of care delivered.
Our findings are consistent with previous reports. Ramkumar et al. [
] conducted a systematic review and meta-analysis of 15 reports on PROMs after TJA. The group found that there were no statistically significant changes in Harris Hip Score, WOMAC, Knee Society Score, and SF scores (SF-12, SF-36, and SF-6D) from 12 to 24 months. In our study, we found that the improvements in outcomes began to plateau even sooner, as early as 6 months. Kagan et al. [
The recovery curve for the patient-reported outcomes measurement information system patient-reported physical function and pain interference computerized adaptive tests after primary total knee arthroplasty.
] prospectively reviewed 91 patients undergoing primary TKA using Patient-Reported Outcomes Measurement Information System physical function and pain interference tests. The authors found that 89% of improvement in physical function and 90% of pain interference occurred by 6 months after surgery. Data from international studies also support this trend. Van der Wees et al. [
] collected Harris Hip Score, OHS, WOMAC, and Knee Society Score Function Score following TJA at 3, 6, and 12 months postoperatively. The group showed a plateau in improvement for all four PROMs around 6 months postoperatively. In another study, Dailiana et al. [
] reported on WOMAC scores following TJA at 6 weeks as well as 3, 6, and 12 months postoperatively. The authors likewise showed a plateau effect around 6 months after surgery.
The data presented in this study by no means rules out continued clinical improvement beyond 6 months postoperatively. However, it provides evidence that a patient’s response to TJA at 6 months may be a good indicator of value of the surgical intervention as the vast majority of improvement in PROMs was reached by this time point. The possibility of making accurate estimates of overall outcomes at an earlier time point has multiple implications. The collection of PROMs and maintenance of a total joint registry is no trivial matter and poses many obstacles both financially and logistically as has been described by others [
]. Minimizing the number of data collection points, as well as collecting data at an earlier time point should help make maintaining a registry a more feasible and affordable feat, hopefully incentivizing more centers to collect such data. Measuring outcomes at 6 months rather than 12 or 24 months also provides a distinct advantage of increased response rates as our study showed a significant attrition beyond 6 months. It should be noted that our findings pertain to short-term outcome collection. This does not preclude the importance of obtaining mid- and long-term data, as this may help provide an additional metric for clinical performance and implant surveillance as some of the major modes of failure typically occur beyond the short-term window [
A central challenge identified by this study is how to improve survey completion rates. The first important area for improvement is minimizing the burden placed on both the patient and the institution to complete the surveys. Fifteen percent of our patients opted out of the registry due to the time-consuming nature of these surveys. Limiting the data collection to 1 general health questionnaire and 1 disease-specific questionnaire may help decrease survey fatigue as well as administrative burden for data entry and maintenance. A second area for improvement is administrating the surveys in a digital platform that allows patients to complete them at their convenience. Our pen and paper method proved to be not cost-effective at later time points as it required both the patient and the research assistant to be present to ensure the questionnaires are fully answered. However, as our study and previous reports indicate [
The recovery curve for the patient-reported outcomes measurement information system patient-reported physical function and pain interference computerized adaptive tests after primary total knee arthroplasty.
], evidence to date show that limiting data collection to 6 months is both clinically relevant and cost-efficient.
This study has some important limitations. The results reflect the experience of a single public academic institution which treats a patient population with increased disease burden. Its conclusions are dependent on collection of several questionnaires using the pen and paper method. As discussed, this approach resulted in a significant attrition rate at 12 and 24 months. The implementation of alternative methods such as simplified questionnaires and electronic administration may improve response rates and has the potential to alter the trends seen in this study. This could also have effects on registry maintenance costs and ultimately return on investment.
Conclusions
In conclusion, this study adds support to previous reports that most of the improvement in PROMs after primary TJA occurs within the first 6 months. In addition, limiting PROMs collection to 6 months appears to be cost-efficient due to increased attrition rates beyond this time interval. Although this conclusion is limited to the data collection method we used, it sheds light on an important and elusive element in PROMs collection—defining the optimal collection window! Future research on PROMs should bridge this gap in our knowledge given its important clinical, economic, and health policy implications in a value-based healthcare system.
Patient-reported outcome measures in arthroplasty registries Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries Part II. Recommendations for selection, administration, and analysis.
Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee.
The recovery curve for the patient-reported outcomes measurement information system patient-reported physical function and pain interference computerized adaptive tests after primary total knee arthroplasty.
One or more of the authors of this paper have disclosed potential or pertinent conflicts of interest, which may include receipt of payment, either direct or indirect, institutional support, or association with an entity in the biomedical field which may be perceived to have potential conflict of interest with this work. For full disclosure statements refer to https://doi.org/10.1016/j.artd.2019.10.003.
30140-2&id=gr1.jpg){kind=link}