Online Ratings and Reviews of American Orthopaedic Foot and Ankle Surgeons

Introduction

In 2018, a law suit for defamation was filed by a New York surgeon against a patient who wrote a negative review about the practice on Yelp. ¹ The doctor claimed that this review was false and designed to inflict emotional harm, loss of income, and damage to his practice. ¹ It made national news, with most of the readers siding with the patient and accusing the doctor of harassment. Although this is an extreme example, it represents a challenge faced by both patients and doctors in the age of social media and online reviews. Medicine is increasingly seen as a customer service industry and is being evaluated online as such, similar to other industries.

Internet use by patients continues to increase, particularly in the realm of consumer-driven health care. ² Recent reports show that 85% of American adults use the internet at home or at work, and 59% state that they have searched online for health information.^3,4 A more recent rising trend in health care is the use of physician rating websites by patients.^5-7 In 2010, 37% of Americans reported consulting online ratings when choosing a health care provider, and 30% of Americans compare online ratings of multiple physicians before making a decision.^8-10 Similarly, 35% reported avoiding a physician with poor ratings and reviews.^11,12 It is clear that among certain segments of the population, online rating sites influence patient preferences.^13-15

Rating sites appear to be of particular interest to patients when selecting procedural physicians such as surgeons. Recent studies demonstrate that up to 99% of subspecialized surgeons have online ratings.^16,17 Although studies have evaluated and found that the majority of general orthopaedic surgeons have positive ratings, there has not been any specific evaluation of the online reviews for foot and ankle surgeons.^18,19 Moreover, there has been little evaluation of the content of the written comments describing patients’ experience with the physician.^20-22

Prior studies have demonstrated that positive comments relate more to surgeon-dependent factors (eg, clinical outcome, professional competence) and negative comments reflect more surgeon-independent factors.^20-22 These prior reports found that doctors are generally well liked with high ratings, but they fail to acknowledge that the review system is highly skewed and does not represent a normal distribution of patient experience. Often rather, these reviews capture the very displeased patients who wish to vent and the very happy who desire to praise. Undoubtedly, some of these later patients were solicited by the doctor or office staff to provide favorable reviews. Regardless of the reason for the review, it is clear that the rating sites are here to stay and that patients will be increasingly interested in using them to gain information about potential providers, regardless of the accuracy contained within. As such, physicians must recognize this reality and critically evaluate how this system affects their practices.

Therefore, the objectives of this study are to examine the online patient ratings and comments for orthopaedic foot and ankle surgeons and analyze characteristics of written comments that are associated with positive and negative ratings. We hypothesized that most foot and ankle surgeons would have favorable online patient ratings and that positive written comments would more likely reflect surgeon-dependent factors, whereas negative comments would more likely reflect surgeon-independent factors.

Methods

A total of 210 orthopaedic foot and ankle surgeons who are members of the American Orthopaedic Foot and Ankle Society were selected for review. A total of 15 major metropolitan areas from across the United States were selected to achieve reasonable regional diversity. Surgeons within a 50-mile radius of the location were randomly selected for review; 210 surgeons were chosen for evaluation based on prior published studies examining a similar number of surgeons.^17,21 Demographic information, ratings, and comments were reviewed on the 3 most visited public domain physician rating websites ¹⁸ —HealthGrades.com (HealthGrades), Vitals.com (Vitals), and Ratemds.com (Ratemds)—between July 18 and August 8, 2018. Demographic information obtained from rating websites included physician gender, age, and year of graduation from medical school. Information about practice setting (academic, private, or private/academic) was obtained from the professional websites of the providers.

The number of reviews, each ratings’ numerical value, and the number of reviews with written comments were recorded from the above websites for each surgeon. Written comments were categorized as being positive or negative and categorized as relating to physician’s personality (eg, “nice,” “arrogant,” “warm”), knowledge, skills (eg, surgical, diagnostic), communication (eg, “good listener,” “thorough explanations,” “informative”), bedside manner (eg, “terrible” or “good” bedside manner), and time spent with patient; facilities (eg, easy parking, clean offices); midlevel practitioners; office staff; waiting time; logistics (eg, easy appointment scheduling, billing, paperwork); and outcome (eg, “no more pain,” “unable to walk normally”).

Student t-tests were used to evaluate differences in mean star ratings, mean number of ratings, and mean number of comments between genders. χ² Tests were used to evaluate the proportions of positive and negative comments across the various categories. Kruskal-Wallis tests were used to compare mean number of comments and reviews across websites and mean rating values based on practice setting. Statistical significance was defined at P <.05.

Results

A total of 182 male and 28 female orthopaedic foot and ankle surgeons were identified in 107 cities and towns covering the West, Midwest, South, and Northeast regions of the United States on 3 physician review websites. Summary data of physician ratings and comments are presented in Table 1. Of the 210 surgeons evaluated, there were 3236 reviews on HealthGrades, 4006 reviews on Vitals, and 627 reviews on Ratemds. The mean number of reviews per surgeon were 15.4 ± 12.7, 19.1 ± 15.1, and 3.0 ± 4.5 for HealthGrades, Vitals, and Ratemds, respectively (P < .001). There were 1144 written comments on HealthGrades, 1012 written comments on Vitals, and 617 written comments on Ratemds. The mean number of comments per surgeon were 4.8 ± 5.9, 5.4 ± 5.5, and 3.0 ± 4.5 for HealthGrades, Vitals, and Ratemds, respectively (P < .001).

OPEN IN VIEWER

Table 1.

Summary Table of Number of Ratings and Comments From HealthGrades.com, Vitals.com, and Ratemds.com.

	Number of Reviews	Reviews With Comments
HealthGrades	3236	1144
Vitals	4006	1012
Ratemds	627	617
Total	7869	2773

There were 2773 written comments in total. Of these, 1842 were positive, 697 were negative, and 234 were neutral. The mean number of positive, negative, and neutral comments per surgeon were 8.8 ± 8.4, 3.3 ± 4.0, and 1.1 ± 1.4, respectively (P < .001). Of the 2773 comments, 36 (0.01%) had responses from surgeons or staff members on behalf of the surgeon. A total of 19 surgeon profiles had responded to these written comments; 17 of the 36 responses were to comments with 1-star ratings, whereas 14 responses were to comments with 5-star ratings.

Across all sites, surgeons had a mean rating of 4.03 ± 0.57 stars (median = 4.08; range = 2.07-5), a mean number of 37.5 ± 26.1 ratings (median = 33.5; range = 2-127), and 13.2 ± 11.1 comments (median = 11.0; range = 0-73). The distribution of patients’ ratings of orthopaedic foot and ankle surgeons across all 3 sites are presented in Figure 1. Of 7869 ratings, 5238 (67%) were 5-star ratings and 1368 (17%) were 1-star ratings.

OPEN IN VIEWER

Figure 1.

Graph of total number of ratings for each star.

The average rating was 3.98 ± 0.63 stars for female surgeons and 4.03 ± 0.56 stars for male surgeons (P = .43). The mean number of ratings based on gender were 26.2 ± 20.7 and 39.2 ± 26.4 for female and male surgeons, respectively (P = .01). The mean number of comments based on gender was 9.86 ± 8.51 and 13.7 ± 11.3 for female and male surgeons, respectively (P = .04).

A total of 71 surgeons practice in an academic setting, 110 in private practice, and 29 in a private/academic setting. The average ratings for surgeons based on practice setting were 4.03 ± 0.55, 4.03 ± 0.62, and 4.03 ± 0.46 stars for academic, private, and private/academic, respectively (P = .969). The mean number of ratings based on practice setting was 39.1 ± 27.6, 33.2 ± 23.8, and 49.8 ± 27.2 for academic, private, and private/academic, respectively (P = .008). The mean number of comments based on practice setting was 12.8 ± 11.6, 11.7 ± 9.58, and 19.8 ± 13.0 for academic, private, and private/academic, respectively (P = .003).

The distributions of positive and negative comments are presented in Figure 2 and Table 2. The top 3 categories for positive comments were outcome (21.1% or all positive comments), physician personality (20.1%), and communication (16.5%). The top 3 categories for negative comments were outcome (18.9% of all negative comments), bedside manner (15.2%), and waiting time (11.1%). Table 2 demonstrates the breakdown of review content based on positive and negative written comments with χ² analysis and P values next to each category.

OPEN IN VIEWER

Figure 2.

Distribution of positive and negative written comments across different review categories.^a

OPEN IN VIEWER

Table 2.

Analysis of Association of Positive and Negative Comments With Different Review Categories.

	Positive	Negative	Total	χ2 Test P Value
Physician personality	859	133	992	<.001
Physician knowledge	372	53	425	<.001
Physician skills	225	50	275	.006
Facilities	23	4	27	.208
Midlevels	57	20	77	.899
Staff	413	116	529	.071
Logistics	74	100	174	<.001
Waiting time	51	157	208	<.001
Bedside manner	248	215	463	<.001
Time spent with patient	279	147	426	<.001
Communication	688	153	841	<.001
Outcome	882	268	1150	.112
Total	4171	1416	5587

Discussion

Patients are increasingly using physician rating websites to rate, evaluate, and choose surgeons.^16,17 These websites have the potential to strongly influence patient preferences.^13-15 More than a third of patients have avoided a potential physician with poor online ratings.^11,12 This emphasizes that physicians must be aware of their online presence, ratings, and written reviews.

The compiled mean online rating for orthopaedic foot and ankle surgeons was 4.03 ± 0.57 out of 5 stars across all 3 websites. This generally favorable rating along with overall positive written comments is consistent with previous findings for orthopaedic surgeons as a whole as well as for those in other surgical fields.^17,19-22 Interestingly, a high percentage (84%) of the total number of ratings was either a 1-star (17% of total) or a 5-star (67%) rating. This pattern has been shown in previous studies and suggests that patients who perceive extremes of care are more likely to rate a physician online. ²³ Additionally, 67% of the total number of ratings were 5 stars, which may suggest that surgeons or their offices are soliciting patients for positive online reviews in order to achieve such a preponderance of 5-star ratings. In fact, HealthGrades recommends encouraging satisfied patients to provide feedback on physician rating sites. ²⁴ Furthermore, it has recommended repeatedly that positive patient reviews should be directly solicited simply to combat and balance out any negative comments. ²⁵ This of course, only further highlights the concern about the validity of these reviews.

Prior studies have demonstrated mixed results regarding average ratings based on practice setting. Some have shown that academic surgeons have higher ratings than nonacademic surgeons,^17,19 whereas others show no differences at all.^20,22,26 Our study found no differences in ratings among orthopaedic foot and ankle surgeons based on practice setting. However, private/academic surgeons did have more ratings and comments than the other groups.

Regarding written reviews, the authors found that positive comments were primarily written about outcome, physician personality, and communication. In addition, analyses demonstrated a significantly higher proportion of comments related to physician personality, physician knowledge, physician skills, time spent with patient, and communication in positive comments as opposed to negative comments (Table 2). These findings are consistent with previous studies that also found correlations between positive comments and surgeon-dependent factors (eg, surgical outcome, surgeon likeability/character).^20-22

Conversely, negative comments most commonly addressed outcome, bedside manner, and waiting time. Analysis also showed a significantly higher proportion of negative comments relating to bedside manner, waiting time, and logistics. Similarly, previous studies identified that negative comments pertained more to surgeon-independent factors such as office environment, parking, ease of scheduling, and ancillary staff interactions.^20,21 Perceived poor bedside manner is a concern among patients seeing orthopaedic foot and ankle surgeons, which has been similarly demonstrated in other studies. ²⁰ Bedside manner has overlap with physician personality and may in fact represent a distinctive description used by patients when discussing this essential physician-patient interaction. The authors chose to keep this as a separate category because of the number of times “bedside manner” was specifically mentioned, particularly in a negative connotation.

With this in mind, physicians should realize that their ratings do not depend on clinical or surgical care and outcomes alone. It is evident that ratings are multifactorial and based on the patient’s unilateral subjective experience. Surgeons should understand that difficulty scheduling, long waiting times for appointments, billing issues, and other factors out of the surgeon’s control can drive negative online ratings. Therefore, surgeons and practice managers should consider establishing systems to regularly monitor these factors and assess for areas of improvement.

It is also important to note that although the χ² tests did not reveal a statistically significant difference in proportions of positive and negative written comments regarding outcomes, this category was the most written about for both. This further corroborates a common theme that patients with either excellent or poor outcomes are driven to write reviews. ²³ Although a high percentage of written comments related to outcomes, studies have shown no correlation between online ratings of physicians and patient clinical outcomes and satisfaction.^27,28

There are several limitations to this study. First, the small sample size of 210 orthopaedic foot and ankle surgeons may not be sufficient to fully represent the ratings and comments of these surgeons as a whole. Second, the identified surgeons in this study practice in or near a metropolitan area, which may differ from surgeons practicing in more rural areas. However, our findings were similar to previous studies regarding overall positive ratings and comments.^18,19

With these observations, we provide a few recommendations for how surgeons can manage negative online reviews and their online profile.

In conclusion, this is the first study to analyze online ratings and written comments of orthopaedic foot and ankle surgeons. We found that the majority of foot and ankle surgeons had a favorable online rating. Furthermore, written comments were mostly positive, and such comments pertained to surgeon-dependent factors. Conversely, negative comments mainly related to surgeon-independent factors, particularly waiting time and logistics. Physician rating websites have no mechanism of verifying patients and their experiences, and information posted is likely unreliable, but it is nevertheless used by patients. We found that 90% of surgeons reviewed had at least one 1-star rating, including nationally renowned and esteemed surgeons well respected in the orthopaedic foot and ankle community. This serves to demonstrate that each patient experience is individual and isolated. It also shows that every surgeon, regardless of age, location, practice setting, experience, ability, and likability, will undoubtedly disappoint patients during their career, some of whom will vent their frustration online as a negative review. With the inevitable increased use of these websites, it is important for foot and ankle surgeons to participate in monitoring their online profiles, address legitimate patient concerns, provide accurate information, and protect their practice, business, and livelihood. The reader should take heart, when they too receive a 1-star review, that 90% of all of the surgeons reviewed here have also received 1-star ratings and they are in good company.