Abstract
Aim:
We reviewed intraoperative video recordings (IVRs) of laparoscopic percutaneous extraperitoneal closure (LPEC) for inguinal hernia in children blindly to assess performance.
Methods:
IVRs of 183 LPEC performed between April 2013 and March 2016, graded by the operating surgeon as difficult (D; n = 8), straightforward (S; n = 96), or easy (E; n = 79), were scored by a panel of reviewers with advanced (group A; >400 LPEC cases; n = 5), intermediate (group I; 50–150 cases; n = 5), and basic (group B; <10 cases; n = 5) experience, according to suturing, dissection plane, vas/vessel dissection, bleeding, and peritoneal injury. They also allocated a recurrence risk rank (RRR; highest = 6; lowest = 1) for each IVR. Mean score variance for each IVR was also compared between reviewers.
Results:
There was one recurrence (R; 4-year-old male; level E). RRR were: 1, 2, and 2 for reviewers A, I, and B, respectively. Reviewer A scores for “suturing” and “bleeding,” and reviewer I scores for “dissection plane” and “peritoneal injury” correlated significantly with RRR. No reviewer B scores correlated with RRR. Score variance between A and I and A and B for cases D1 and D2 were statistically significant.
Conclusion:
Advanced reviewers showed greatest variance, questioning the validity of whether experience alone improves surgical technique.
Introduction
Since laparoscopic percutaneous extraperitoneal closure (LPEC) was first reported by Takehara et al. 1 It has been performed widely in Japan with a corresponding increase in reports about technical modifications. LPEC would appear to be plagued by higher recurrence rates according to some reports, but despite proactive research, details of incidence, and etiology remain inconclusive, and guidelines for prevention have yet to be established.
One of the coauthors (G.M.), who completed standard surgical training at the Juntendo group of hospitals, began using LPEC in April 2013 after standard instruction and supervised performance typical of training for a new procedure. Between April 2013 and January 2016, he was the operative surgeon in 183 cases and at the time of writing, all have been followed up for a minimum of 12 months. During this period there was one recurrence in a 4-year-old boy (case R). Because we could not attribute the recurrence to any particular shortcoming, it prompted us to propose a study to determine differences between LPEC with recurrence and LPEC without recurrence by having a panel of reviewers with different levels of experience of LPEC review intraoperative video recordings (IVRs) blindly to help reveal some factors that may affect outcome.
Methods
G.M. categorized his 183 cases of LPEC according to complexity retrospectively, using three levels: difficult (D; n = 8), straightforward (S; n = 96), and easy (E; n = 79). Case R was a level E case. IVRs used in this study were selected as follows. Two IVRs of male patients <5 years old were chosen randomly from level D cases (D1 and D2) and level S cases (S1 and S2). An IVR of a male patient <5 years old was chosen randomly from a level E case (E1). IVRs from cases R, D1, D2, S1, S2, and E1 were recorded on one DVD for distribution to each member of a panel of reviewers comprising 15 specialist pediatric surgeons from the Juntendo group of hospitals and two affiliated children's hospitals who were divided into three groups according to their personal experience of performing LPEC as the operating surgeon; advanced Table 1.
Laparoscopic Percutaneous Extraperitoneal Closure Video Evaluation Scoring Protocol
(A; >400 cases; n = 5), intermediate (I; 50–150 cases; n = 5), and basic (B; <10 cases; n = 5). They were each asked to score all IVRs using a standardized protocol (Table 1) then allocate each IVR a risk for recurrence rank (RRR) with one being the least likely to recur and six being the most likely to recur. The variance of each reviewer's mean total score for each of the six IVRs was also evaluated for each reviewer then compared between reviewer groups (A, I, and B).
LPEC was performed according to the standard technique reported by Takehara et al. 1 In brief, the patient is placed supine and general anesthesia is induced. A 3 mm trocar for the laparoscope is inserted in the umbilicus, and a 2 mm trocar for grasping forceps is placed on the right side of the umbilicus. Then, a 19-gauge LPEC needle (Lapaherclosure; Hakko Medical Co., Nagano, Japan) with a wire loop at the tip to hold nonabsorbable suture material is inserted at the approximate position of the internal inguinal ring. The orifice of the hernia sac is closed extraperitoneally by suturing circumferentially around the internal inguinal ring using the LPEC needle.
Pearson correlation coefficients were calculated to test correlations between technical criteria used for scoring and RRR. Levene test was used to compare the variance of each reviewer's scores between reviewer groups. Statistical significance was set at P < .05.
This study was approved by the Juntendo University School of Medicine Institutional Review Board (IRB no: 16-253).
Results
Reviewer scores for each criterion are shown per group of reviewers (Table 2). Mean RRR for each IVR are also presented. Case R was allocated RRR of 1, 2, and 2 by reviewer groups A, I, and B, respectively. In other words, case R was considered to be least likely or unlikely to recur by all reviewers. In A, differences in “suturing” and “bleeding” were statistically significant for determining recurrence, whereas the other three criteria were not. In I, differences in “dissection plane” and “peritoneal injury” were statistically significant for Figure 1 determining recurrence while the other three criteria were not. B made no valid assessments, and no criteria were statistically significant for determining recurrence. Figure 1 shows the variance of each reviewer group's mean total scores for each of the six IVRs. For R, E1, S1, and S2, there was a tendency for variance to be greater in group A than in group I and greater in group I than in group B, but differences were not significant. For D1 and D2, differences in variance were statistically significant between reviewer groups A and I and between reviewer groups A and B.

Variance of each reviewer group's mean total scores for each of the six IVRs. For R, E1, S1, and S2, there was a tendency for variance to be greater in group A than in group I and greater in group I than in group B, but differences were not significant. For D1 and D2, differences in variance were statistically significant between reviewer groups A and I and between reviewer groups A and B. IVRs, intraoperative video recordings.
Mean Laparoscopic Percutaneous Extraperitoneal Closure Video Evaluation Scores
1 = least likely, 6 = most likely.
Average of reviewer risk ranks for each video.
Criterion with statistical significance for recurrence.
LPEC, laparoscopic percutaneous extraperitoneal closure.
Discussion
Miyake et al. 2 reported the results of a study comparing open repair and LPEC, in which LPEC had a lower recurrence rate than open repair. In their series, there were 0.48% (8/1653) recurrences of hernia, which is consistent with other reports in the literature.3,4 They also analyzed risk factors for recurrence and found all eight recurrence cases were male, and on multivariate analysis, aged <1 year. Operator experience or side of repair were not statistically significant factors. 5 In other words, even from their large series it is difficult to determine the possible risk factors for recurrence. During LPEC, there is a tendency for the peritoneum to be injured while working around the testicular vessels and spermatic duct. If there is some minor peritoneal insult, thorough circumferential suturing and secure circumferential ligation of a hernia sac become impossible and increase the risk for recurrence. Furthermore, if a gap is left in the peritoneum where the lateral and median edges should join, it could contribute to recurrence even if it is miniscule. These concepts are general common knowledge to surgeons with experience of LPEC for hernia repair; that is, recurrence is possible if the peritoneum is compromised, but R's repair did not involve any of these known causes. In fact, R's RRR was consistently low for each reviewer group, suggesting that he would be very unlikely to recur, and the operating surgeon (G.M.) graded his hernia repair as easy. We are somewhat at a loss trying to explain what went wrong, but hypothesize that recurrence in case R may have been caused by the LPEC procedure progressing too smoothly with no peritoneal injury, no bleeding, next to no inflammation around the surgical site, and few adhesions. In other words, the seal between tissues may not have been inadequate, predisposing tissue planes to loosen over time and possibly cause recurrence. The etiology of recurrence would appear to be multifactorial and seemingly unrelated to experience.
In 2003, a qualification system for endoscopic surgical skills began in the fields of gastrointestinal surgery, urology, and gynecology in Japan. The standard requirement for all candidates is to complete common procedures relevant to their specialty, laparoscopically. A similar qualification system was established for pediatric surgery in 2008, and is based on peer review of an IVR of an entire laparoscopic fundoplication. Video review is beneficial for evaluating surgical competency because reviewers can judge several parameters required for safe surgery simultaneously. 6 However, IVR review has been hindered by poor consensus regarding penalties for unsafe or dangerous maneuvers. Different reviewers can assess the same IVR differently as we saw in this study. A reviewer must be objective in the first instance and strive to ensure their assessments are accurate and reproducible consistently; otherwise, the entire review process itself loses credibility. Thus, the technical skill of a reviewer, his/her flexibility and objectivity, and finally decision-making ability can combine to cause discrepancies in scores. Recent studies suggest that experienced laparoscopic surgeons have a short learning curve to become proficient in advanced laparoscopic surgery.7,8 A short learning curve is only of value if quality and safety are maintained. Although it has been reported that a surgical fellow may need to perform 30 LPEC to improve as measured by operative time 9 we specifically chose our reviewers to have widely varying levels of experience; that is, >400 cases for advanced level (A), 50–150 for intermediate level (I), and <10 for basic level (B) reviewers, hoping that bias would be eliminated; however, each group seemed to assess each criteria differently, which might reflect differences in judgment based on the experience of the reviewer. For example, a basic level reviewer may not notice any problem because of limited experience or focus on minor issues that are of little consequence. In other words, reviewers themselves may have tried to modify their decisions to match the apparent skill of the operating surgeon or they might have thought their own sense of reliability and reproducibility may not be rigid enough to allow credible evaluation or they themselves may be too regimented to be flexible? It is very difficult to determine. Nevertheless, successful IVR review may require multiple blind viewings of a single candidate's IVR over time with some option to assess postoperative care and skill of managing other health care workers as a team to ensure optimal patient recovery.
From our results, the spectrum of reviewer responses obtained would indicate that there was a lack of uniformity in scoring. Since the golden rule of all surgery is successful repair; and of minimally invasive surgery is successful repair with minimal surgical stress, instrument manipulation must be meticulous during LPEC to maintain safety. However, since Takehara's original report, 1 many modifications have been purported and justified and now involve all aspects of LPEC from choice of instruments to how needles should be used, in which layers, and even where dissection should commence. In other words, LPEC is constantly evolving without a focused goal and as such it is very difficult to establish what caused a recurrence because the standard procedure has changed to such an extent that IVR review sessions may be the only way to ensure consistency of technique and reliability of outcome.
Reviewer score variance is an unexpected issue that was highlighted by our study. Surprisingly, variance was greatest in the most experienced accomplished group of reviewers, and we can only assume that group A reviewers have tried many modifications over time and as a result are more flexible, provided there is no detrimental effect on outcome. Variance was significantly higher only in group-A reviewers, with each reviewer choosing different criteria for assessing skill. This finding raises controversial questions about our experience-based supervising/teaching programs where the fundamentals of standard surgical education are explained to trainees, who must learn to adapt and master numerous skills. It somewhat undermines the tradition of learning by experience that is the core of surgical training. To ensure the well-experienced teachers and supervisors keep in touch with reality, it might be beneficial for very well-experienced surgeons to refer back to original techniques for procedures periodically and compare their performance with the original (assuming the original is the best). By doing so, they are able to focus on the shortcomings of a trainee's IVR for completing a procedure safely and successfully, not view an IVR as a creative opportunity or as a source of inspiration.
From this study, IVR evaluation by blind review is useful to trainees because reviewers provide both positive and negative feedback, but IVR can also be reviewed by trainees themselves, or even senior surgeons themselves to help improve confidence and overcome technical difficulties that may not be noticed at the time of surgery.
Footnotes
Disclosure Statement
No competing financial interests exist.
Funding Information
No funding was received for this article.
