Studies have shown that using surgical simulators to improve skills in the operating room (OR) can be effective for surgeons in training. To maximize the effectiveness of simulation training, a proficiency-based training paradigm in which trainees are required to achieve expert-derived performance goals has been suggested. This type of training is tailored to individual needs and ensures the acquisition of uniform skills. However, while proficiency-based curricula have been effective in improving operative performance, studies show that simulator-trained learners do not always reach expert performance in the OR. “Simulators cannot always reliably predict when skill acquisition is complete because the currently used metrics of performance may have limited sensitivity,” explains Dimitrios Stefanidis, MD, PhD, FACS. “This can be problematic should problems emerge in the demanding environment of the OR.”
According to Dr. Stefanidis, most simulation curricula traditionally use time and errors as metrics of performance. “The problem with these metrics is they don’t provide a complete picture of trainee performance. They give no information about the attention demands required by the task, the effort the trainee had to invest to achieve a level of performance, and the quality of the learning that occurred. While two different surgeons may get equal results on time and accuracy measurements, they may have substantial differences in workload, attention demands, and physiologic parameters that reflect differences in learning, true skill level, and experience.”
Defining Automaticity in Surgery
One of the key characteristics that distinguishes experts from novices is their ability to engage in certain activities without requiring significant attention. “Psychologists have called this ‘automaticity,’ ” says Dr. Stefanidis. “Many habitual or highly practiced motor acts can be performed automatically, leaving enough spare attention capacity for engaging in multiple activities. Automaticity is achieved through repeated practice on tasks with consistently mapped characteristics.” In order to measure automaticity, some training programs use a secondary task that assesses spare attention capacity when the primary task is being performed.
New Data Using Simulator Training
In a study published in the January 2012 Annals of Surgery, Dr. Stefanidis and colleagues tested a unique intervention to see if trainees could benefit from simulator training to automaticity. For the intervention group, novices practiced on a suturing task until they achieved expert levels of time and errors. These novices were tested on a live porcine fundoplication model. They continued simulator training until they achieved expert levels on a visual spatial secondary task (automaticity) and were retested on the OR model. The control group participated only during testing sessions.
According to the results, suturing performance improved significantly for the intervention group after proficiency-based training when compared with their baseline (Figure 1). This was anticipated and has been demonstrated before. However, the investigators also demonstrated that continued training to automaticity beyond this initial level of proficiency led to further improvements in OR performance. Control group performance did not change significantly between the two OR tests for any of the recorded parameters. Moreover, the intervention group caused fewer inadvertent injuries during the second OR test after automaticity training (Figure 2).
“Our study demonstrated that training novices to expert levels of secondary task performance above and beyond the standard proficiency levels of time and errors (automaticity) can lead to superior skill acquisition and transfer,” Dr. Stefanidis says. “All intervention participants demonstrated improvements in their secondary task performance.”
It should be noted, however, that the achievement of automaticity on simulators requires an extensive amount of training beyond what is needed to reach proficiency as it is currently defined. On the basis of the number of repetitions, study participants who achieved automaticity did so after training that was twice as long as their initial training to proficiency. Also, at least half of the participants were unable to achieve expert levels of secondary task performance despite longer training, indicating that they needed even longer training times.
Achieving Automaticity on Simulators
Considering the potential benefits gained by achieving automaticity on simulators, Dr. Stefanidis suggests that simulator training go beyond the initial achievement of proficiency in surgical curricula. “It should be highlighted that the safety achieved with automaticity training was an important finding,” he says. “Significantly fewer inadvertent injuries occurred to surrounding structures when compared with initial proficiency training. This finding indicates that proficiency training alone may leave some surgeons unprepared for transitioning to the clinical environment.”
Dr. Stefanidis recommends that skills curricula incorporate secondary tasks that document achievement of automaticity. “Metrics obtained from a secondary task provide a more comprehensive assessment of trainee performance and multi-task ability. Adding these metrics to the traditional metrics of time and errors during simulator training is promising for improving learning. While more multi-institutional studies are needed to confirm our findings, it appears that this strategy could yield positive dividends in the surgical community.”
Readings & Resources (click to view)
Stefanidis D, Scerbo MW, Montero PN, et al. Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: a randomized controlled trial. Ann Surg. 2012;255:30-37. Available at: http://www.medscape.com/viewarticle/757914_print or at http://journals.lww.com/annalsofsurgery/Abstract/2012/01000/Simulator_Training_to_Automaticity_Leads_to.6.aspx.
Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. N Engl J Med. 2006;355:2664-2669.
Stefanidis D. Optimal acquisition and assessment of proficiency on simulators in surgery. Surg Clin North Am. 2010;90:475-489.
Sachdeva AK, Pellegrini CA, Johnson KA. Support for simulation-based surgical education through American College of Surgeons—accredited education institutes. World J Surg. 2008;32:196-207.
Stefanidis D, Scerbo MW, Sechrist C, et al. Do novices display automaticity during simulator training? Am J Surg. 2008;195:210-213.
RosenthalME, Ritter EM, GoovaMT, et al. Proficiency-based fundamentals of laparoscopic surgery skills training results in durable performance improvement and a uniform certification pass rate. Surg Endosc. 2010;24:2453-2457.
Risucci D, Cohen JA, Garbus JE, et al. The effects of practice and instruction on speed and accuracy during resident acquisition of simulated laparoscopic skills. Curr Surg. 2001;58:230-235.