Ranking surgical skills using an attention-enhanced Siamese network with piecewise aggregated kinematic data

Purpose: Surgical skill assessment using computerized methods is considered to be a promising direction in objective performance evaluation and expert training. In a typical architecture for computerized skill assessment, a classification system is asked to assign a query action to a predefined category that determines the surgical skill level. Since such systems are still trained by manual, potentially inconsistent annotations, an attempt to categorize the skill level can be biased by potentially scarce or skew training data. Methods: We approach the skill assessment problem as a pairwise ranking task where we compare two input actions to identify better surgical performance. We propose a model that takes two kinematic motion data acquired from robot-assisted surgery sensors and report the probability of a query sample having a better skill than a reference one. The model is an attention-enhanced Siamese Long Short-Term Memory Network fed by piecewise aggregate approximation of kinematic data. Results: The proposed model can achieve higher accuracy than existing models for pairwise ranking in a common dataset. It can also outperform existing regression models when applied in their experimental setup. The model is further shown to be accurate in individual progress monitoring with a new dataset, which will serve as a strong baseline. Conclusion: This relative assessment approach may overcome the limitations of having consistent annotations to define skill levels and provide a more interpretable means for objective skill assessment. Moreover, the model allows monitoring the skill development of individuals by comparing two activities at different time points.