The Input Is Reflected in the Output: Evaluating Neurophysiologic Monitors With Simulated Data
To show the applicability of their device, the authors played back modulations of Perlin noise to mimic different levels of anesthesia and to examine the tested monitors’ index behavior. We feel that this approach may unfortunately be too oversimplified to draw conclusions regarding monitor performance. Different levels of anesthesia were simulated by manipulating k, a parameter that linearly reflects the 1/f appearance of the EEG spectrum. Because the EEG during anesthesia contains so many nonlinearities, it is unclear how their results apply in the clinical situation. In addition to a steeper linear slope in the 1/f power spectrum, other critical EEG features appear during the transition to unconsciousness—for instance, a peak in alpha power develops.3 This peak is not captured by the simulation. Furthermore, there is not enough evidence to assume that any EEG changes occur in a linear fashion with increasing depth/dose of anesthesia. Several studies suggest the correlation is poor and a nonlinear relationship is more likely.4
The authors demonstrate that SE/RE (GE Healthcare, Little Chalfont, UK) and NeuroSENSE (NeuroWave Systems, Cleveland, OH) show a monotonic and partially linear trend for mapping manipulations of the simulated signal, whereas the BIS shows plateaus, hysteresis, and index oscillations for BIS > 50. We think that these findings are not incriminatory to the BIS monitor but more likely reflect the noncompatibility between the simulated signal and the BIS algorithm, that is, the simulation’s signal modulation fails to contain specific nonlinear properties present in a real EEG recording. Specifically, the simulated signal only reflects one particular aspect of anesthetic-induced EEG changes (ie, a more uniformly distributed power spectrum). Consequently, there will be a bias toward algorithms that exploit this particular aspect. Although the authors appropriately concede that it is necessary to verify their results with actual EEG, we think that the composition of the simulated signal renders these data of limited use for comparing monitors.
Furthermore, the presented replaying device was not examined over a relevant range of clinical conditions, including anesthesia concentrations that might result in burst suppression. Most monitors are primed for burst suppression detection, a feature not considered in the simulation. Hence, the idea of a smartphone-based EEG player to evaluate the monitors is appealing, but we fear that the simulation approach presented may not reflect a patient’s EEG in a close enough fashion to draw conclusions from the indices that were calculated based on these simulations.