The Simon task is one of the most prominent interference tasks and has been extensively studied in experimental psychology and cognitive neuroscience. Despite years of research, the underlying mechanism driving the phenomenon and its temporal dynamics are still disputed. Within the framework of the review, we adopt a model-based cognitive neuroscience approach. We first go over key findings in the literature of the Simon task, discuss competing qualitative cognitive theories and the difficulty of testing them empirically. We then introduce sequential sampling models, a particular class of mathematical cognitive process models. Finally, we argue that the brain architecture accountable for the processing of spatial (‘where’) and non-spatial (‘what’) information, could constrain these models. We conclude that there is a clear need to bridge neural and behavioral measures, and that mathematical cognitive models may facilitate the construction of this bridge and work towards revealing the underlying mechanisms of the Simon effect.