In the absence of external stimuli or overt behavior, the activity of the left and right cortical hemispheres shows fluctuations that are largely bilateral. Here, we show that these fluctuations are largely responsible for the variability observed in cortical responses to sensory stimuli. Using widefield imaging of voltage and calcium signals, we measured activity in the cortex of mice performing a visual detection task. Bilateral fluctuations invested all areas, particularly those closest to the midline. Activity was less bilateral in the monocular region of primary visual cortex and, especially during task engagement, in secondary motor cortex. Ongoing bilateral fluctuations dominated unilateral visual responses, and interacted additively with them, explaining much of the variance in trial-by-trial activity. Even though these fluctuations occurred in regions necessary for the task, they did not affect detection behavior. We conclude that bilateral ongoing activity continues during visual stimulation and has a powerful additive impact on visual responses.