The advent of virtual reality technology has provided a new approach for assessing and training cognitive processing ability, with the design of simulations used to replicate real events in everyday lives. To better understand how head mounted displays/flat screen displays (HMDs/FSDs) and differences in the individuals who use them affect cognitive performance and the use of VR systems, our research group created the Stroop task-embedded virtual reality driving system (STEVRDS) and conducted a 2 × 2 between-group factorial design experiment among college students. The study examined the effects of HMDs and FSDs that differ in monovision/stereovision and field of view, the impact of gender (males vs. females) on users' performances in virtual driving and Stroop trials, and users' psychophysiological responses while using the system. The participants' subjective perceptions toward STEVRDS were also assessed to support the analyses/interpretations of cognitive performance, as well as provide empirical data relating to user experiences. The statistical analyses showed both main and interaction effects of HMDs/FSDs and gender on task performance, psychophysiological responses, and user evaluations of the system. The psychophysiological patterns exhibited during the use of STEVRDS further extended the findings. Overall, our results were comparable with cognitive phenomena reported in other studies/in real-life experiences or explained by logical reasoning, which suggests that the design/development of the STEVRDS is suitable for cognitive assessment/training. Practical implications are discussed for the application of HMDs and FSDs in evaluating and enhancing cognitive processing ability and the need for specific tailoring for male and female users.