Recent progress in building intelligent systems have revealed great opportunities in use of computer vision based methods. Examples of such intelligent systems include autonomous vehicles that need depth perception and Visual Question Answering systems that need to solve scene understanding. However, many of the state-of-the-art approaches that use techniques such deep learning and reinforcement learning need large amounts of training data. There have been promising approaches that show that it might be feasible to generate artificial data via simulators in order to augment the training set. In addition, recent advance in graphics and hardware has enabled development of engines that are not only photorealistic but also run in real-time, enabling rapid training and testing of models.

While such ideas are promising, there are several research challenges that still need to be addressed. For example, first is the question of how can we build a simulator that can indeed generate data that is realistic and can be useful to solve vision tasks. Secondly, techniques such as reinforcement learnings have been very useful in solving gaming tasks such as AlphaGo, Atari Games etc. it is still not clear how such techniques can enable training of systems that are deployed in real-world. The role of building and effectively using realistic simulators can be critical in bridging such simulation to reality gap.