Recent progress in machine learning has led to many advances in engineering and science fields, including computer vision and graphics. Most remarkable successes have been shown in supervised learning with deep neural networks. Despite these successes, as a consequence of using high-capacity models, we are struggling with a lack of high-quality annotations in supervised methods or limitations of hardware resources. Directly dealing with these bottlenecks is often very expensive or challenging. In this talk, I will present a few potential strategies by showing my past and recent research, which include leveraging synthetic data, injecting prior knowledge into the model, and leveraging worldwide user resources.