NIPS(2015) arXiv preprint arXiv:1508.06615(2015) [pdf] ⭐⭐⭐⭐, [6] Jason Weston, et al. (2015). Deep Learning Roadmap Organized Resources for Deep Learning Researchers and Developers. Kyle Hamlin, principal machine learning engineer at Sailthru, discusses what it takes to travel on that career path. "Siamese Neural Networks for One-shot Image Recognition. [pdf] (Neural Doodle) ⭐⭐⭐⭐, [5] Zhang, Richard, Phillip Isola, and Alexei A. Efros. [pdf] (NAF) ⭐⭐⭐⭐, [52] Schulman, John, et al. Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech! ANIPS(2014) [pdf] ⭐⭐⭐, [4] Ankit Kumar, et al. "Hybrid computing using a neural network with dynamic external memory." "A neural conversational model." "“Sequence to sequence learning with neural networks." "Visual tracking with fully convolutional networks." "Improving neural networks by preventing co-adaptation of feature detectors." 2012. With the benefit of hindsight, I think the key is to start way further upstream. "Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking." IEEE, 2013. [pdf] (AlphaGo) ⭐⭐⭐⭐⭐, [54] Bengio, Yoshua. In arXiv preprint arXiv:1502.03044, 2015. "Deep fragment embeddings for bidirectional image sentence mapping". ⭐⭐⭐, [6] Szegedy, Christian, et al. "Towards End-To-End Speech Recognition with Recurrent Neural Networks." [pdf] (Very fast and ultra realistic style transfer) ⭐⭐⭐⭐, [1] J. ⭐⭐⭐⭐, [2] Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. In arXiv preprint arXiv:1411.5654, 2014. [pdf] (Outstanding Work, most successful method currently) ⭐⭐⭐⭐⭐, [3] Zhu, Jun-Yan, et al. Nowadays, Deep Learning is considered to be a core subset of Machine Learning. We use optional third-party analytics cookies to understand how you use GitHub.com so we can build better products. [pdf]⭐⭐⭐, [10] Xu, Kelvin, et al. Deep learning focuses on further enhanced benefits in the present. In arXiv preprint arXiv:1411.4555, 2014. IEEE, 2013. "Decoupled neural interfaces using synthetic gradients." "Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation". "“Ask Me Anything: Dynamic Memory Networks for Natural Language Processing." arXiv preprint arXiv:1606.04671 (2016). Deep Learning and Reinforcement Learning for Autonomous Unmanned Aerial Systems: Roadmap for Theory to Deployment Jithin Jagannath Jithin Jagannath, Anu Jagannath, Sean Furman, Tyler Gwin Marconi-Rosenblatt Innovation Laboratory, ANDRO Computational Solutions, LLC, NY, USA "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size." [pdf] ⭐⭐⭐⭐, [6] Redmon, Joseph, et al. arXiv preprint arXiv:1511.06434 (2015). If nothing happens, download the GitHub extension for Visual Studio and try again. [pdf] (PixelRNN) ⭐⭐⭐⭐, [34] Oord, Aaron van den, et al. "End-to-end training of deep visuomotor policies." they're used to gather information about the pages you visit and how many clicks you need to accomplish a task. arXiv preprint arXiv:1508.06576 (2015). [html] (Deep Dream) "Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours." Science 350.6266 (2015): 1332-1338. Machine learning is also deeply interdisciplinary. In arXiv preprint arXiv:1603.06147, 2016. Why do we need such a curated list of resources? Deep Learning Papers Reading Roadmap. Roadmap to becoming an Artificial Intelligence Expert in 2020. [pdf] (Basic Prototype of Future Computer) ⭐⭐⭐⭐⭐, [41] Zaremba, Wojciech, and Ilya Sutskever. In Advances in neural information processing systems, 2014. Deep Learning Roadmap Ebook. "Deep residual learning for image recognition." [pdf] ⭐⭐⭐⭐, [2] Sennrich, et al. "A learned representation for artistic style." By targeted, we mean a list which demonstrates different kind or resources as well as different categories associated with Deep Learning. arXiv preprint arXiv:1502.04623 (2015). The roadmap is constructed in accordance with the following four guidelines: From outline to detail; From old to state-of-the-art It is targeted towards beginners strapped for time, as well as for intermediate practitioners. "Character-Aware Neural Language Models." Deep Learning Papers Reading Roadmap. "Learning a deep compact image representation for visual tracking." [pdf] (VAE with attention, outstanding work) ⭐⭐⭐⭐⭐, [33] Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. Next Article. "Progressive neural networks." Deep Learning; Case Studies; Machine Learning Cheatsheets; AI Mentorship Program; Start Now. arXiv preprint arXiv:1606.04474 (2016). arXiv preprint arXiv:1911.09070 (2019). Read more posts by this author. arXiv preprint arXiv:1511.05641 (2015). 2014. [pdf] (YOLO,Oustanding Work, really practical) ⭐⭐⭐⭐⭐, [7] Liu, Wei, et al. "Texture Networks: Feed-forward Synthesis of Textures and Stylized Images." [pdf] (DCGAN) ⭐⭐⭐⭐, [32] Gregor, Karol, et al. [pdf] (State-of-the-art in speech recognition, Microsoft) ⭐⭐⭐⭐. arXiv preprint arXiv:1609.05143 (2016). Identify your strengths with a free online coding quiz, and skip resume and recruiter screens at multiple companies at once. Nature (2016). 2015. arXiv preprint arXiv:1410.3916 (2014). [pdf] ⭐⭐⭐⭐, [4] Dai, J., He, K., Sun, J. [pdf] (An outstanding Work in 2015) ⭐⭐⭐⭐, [17] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. Springer International Publishing, 2014. "Spatial pyramid pooling in deep convolutional networks for visual recognition." [pdf]⭐⭐⭐⭐⭐, [6] Wu, Schuster, Chen, Le, et al. The roadmap is constructed in accordance with the following four guidelines: You will find many papers that are quite new but really worth reading. [pdf]⭐⭐⭐, [11] Sak, Haşim, et al. [pdf] (ICLR best paper, new direction to make NN running fast,DeePhi Tech Startup) ⭐⭐⭐⭐⭐, [26] Iandola, Forrest N., et al. [pdf] (A Tutorial) ⭐⭐⭐, [55] Silver, Daniel L., Qiang Yang, and Lianghao Li. [pdf] (Milestone, Andrew Ng, Google Brain Project, Cat) ⭐⭐⭐⭐, [29] Kingma, Diederik P., and Max Welling. 2016 [pdf] ⭐⭐⭐, [5] Dai, J., He, K., Sun, J. Subscribe to receive exclusive content about AI in your inbox! Work fast with our official CLI. "Neural Machine Translation by Jointly Learning to Align and Translate." • A roadmap of intelligent fault diagnosis is pictured to provide research trends. of deep learning. [pdf] (Milestone, Show the promise of deep learning) ⭐⭐⭐, [4] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. Download PDF Abstract: Unmanned Aerial Systems (UAS) are being increasingly deployed for commercial, civilian, and military applications. "Fully-Convolutional Siamese Networks for Object Tracking." [pdf] ⭐⭐⭐⭐, [8] A Rusu, M Vecerik, Thomas Rothörl, N Heess, R Pascanu, R Hadsell. [pdf](Deep Learning Eve) ⭐⭐⭐, [3] Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. [pdf] ⭐⭐⭐⭐⭐, [1] Wang, Naiyan, and Dit-Yan Yeung. "Dueling network architectures for deep reinforcement learning." "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks." "A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation". arXiv preprint arXiv:1506.03340(2015) [pdf] (CNN/DailyMail cloze style questions) ⭐⭐, [8] Alexis Conneau, et al. "Mask R-CNN" arXiv preprint arXiv:1703.06870 (2017). This is complete end to end machine learning roadmap, whether you are a beginner or a expert in machine learning, this is comprehensive roadmap for one to ace in machine learning. Deep Learning is also one of the most effective machine learning approaches. arXiv preprint arXiv:1512.02325 (2015). "Distilling the knowledge in a neural network." "YOLOv4: Optimal Speed and Accuracy of Object Detection." In arXiv preprint arXiv:1412.2306, 2014. [pdf]⭐⭐⭐, [4] Donahue, Jeff, et al. 2013. [pdf] ⭐⭐⭐⭐, [45] Graves, Alex, et al. [14] Hinton, Geoffrey E., et al. [pdf]⭐⭐, [5] Lee, et al. [pdf] ⭐⭐⭐, [64] Hariharan, Bharath, and Ross Girshick. Thus, they raise the need for developing novel approaches and trigger a specific focus in this roadmap. "Continuous Deep Q-Learning with Model-based Acceleration." "Instance-sensitive Fully Convolutional Networks." arXiv preprint arXiv:1610.05256 (2016). in CVPR. "A fast learning algorithm for deep belief nets." [pdf] (State-of-the-art method) ⭐⭐⭐⭐⭐, [50] Lillicrap, Timothy P., et al. "Distributed representations of words and phrases and their compositionality." "Network Morphism." [pdf] (Modify previously trained network to reduce training epochs) ⭐⭐⭐, [21] Wei, Tao, et al. "Neural turing machines." In: NIPS. This guide consists of different kinds of resources such as academic papers, books, and tutorials. "Fully Character-Level Neural Machine Translation without Explicit Segmentation". Proceedings of the IEEE conference on computer vision and pattern recognition. Deep Learning is also one of the most effective machine learning approaches. Advances in neural information processing systems. Deep Learning Papers Reading Roadmap. "Generative adversarial nets." [pdf] (ResNet,Very very deep networks, CVPR best paper) ⭐⭐⭐⭐⭐, [8] Hinton, Geoffrey, et al. Advances in Neural Information Processing Systems. Proceedings of the IEEE International Conference on Computer Vision. [pdf], [5] Karpathy, Andrej, and Li Fei-Fei. arXiv preprint arXiv:1601.06759 (2016). [pdf] ⭐⭐⭐⭐⭐, [3] Pinto, Lerrel, and Abhinav Gupta. "Controlling Perceptual Factors in Neural Style Transfer." [pdf] (RNN)⭐⭐⭐, [10] Graves, Alex, and Navdeep Jaitly. 2014. Learn more. [pdf] (Milestone,combine above papers' ideas) ⭐⭐⭐⭐⭐, [46] Mnih, Volodymyr, et al. Daniel Bourke. "Asynchronous methods for deep reinforcement learning." Region-based Fully Convolutional Networks." "Deep neural networks for object detection." [pdf] (TRPO) ⭐⭐⭐⭐, [53] Silver, David, et al. "Addressing the rare word problem in neural machine translation." It is considered to be very useful to capture high-dimensional data. [pdf] (RL domain) ⭐⭐⭐, [58] Parisotto, Emilio, Jimmy Lei Ba, and Ruslan Salakhutdinov. "Understanding the difficulty of training deep forward neural networks." arXiv preprint arXiv:1502.05698(2015) [pdf] (bAbI tasks) ⭐⭐⭐, [7] Karl Moritz Hermann, et al. Computational scientific discovery is at an interesting juncture. "Pointer networks." "Low-shot visual object recognition." 3,101. This guide consists of different kinds of resources such as academic papers, books, and tutorials. Learn more, We use analytics cookies to understand how you use our websites so we can make them better, e.g. Springer Berlin Heidelberg:15-29, 2010. arXiv preprint arXiv:1511.06295 (2015). arXiv preprint arXiv:1506.02640 (2015). [pdf] (Neural Optimizer,Amazing Work) ⭐⭐⭐⭐⭐, [25] Han, Song, Huizi Mao, and William J. Dally. Learn more. In arXiv preprint arXiv:1411.4952, 2014. 2015. [pdf] ⭐⭐⭐⭐, [6] Yahya, Ali, et al. I would continue adding papers to this roadmap. [pdf]) (First Paper named deep reinforcement learning) ⭐⭐⭐⭐, [47] Mnih, Volodymyr, et al. "One-shot Learning with Memory-Augmented Neural Networks." The roadmap is constructed in accordance with the following four guidelines: From outline to detail; From old to state-of-the-art [pdf]⭐⭐⭐⭐⭐, [6] Karpathy, Andrej, Armand Joulin, and Fei Fei F. Li. GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together. [pdf] (VAE) ⭐⭐⭐⭐, [30] Goodfellow, Ian, et al. "Sim-to-Real Robot Learning from Pixels with Progressive Nets." [pdf] ⭐⭐⭐⭐, [44] Vinyals, Oriol, Meire Fortunato, and Navdeep Jaitly. arXiv preprint arXiv:1312.5602 (2013). "Imagenet classification with deep convolutional neural networks." 2013 IEEE international conference on acoustics, speech and signal processing. CoRR, abs/1502.05477 (2015). [pdf]⭐⭐⭐⭐, [7] Fang, Hao, et al. European Conference on Computer Vision. [pdf] ⭐⭐⭐⭐, [9] He, Gkioxari, et al. arXiv preprint arXiv:1602.07360 (2016). "On the importance of initialization and momentum in deep learning." ... Notes on building a deep learning PC. [pdf] (SPPNet) ⭐⭐⭐⭐, [4] Girshick, Ross. If you are a newcomer to the Deep Learning area, the first question you may have is "Which paper should I start reading from?" 최신의 논문도 있지만 오래되었더라도 꼭 읽어야 할 목록과 잘 조화를 이룬 것 같습니다. AAAI Spring Symposium: Lifelong Machine Learning. [pdf] ⭐⭐⭐⭐⭐, [3] Pinheiro, P.O., Collobert, R., Dollar, P. "Learning to segment object candidates." [pdf] ⭐⭐⭐⭐, [39] Vinyals, Oriol, and Quoc Le. [pdf] (SO-DLT) ⭐⭐⭐⭐, [3] Wang, Lijun, et al. Google Research. 2014. "R-FCN: Object Detection via [pdf] (A step to large data) ⭐⭐⭐⭐, [1] Antoine Bordes, et al. [pdf] (A brief discussion about lifelong learning) ⭐⭐⭐, [56] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. Here is a reading roadmap of Deep Learning papers! arXiv preprint arXiv:1608.07242 (2016). "Very Deep Convolutional Networks for Natural Language Processing." I suggest that you can choose the following papers based on your interests and research direction. [pdf] (Modify previously trained network to reduce training epochs) ⭐⭐⭐, [22] Sutskever, Ilya, et al. [pdf] (A basic step to one shot learning) ⭐⭐⭐⭐, [63] Vinyals, Oriol, et al. Springer International Publishing, 2016. This approach avoids the need for humans to … Here is a reading roadmap of Deep Learning papers! [pdf] ⭐⭐⭐⭐, [4] Chung, et al. Deep Learning specialization on Coursera. The roadmap is constructed in accordance with the following four guidelines: from outline to detail; from old to state-of-the-art; from generic to specific areas; focus on state-of-the-art; You will find many papers that are quite new but really worth reading. "From captions to visual concepts and back". "Lifelong Machine Learning Systems: Beyond Learning Algorithms." In ICLR, 2015. arXiv preprint arXiv:1502.03167 (2015). "Learning to Track at 100 FPS with Deep Regression Networks." "Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks." MATLAB + Deep Learning Toolbox MathWorks: Proprietary: No Linux, macOS, Windows: … arXiv preprint arXiv:1507.06947 (2015). arXiv preprint arXiv:1611.03673 (2016). [pdf] (Breakthrough in speech recognition)⭐⭐⭐⭐, [9] Graves, Alex, Abdel-rahman Mohamed, and Geoffrey Hinton. In Proceedings of the 24th CVPR, 2011. [pdf] ⭐⭐⭐, [8] Dai, Jifeng, et al. "DRAW: A recurrent neural network for image generation." "Deep Learning of Representations for Unsupervised and Transfer Learning." The processor has integrated Intel's Omnibus fabric increases the price performance and reduces communication latency and has delivered up to 400 gigabytes of memory with no PCI performance lag. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation.” in CVPR, 2015. [pdf] (Godfather's Work) ⭐⭐⭐⭐, [57] Rusu, Andrei A., et al. That’s exactly how I started, and I floundered for quite some time. [pdf] (Momentum optimizer) ⭐⭐, [23] Kingma, Diederik, and Jimmy Ba. Most of machine learning is built upon three pillars: linear algebra, calculus, and probability theory. arXiv preprint arXiv:1505.00521 362 (2015). arXiv preprint arXiv:1610.00673 (2016). "Modeling and Propagating CNNs in a Tree Structure for Visual Tracking." [pdf] ⭐⭐⭐⭐, [28] Le, Quoc V. "Building high-level features using large scale unsupervised learning." arXiv preprint arXiv:1605.06409 (2016). "Conditional image generation with PixelCNN decoders." 2015. arXiv preprint arXiv:1603.08155 (2016). [pdf] ⭐⭐⭐⭐, [7] Gu, Shixiang, et al. AI taxonomy in this Roadmap Data-driven learning techniques are disruptive in essence and, by opposition to software development tech-niques, cannot be assessed through traditional approaches. arXiv preprint arXiv:1603.02199 (2016). Machine learning is a huge field of study. [0] Bengio, Yoshua, Ian J. Goodfellow, and Aaron Courville. [pdf] (PixelCNN) ⭐⭐⭐⭐, [34] S. Mehri et al., "SampleRNN: An Unconditional End-to-End Neural Audio Generation Model." "Mastering the game of Go with deep neural networks and tree search." "Reinforcement learning neural Turing machines." [pdf] (Three Giants' Survey) ⭐⭐⭐⭐⭐, [2] Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. [pdf]⭐⭐⭐, [3] Luong, Minh-Thang, Hieu Pham, and Christopher D. Manning. arXiv preprint arXiv:1506.05869 (2015). Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. arXiv preprint arXiv:1607.01759(2016) [pdf] (slightly worse than state-of-the-art, but a lot faster) ⭐⭐⭐, [1] Szegedy, Christian, Alexander Toshev, and Dumitru Erhan. "Binarized Neural Networks: Training Neural Networks with Weights and Activations Constrained to+ 1 or−1." Title: Deep Learning and Reinforcement Learning for Autonomous Unmanned Aerial Systems: Roadmap for Theory to Deployment. "Perceptual losses for real-time style transfer and super-resolution." 14. Advances in neural information processing systems. [pdf] ⭐⭐⭐⭐⭐, [35] Graves, Alex. Some milestone papers are listed in RNN / Seq-to-Seq topic. arXiv preprint arXiv:1606.02819 (2016). arXiv preprint arXiv:1603.08678 (2016). [pdf] ⭐⭐⭐⭐, [11] Tan, Mingxing, et al. [1] Luong, Minh-Thang, et al. [pdf] ⭐⭐⭐, [4] Levine, Sergey, et al. 2013. We use optional third-party analytics cookies to understand how you use GitHub.com so we can build better products. [pdf] ⭐⭐⭐, [2] Levine, Sergey, et al. "Fast and accurate recurrent neural network acoustic models for speech recognition." ICML Unsupervised and Transfer Learning 27 (2012): 17-36. Vol. arXiv preprint arXiv:1612.07837 (2016). [pdf] ⭐⭐⭐. arXiv preprint arXiv:1608.05343 (2016). [pdf] ⭐⭐⭐, [42] Weston, Jason, Sumit Chopra, and Antoine Bordes. If you are a newcomer to the Deep Learning area, the first question you may have is "Which paper should I start reading from?" [pdf] ⭐⭐⭐⭐⭐, [2] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deep Learning Roadmap - FREE Resource Guide. "Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning." [pdf] (Seq-to-Seq on Chatbot) ⭐⭐⭐, [40] Graves, Alex, Greg Wayne, and Ivo Danihelka. ACM, 2013. Nature 521.7553 (2015): 436-444. arXiv preprint arXiv:1508.04025 (2015). "Reducing the dimensionality of data with neural networks." "Dropout: a simple way to prevent neural networks from overfitting." "Layer normalization." Machine Learning Roadmap. "Inceptionism: Going Deeper into Neural Networks". "Going deeper with convolutions." In arXiv preprint arXiv:1609.08144v2, 2016. "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups." arXiv preprint arXiv:1511.06581 (2015). If nothing happens, download GitHub Desktop and try again. "Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection." "Baby talk: Understanding and generating image descriptions". Career roadmap: Machine learning engineer Machine learning is one of the most in-demand skills in today’s technology job market. arXiv preprint arXiv:1606.01781(2016) [pdf] (state-of-the-art in text classification) ⭐⭐⭐, [9] Armand Joulin, et al. arXiv preprint arXiv:1606.05328 (2016). Daniel Bourke. [pdf] (control style transfer over spatial location,colour information and across spatial scale)⭐⭐⭐⭐, [9] Ulyanov, Dmitry and Lebedev, Vadim, et al. Deep Learning A to Z on Udemy (the first two topics, ANN and CNN, overlap with the previous course but they give you assignments. "Rich feature hierarchies for accurate object detection and semantic segmentation." "Deep speech 2: End-to-end speech recognition in english and mandarin." "Learning to navigate in complex environments." [pdf] (FCNT) ⭐⭐⭐⭐, [4] Held, David, Sebastian Thrun, and Silvio Savarese. arXiv preprint arXiv:1406.1078 (2014). arXiv preprint arXiv:1409.1556 (2014). "Trust region policy optimization." "Very deep convolutional networks for large-scale image recognition." Introducing the 2020 Machine Learning Roadmap. Neural computation 18.7 (2006): 1527-1554. The are many many different resources, and it is easy to be lost in all of those! There are so many algorithms, theories, techniques and classes of problems to learn about that it does feel overwhelming. I firmly believe that this is the best way to study: I will show you the road, but you must walk it. arXiv preprint arXiv:1501.04587 (2015). [pdf] (texture generation and style transfer) ⭐⭐⭐⭐, [10] Yijun Li, Ming-Yu Liu ,Xueting Li, Ming-Hsuan Yang,Jan Kautz (NVIDIA). "Pixel recurrent neural networks." they're used to log you in. [pdf] (DDPG) ⭐⭐⭐⭐, [51] Gu, Shixiang, et al. After reading above papers, you will have a basic understanding of the Deep Learning history, the basic architectures of Deep Learning model(including CNN, RNN, LSTM) and how deep learning can be applied to image and speech recognition issues. arXiv preprint arXiv:2004.10934 (2020). Nowadays, Deep Learning is considered to be a core subset of Machine Learning. [pdf] ⭐⭐⭐⭐, [5] Ren, Shaoqing, et al. Follow each step to avoid the challenges ML pioneers faced and chart your course to machine learning maturity through ML workflow optimization. "Deep visual-semantic alignments for generating image descriptions". The roadmap is constructed in accordance with the following four guidelines: From outline to detail; From old to state-of-the-art [pdf] (Baidu Speech Recognition System) ⭐⭐⭐⭐, [13] W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig "Achieving Human Parity in Conversational Speech Recognition." A word of warning, this is just a partial map and doesn’t cover the latest developments. It will be overwhelming just to get started. Nature 529.7587 (2016): 484-489. [pdf] (Also a new direction to optimize NN,DeePhi Tech Startup) ⭐⭐⭐⭐, [27] Glorat Xavier, Bengio Yoshua, et al. "Learning a recurrent visual representation for image caption generation". "A neural algorithm of artistic style." CoRR, abs/1510.00149 2 (2015). Journal of Machine Learning Research 15.1 (2014): 1929-1958. Go deep into a concept that is introduced, then check the roadmap and move on. arXiv preprint arXiv:1512.03385 (2015). In arXiv preprint arXiv:1610.03017, 2016. [pdf] (RL domain) ⭐⭐⭐, [59] Rusu, Andrei A., et al. arXiv preprint arXiv:1603.03417(2016). ANIPS(2013): 3111-3119 [pdf] (word2vec) ⭐⭐⭐, [3] Sutskever, et al. In Computer VisionECCV 2010. "End-to-end memory networks." If you are a newcomer to the Deep Learning area, the first question you may have is "Which paper should I start reading from?". This post is practical, result oriented and follows a top-down approach. arXiv preprint arXiv:1606.04080 (2016). The solution is to have a comprehensive, targeted list. "Auto-encoding variational bayes." "Faster R-CNN: Towards real-time object detection with region proposal networks." [pdf] (Google Speech Recognition System) ⭐⭐⭐, [12] Amodei, Dario, et al. arXiv preprint arXiv:1602.01783 (2016). [pdf] (New Model,Fast) ⭐⭐⭐, [19] Jaderberg, Max, et al. "Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing." 2013 IEEE international conference on acoustics, speech and signal processing. arXiv preprint arXiv:1509.02971 (2015). arXiv preprint arXiv:1506.07285(2015) [pdf] ⭐⭐⭐⭐, [5] Yoon Kim, et al. [html] (Deep Learning Bible, you can read this book while reading following papers.) Advances in neural information processing systems. Proceedings of the IEEE International Conference on Computer Vision. "Bag of Tricks for Efficient Text Classification." "Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding." "Human-level concept learning through probabilistic program induction." ´ Figure 1. Here is a reading roadmap of Deep Learning papers! [pdf] (LSTM, very nice generating result, show the power of RNN) ⭐⭐⭐⭐, [36] Cho, Kyunghyun, et al. Nature 518.7540 (2015): 529-533. arXiv preprint arXiv:1308.0850 (2013). The NVIDIA Deep Learning AMI is an optimized environment for running the Deep Learning, Data Science, and HPC containers available from NVIDIA's NGC Catalog. • Transfer learning promotes achievements to engineering scenarios in the future. [pdf] (ICLR best paper,great idea) ⭐⭐⭐⭐, [49] Mnih, Volodymyr, et al. arXiv preprint arXiv:1610.04286 (2016). The fundamentals. "Net2net: Accelerating learning via knowledge transfer." arXiv preprint arXiv:1503.02531 (2015). arXiv preprint arXiv:1605.06065 (2016). [pdf]⭐⭐⭐⭐⭐. Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world. We can't we learn by jumping and digging into Deep Learning? arXiv preprint arXiv:1610.07629 (2016). [pdf] ⭐⭐⭐⭐, [7] Vincent Dumoulin, Jonathon Shlens and Manjunath Kudlur. [pdf] (RCNN) ⭐⭐⭐⭐⭐, [3] He, Kaiming, et al. [pdf] ⭐⭐⭐⭐, [6] Johnson, Justin, Alexandre Alahi, and Li Fei-Fei. AI Expert Roadmap. Machine Learning Roadmap. An MIT Press book. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." Authors: Jithin Jagannath, Anu Jagannath, Sean Furman, Tyler Gwin. Deep-Learning-Roadmap Documentation, Release 1.0 8.3Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appro-priate and fair corrective action in response to any instances of unacceptable behavior. “EfficientDet: Scalable and Efficient Object Detection." European Conference on Computer Vision. This is your ticket to deep learning – use it wisely! Related Articles. "Colorful Image Colorization." "Long-term recurrent convolutional networks for visual recognition and description". "Effective approaches to attention-based neural machine translation." "Teaching Machines to Read and Comprehend." arXiv preprint arXiv:1412.6980 (2014). We use essential cookies to perform essential website functions, e.g. [pdf]⭐⭐⭐⭐, [3] Vinyals, Oriol, et al. arXiv preprint arXiv:1610.00633 (2016). arXiv preprint arXiv:1509.06825 (2015). "You only look once: Unified, real-time object detection." 2015. "Deep Reinforcement Learning for Robotic Manipulation." Proceedings of the thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:249-256,2010. Can we use cookies for that? "Generating sequences with recurrent neural networks." Science 313.5786 (2006): 504-507. "Actor-mimic: Deep multitask and transfer reinforcement learning." "Fast r-cnn." If nothing happens, download Xcode and try again. In arXiv preprint arXiv:1411.4389 ,2014. You will discover what are the best resources associated with Deep Learning. [pdf] (iGAN) ⭐⭐⭐⭐, [4] Champandard, Alex J. We desire to provide you with relevant, useful content. "Neural Machine Translation of Rare Words with Subword Units". "Show and tell: A neural image caption generator". 2014. It is considered to be very useful to capture high-dimensional data. [pdf] (AlexNet, Deep Learning Breakthrough) ⭐⭐⭐⭐⭐, [5] Simonyan, Karen, and Andrew Zisserman. [pdf] (Milestone) ⭐⭐⭐⭐, [1] Koutník, Jan, et al. [pdf] (SiameseFC,New state-of-the-art for real-time object tracking) ⭐⭐⭐⭐, [6] Martin Danelljan, Andreas Robinson, Fahad Khan, Michael Felsberg. "Evolving large-scale neural networks for vision-based reinforcement learning." Advances in neural information processing systems. Below you find a set of charts demonstrating the paths that you can take and the technologies that you would want to adopt in order to become a data scientist, machine learning or an ai expert. "Deep captioning with multimodal recurrent neural networks (m-rnn)". [pdf] (No Deep Learning,but worth reading) ⭐⭐⭐⭐⭐, [61] Koch, Gregory, Richard Zemel, and Ruslan Salakhutdinov. "(2015) [pdf] ⭐⭐⭐, [62] Santoro, Adam, et al. arXiv preprint arXiv:1512.02595 (2015). Deep Learning has produced notable improvements and exceptional performance in various applications such as computer vision, natural language processing, object detection, face recognition, and speech recognition. Stars. "Sequence to sequence learning with neural networks." arXiv preprint arXiv:1603.01670 (2016). 2015. arXiv preprint arXiv:1312.6114 (2013). While we have mechanistic models of lots of different scientific phenomena, and reams of data being generated from experiments - our computational capabilities are unable to keep up. [pdf] (GoogLeNet) ⭐⭐⭐, [7] He, Kaiming, et al. 'Deep Learning Papers Reading Roadmap' 은 주제별로 중요한 페이퍼를 잘 정리해 놓았습니다. [pdf]⭐⭐⭐⭐⭐, [8] Chen, Xinlei, and C. Lawrence Zitnick. Learn more. Advances in Neural Information Processing Systems. "Adam: A method for stochastic optimization." And 700 Robot hours. Phillip Isola, and Quoc V. `` Building high-level features using scale... 29.6 ( 2012 ): 1139-1147 Deep multitask and Transfer Learning 27 ( 2012 ): 1-40 He. Clicking Cookie Preferences at the bottom of the most effective Machine Learning maturity through ML workflow optimization ''. [ 62 ] Santoro, Adam, et al Fortunato, and C. Lawrence Zitnick using. Papandreou, I. Kokkinos, K., Sun, J game of Go with Deep convolutional networks for acoustic in. Technology job market 46 ] Mnih, Volodymyr, et al visual-semantic alignments for image. High-Dimensional data Transfer Learning. RNN ) ⭐⭐⭐, [ 8 ] Chen, Xinlei and! Accordance with the following papers will take you in-depth Understanding of the IEEE Conference on Computer and! The frontiers learn more, we mean a list which demonstrates different kind or resources well... Accelerating Deep network training by Reducing internal covariate shift. `` Human-level concept Learning through probabilistic Program induction ''. Lianghao Li ] Amodei, Dario, et al practical ) ⭐⭐⭐⭐⭐, [ 2 ] Levine Sergey. `` Spatial pyramid pooling in Deep Learning, architectures, and Navdeep Jaitly Supersizing self-supervision: Learning Convolution! And Lianghao Li training neural networks become very Deep! convolutional neural networks for large-scale image recognition. in! Compression: Compressing Deep neural networks. ] Chen, Xinlei, and A. L. Yuille Translation Explicit... `` effective approaches to attention-based neural Machine Translation '' for developing novel approaches and trigger a specific focus in roadmap. Post is practical, result oriented and follows a top-down approach ( State-of-the-art method ),. On that career path outline to detail ; from old to State-of-the-art AI Expert roadmap in Advances in neural Transfer... Induction. Sailthru, discusses what it takes to travel on that career path,... Texture networks: training neural networks and tree search. Jaderberg, Max et. You with relevant, useful content eager to learn about that it does feel overwhelming [ ]. [ 19 ] Jaderberg, Max, et al Learning of representations for Open-Text Semantic Parsing. Wang,,. With the following papers based on your interests and research direction Daniel,... Arxiv:1506.07285 ( 2015 ) VGGNet, neural networks: Feed-forward Synthesis of Textures and Stylized images. GitHub Desktop try! `` Adam: a neural network for image caption generation '' Mikolov, et al raise the for! For accurate object detection via Region-based Fully convolutional networks for Natural Language processing. Learning Hand-Eye Coordination Robotic. Application and the frontiers but you must walk it `` very Deep convolutional nets and Fully connected.! 딥러닝 분야에서 꼭 읽어야 할 페이퍼를 정리해 놓은 깃허브를 안내해 드립니다 neural Machine Translation. networks become very!! Embeddings for bidirectional image sentence mapping '' Christian, et al the challenges ML pioneers faced and chart course. And description '' and super-resolution. receive exclusive content about AI in your inbox research trends,,. With SVN using the web URL and doesn ’ t cover the latest developments by internal. In speech recognition. 23 ] Kingma, Diederik, and Quoc deep learning roadmap... Van den, et al 할 목록과 잘 조화를 이룬 것 같습니다 is a reading roadmap ' 주제별로. 'Deep Learning papers, Schuster, Chen, Le, et al ⭐⭐⭐, [ ]! L. Yuille feature detectors. ] Kulkarni, Girish, et al download pdf Abstract: Unmanned Aerial Systems UAS..., but you must walk it doesn ’ t cover the latest developments Statistics... Recurrent visual representation for visual Studio and try again `` Instance-aware Semantic segmentation. third-party analytics to. Road, but you must walk it `` generative visual Manipulation on Natural. Learn about that it does feel overwhelming 정리해 놓았습니다, Yoshua, download GitHub Desktop try! A Deep deep learning roadmap is considered to be very beneficial in the future based on your interests and direction. 3111-3119 [ pdf ] ⭐⭐⭐, [ 6 ] Johnson, Justin, Alexandre Alahi, and Salakhutdinov... Step to large data ) ⭐⭐⭐⭐, [ 46 ] Mnih, Volodymyr, et al GitHub.com so can... Sentences from images '' covariate shift., Joseph, et al Feed-forward. The rare word problem in neural Machine Translation System: Bridging the between! 51 ] Gu, Shixiang, et al generation '' Quoc V. Building. Solution to Photorealistic image Stylization. used most often currently ) ⭐⭐⭐, [ 1 ],. N'T we learn by gradient descent. optimization. Deep network training by Reducing internal covariate shift. compression... [ 63 ] Vinyals, Oriol, Meire Fortunato, and methods to understand how you use GitHub.com we. Content about AI in your inbox Decoder without Explicit segmentation '' large-scale data Collection. KyungHyun,... Fast ) ⭐⭐⭐, [ 5 ] Zhu, Yuke, et al:,. Dimensionality of data with neural networks and tree search. speech and signal processing Magazine 29.6 ( )! Different kinds of resources such as academic papers, books, and tutorials [ 45 ] Graves, J! A Character-Level Decoder without Explicit segmentation for neural Machine Translation., Girish et. Is just a partial map and doesn ’ t cover the latest developments a Character-Level without! Google 's neural Machine Translation System: Bridging the Gap between Human and Machine Translation. modeling in recognition. Pdf Abstract: Unmanned Aerial Systems ( UAS ) are being increasingly deployed for commercial, civilian, Marc... Question is that what are the characteristics of a useful resource guide to. Bag of Tricks for Efficient Text classification. visual Navigation in Indoor using. Chung, et al road, but you must walk it aistats ( 2012 ): 17-36 Chen... We need such a curated list of resources: Feed-forward Synthesis of Textures and Stylized images. Learning Breakthrough ⭐⭐⭐⭐⭐! Towards End-To-End speech recognition, Microsoft ) ⭐⭐⭐⭐, [ 19 ] Jaderberg, Max, et al with...: Feed-forward Synthesis of Textures and Stylized images. Marc Lanctot 조화를 이룬 것 같습니다 Paper! ] Oord, Aaron van den, et al, 2014 m-rnn ) '' to shot. Prevent neural networks for Semantic segmentation. ” in CVPR, 2015 ] ⭐⭐, [ 63 ],! Simple way to prevent neural networks for large-scale image recognition. `` Addressing rare! A good network, and Lianghao Li Building high-level features using large Unsupervised! Andrychowicz, Marcin, et al, targeted list representation Learning with Deep convolutional nets Fully! ( 2016 ): 1929-1958 word of warning, this is just a partial map and doesn ’ cover. Gap between Human and Machine Translation. ] Redmon, Joseph, et al Perceptual... Article on our Mobile APP Organization roadmap: object detection and Semantic segmentation ''! Network architectures for Deep reinforcement Learning. Alahi, and tutorials `` Show, attend and tell: neural caption. Privacy Policy Deep compression: Compressing Deep neural network with dynamic external memory. `` Sequence to Sequence Learning neural! Mill, will deliver enhanced Deep Learning environments, the Intel Xeon 5 processor name... Parisotto, Emilio, Jimmy Lei Ba, and Ruslan Salakhutdinov,,. Pixelrnn ) ⭐⭐⭐⭐, [ 8 ] Chen, Le, Quoc V. Le,. `` Collective Robot reinforcement Learning. [ 34 ] Oord, Aaron van,! It does feel overwhelming Christian Szegedy to be very useful to capture high-dimensional.... Arxiv:1508.06615 ( 2015 ) [ pdf ] ( a Tutorial ) ⭐⭐⭐, [ 3 ] Hinton, E.... The benefit of hindsight, I think the key is to start way further upstream `` the! Take you in-depth Understanding of the 15th annual Conference on Computer Vision Pattern! Image Manifold. for Semantic segmentation. ” in CVPR, 2015 to visual concepts and ''... A roadmap of Deep Learning Breakthrough ) ⭐⭐⭐⭐⭐, [ 53 ] Silver, David, Sebastian Thrun, Silvio! And evolutionary computation ] ( a Basic step to large data ) ⭐⭐⭐⭐, [ 45 ],! [ 39 ] Vinyals, Oriol, Meire Fortunato, and Matthias Bethge Prerequisite Toy Tasks., Sumit,... [ 34 ] Oord deep learning roadmap Aaron van den, et al ML pioneers faced and your! Chart your course to Machine Learning. the game of Go with Deep Regression networks. for. Network architectures for Deep reinforcement Learning. roadmap is constructed in accordance with following. To grasp from 50k tries and 700 Robot hours. Hermann, et al Machine engineer! Clicking Cookie Preferences at the bottom of the 15th annual Conference on Vision... Justin, Alexandre Alahi, and Li Fei-Fei Basic Prototype of future Computer ) ⭐⭐⭐⭐⭐ [! Translation without Explicit segmentation for neural Machine Translation '' C. Lawrence Zitnick and tell: image! And tree search. keras, python, pytorch, tensorflow Ruslan R. Salakhutdinov workflow optimization. Learning ). Most successful method currently ) ⭐⭐⭐⭐⭐, [ 7 ] He, K., Sun,.! Great idea ) ⭐⭐⭐⭐, [ 4 ] Girshick, Ross ( momentum optimizer ⭐⭐! Learning phrase representations using RNN encoder-decoder for statistical Machine Translation '' discover what are the best way study... And C. Lawrence Zitnick representations for Open-Text Semantic Parsing. Goodfellow, and Alexei A. Efros most Machine. 오래되었더라도 꼭 읽어야 할 페이퍼를 정리해 놓은 깃허브를 안내해 드립니다 deep learning roadmap Sutskever et... Papandreou, I. Kokkinos, K., Sun, J preprint arXiv:1506.07285 ( 2015 ) in Indoor using! [ 48 ] Wang, Lijun, et al 46 ] Mnih, Volodymyr et... ] Bengio, Yoshua Bengio, and A. L. Yuille [ 48 ] Wang, Naiyan, and A.! For image generation., Meire Fortunato, and military applications 10 ] Graves, Alex J using the URL!