Self-Study Guide to Machine Learning

 There are lots of things you can do to learn about machine learning.

There are resources like books and courses you can follow, competitions you can enter and tools you can use.

In this post I want to put some structure around these activities and suggest a loose ordering of what to tackle when in your journey from programmer to machine learning master.

Four Levels of Machine Learning

Consider four levels of competence in machine learning. This is a model to help us think about the resources and activities available and when a good time to tackle them might be.

1. Beginner
2. Novice
3. Intermediate
4. Advanced

I want to separate beginner from novice here because I want to show that an absolute beginner (a programmer with an interest in the field) has a path before them if they choose.

We are going to tour through each of these four levels and look at resources and activities that can help someone at one level learn more and progress their understanding and skill levels.

The breakdown is just a suggestion, and it is very likely that some activity or resource at a level before or after can be very useful and appropriate at a given level in the breakdown.

I think the overall structure is useful, I’m keen to hear what you think, leave a comment below with your thoughts.

Credited to pugetsoundphotowalks, some rights reserved

Beginner

A beginner is a programmer with an interest in machine learning. They may have started to read a book, Wikipedia page, or taken a few lessons in a course, but they don’t really “get it” yet. They’re frustrated because the advice they are getting is for intermediates and advanced levels.

Beginners need a gentle introduction. Away from code and textbooks and courses. They need the whys and whats and hows pointed out first to lay the foundation for novice-level material.

Some activities and resources for the absolute beginner are:

• Introductions from Books: Read introductions to good machine learning and data mining books for programmers, such as:
  • Machine Learning for Hackers
  • Programming Collective Intelligence
  • Data Mining: Practical Machine Learning Tools and Techniques
• Overview Videos: Watch presentations that give an overview of machine learning to laymen audiences. Some examples include: Interview with Tom Mitchell and Peter Norvig on big data Facebook Tech Talk.
• Talk to People: Ask how they got started in the field, what resources they recommend for beginners, what excites them about the field.

Novice

A novice has had some contact with the field of Machine Learning. They have read a book or taken a course. They know they are interested and they want to know more. They are starting to get it and want to start to get things done.

Novices need something to do. They need to be put into action to have the material grounded and integrated into existing knowledge structures like the programming languages they know or the problems they are used to solving.

Some activities and resources for the novice are:

• Complete a Course: Take and complete a course like the Stanford Machine Learning course. Take a lot of notes, complete the homework if possible, ask a lot of questions.
• Read some Books: Not textbooks, but friendly books like those listed above targeted at beginner programmers.
• Learn a Tool: Learn to drive a tool or library like Scikit-Learn, WEKA, R or similar. Specifically, learn how to use an algorithm you have read or learned about in a book or course. See it in action and get used to trying things out as you learn them.
• Write Some Code: Implement a simpler algorithm like a perceptron, k-nearest neighbour or linear regression. Write little programs to demystify methods and learn all the micro-decisions required to make it work.
• Complete Tutorials: Follow and complete tutorials. Start building up a directory of small projects that you have completed with datasets, scripts and even source code you can look back on, read and think about.

Intermediate

A novice has read some books and completed some courses. They know how to drive some tools and have written a bunch of code both implementing simple algorithms and completing tutorials. An intermediate is breaking out on their own, devising their own projects to learn new techniques and interacting and learning from the greater community.

The intermediate is learning how to implement and wield algorithms accurately, competently and robustly. They are also building the skills of spending a lot of time with data up front, cleaning, summarizing and thinking about the types of questions that it can answer.

Some activities and resources for the intermediate are:

• Small Projects: Devise small programming projects and experiments where machine learning can be used to solve a problem. This is like designing and executing your own tutorials in order to explore a technique you’re interested in. You may implement an algorithm or link to a library that provide the algorithm. Learn more about small projects.
• Data Analysis: Get used to exploring and summarizing datasets. Automate reports, know which tools to use when, and look for data you can explore, clean, and on which you can practice techniques and communicate something interesting.
• Read Textbooks: Read and internalize textbooks on machine learning. This may very well require skills to grok mathematical descriptions of techniques and acknowledging formalisms that describe classes of problems and algorithms.
• Write Plugins: Write plugins and packages for open source machine learning platforms and libraries. This is an exercise in learning how to write robust and production level algorithm implementations. Use your own plugins on projects, ask for code reviews from the community and work to get the code included into the platform if possible. Getting feedback and learning is the goal.
• Competitions: Participate in machine learning competitions, such as those associated with conferences or offered on platforms like Kaggle. Get involved in discussions, ask questions, learn how other practitioners are approaching the problem. Add to your repository of projects, methods and code from which you can draw.

Advanced

An advanced practitioner has written a lot of code either integrating machine learning algorithms or implementing algorithms themselves. They may have competed in competitions or written plugins. They have read the textbooks, completed the courses and have a broad knowledge of the field, as well as a deep knowledge on a few key techniques of which they prefer.

The advanced practitioner builds, deploys and maintains production systems that use machine learning. They keep abreast of new developments in the fields and eagerly seek out and learn the nuances of a method and tips passed around from other frontline practitioners like themselves.

Some activities and resources for the advanced practitioner are:

• Customizing Algorithms: Modify algorithms to meet their needs, which may involve implementing customizations outlined in conference and journal papers for similar problem domains.
• New Algorithms: Devising entirely new methods based on the underlying formalisms to meet the challenges they encounter. It is more about getting the best results possible rather than advancing the frontier of the field.
• Case Studies: Read and even recreate case studies completed for machine learning competitions and by other practitioners. These “how I did it” papers and posts are usually chock full of subtle pro tips for data preparation, feature engineering and technique usage.
• Methodology: Systemizing of processes, whether formally or for themselves. They have a way to approach problems and get results at this point and they are actively looking for ways to further refine and improve that process with tips, best practices and new and better techniques.
• Research: Attending conferences, reading research papers and monographs, having conversations with experts in the field. They may write up some of their work and submit it for publication, or just drop it in a blog post and get back to work.

Mastery is continuous, the learning does not end. One could pause and detour at any point along this journey and become the “competition guy” or the “pro library guy“. In fact, I expect such detours to be the norm.

This breakdown could be read as a linear path of the technicians journey from beginner to advanced level, it’s intentionally programmer centric. I’m keen to hear criticisms of this reading so that I can make it better. This breakdown is just my suggestions of the types of activities to tackle if you find yourself hungering for more at a given level.

So what level are you and what are you going to take on next? Leave a comment!