Welcome to part two of my three-part blog series. In blog one I reviewed some common AI terminology at a high level. In this blog I will explore the differences between machine learning and decision optimization.
If you don’t possess an advanced degree in a STEM field, you will be forgiven for not knowing the difference between Machine Learning and Decision Optimization. In fact, you might have heard your boss say some version of the following: “we need to optimize this process with machine learning”. Does this verbiage make any sense?
Yes and no. Technically speaking, optimizations can exist without the presence of Machine Learning. Machine Learning on the other hand can rarely exist without some form of optimization. Allow me to explain:
Machine Learning is a subset of AI which uses statistical modelling to enable machines to learn from data. You provide data to a pre-defined mathematical structure and ask it to learn the necessary elements to perform a task well. How does this learning occur? In most cases, the authors of the algorithm created what’s called a cost function to evaluate how well the machine is currently performing. If the cost function has a high value, the machine is performing poorly, and if the cost is low, the machine is performing well. We then use some form of optimization to adjust the parameters of the model to minimize the cost. This tuning leads to a machine that performs better than its previous state, which is commonly referred to as “learning”.
What do you mean by “pre-defined mathematical structure”?
When an academic releases a paper describing a new Machine Learning method, they often start with some non-optimized structure, and describe how it can be optimized with your data to perform a task well. Let’s say for example you wanted to create a machine learning model to predict the price of a house based on square footage. You might start with a line-of-best-fit approach, and say the price can be modelled with the following structure:
If we have data, we know for each house the price and the square footage, but what about these mysterious quantities’ alpha and beta? This is where optimization comes in: the author will pick a cost function (in this case usually the squared error of your prediction) and minimize this cost by adjusting those parameters. This leads towards an optimized/trained model.
Ok, so we know optimization is used in machine learning, but how is Machine Learning different from pure decision optimization?
In decision optimization, it’s up to you to define your structure. Every time you start a new optimization, you need to mathematically define all the variables present in your system, the constraints on those variables, and the objective function you are trying to optimize.
For example: In the 1930’s the United States army decided they wanted to apply decision optimization to minimize the cost of feeding their troops. They constructed the function they wanted to minimize: the sum of the cost all food they are feeding one soldier. Obviously with a problem like this the minimum cost to feed a soldier would be $0, as you could choose to feed them nothing. This highlights the need for us to set our constraints: each soldier has a minimum amount of each type of nutrient they must consume every day, and a recommended caloric intake. With these variables and constraints defined, they were able to optimize this diet by minimizing cost to the US government, while still maintaining healthy soldiers.
To-Do List
You might have noticed that there is more complexity in the Decision Optimization problem when compared to the machine learning model. This is because there is more power given to the person applying optimization. With Machine Learning, there is typically software that automates the solution and the structure, but in optimization, the structure is completely up to you. This leads to applications of decision optimization being more susceptible to the pitfall of “garbage in, garbage out”: the optimal solution of a poorly defined structure is meaningless.
Stay tuned for part 3: How can you apply cutting-edge AI in your business?
Oliver is a Senior data science consultant, helping clients maximize the value of business data. After graduating with a Bachelor of Mechanical Engineering from McGill University, he later went on to study at Georgia Institute of Technology in their Master of Science in Analytics program. Since then, he has been focused on delivering production-ready data science solutions to domains such as manufacturing, engineering, finance, and retail.
