# Reinforcement learning and its relationship to supervised

### Reinforcement Learning and Supervised Learning: A brief comparison

Most beginners in Machine Learning start with learning Supervised However, one of the most important paradigms in Machine Learning is. This chapter contains sections titled: Introduction. Supervised Learning. Reinforcement Learning. Sequential Decision Tasks. Supervised. I've been following the Machine Learning space for a while now, and it's In supervised algorithms, you may not know the inner relations of the.

This scalar value tells us whether the outcome of whatever we did was good or bad. Hence, the goal of RL is to take actions in order to maximize reward.

This process is memoryless, so everything we care about we know through the current state. The RL setup can be visualized like this: In Supervised Learning, given a bunch of input data X and labels Y we are learning a function f: The function will be able to predict Y from novel input data with a certain accuracy if the training process converged.

We are given a set of states S and a set of actions A. P is the state transition probability. The reward is a value that tells us how good we did in terms of the goal we want to optimize towards. It is given by a reward function R: This function is called the policy function. The objective is now to find an optimal policy that maximizes the expected sum of rewards. This is also called the control problem.

The game of Go can be modeled with this approach in the following way: Position of all pieces Actions: Where the player put its piece down Reward: Was it the move we made three actions before or the current one? We call this the credit assignment problem. Our optimization problem is maximizing the expected sum of discounted rewards. Thus, the optimal policy can be found by calculating the result of this equation: To learn the optimal policy, there are different approaches such as policy gradient and Q-Learning.

While policy gradient tries to learn the policy directly, Q-Learning is learning a function of state-action pairs. I will delay a detailed explanation of these algorithms to a future post.

In Supervised Learning, we use Deep Learning because it is unfeasible to manually engineer features for unstructured data such as images or text. Algorithms for performing binary classification are particularly important because many of the algorithms for performing the more general kind of classification where there are arbitrary labels are simply a bunch of binary classifiers working together.

For instance, a simple solution to the handwriting recognition problem is to simply train a bunch of binary classifiers: The classifier just outputs the digit whose classifier has the highest certainty. A large subclass of unsupervised tasks is the problem of clustering. Clustering refers to grouping observations together in such a way that members of a common group are similar to each other, and different from members of other groups. A common application here is in marketing, where we wish to identify segments of customers or prospects with similar preferences or buying habits.

## Reinforcement Learning and Supervised Learning: A brief comparison

A major challenge in clustering is that it is often difficult or impossible to know how many clusters should exist, or how the clusters should look. Generative models are models that imitate the process that generates the training data.

A good generative model would be able to generate new data that resembles the training data in some sense. This type of learning is unsupervised because the process that generates the data is not directly observable—only the data itself is observable.

Recent developments in this field have led to startling and occasionally horrifying advances in image generation. The image here is created by training a kind of unsupervised learning model called a Deep Convolutional Generalized Adversarial Network model to generate images of faces and asking it for images of a smiling man.

In reinforcement learning, we do not provide the machine with examples of correct input-output pairs, but we do provide a method for the machine to quantify its performance in the form of a reward signal.

Reinforcement learning methods resemble how humans and animals learn: Reinforcement learning is useful in cases where the solution space is enormous or infinite, and typically applies in cases where the machine can be thought of as an agent interacting with its environment. One of the first big success stories for this type of model was by a small team that trained a reinforcement learning model to play Atari video games using only the pixel output from the game as input.

In order to implement a supervised learning to the problem of playing Atari video games, we would require a dataset containing millions or billions of example games played by real humans for the machine to learn from.

By contrast, reinforcement learning works by giving the machine a reward according to how well it is performing at its task.

Simple video games are well suited to this type of task since the score works well as a reward. The machine proceeds to learn by simulation which patterns maximize its reward.

Often, hybrid approaches between some or all of these different areas lead to good results. For instance, an important task in some areas is the task of anomaly detection. An anomaly detection algorithm monitors some signal and indicates when something weird happens. A good example is fraud detection.

### 5 Things You Need to Know about Reinforcement Learning

We want an algorithm that monitors a stream of credit card transactions and flags weird ones. But what does weird mean? There are certainly some known patterns that we would like the algorithm to be able to detect, and we can train a supervised learning model by showing it examples of the known fraud patterns.