Tag Archives: homer simpson

Homer Simpson Guide to Backpropagation

IQ-homer

Homer Simpson has a low IQ of 55

Backpropagation algorithm is based on complex mathematical calculations. That’s why, it is hard to understand and that is the reason why people have an edge on neural networks. Adapting the concept into the real world makes even Homer Simpson easier to figure out. In this post, we’ll mention how to explain backpropagation to beginners.

What if an approved loan application would be outstanding loan (or dead loan)? The bank loses money. So, how can this financial institution derive lesson from this mistake?

Loan application is a process. In other words, an application is required to be examined by multiple authorized employees respectively. For instance, a customer makes an application to bank branch agent, then agent delivers the application to branch supervisor or branch manager. After then, head office employees examine the application when branch manager approved. To sum up, a loan application follows a path and comes to hands of in charged of employees. Should these employees responsible for the lose? The answer is yes based on backpropagation.

Backpropagation algorithm proposes to reflect the lost money amount on the same path, but backwardly. That’s why, it is named as back-propagation. Fine head office employees first, then punish branch manager, supervisor, and agent respectively. What’s more, how much the total lose amount should be reflected to a branch agent? Total lose amount should be divided between in charged of employees based on their contributions on total lose. (Actually, that is the derivative of total lose amount with respect to the employee. E.g. ∂TotalLoseAmount / ∂BranchAgent).  In this way, these employees would be more careful in the next time. That is the principle of backpropagation algorithm. Thus, examination process would progress in time.

As the phrase goes, backpropagation advices slapping ones who are on the tracked path backwardly and in the ratio of their contribution on total error. I would like to thank Dr. Alper Ozpinar for this metaphor.

batman-slap-v2

Batman backpropagates Robin

Homer Sometimes Nods: Error Metrics in Machine Learning

Even the worthy Homer sometimes nods. The idiom means even the most gifted person occasionally makes mistakes. We would adapt this sentence to machine learning lifecycle. Even the best ML-models should make mistakes (or else overfitting problem). The important thing is know how to measeure errors. There are lots of metrics for measuring forecasts. In this post, we will mention evalution metrics meaningful for ML studies.

homer-doh-small

Homer Simpson uses catchphrase D’oh! when he has done something wrong

Sign of actual and predicted value diffence should not be considered when calculation total error of a system. Otherwise, total error of a series including equally high underestimations and overestimations might measure very low error. In fact, forecasts should include low underestimations and overestimations, and total error should be measured low. Discarding sign values provides to get rid of this negative effect. Squaring differences enables discarding signs. This metric is called as Mean Squared Error or mostly MSE.

Continue reading