We all talk about artificial intelligence nowadays. Even innovations often named including AI or machine learning terms. Have you ever think about that what makes AI too popular today? Improvements in software might contribute but dominant accelerator and driver were definitely hardware. We will intellectualize this argument.
We have already known how to train computers since 30 years. The algorithm was found but data volume wasn’t enough and it starved for really big data. It also has required high computation power. Old hardwares cannot serve performly for that level of computations. Therefore, we could not run the algorithm as we wish. Then, we realized that processing units developed for 3D game lovers can also run complex neural networks computations because these units designed to apply large matrix operations. Moreover, everyone shares photos and information (check-in, comment) on internet. In this way, data shortage is solved. Finally, we have been modeling neural networks with 3 layers. Increasing number of layers created deep learning. Deep term came from here. So, all of these modifications have worked well unexpectedly. Now, we can teach everything if we feed data to these systems. – Cem Say, Watson Istanbul Summit 2017
🙋♂️ You may consider to enroll my top-rated machine learning course on Udemy
Supportingly, Barbara‘s deep learning definition is my favorite. She defines deep learning as matrix multiplication, a lot of matrix multiplication.
Why Apple Succeed
Alan Kay means a lot for technology enthusiasts. He pioneers to develop first GUI concept, first object oriented programming language, and he designed first tablet PC in 70s. Beyond these inventions, he considers hardware as important as software.
People who are really serious about software should make their own hardware – Alan Kay, Creative Think Seminar (1982)
You can see that Apple adopts this quote as a principle in its products. Apple has already cited this quotation in ipad launching in 2007.
He was invited to the premiere. It might be because he is the inventor of prototype of a tablet computer or legacy version of iPad.
Data Structures for machine learning
If you define a constant variable, it would be scalar. Matrices store arrays of scalar values in multi dimensional space. A vector is a 1-dimensional matrix. Herein, tensors state a matrix where each item is a matrix. You can model a neural network with scalar variables but this increase the computation time radically. Instead of this, if you model your network with Tensors, this will reduce the computation time dramatically. The name of Google’s deep learning framework TensorFlow
Hardware for machine learning
We’ve mentioned that trigger of machine learning adoption was graphic processing units but GPUs were not created for machine learning studies. Herein, tensor processing units are hardware designed to compute complex tensor operations as well. In other words, TPUs are hardware that specialized to accelerate machine learning workload.
Today, Google produces its own TPUs just like Alan Kay declared and Apple adopted. This technology was first announced in Google IO 2016. Even imagination is hard how these hardware accelerate the process.
If you are training an image recognition model, Let’s say ResNet 50, it is a sort of standard benchmark right now. It was state of the art, not too long ago. And if you want to train that to 75% accuracy, which is what you expect from publication on the subject, that might previously have taken days a few years ago, when that paper was published. Now, that is down to about 12.5 hours on one of those cloud TPUs. And on the full TPU pod, you can do that in less than 12.5 minutes – Zak Stone, TensorFlow Dev Summit 2018
BTW, these are not typical hardware. It cannot be bought anywhere, Google offers it on cloud instead of on-premise.
Respect to Alan Kay. Apparently, he still continues to change the world, even for the fields he hasn’t involved.
Like this blog? Support me on Patreon
