Deep learning, MIT intro

www.deeplearningbook.org

Early days of AI solved problems difficult for human but relative straightforward for computers, which can be described by a list of formal, mathematical rules. The true challenge to AI proved to be solving tasks that are easy for people to perform but hard for people to describe formally, problems we solve intuitively, that feel automatic, like recognizing spoken words or faces in images.

Deep learning is a solution to these more intuitive problems, which allow computers to learn from experience and understand the world in terms of a hierarchy of concepts.

Ironically, abstract and formal tasks, while mentally difficult for a human, are among the easiest for a computer.

A person’s everyday life requires an immense amount of knowledge about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. So one of the key challenge in AI is how to formalize this informal knowledge.

Simple machine learning algorithms depends heavily on the representation of the data, which is known as features.

For many tasks, it’s difficult to know what feature should be extracted. The approach is known as representation learning.

The main reason for the diminished role of neuroscience in deep learning research today is that we simply do not have enough information about the brain to use it as a guide.

my comment

I will make a stop of this book here due to the time constraint. My largest gain in this introduction is the awareness of informal knowledge. This reminds me that there are so many things that a school education failed to teach (at least at this moment) but are vital to a human’s life. These knowledges include emotional intelligent, time management, marriage fitness, culture shock, spiritual growth, etc. Unfortunately, we usually regard them as common sense without a systematical understanding.
​
The deep learning approach shines some light on these understanding. We can always insert arbitrary hidden layers between what we have and what we want. These hidden layers serve as thought-provoking buffer which allow us for creative ideas without directly jumping into the conclusion. I will practice this method to draw some mind maps.
​

​