Intro to AI

finish 40% in 3 days, may come back if needed.

1 AI in general

• Finace: making trading decisions
• Web
• Game
• Robotics
• medicine: diagnostic
• languague translation

Terminology

• Fully vs. Partially Observable
• Deterministic vs. Stochastic
• Discrete vs. Continuous
• benign vs. Adversarial environment

AI as uncertainty management:

• sensor limit
• ignorance
• stochastic

definition of a problem

• initial state
• actions(s)
• result(s, a): return new state
• GoalTest(s): return True/False
• Path cost/step cost

2 finding the best route

Breath first search vs Depth first search

3 Beyes Network

complementary, independence,

conditional probability

joint probability

Bayes Rule

A is not directly observable, truth or fact of a cancer, B is directly obserbale, testing result of cancer. In Beyes Network (A—>B), It takes 1+2 parameters to specify. The core calculation depends on joint probability.

Posterior= Likelihood*Prioir/Marginal likelihood

Away effect:

indepedence does not imply conditional independence

e.g. S and R are independent, they are both necessary conditions to H. If we know H, then the existence of S or R will affect each other.

4 Probabilistic inference

• Evidence
• Hidden
• Query

Enumeration

variable elimination: joining table, sum out, rejection sampling, likelihood weighting.

Monty Hall problem: 1/3 vs 2/3

5 Machine learning

• Bayes networks: reason with known models
• machine learning: learn models from data

Stanley self-driving car: to see 200 meters far is a key factor to win the 2005 DAPRA race.

• Classification: output is binary or a fixed number of classes
• regression: output is continuous

Overfitting is a major source of poor performance of a machine learning algorithm.

spam detection: maximum likelihood, laplace smoothing.

Naive Bayes model, but not powerful enough to counter spam.d

logistic regression is slightly more complex than linear regression

gradient decent

perceptron algorithm

support vector machine: kernel trick

K nearest neighbors: nonparameter

6 Unsupervised learning

7 Representation with Logic

Agent design: reflex-based, goal-based, utility-based

complexity of environment

representation: logic

propositional logic:

• V: or ;
• ^: and

limitations

• can’t handle uncertainty
• can’t handle objects
• no short cuts

first-order logic: relation to object

8 planning

problem solving: in deterministic and fully obserable environment, have starting point and end goal, find a path.

In your real life, you do your planning ahead of time, you have the complete goal, then without interacting with the world, without sensing it at all, start execute the path.

However, if people have gone out and blindfolk walk, they can never walk in a straight line, without any feedback from the environment.

In a stochastic environment, we must react to unexpected result. If the road is closed, we don’t know our state, which have partial observability.

19 Robotics

Robotics is the science of bridging the gap between sensor data and actions.

Kinematic state of a car: 3D of state space: xy plane + steer direction

Dynamic state of a car: 3+ foward veloctiy + raw rate

Kinematic state of an airplane, 6: x,y,z+ yaw rate+tilt+ roll

Dynamic state of an airplane:12

Monte Carlo Localization

particle filter

look at measurement, compute weight, sample, predict

office hour Q&A

Mahout: machine learning toolkit

sebastian: I never use toolkit. I’ve always coded straight in C++ or Java. More recently, as I aged, more in Matlab, which is really nice for rapid prototyping. It doesn’t give me the AI I want, but the tools to process the data, visualize data really quickly. So it’s fantastic to try out an idea.

next challenging?:

• Object recognition.
• Chess problem is only search problem.
• perception problem. Do you understand the situation in fuzzy situations?
• have an assistant. make everybody twice effective .

What hardware advancement expected to improve AI:

my 3-year old child is able to process huge amount of data. The way is: use existing computers and be really smart how to process massive amounts of data.

XX algorithms are hard to train?

Your laptop today was supercomputer of a a decade ago.

S: this is one of my frustration in scientific field, which in the past has looked into lots of small data sets, and very few have looked at very large data sets. As a result, the algorithms are mathematically beautiful and elegant, but they don’t scale.

I am pretty much self-taught. Look into the data first before applying any algorithm.

paths to professional in AI?

a lot of titles like computer scientist, game developer, research scientist, data scientist. No matter what the tools take, you’ve got to be able to get the job done.

The sexiest job in next 10 years will be statistician or machine learning person.

​

Intro to DevOps

To create value, first you need an idea for a product or service. But having an idea isn’t nearly enough. Customers don’t pay for just ideas. They want to see something that does what they need, a functioning product, a working service. An idea that you don’t actually ship is nothing but vapor water.

Classic organizational structure has strict borders between development, QA and operations. No matter how efficient the process within single departments, if there are high walls in between them, problems and inefficiencies creep in. All too often, there’s not a lot of communication between departments. The attitude might be, throw it over the wall it’s not our problem anymore, we met our deadline. But if the customer throw back, saying this is not I want. then a nasty cycle of blame can start:

• lazy technical staff
• unrealistic project deadline
• inadequate server budget
• sloppy coding practices

results in :

• unhappy
• slow and ineffective at delivery
• can’t learn from failure (people blame each other!)
• hard to make changes
• operation problems stack up

In 2009, a grassroots movement of practitioners started, which known as DevOps.

The scariest phase in English language is we’ve always done it that way. With the advancement of hardware and software, the pace of innovation has only accelerated. Old methodologies simply can’t keep pace.

People just expect new features and fixes to happen quickly without disruption to their daily lives. A development process with a long detail design phase, and a planning phase, and then another long implementation phase is slow, inefficient and error-prone.

Jez Humble Keynote

Agile development: constant feedback with an iterative approach allows to find problems between plan, code and test. But it’s not enough, like in deploy, operate.

One of agile’s key points: people > process> tools. If you have the best teams, but there’s no feedback or collaboration between teams, it can lead to a huge waste of time and resources.

hurry up and wait.

Everyone needs to have a shared view on what the current situation is. What are the goals? shared input generally creates shared responsibility and ownership. People are willing to work together if their thoughts and opinions are being heard. It’s important to have a shared view and identify and eliminate bottlenecks. The bottlenecks can be inside or between department. It’s not really important as long as they’re being addressed. It’s also important to have shared benefits among them. You can’t improve what you can’t measure.

roles and perspectives

Developer role: implement feature requests, fix bugs. feature-oriented. Building new features might be dependent on exploring new technologies or approaches that have not been used before. take new risks.

Operations people: run services. maintain services, troubleshooting/diagnosis. stability-oriented.usually invisible to the company, when everything works fine. Ops is noticed when something doesn’t work.

Conflicting interests.

• Dev can feel like ops is blocking their work by not providing resources fast enough or not being supportive of new cutting edge tech.
• Ops can feel like devs are being irresponsible and reckless and just implement new features without thinking about performance and security.
• Product managers may be even more feature oriented than devs.
• Security team more in the fear of introducing new, unknown vulnerabilities.
• business managers: cost-benefit.

prepare development environment

brew install packer
git clone https://github.com/jychstar/devops-intro-project.git devops

follow README.md in the repo.

• Run packer build -only=virtualbox-iso application-server.json It took me 3 hours to finish this step.
• Run cd virtualbox
• Run vagrant box add ubuntu-14.04.4-server-amd64-appserver_virtualbox.box --name devops-appserver
• Run vagrant up
• Run vagrant ssh to connect to the server

the most popular cloud providers that have free trials:

• Google Cloud Platform
• Amazon Web Services
• Microsoft Azure.

problems

1. When I in lesson 2 “using the image locally”, run vagrant up could not get through.
2. In lesson 2”choose a cloud provider”, I could not follow through.
3. In lesson 3” quiz: setting up Jenkins”, this is a following task

packer

packer can create golden images for different build targets, like VirtualBox and VMware or use on your workstation, also for cloud provider like AWS.

A template is a JSON file that defines one or more builds by configuring the various components of Packer. Packer use these information to create multiple machine images.

• variables
• builders
• provisioners
• post-processors

continuous integration

Jenkins

​

Deep learning, MIT intro

www.deeplearningbook.org

Early days of AI solved problems difficult for human but relative straightforward for computers, which can be described by a list of formal, mathematical rules. The true challenge to AI proved to be solving tasks that are easy for people to perform but hard for people to describe formally, problems we solve intuitively, that feel automatic, like recognizing spoken words or faces in images.

Deep learning is a solution to these more intuitive problems, which allow computers to learn from experience and understand the world in terms of a hierarchy of concepts.

Ironically, abstract and formal tasks, while mentally difficult for a human, are among the easiest for a computer.

A person’s everyday life requires an immense amount of knowledge about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way. So one of the key challenge in AI is how to formalize this informal knowledge.

Simple machine learning algorithms depends heavily on the representation of the data, which is known as features.

For many tasks, it’s difficult to know what feature should be extracted. The approach is known as representation learning.

The main reason for the diminished role of neuroscience in deep learning research today is that we simply do not have enough information about the brain to use it as a guide.

my comment

I will make a stop of this book here due to the time constraint. My largest gain in this introduction is the awareness of informal knowledge. This reminds me that there are so many things that a school education failed to teach (at least at this moment) but are vital to a human’s life. These knowledges include emotional intelligent, time management, marriage fitness, culture shock, spiritual growth, etc. Unfortunately, we usually regard them as common sense without a systematical understanding.
​
The deep learning approach shines some light on these understanding. We can always insert arbitrary hidden layers between what we have and what we want. These hidden layers serve as thought-provoking buffer which allow us for creative ideas without directly jumping into the conclusion. I will practice this method to draw some mind maps.
​

​

Why AlghaGo?

quanta magazine

why alpha go is really such a big deal

Knight=bishop= 3 pawns, rook-5 pawns, queen= 9 pawns, king =$$ pawns

The notionof value is crucial in computer chess. The goal is for the program to find a sequence of moves that maximizes the final vlaue of the program’s board position, no what what the opponent do.

Ideas like this(a pawn blocking the rook devalue the rook) depend on detailed knowledge of chess and were curcial to deep blue’s success.

What happends if you apply this strategy to Go? .. Top Go players use a lot of intuition in juding how good a particualr board position is. And it’s not immediately clear how to express this intuition in simple, well-defined systems like the valuation of chess pieces. In 2006, Monte Carlo tree search algorithms was introduced, based on a clever wayof randomly simulating games.But it still fell far short of human player.

The mechanics behind AlphaGo is published in Nature in Jan. 2016.

AlphaGo learned in 2 stages:

1. AlphaGo was trained by 150 k games played by good human players(6~9 dan), and used an artificial neural network to find patterns in those games. It learned to predict with high probability what move a human player would take in any given position.
2. Improve the neural network by repeatedly playing it against earlier version of itself, adjusting the network so it gradually improved its chance of winning.

The neural network is a very complicated mathematical model, with millions of parameters to tune. When the network learned, it kept making tiny adjustments to the parameters in the model, trying to find a way to make corresponding tiny improvements in its play. This sounds like a crazy strategy—repeatedly tiny tweaks to enormously complicated function. But if you do this for long enough, with enough computing power, the network gets pretty good. And here’s the strange thing: it gets good for reasons no one really understands, since the improvements are a consequence of billions of tiny adjustments made automatically.

However, the core idea is how to get a valuation of the position. While the valuation system of Deep Blue based on lots of detailed knowledge, Alphago did it by analyzing thousands of prior games and engaing in a lot of self-play. Alphago created a policy network through billions of tiny adjustments, and build a valuation system similar to a good player’s intuition abuout the value of different board positions.

However, neural network have drawbacks. It can be fooled. It needs too many training data than human players.

​

Misleading thinking: Criticizing each other only creates hate

As human nature, it’s easier to be dissatisfied and criticize, it’s harder to build and appreciate, which requires more efforts.

I am very thankful to Bill and Pam for inviting me to the watch party and preparing nice light refreshments(celery, carrot,grape, thin crackers, green tea).

From my perspective, the documentary film “Hillary’s America: The secret History of the democratic party” is highly biased. It tells the audience that Democratic is trying everything to weaken US, such as Indian removal, slavery, segregation. And GOP is always fighting against these wrongs. He twisted the truth so much that I have to come out my version.

1st party system: 1792-1824

aristocrat vs democrat

Federalist Party, created by Alexander Hamilton, appeals to business community

Democratic-Republican Party, by Thomas Jefferson and James Madison, appeal to the southern planters and farmers.

Starting from 1806, DR party had dominated both House and Senate with >80% seats.

In election of 1824, there’s only 1 party: Republican-democratic, but 4 candidates, none of them received over 50% electoral votes. Then decision comes to the hands of house of representatives. Top 3 entered the finalist with Jackson ranking No. 1. However, the final votes by representative choose Adam who was originally No.2. Adam won 13 out of 24 votes. (there were only 24 states at that time, TX belonged to Mexico).

2nd party system

Jackson began his revenge, formed a new party and won the election in 1828. He used spoils system to benefit his men.

• spoils system, or patronage system: when a political party wins an election, the government jobs are given to its supporters, friends and relatives as a reward and an incentive to keeping working for the party.

The modern Republican Party) was formed in 1854 to oppose the expansion of slavery.

In political science, there are 3rd, 4th, 5th, 6th party systems, which are used to indicated that these two parties evolved or changed their respective ideology or campaign strategy over time.

donation limitation

Candidate Committee: 2700 for primary electron, another 2700 for general election, as you can see in Hillary campaign.

PAC: $5000, PAC usually represents business, labor or ideological interests

Super PAC: unlimited. But Super PAC makes no contribution to candidates or parties. They run independently.

D’Souza lied about what is his felony

He used a “straw donor” trick, which donated money under other’s name and that person get reimbursed.

“Mr. D’Souza agreed to accept responsibility for having urged two close associates to make contributions of $10,000 each to the unsuccessful 2012 senate campaign of Wendy Long and then reimbursing them for their contributions. Given the technical nature of the charge, there was no viable defense,” D’Souza Attorneys Benjamin Brafman and Alex Spiro said in a statement.

D’Souza argued for the charges to be dismissed on grounds of selective prosecution. Last week, a judge denied that motion, citing “no evidence” to support it.

“Following the court’s ruling denying Dinesh D’Souza’s baseless claim of selective prosecution, D’Souza now has admitted, through his guilty plea, what we have asserted all along – that he knowingly and intentionally violated federal election laws,” Bharara said in a statement.

my point of view

D’Souza’s supporters claimed that he was the victim of double standard/ selected prosecution. I think D’Sourza was so shameless:

1. Others doing the same crime without being prosecuted, doesn’t make you innocent. You are just trying to shift the point, the fact that you committed a felony.
2. If you know others doing the same crimes, why don’t you make a movie expose that? Why don’t you spend effort to improve the social justice, make the whole election system more transparent? The answer is very simple : no one is going to buy the ticket. You know it’s not a good business. You’re good at selling hate.

​

Neural network and deep learning 4, convolutional network

[TOC]

4 A visual proof that neural nets can compute any function

As the title says, it’s a visual proof.

5 Why are deep neural networks hard to train?

vanishing gradient problem

neurons in the earlier layers learn much more slowly than neurons in later layers.

6 Deep learning

convolutional networks

It’s strange is use networks with fully-connected layers to classify images, because it doesn’t take into account the spatial structure of the images. Instead, the learning speed could be improved by convolutional nets, which is based on 3 basic ideas:

• local receptive fields
• shared weights
• pooling

Local receptive fields

In conventional approach, every input pixel is connected to every hidden neuron. Instead, taking advantage of the spatial structure, we only use a small window filter to make connections of the input images. This small window, i.e. a 55 region, is called *local receptive field. This filter is called feature map, which only requires 25+1= 26 parameters, which significantly reduce the calculation. In some sense, this way look like a pyramid hierarchy.

All the local receptive fields can share a set of weights and biases, and the resultant hidden layer is called a feature map. For image recognition, a complete convolutional layer consists of several different feature maps, corresponding to different kernels (shared weights/biases)

A big advantage of sharing weights and biases is it greatly reduces the number of parameters.

A pooling layer, or a condensed feature map, is the next layer after feature map. For max-pooling, a pooling unit is the maximum activation in the 2*2 input region. The intuition is that once a feature has been found, its exact location isn’t as important as its rough relative location. For L2 pooling, we take the square root of the sum of the squares of the activations.

code implementation

The difference between network 3 and network1/2 are well explained in its doc string. The dirty job is done by theano.function within Network. SGD().

Briefly, the results for different hyper-parameters are:

main code	hyper-parameter	accuracy	time
network3.py	FullyConnected, 60 epoch	97.8	223 s
network3.py	ConvPoolLayer, 60 epoch	98.78	1800 s
network3.py	ConvPoolLayer*2, 60 epoch	99.06
network3.py	ConvPoolLayer*2, 60 epoch,ReLU	99.23
network3.py	above + expand dataset by distortion	99.37
network3.py	above + extra fullyConnected	99.43
network3.py	above + dropout, 40 epoch	99.60
network3.py	above + ensemble	99.67
network.py	Network(784,30,10), sigmoid,30 epoch	95.42	330 s
network.py	Network(784,100,10), sigmoid,30 epoch	96.59
SVM		94.35	572 s
network2.py	Network(784,30,10),crossEntropy, 30	95.49
network2.py	Network(784,30,10),crossEntropy, 30	96.49
network2.py	Network(784,100,10),crossEntropy, 30	97.92

2013 record : 99.79% by http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html

import network3 
from network3 import Network
from network3 import ConvPoolLayer, FullyConnectedLayer,SoftmaxLayer
training_data,validation_data,test_data= network3.load_data_shared()
mini_batch_size =10

# fully connected layer as baseline
net= Network([FullyConnectedLayer(n_in=784,n_out=100),
            SoftmaxLayer(n_in=100, n_out=10)], mini_batch_size)
net.SGD(training_data,60, mini_batch_size,0.1,validation_data,test_data)

# convolutional networks
net= Network(
    [ConvPoolLayer(image_shape=(mini_batch_size,1,28,28),filter_shape=(20,1,5,5),poolsize=(2,2)),FullyConnectedLayer(n_in=20*12*12,n_out=100),SoftmaxLayer(n_in=100, n_out=10)], mini_batch_size)
net.SGD(training_data,60, mini_batch_size,0.1,validation_data,test_data)

# insert a 2nd convolutional-pooling layer
net= Network(
    [ConvPoolLayer(image_shape=(mini_batch_size,1,28,28),filter_shape=(20,1,5,5),poolsize=(2,2)),
     ConvPoolLayer(image_shape=(mini_batch_size,20,12,12),filter_shape=(40,20,5,5),poolsize=(2,2)),
     FullyConnectedLayer(n_in=40*4*4,n_out=100),
     SoftmaxLayer(n_in=100, n_out=10)], mini_batch_size)
net.SGD(training_data,60, mini_batch_size,0.1,validation_data,test_data)

# rectifield linear units
from network3 import ReLU
net = Network([
    ConvPoolLayer(image_shape=(mini_batch_size, 1, 28, 28),
                  filter_shape=(20, 1, 5, 5),
                  poolsize=(2, 2), activation_fn=ReLU),
    ConvPoolLayer(image_shape=(mini_batch_size, 20, 12, 12), filter_shape=(40, 20, 5, 5),
                  poolsize=(2, 2),
                  activation_fn=ReLU),
    FullyConnectedLayer(n_in=40*4*4, n_out=100,
                        activation_fn=ReLU),
    SoftmaxLayer(n_in=100, n_out=10)], mini_batch_size)
net.SGD(training_data, 60, mini_batch_size, 0.03,
        validation_data, test_data, lmbda=0.1)

Why rectified linear activation function is better than the sigmoid or tank functions? The adoption is empirical. A heuristic justification is that ReLU doesn’t saturate in the limit of large z, which helps it continue learning.

Recent progress in image recognition

In 1998, MNIST was introduced. It took weeks to train by a state-of-the-art workstation. Now it becomes a problem good for teaching and learning purposes.

2011-2015 is an era of huge breakthrough for computer vision. It’s a blit like watching the discovery of the atom, or the invention of antibiotics.

2014 ILSVRC competition

ImageNet Large-Scale Visual Recognition Challenge.

A training set of 1.2 million images in 1000 categories, from original 16 million images.

GoogLeNet achieve 6.8% error rate.

Recurrent neural networks

It has feedback loop which can save dynamic change over time.

useful in speech recognition.

Neural networks have done well at pattern recognition problems, not implementing web server or database program.

long short-term memory units

solve the issue of unstable gradient.

A LSTM block contains “forget gate”. Based on the sigmoid activation value, it decides whether the value is significant enough to remember, or block the value from entering into the next layer.

deep belief nets, generative models, Boltzmann machines

It can learn to write— generate images.

It can do unsupervised and semi-supervised learning.

Though interesting and attractive, DBN lessened in popularity. The marketplace of ideas often functions in a winner-take-all fashion, with nearly all attention going to the current fashion-of-the-moment in any given area. It can become extremely difficult for people to work on momentarily unfashionable ideas, even when those ideas are obviously of real long-term interest.

intention-driven user interface

An impatient professor: Don’t listen to what I say; listen to what I mean

Historically, computers are like confused student. Now google search is able to suggest the corrected query.

Products in the future would tolerate imprecision, while discerning and acting on the user’s true intent.

data science

The biggest breakthrough will be that machine learning research becomes profitable, through applications to data science and other areas.

Machine learning is an engine driving the creation of several major new markets and areas of growth in technology.

What next?

We understand neural networks so poorly.

The ability to learn hierarchies of concepts, building up multiple layers of abstraction, seems to be fundamental to making sense of the world.

Will deep learning soon learn to AI?

Conway’s law:

Any organization that design a system, will inevitably produce a design whose structure is a copy of the organization’s communication structure.

This means, the design and engineering of systems reflect the understanding of the likely constituent parts, and how to build them. Deep learning can’t be applied directly to the development of AI, because we don’t know what the constituent parts are. Indeed, we’re not even sure what basic questions to be asking. At this point, AI is more a problem of science than of engineering.

Wernher von Braun

basic research is what I’m doing when I don’t know what I’m doing.

As our knowledge grew, people were forced to specialize. Many deep new ideas, such as germ theory of disease, how antibodies work, what forms a complete cardiovascular system. Such deep insights formed the basis for subfields such as epidemiology, immunology, and the cluster of inter-linked fields around the cardiovascular system. And the structure of our knowledge has shaped the social structure of medicine, due to the realizing the immune system exists.

The field start out monolithic, with just a few deep ideas. Early experts can master all those ideas. But as time passes, we discover many deep new ideas, too many for any one person to really master. So the structure of our knowledge shapes the social organization of science, which in turn constrains and helps determine what we can discover.

Deep learning is the latest super-special weapon I’ve heard used in such arguments.

Deep learning is an exciting and fast-paced but also relatively monolithic field. What we don’t yet see is lots of well-developed subfields, each exploring their own sets of deep ideas, pushing deep learning in many directions. still a rather shallow field. It’s still possible for one person to master most of the deepest ideas in the field.

How complex and powerful a set of ideas will be needed to obtain AI?

No one knows for sure.

We are at least several decades from using deep learning to develop general AI.

This indefinite conclusion will no doubt frustrate people who crave certainty.

If you ask a scientist how far away some discovery is?

They say “10 years”.

What they mean is “I’ve got no idea”

​