Friday, March 17, 2017

The business of deep learning

The Business of Deep Learning
Understanding Deep Learning and Discovering Real World Applications
By Matt Coatney
Publisher: O’Reilly Media
Final Release Date: February 2017
Run time: 3 hours 30 minutes
Video: $64.99

course note

Overview: The first 2 lessons (~30 min)are free and give you an introduction and the 5 areas that will be covered:
  • characterize the world (36 min)
  • identify and group things together (27 min)
  • predict outcome and behavior (18 min)
  • human-machine interaction (30 min)
  • robotics and real-world interaction (22 min)
The details are charged. Then ends with implications (30 min) and conclusion(14 min).

Why now?
  1. Business need: hyper competition, knowledge economy
  2. data: natively digital, sensors, internet of things
  3. methods: large neural nets, ensemble/panel of experts
  4. compute power: exponential growth, GPU
AI:
  • more than just deep learning
  • ensemble of techniques approaching human intelligence
  • no clear definition or finish
4 possibilities for business models: (risk from high to low)
  1. platform( expensive, IBM Watson, Google tensor flow, Microsoft cntk, caffe, torch)
  2. consulting service (customized solutions, people intensive, not scalable)
  3. technique core (backbone for other specific technology)
  4. enablement (integration in application: Netflix,google, facebook, amazon)
questions:
  • which business models are most applicable to your industry?
  • what initial ideas do you have for monetizing deep learning?

characterize the world

extract features: image, audio, video, text

identifying and grouping things:

  • segmenting(clustering)
  • categorizing(classification)
dimensionality reduction: principle component analysis,
predicting outcomes/behavior
  • making recommendations
  • making prediction

building buy-in for AI projects

  • fear of change
  • identify a pressing,burning need
  • avoid pitching/building a platform
  • incrementally target new high-value needs
  • use past success as internal social proof
  • look for adjacent opportunity
  • follow the money

Driving User Acceptance

  • Teach: time, empathy, awareness, communication, help
  • If you communicate about 10 times as much as you should, you are starting to get it right.
  • create the climate for change: create urgency, form a powerful coalition, create a vision for change.

Thursday, March 16, 2017

A hands-on tutorial for Xap beginners

Not every internet product survives in today’s fierce competition. I believe the survival of Exaptive platform in the next 5 years will require a design paradigm shift. This tutorial is not only for Xap users, but also for Xap developers. The overarching theme is building a minimum viable product (MVP). With such a skeleton at hand, users will have a lot of fun by tinkering here and there.
Data scientists come from different backgrounds and have their own preferred technology stacks. Exaptive platform aims to bridge the gap between offline data visualization and online data visualization. The latter means web-based, which leverage the power of fast-growing JavaScript libraries such as D3, Dimple and Plotly.
To positioning this tutorial, the targeted learners in my mind are frequent Python users, who are already familiar with data structure like dictionary, and module like pandas. From my learning experience, there are 2 major knowledge gaps:
  1. HTML/JavaScript
  2. Data types and conversion in Exaptive environment

HTML/JavaScript

These are prerequisites for any web applications that are tailored to exactly what you need. It is easy to learn because you can practice it in the browser and get immediate feedback. You can learn all the fundamentals at https://www.w3schools.com/. My learning note is here.
Once you know how to control the DOM by JavaScript, and how to use jQuery or D3 to draw a simple shape, you are good to go.

Data types and conversion in Exaptive

This is the most confusing and buggy thing for a beginner in Exaptive. I will provide some easy-to-follow steps to walk you through the mist. I am going to open every black box to help you get a clear understanding of the magic.

Get familiar with the platform

Open https://exaptive.city/, there are 4 options at the top:
  • Home. This is the warehouse for storing your Xap.
  • Studio. This is the warehouse for storing your Xap and Components. If you click + button, there pops up there things: Xap, Component, Asset. Component is the basic building block. Xap is the product when you piece required components together. Asset is a dataset, code snippet, pictures that are used to build a component or feed data. The current version of Studio is 4.0.24, released on 2016.8.29
  • Explore. A collection of xap, component and asset that have been built and published. You can add them to your own Studio, reuse or learn from them. I expect there will be better grouping here, such as function type (data wrangling, data mining, data visualization) and language (JavaScript, Python, R).
  • Learn. Tutorials, Documentations, Discussion. This is a place you can gain more detailed answers.

Get family with the JavaScript Component and Xap

  1. At Studio page, add a JavaScript Component, name it “tutorial”.
  2. Open the component, in the “info” tab, you can see the inputs, outputs, layout and dependencies. Their functions are self-explained by the name. And “dependencies” shows a dollar sign and an URL, which means $ is claiming the namespace and the JavaScript library is imported from the URL. In the “edit” tab, you will see more items shown on the left. The most important one is “script”, a place where we will get our hands dirty.
  3. Right now we don’t do any changes. Just save and go back to Studio, add a Xap, name it “Xap_0”. Open it by clicking the hammer. Use your curiosity to click everything before we settle down on the “edit” page and “dataflow” tab.
  4. On the left bar, click “add component” and drag our beloved “tutorial” component to the blank background. Now the little cute square box appears with the name “tutorial_0”. It has a “doSomthing” input port on the left and a “data” output port on the right. use your curiosity again to explore the box. click the port or double click the box to expand the box.And the button to collapse the box. Mouse hover the box to find the “edit” entrance.
  5. Now click the input port and find an arrow button when you mouse hover near the output port. click the arrow to trigger the component. Notice the blue light up on the port, or red flash if something goes wrong. Notice the log icon on the right bottom corner of the page. This is actually the debugging tool for the component. Explore what’s there.
  6. click the input port again. Notice that under “doSomething” shows ‘string’, which indicates the data type of the input port. Write some string in the empty underscore. Then click the “…” sign to open the edit page. Replace the script by:
    export default {
        doSomething() {
            let state = this.api.inputState.export();
            let s = state.doSomething;
            this.api.log("LOG",s);
            this.api.output("data", "Hello World");
        }
    };
    
    There are several things to pay attentions here:
    • doSomething() and state.doSomething must match the input port name to correctly trigger the method and capture the data.
    • let is introduced in ECMAScript 6 and is similar to the use of var They have slightly difference in scoping.
    • trigger the component by clicking the arrow at the input. Observe what’s happening in the log message and the output port.

Data type

  1. open the scirpt of “tutorial_0” component and replace the line this.api.out() with the following:
    var ent = {
        "dream_car": "Tesla Model 3", 
        "price ": "35000",
          "schedule":"July 2017"
      }; // JSON format/Python dictionary/Java hashmap
    this.api.output ('data',ent);
    
    Trigger the input port and check what you get at the outputs port. After the “data”, you will see “3 attributes”. Click on it and the meat comes out, which is exactly the variable ent that we just defined. Exaptive calls data type as entity. You may feel confused about this fancy name. But it is essentially the same with JSON in JavaScript, or dictionary in Python, or hashmap in Java. For R users, it is similar to a list or 2-column data frame.
  2. The value type for input or output that you are allowed to set in Exaptive is: boolean, integer, float, string, tag, null, type, entity, multiset, dynamic, nullable. If you are not sure which one to use, dynamic is the one-size-for-all choice. Try to change the value type of output in the edit page and check whether it still works.
  3. Now change the this.api.output ('data',ent); to this.api.output ('data',[ent,ent]); and check what changes at the output. You will see 2 entities/ 3 attributes. Keep in mind that this will be the most frequent datatype we will encounter in the future.

Parsing string

Draw the graph

Note: Sorry I am not able to finish this tutorial, because the position that I have been preparing for a full month is cut right after my final interview. It is a shock to me when I was expecting an offer negotiation and the celebration of my 100th post.
100post

Wednesday, March 15, 2017

How much money has OK spent on common education


Norman Public Schools is a school district near the University of Oklahoma. NPS serves nearly 16 k students in pre-Kindergarten through 12th grade. It has 17 elementary schools, 4 middle schools, 2 high schools(Norman High, Norman North High), 1 alternative school and 1 online program. For example, Norman High, 4-year. 127 faculties, 1.9 k student, a ratio of 1:18.

Finacial report by Brenda Burkett on 2016.2.10

For the whole Oklahoma state, how funding is changed during the past 8 years:
2008 2016
total state aid funding, 2.05 B 1.85 B
total students 641.7 k 692.8 k
aid per student 3.2 k 2.70 k
state aid formula, per weighted student 3.29 k 3.05 k
total spending 5.96 B
spending per pupil 8.85 k
As I noted in my previous post. OK government has a total appropriation of 5.7B. It spends 3.4 B in education, among which 0.96 B is in higher education. The about 1/3 of government money is spent on k-12 education, or common education.
The school fundings have many different sources with different targeted purposes:
  • general fund. basic operational can support service, such as teacher and support salaries, health insurance, instructional supplies, textbooks, transportation.
  • building fund.
  • child nutrition fund.
  • bond fund.
  • sinking fund
  • school activity fund.
GeneralFund
The school districts in richer cities has more percentage of money from local.

top 10 question about school funding

state aid funding formula is a metric used to allocate state money to individual district by the student number. The weight factor includes grade level, special education, economically disadvantaged, bilingual, district characteristics. It is a way to equalize revenue among districts. For example, more local revenue collection, less aid from the state.
schools are affected by the sequestration due to the Federal funding cut.
Although Oklahoma law does mandate max class size (20~25), the inadequate funding to staff leads to the fact that legislation is passed to remove the penalties for non-compliance.

Amended school budget and financing plan

total amended budget of appropriated funds is $ 112 M, which includes 110 M General fund, 4.6 M for building fund, and 4.9 M for Child Nutrition Fund. The name of the fund is pack something together as a black box. Specifically, 31.5 M is from Ad Valorem Tax, a latin name for “property tax”. 48 M is from State aid for general operations. Other sources are less than 4 M each.
As OKpolicy explains, Property tax, also known as ad valorem tax, is an annual tax paid by property owners to local government. Property tax collections in Oklahoma totaled $2.2 billion in 2011 and are the single largest source of local government revenue. The math is simple: property tax = (property valuation * assessment ratio - exemption) * millage levy. For example, a house worth 100 k will pay annual tax = (100 k * 0.12 - 1 k) * 0.11 = 1.21 k. It is interesting that mileage levy has a unit of 0.001.
According to salary.com, Oklahoma Teacher High School Salaries range from 50 k to 56 k. Note the median household income is 46 k for Oklahoma and 52 k for the US.

Tuesday, March 14, 2017

Plotly is eating the data science market

Plotly is my new love after dimple failed to draw a simple pie chart. It turns out that, Plotly is doing the best job than any others that make every data analytics tool (e.g., python, R, Matlab) immediately ready for web-based data visualization. If you have complicated data and want to publish your finding online, look no further, python+ plotly is your best choice.
The founder of Plotly, Alex Johnson, has an interesting career path. He got his Harvard PhD in Physics in 2005, research on “Charge Sensing and Spin Dynamics in GaAs Quantum Dots” with a National Science Foundation graduate research fellowship. After one year of postdoc, he went to Harvard Environment Center for 3 years. He was trying to develop novel thin-film solid oxide fuel by applying semiconductor techniques such as microfabrication. He then spent 1 year at C12 Energy design and built a database and web interface for screening and forecasting enhanced oil recovery projects.
In 2012, he founded Plotly, which is a JavaScript graphing library:
  • comparable in scope and features to MATLAB or Python’s matplotlib.
  • It has D3 and WebGL for backend. no need for jQuery.
  • use JSON schema,It focuses on the chart’s physical attributes and attempts to leave the chart data separate.
  • In contrast, The vega and vega-lite schemas are more opinionated in prescribing how the chart data is grouped, sliced, or statistically processed before graphical display. This allows for complicated chart display with a concise JSON description, but leaves less control to the user. Neither approach is more “correct”—they’re just different.
Since 2015. 11, Plotly was open-source at https://github.com/plotly/plotly.js The business model for plotly is by charging the API of python. matlab, R, similar to the charge of Google map API. The community version is free for upto 50 API calls per day. More advanced plotting and more API calls are charged. Personal plan is $33 per month and student plan is $5 per month.
Due to time constraint, I only have a quick practice with the “getting started” for each language. Much more APIs are found here: https://plot.ly/api/

JavaScript

<script src=”https://cdn.plot.ly/plotly-latest.min.js”>
Basic Box plot
var y0,y1;
for (var i = 0; i < 50; i ++) {
    y0[i] = Math.random();
    y1[i] = Math.random() + 1;
}
var trace1 = {
  y: y0,
  type: 'box'
};
var trace2 = {
  y: y1,
  type: 'box'
};
Plotly.newPlot('myDiv', [trace1, trace2]);

python API

install the python API: pip install plotly
import plotly
plotly.tools.set_credentials_file(username='jychstar', api_key='1GPp9Dwmnsf897Z3kX8Q')  # now I use a free api key, info stored at .plotly/.credentials file in  home directory
plotly.tools.set_config_file(world_readable=False,
                             sharing='private') # public, private, or secret
import plotly.plotly as py
import plotly.graph_objs as go
trace0 = go.Scatter(
    x=[1, 2, 3, 4],
    y=[10, 15, 13, 17]
)
trace1 = go.Scatter(
    x=[1, 2, 3, 4],
    y=[16, 5, 11, 9]
)
figure = go.Figure(
    data = [trace0, trace1],
    layout = go.Layout (title = "hello world" )
)

py.plot(figure, filename = 'basic-line')  # plot online
py.iplot(figure, filename = 'basic-line') # plot inline
plotly.offline.plot(figure) # plot offline in browser
plotly.offline.iplot(figure) # plot offline in notebook
If you are interested what methods can be called by go, using dir(plotly.graph_objs)will give 58 items such as Area, Bar, Box, Candlestic,, HeatMmap, Histogram, Line, Pie, Scatter, Trace.
pie
labels=['Oxygen','Hydrogen','Carbon_Dioxide','Nitrogen']
values=[4500,2500,1053,500]
colors = ['#FAEE1C', '#F3558E', '#9C1DE7', '#581B98']
trace = go.Pie(labels=labels,values=values,marker={'colors': colors})
url_2 = py.plot([trace], filename='pie-for-dashboard', auto_open=True)
py.iplot([trace], filename='pie-for-dashboard')

pandas

use a fresh-bake model cufflinks , pandas can directly access plotly method.
import cufflinks as cf
import pandas as pd
cf.set_config_file(world_readable=True,offline=False)
pie=cf.datagen.pie()
pie.head()
pie.iplot(kind='pie',labels='labels',values='values')
df.iplot(kind='box', filename='box-plots')
df.iplot(kind='histogram', barmode='stack', bins=100, histnorm='probability', filename='histogram-binning')

matplotlib, seaborn

use existing plotting libraries, pass the handler to plotly.
import matplotlib.pyplot as plt
import numpy as np
import plotly.plotly as py
py.sign_in('jychstar', '1GPp9Dwmnsf897Z3kX8Q')
x= np.array(range(1,10))
y = np.sin(x)
fig, ax = plt.subplots()
ax.plot(x,y)
plot_url = py.plot_mpl(fig)  # publish fig online
## try seaborn, found the filling area is missing
import seaborn as sns
tips = sns.load_dataset("tips")
fig, ax = plt.subplots()
ax = sns.boxplot(x="day", y="total_bill", data=tips)
plot_url = py.plot_mpl(fig)

MatLab API

Download and uncompress the Plotly MATLAB library. 210K.
cd ~/Downloads/matlab_api/
plotlysetup('jychstar', '1GPp9Dwmnsf897Z3kX8Q')
[X,Y,Z] = peaks;
contour(X,Y,Z,20);
fig2plotly()  % push fig online
getplotlyoffline('https://cdn.plot.ly/plotly-latest.min.js') % download the offline plotly bundle
fig2plotly(gcf, 'offline', true) % generate html in current working directory
% try some simple plot
x= linspace(0, 5, 101);
plot(x, sin(x.^2), '-pb', 'linewidth',2)
fig2plotly()

R API

download latest R from CRAN
I happened to have a R version 3.3.1(2016.6.21). Because R 3.3.2 was release on 2016.10.31, I guess R API was released after that.
R.version  # check, plotly require R version 3.3.2
install.packages("ggplot2")  # plotly require version >2.1.0
install.packages("plotly")
library(ggplot2)
library(plotly)
Sys.setenv("plotly_username"="jychstar")
Sys.setenv("plotly_api_key"="1GPp9Dwmnsf897Z3kX8Q")
fig = plot_ly(midwest, x = ~percollege, color = ~state, type = "box")
plotly_POST(fig, filename = "midwest-boxplots")
# try to use other plotting library other than plotly
t = seq(0,10,0.1)
fig = qplot(t,sin(t),geom="path", xlab="time", ylab="Sine wave")
plotly_POST(fig, filename = "sine_wave")

Plotly vs Xap

Plotly Xap
web-based visualization yes yes
hosting final graph data pipeline
web presentation layout one full-size graph customized
plotting tool plotly.js + naive plotting library Component + JS library
generate take-away html, png files yes no
Python, R, Matlab works in naive environment yes no
open source yes ?
business model community/advanced consulting