Saturday, August 13, 2016

IPND, stage 5, front-end

Update on 2017-3-6

7 months I had a quick feeling of JavaScript. I didn’ t process because I am more interested in the content than the face. And lots of tricky details and fine tunes are somewhat overwhelming to me.

Now I have the content, the knowledge discovered by applying machine learning models to lots of data. I realize the last mile is the data presentation, the packaging, the user experience. It’s worth to spend time on polishing the high-quality content, which helps spread the insight to more viewers.

JavaScript

Some basic syntax.

console.log("Hello World!");
var a="yourname";
var b= a.replace("your","my");
var y=function(x){ return z};
array.length;
array.slice;
a.toUpperCase();
array.pop();
array.push();
.split(" ");
.join(" ");
no class but object {};
var bio={"name":"James",
"age":32};
bio.city="Norman";  // dot notation
bio["city"]="Norman"; // square bracket notation is better because special characters or space is tolerent.
if (condiction){}
else{}

jQuery

jQuery is a popular JavaScript library for reading and making changes to the Document Object Model (DOM). The DOM is a tree that contains information about what is actually visible on a website.

While HTML is a static document, the browser converts HTML to the DOM and the DOM can change. In fact, JavaScript’s power comes from its ability to manipulate the DOM, which is essentially a JavaScript object. When JavaScript makes something interesting happen on a website, it’s likely the action happened because JavaScript changed the DOM. jQuery is fast and easy to use, but it doesn’t do anything you can’t accomplish with vanilla (regular) JavaScript.

jQuery was first released in 2006. The latest version is 3.1.1, released on 2016.9.22. It occupies 96% of the JavaScript library market share and is deployed in 70 M websites. The runner-up, “Bootstrap”, has 7 M websites. To my surprise, D3 is only deployed in 12 K websites, probably because it is new.

jQuery’s syntax is designed to make it easier to navigate a document, select DOM elements, create animations, handle events), and develop Ajax) applications.

the basics

http://learn.jquery.com/

<!doctype html>
<html>
<head>
    <meta charset="utf-8">
    <title>Demo</title>
</head>
<body>
    <a href="http://jquery.com/">jQuery</a>
    <script src="jquery.js"></script>
    <script>
    // Your code goes here.
    </script>
</body>
</html>

To run code as soon as the document is ready to be manipulated,

$( document ).ready(function() {});

$ is simply an alias for jQuery because it is shorted and faster to write. It is essentially a window object.

if other JavaScript library wants to use the $ namespace, you can redefine an alias $j for jQuery:var $j = jQuery.noConflict();

Alternatively, you can use a locally-scoped $

jQuery.noConflict();
jQuery( document ).ready(function( $ ) {
    // locally-scoped $ as an alias to jQuery.
    $( "div" ).hide();
});

// The $ variable now has the prototype meaning, which is a shortcut for
// document.getElementById(). mainDiv below is a DOM element, not a jQuery object.
window.onload = function(){
    var mainDiv = $( "main" );
}

.attr() method is a setter for 2 inputs, and a getter for 1 input:

$( "a" ).attr( "href", "allMyHrefsAreTheSameNow.html" );
$( "a" ).attr({
    title: "all titles are the same too!",
    href: "somethingNew.html"
});
$( "a" ).attr( "href" );

.html() – Get or set the HTML contents.
.text() – Get or set the text contents; HTML will be stripped.
.attr() – Get or set the value of the provided attribute.
.width() – Get or set the width in pixels of the first element in the selection as an integer.
.height() – Get or set the height in pixels of the first element in the selection as an integer.
.position() – Get an object with position information for the first element in the selection, relative to its first positioned ancestor. This is a getter only.
.val() – Get or set the value of form elements.

jQuery Object

working directly with DOM elements is awkward. By wrapping it in a jquery object, life is much easier. Following are equivalent codes that are implemented by raw JavaScript and jQuery.

var target = document.getElementById( "target" );
target.innerHTML = "<td>Hello <b>World</b>!</td>";
$( target ).html( "<td>Hello <b>World</b>!</td>" );

var target = document.getElementById( "target" );
var newElement = document.createElement( "div" );
target.parentNode.insertBefore( newElement, target.nextSibling );
$( target ).after( newElement );

jQuery UI

This is really cool. With a single line of code, you have a pull-down calendar. Detailed introduction deserves another post.

JSON

JavaScript Object Notation. JSON is very handy to store hierarchal information. The highly flexible means vulnerable to bugs. Use http://jsonlint.com/ to correct bugs.

{
  "Schools":[
    {
      "name":"Beijing University of Posts and Communications",
      "city":"Beijing, China",
      "degree":"BS",
      "major":["Applied Physics"]
    },
    {
      "name":"University of Oklahoma",
      "city":"Norman, OK, US",
      "degree":"PhD",
      "major":["Electrical and Computer Engineering"]
    }
  ]
}

Project: online resume

In the head block, loads style.css and possible script from google map api(You can obtain your own Google Maps API key here)

In the body block, the “main” block has 5 sub tags: with id = header, workExperience, projects, education, mapDiv, lets-connect. Then comes 3 script files: jQuery.js, helper.js, resumeBuilder.js. At last comes the real script, and if certain element is empty, set .style.display = "none";. Alternatively, we can also set .style.backgroundColor = "black";

To avoid script attack, you may use regular expression to catch all the <and > and replace:

var charEscape = function(_html) {
    var newHTML = _html;
    newHTML = _html.replace(/</g, "&lt;");
    newHTML = newHTML.replace(/>/g, "&gt;");
    return newHTML;
};

frustrated by tons of errors, install a local JavaScript IDE: webStorm

script path: /usr/local/bin/webstorm

google map API is cool!. “initializeMap” function is 100 lines of code, really a monster, and many imbedded functions like locationFinder, createMapMarker,callback, pinPoster.

Friday, August 12, 2016

IPND, graduation & final reflection survey

I spend about 2 weeks to finish this nanodegree. I learned a lot ~

14. What do you now want to do with these new skills and your Nanodegree in hand?

try to get a job and apply these skills.

15. How have the skills you learned in the Nanodegree program made you a better candidate for a career in your desired field?

It is all about building confidence.

I guess the few projects are not enough to get a real job. But it is a good starting point.

17. Please tell us why you are interested/not interested in career services.

Because I want to know whether I am on track to what I want to do.

18. Did you learn anything new in the Nanodegree program that changed your view about the field or your career aspirations?

Yes, the thinking in programming language. Object oriented, detail oriented, concurrent thread, etc. All these concepts are important to open my mind.

19. Collaboration and mentoring are key to success in the workplace. Udacity instructors and your peers helped foster a sense of community in the forums. What lessons will you take from your experience with your cohort if/when you mentor someone in the field?

I don't spend much time in forum because it's not very efficient.

20. Based on your experience, what tips do you have for students embarking on their Nanodegree journey?

Have a peek what you are going to learn first, then commit your time to learn it.

Make notes and do quiz. Keep practice.

Some of the learning materials are from the free courses. You can use the free courses to build a foundation or get a basic idea. I guess nanodegree is a highlight to all these sources.

Wednesday, August 10, 2016

IPND, stage 5, back-end, SQL

[TOC]

SQL

Structured query language

Database has several tables for different purposes.

basic query

select food from diet where species = "orangutan";
select * from tablename limit 10 offset 2;
select name, birthdate from animals where species != 'gorilla';
insert into tablename  values ( val1, val2, … );

select name from animals, diet where animals.species=diet.species and diet.food='fish';

select count (species),species from animals group by species order by count(species) DESC;
select animals group by species having num=1; # after aggregation

select ordernames.name, count(*) as num
  from animals, taxonomy, ordernames
  where animals.species = taxonomy.name and taxonomy.t_order = ordernames.t_order
  group by ordernames.name
  order by num desc;

update posts set content='cheese' where content like '%spam%';

delete from posts where content='cheese';

Create/drop database/table, primary key, references

create database name;
create table fish(id serial primary key, name text);
create table postal_places(postal_code text,country text, name text, primary key(postal_code, country));
create table sales(sku text references products, sale_date date, count integer); # use references for foreign key

drop database name;
drop table name;

delete from tablename;
insert into tablename values ('This is text!');

\c name # connect to database

join

select animals.name, animals.species, diet.food 
from animals join diet 
on animals.species = diet.speices
where food = 'fish';
-- show item with 0 count
select products.name, products.sku, count(sales.sku) as num
  from products left join sales
    on products.sku = sales.sku
  group by products.sku;
-- self join
select a.id, b.id, a.building, a.room
       from residences as a, residences as b
 where a.building = b.building
   and a.room = b.room
   and a.id > b.id
 order by a.building, a.room;

subQuery

select name, weight from players, (select avg(weight) as av from players) as subq where weight< av;
create view name as select ...

Python DB-API

use PosreSQL

always use 2 command windows

for python

python forum.py
http://localhost:8000/ # you can open it on a browser

for psql

psql forum   # load forum database, equal to" \i forum.sql"
select 2+2 as a, 4+4 as b;
select * from posts;
select * from posts \watch  # update very 2 seconds, so you can see what's added
\d posts  # get columns and type
\dt  # list all tables
\H #switch b/w plain text vs HTML

loop hole

something fun: http://xkcd.com/

sql injection attack: '); delete from posts; —

script injeciton attack

DB-API using sqlite3

import sqlite3
with sqlite3.connect("chinook.db") as conn:
    cursor = conn.cursor()
    rows = cursor.execute('select * from artist limit 10;').fetchall()
    conn.commit() # required if insert new data
for row in rows: print row

normalized design, which makes it easier to write effective code using a database.
A Simple Guide to Five Normal Forms in Relational Database Theory

normalized table

Every row has the same number of columns,
There’s a unique key. Everything in a row says something about the key.
Facts that don’t relate to the key belong in different tables,
Tables should’t imply relationships that don’t exist.

Saturday, August 6, 2016

IPND, stage 5, data analyst, notebook

update 2017-2-27

It is almost 7 months after the original post, which is really a mess from my current knowledge. I realize there is a huge cognitive gap between the learner and the teacher. This is where the “curse of knowledge” come in. Because we stand on the shoulder of giant. But the “giant” is hugely different for individuals. Different background, different culture, different learning pace, different learning style. We build new knowledge on top of what we already know. And the way we construct our knowledge is more like a Graph database. Knowledge is stored at the vertexes or edges. Our brain doesn’t store information in the SQL style.

Pandas

https://github.com/alfredessa/pdacookbook

auto = pd.read_csv('data/auto.csv')
auto.head()
auto.describe()
auto.mpg.describe()
auto['mpg'].describe()
auto.mpg.std()
auto.price.hist()
auto.boxplot(column='price')
grouped = titanic.groupby('Sex')
grouped.Age.describe()

# add 2 pd.Series and fill missing value
s= s1.add(s2, fill_value = 0)
# def fun(x): return x**2
s. apply(fun)

df.loc[] # label based position
df.iloc[] # integer position
df.sort_values(ascending = False)

matplotlib

http://matplotlib.org/api/pyplot_api.html
http://matplotlib.org/examples/

import matplotlib.pyplot as plt
plt.hist(list) or Series.hist()
x = np.arange(0, 5, 0.1)
y = np.sin(x)
plt.plot(x,y)
plt.xlabel()
plt.ylabel()
plt.title()
plt.show() #show plot in a new window

df.plot(kind='hist',title="passengers vs. sex")
plt.xlabel("gender, 0=male, 1=female")
plt.ylabel("number of passengers")

plt.legend(["dead","survived"])

seaborn

http://seaborn.pydata.org/

Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics. Full customization of the figures will require a sophisticated understanding of matplotlib objects.

histogram
boxplot
kernel density estimation
violin plot
cumulative distribution function

import seaborn as sns
plt.hist(titanic.Age.dropna(),bins=25)
sns.boxplot(titanic.Age, titanic.Sex, vert=False)
sns.kdeplot(titanic.Age.dropna(), shade=True)
sns.distplot(titanic.Age.dropna())  # density + hist
sns.violinplot(titanic.Age.dropna()) # density + box
sns.kdeplot(titanic.Age.dropna(), cumulative=True)