Showing posts with label programming language. Show all posts
Showing posts with label programming language. Show all posts

Monday, May 22, 2017

Java refresher


Java version of hello world is like this:
import java.util.*;
public class vanilla {
    public static void main(String args[]) {
    System.out.println("hello world");
    }
}
Notes:
  1. java.util is the basic package of Java, which contains many useful classes such as Arrays, ArrayList, Date, HashMap, Hashtable, LinkedList, Random, Set, List. A full description is here.
  2. class is usually public so it can be called. The class name is the same with the java file name. Because we usually have multiple classes work together, a project is build.
  3. There will be a main method, which is also usually public. So the java application runs by the main method.
  4. System is one of the classes in java.lang package, which is the default. A full list of the package is here.
Because Java is compile-run 2-step language, so:
  • we don’t have interactive IDE like Jupyter notebook, Matlab or R studio.
  • It is a production language. It is static type so you declare data type of each variable.
  • It trades development time for run time.
For convenience, I only write the code snippet inside the main method.

2 types of for-loop and array

int [] numbers = {10, 20, 30, 40, 50}; //list of int
for(int x : numbers ) {
    System.out.print( x +"\t");
}
System.out.print("\n");
String [] names = {"James", "Larry", "Tom", "Lacy"};  // list of String
for( String name : names ) {
    System.out.print( name + ",");
} // enumerate style: enhanced
for (int i=0; i<5;i++){
  System.out.println(i);
}  // incremental style: traditional
conditional operator Exp1 ? Exp2 : Exp3;

math

System.out.printf("The value of e is %.4f%n", Math.E);
System.out.printf("sqrt(%.3f) is %.3f%n", x, Math.sqrt(2));
note: printf is formatted print so %.3f means the float variable is formatted as 3 digits after the decimal point.%n is used to create a new line.

string

System.out.print("hello".length());
String Str = new String("Welcome-to-Tutorialspoint.com");
for (String word: Str.split("-")) {
    System.out.println(word);
} // iterate to print an array of string
s.substring(0,2) // get first 2 letters.

pass array to method

public static void printArray(int[] array) {
   for (int a :array) {
      System.out.print(a + " ");
   }
}
printArray(new int[]{3, 1, 2, 6, 4, 2});

regular expression

import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class RegexMatches {
   public static void main( String args[] ) {
      String line = "This order was placed for QT3000! OK?";
      String pattern = "(.*)(\\d+)(.*)"; // 3 groups
      Pattern r = Pattern.compile(pattern);
      Matcher m = r.matcher(line);
      if (m.find( )) {
         System.out.println("Found value: " + m.group(0) );
         System.out.println("Found value: " + m.group(1) );
         System.out.println("Found value: " + m.group(2) );
      }else {
         System.out.println("NO MATCH");
      }
   }
}

args input

public class CommandLine {
   public static void main(String args[]) { 
      for(String s:args) {
         System.out.println(s);
      }
   }
}

file IO

import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class vanilla {
    public static void main(String args[]) throws IOException{
        FileReader in = new FileReader("input.txt");
        FileWriter out = new FileWriter("output.txt");
        int c;
        while ((c = in.read()) != -1) {
            out.write(c);
        }
    }
}
java.io package has a lot of paired classes to perform input and output in terms of streams.
  • Byte Streams: 8-bit bytes. FileInputStream, FileOutputStream
  • Character Streams: 16-bit unicode. FileReader, FileWriter
  • Standard Streams: InputStreamReader, System.out.println

Directories operation

import java.io.File;
...
File d = new File("/tmp/user/java/bin");
d.mkdirs(); //create a directory and all parent directory

File a = new File("/tmp");
String[] paths = a.list();  // list all files, directories
for (String path: paths){
  System.out.println(path);
}

read and parse csv file

This is for a simple case, which you use split() to parse the csv file.
String csvFile = "country.csv";
String line = "";
BufferedReader buffer = null;
try {FileReader file = new FileReader(csvFile);
     buffer= new BufferedReader(file);
    while ((line = buffer.readLine()) != null) {
        String[] words = line.split(",");
        for (String s:words ) {
            if (s.equals("\"China\"")){System.out.println("hello");}
            System.out.print(s+"\t");
            //System.out.print(s.contains("China"));
        }
        System.out.println();
    }
}  catch (IOException e) {e.printStackTrace();}

array vs ArrayList

  1. array such as int[] nums= int[5] has fixed size, but arraylist such as ArrayList<Integer> nums = new ArrayList<Interger>() has elastic size.
  2. array is easy to use and storage efficient but has limited functionality
int[] nums= {1,2,3};
System.out.println(nums.length);

ArrayList<Integer> n = new ArrayList<Integer>();
n.add(1);
n.add(2);
System.out.println(n.size());
double array size:
int [] nums = new int[100];
int size = 0;  // track of current size
value [size] = value;
size++;
if (size>= nums.length){
  nums = Arrays.copyOf(nums, 2*nums.length);
}  // from java.util.Arrays package
System.out.println(Arrays.toString(nums)) // use string to print
Note:
  1. String can be converted to chars by s.toCharArray(), or you access individual char using s.chartAt(i)
  2. length of string is s.length() but length of array is a.length, length of ArrayList is al.size()

scanner

import java.util.Scanner;
System.out.println("Enter your username: ");
Scanner scanner = new Scanner(System.in);
String username = scanner.nextLine();
System.out.println("Your username is " + username);
You can use while(scanner.hasNextDouble()) to check user input for continuous input.

add external jar in intell j

file ->project structure -> modules -> dependecne => +JARs

java system property

java.util.Properties props = System.getProperties();
System.out.println(props.get("os.name"));
System.out.println(props.get("os.version"));
System.out.println(props.get("java.vendor"));
System.out.println(props.get("java.version"));
System.out.println(props.get("java.class.path"));

IDE

IntelliJ is my favourite. BlueJ is also nice due to its class-module visualization.

Downloading BlueJ

Mac
Windows
Linux

Friday, May 12, 2017

Intro to Fortran


Due to a technical interview at Intel, I have to pick up this legacy language.
Fortran, derived from Formula Translation, is a general-purpose, imperative programming language that is especially suited to numeric computation and scientific computing. It was originally developed by IBM in the 1950s. It has still been used in computationally intensive areas due to its fast speed and existing software/packages.
.f file is used in early Fortran program written in a fixed-column format to reflect the 80-column punched-card practice. So it has very weird grammar:
  • in each line, first 1-5 are label fields, it can be c (comment) or number (notation for the code block)
  • 6th column. If it is something other than 0, it means the code continues from the previous line
  • 7~72. Real, independent codes
  • 73-80 are ignored because the IBM 704 card reader only had 72 columns
Smiley face
After Fortran 90, the Free Format is used and the file extension is .f90 In this format, the comment is signaled by ! and each line can be 132 symbols without the need of first 5 empty columns. The between line continuation is signaled by & at the end of the previous line as well as the head of next line.
Most commonly used versions today are: Fortran 77, Fortran 90, and Fortran 95. Newer versions such as Fortran 2008 only adds minor revision.

setup

There are several ways to install Fortran compiler/IDE.

1. GNU compiler

brew install gcc
🍺 /usr/local/Cellar/gcc/7.1.0: 1,485 files, 289.6MB
This GNU version of compiler bundles fortran and c together.

2. intel compiler

Intel® Parallel Studio XE Composer Edition for Fortran macOS*
Serial number : 26BK-MCT25TSK
expire 2018-5-12
But it turns out to be a compiler, which must be used along with Microsoft visual studio or Max Xcode.

3. eclipse Photran

run compiler

gfortran xx.f      # default output file is named a.out
gfortran xx.f -o xx  # customerize file name
./a.out   # execute file

tutorial

stanford

From time to time, so-called experts predict that Fortran will rapidly fade in popularity and soon become extinct. These predictions have always failed. Fortran is the most enduring computer programming language in history. One of the main reasons Fortran has survived and will survive is software inertia. Once a company has spent many man-years and perhaps millions of dollars on a software product, it is unlikely to try to translate the software to a different language. Reliable software translation is a very difficult task.
Use Fortrain 77 compiler on a Unix workstation.
Install libraries? Libraries have file names starting with lib and ending in .a. Some libraries have already been installed by your system administrator, usually in the directories /usr/lib and /usr/local/lib. For example, the BLAS library MAY be stored in the file /usr/local/lib/libblas.a. You use the -l option to link it together with your main program, e.g.
      f77 main.f -lblas

tutorials point

This website provides an online fortran 95 environment.
program title
implicit none  ! let compiler check all variables
real :: a, b, result ! declare variable type
a = 12.0
b = 15.0
result = a + b
print *, "the total is", result  ! * means format
write(*,*) reult ! similar to print, more variety
end program title  ! finish program
fortran is case insensitive.
variable type:
integer a
a = 1
real b
b = 1.0
real(kind=8) c ! declare bite size
c = 1e9
double precision cc ! double precision
cc = 1.578d10
complex d 
d = (3.2,2.5)  ! set complex value
character e ! declare one lette
character(len=10) f ! declear string size
f = "Hello"
logical h
h = .true.  ! note the weird dots
real, parameter :: pi = 3.14159 ! declare tyes and set initial value, using two colons
integer i, j
equivalence (i,j)  ! using the same meomory
mod (b,c)  ! equivalent to % in python
customized type
type :: person  ! begin to define a type person
    character(len=30) :: name
    interger :: age
end type person  ! finish defining the type
type(person) :: a  ! declare a person type variable
write(*,*) "name:" ! prompt user to input name
read(*,*) a%name  ! read user input into name
logical control
if (a>b) then
    print *, "a is larger than b"
else if (a== b) then
    print *, "a is equal to b"
else
    print *, "a is not larger than b"
end if
I happen to have a book “Fortran 95 程序设计” (彭国伦) which I bought 5 years ago but never get a chance to read it until now. It turns out to be extremely good. It not only introduces FORTRAN 95, but also mentions between its improvement over Fortran 77, and how some old-fashion styles such as goto should be discarded. It really helps me to understand some legacy codes within a couple of hours. I remembered how the old-fashion formatted Fortran codes scared me off when I first encounter Fortran. I master it now.

Thursday, March 23, 2017

SAS, University version

Why SAS?

SAS is short for “Statistical Analysis System”.
Timeline:
  • 1966, prototype was developed by Barr and Goodnight, and funded by NIH
  • 1976, they moved from North Carolina State University and founded SAS Institute.
  • 1985, SAS was rewritten in C to allow it run on Unix, MS-DOS, and windows.
  • 2002, Text Miner component was introduced.
  • 2010, a free version for student was introduced.
  • 2010-12, sued world programming, but European Court of Justice ruled that “the functionality of a computer program and the programming language cannot be protected by copyright”
So SAS has a long history and its target customers are enterprise analytics.
Features:
  • It is web browser based. Although starting a local server by virtual machine seems a little complicated, it has the advantage of cross-platform
  • It can be seen as “advanced statistical version“ of Excel, which has rich GUI for people to learn quickly and provides brilliant technical support.
  • Big corporations like SAS because there’s a complete ecosystem that satisfies customers’ every need.
  • its direct competitors are Stata and SPSS (acquired by IBM).
  • You click on the front-end, the corresponding codes are automatically generated in the back-end. This means you can have the code to generate the exact same graph or make changes on that.
  • Integrate with SQL seamlessly.
And the usage differs by industry sectors:

University Edition

This version is free. check here. SAS University Edition includes SAS® Studio, Base SAS®, SAS/STAT®, SAS/IML®, SAS/ACCESS® and several time series forecasting procedures from SAS/ETS®.
There are 2 approaches to get SAS running:
  1. download a .ova file (2.2GB). use virtual box to start a local host and run SAS locally.
  2. use AWS AMI: SAS University Edition. You have to pay EC2 fee ranging from 0.012-0.047 /hr. It’s actually pretty cheap.
Open a new browser window with http://localhost:10080/ And you are good to go.

learn

SAS programs have a DATA step, which retrieves and manipulates data, usually creating an SAS data set, and a PROC step, which analyzes the data.
data highchol;
    set sashelp.heart;
    where Chol_Status = "High";
run;
proc print data = highchol;
run;
proc print data = sashelp.cars;    /*two-level name: library.table */
    by Make;
    var Make Model Type;
run;

create library/ import csv

libname libsas 'S:/datafiles'; /* physical location of the dataset, which can be found in file's property */
data titanic;
    infile '/folders/myfolders/train.csv' dlm=',' firstobs=2; 
    input PassengerId Survived Pclass Name Sex;
run;
use proc import is much more convenient, you don’t need to manually assign the column name. video guide which uses the snippets
/** FOR CSV Files uploaded from Unix/MacOS **/
FILENAME CSV "/folders/myfolders/train.csv" TERMSTR=LF;
/** Import the CSV file.  **/
PROC IMPORT DATAFILE=CSV
            OUT=WORK.MYCSV
            DBMS=CSV
            REPLACE;
run;
/** Print the results. **/
PROC PRINT DATA=WORK.MYCSV; RUN;
/** Unassign the file reference.  **/
FILENAME CSV;
run;
Alternatively, you can use tasks and utilities -> utilities -> import data. Then drag and drop the file from the “server files and folders”.

Graph

scatterplot

ods graphics / reset imagemap;
proc sgplot data=SASHELP.CARS;
    title "Vehicle Statistics";
    scatter x=Horsepower y=MPG_City / group=Origin 
        markerattrs=(symbol=CircleFilled size=12) transparency=0.7 name='Scatter';
    xaxis grid;
    yaxis grid;
    keylegend / location=Inside across=1;
run;
ods graphics / reset;
title;
Other plots like barplot, histogram are similar.

Certification training

The ad is for version 9.3, 2011, while the latest version is 9.4, 2013.
There are several certification packages:
  • Base programming: 3.1 k
  • Advanced programming: 3.8 k/2.45k
  • Predictive Modeling: 2.65 k
  • statistical analysis: 3.05 k