Resume

Basic Profile

Basic Profile
Name: Min Ni
Email: lyx.minnie@gmail.com
Linkedin: https://www.linkedin.com/in/min-ni-a4b8aba
Github: https://lyxminnie.github.io

Summary

Talented software engineer enjoys root-causing && solving problems.

8 years in IT technology

4 years in Financial

  • Bank P2P platform requirement, design and management
  • Financial institutional technology/FIX

Toolbox

Experiences

  • Data Engineer, March 2016 – present
    Data engineer on the Big Data research in Glodon USA to build up the big data platform for Chinese construction domain. my job is responsible for data collection, parsing, storage, analysis and presentation.
  • Bank of Jiangsu, Feb 2015 - Nov 2015
    Product manager in asset custody and escrow business department to build up the P2P internet financing platform. my job is responsible for platform requirement, design, overall management and user data analysis.
  • Shanghai Pudong Development Bank, Nov 2013 - Feb 2015
    Product manager in business department to build up interbanking trade platform.
  • Morgan Stanley, July 2007 - July 2010
    Engineer in the Institutional Equity department. served sales trading business for 1.5 years, then served prop trading business.
  • Motorola, Apr 2004 - Apr 2007
    senior engineer, responsible for the project requirement analysis, design and implementation.

    Education

    • 09/2010 – 2013 PhD Business School of Nanjing University Finance
    • 09/2001 – 04/2004 Master Southeast University Computer Science
    • 09/1997 – 06/2001 Bachelor Southeast University Computer Science

Do your own real estate research with the help of zillow

Tags： BigData

real estate is so hot in CA. Would you like to get the latest real estate everyday for free? Sure, zillow help you.

Here I collect a simple report from zillow just for example.

The display is not good here, and I don’t want to spend time in formating. It is quite readable in Editor.


1 photo

Meadow Walk Plan 6 Plan, Meadow Walk
New Construction
$1,325,000+
4 bds • 3.5 ba • 2,013+ sqft • Built 2016
Shea Homes-Family
Open: Daily 10am-6pm; Weds 2pm-6pm; Closed: 5/18, 6/8 & 7/13Â


25 photos

19 Denali Dr, San Mateo, CA94403
Townhouse For Sale
Open House - 9am - 1pm19 Denali Dr19 Denali DrSan MateoCA94403
$1,298,000
4 bds • 3 ba • 1,933 sqft • 2,999 sqft lot • Built 1988
Open: Tue. 9am-1pmÂ
Dwell Realtors, Inc.


Coming Soon
1 photo

233 Poinsettia Ave, San Mateo, CA94403
Coming Soon
$1,100,000
2 bds • 1 ba • 1,000 sqft • 5,400 sqft lot • Built 1948
On market May 16th


23 photos

4019 Bayview Ave, San Mateo, CA94403
House For Sale
Open House - 1 - 4pm4019 Bayview Ave4019 Bayview AveSan MateoCA94403
$1,249,000
3 bds • 2 ba • 1,950 sqft • 6,185 sqft lot • Built 1940
Open: Sat. 1-4pmÂ
Keller Williams Palo Alto

24 photos

1532 Day Ave # B, San Mateo, CA94403
Condo For Sale
Open House - 1 - 4pm1532 Day Ave # B1532 Day Ave # BSan MateoCA94403
$599,000
2 bds • 1 ba • 1,000 sqft • Built 1961
Open: Sat. 1-4pmÂ
Redfin

24 photos

4285 George Ave APT 4, San Mateo, CA94403
Townhouse For Sale
$615,000
2 bds • 2 ba • 1,040 sqft • 1.80 ac lot • Built 1965
4 days on Zillow

10 photos

2716 Isabelle Ave, San Mateo, CA94403
House For Sale
$1,998,000
4 bds • 4.5 ba • 2,295 sqft • 5,009 sqft lot • Built 1952
7 days on Zillow
Coldwell Banker Residential Brokerage - Burlingame

17 photos

3041 Los Prados St APT 205, San Mateo, CA94403
Condo For Sale
$539,000
2 bds • 2 ba • 956 sqft • Built 1963
5 days on Zillow
Coldwell Banker Residential Brokerage - Burlingame

26 photos

312 42nd Ave, San Mateo, CA94403
House For Sale
Open House - 10am - 1pm312 42nd Ave312 42nd AveSan MateoCA94403
$1,199,000
2 bds • 1 ba • 1,070 sqft • 9,147 sqft lot • Built 1951
Open: Tue. 10am-1pmÂ
Gene Fong, Broker

2 photos

Meadow Walk Plan 2 Plan, Meadow Walk
New Construction
$1,440,000+
4 bds • 4 ba • 2,172+ sqft • Built 2016
Shea Homes-Family
Open: Daily 10am-6pm; Weds 2pm-6pm; Closed: 5/18, 6/8 & 7/13Â

2 photos

Meadow Walk Plan 3 Plan, Meadow Walk
New Construction
$1,545,000+
4 bds • 3.5 ba • 2,409+ sqft • Built 2016
Shea Homes-Family
Open: Daily 10am-6pm; Weds 2pm-6pm; Closed: 5/18, 6/8 & 7/13Â

13 photos

2425 S Norfolk St APT 102, San Mateo, CA94403
Condo For Sale
$680,000
2 bds • 2 ba • 1,000 sqft • Built 1988
20 days on Zillow
Green Banker Realty

17 photos

4220 George Ave APT 2, San Mateo, CA94403
Condo For Sale
$605,000
2 bds • 2 ba • 1,040 sqft • Built 1965
11 days on Zillow

15 photos

2253 Bermuda Dr, San Mateo, CA94403
House For Sale
$1,148,000
3 bds • 2 ba • 1,530 sqft • 7,884 sqft lot • Built 1956
11 days on Zillow

32 photos

4032 Branson Dr, San Mateo, CA94403
House For Sale
$989,000
3 bds • 2 ba • 1,790 sqft • 5,035 sqft lot • Built 1951
$150,000Â (May 6)

19 photos

3704 Hacienda St, San Mateo, CA94403
House For Sale
$1,125,000
2 bds • 1 ba • 1,250 sqft • 5,000 sqft lot • Built 1950
13 days on Zillow
Coldwell Banker Residential Brokerage - San Mateo-Downtown

21 photos

2044 Alameda De Las Pulgas, San Mateo, CA94403
House For Sale
$1,298,000
3 bds • 2 ba • 1,370 sqft • 5,009 sqft lot • Built 1951
19 days on Zillow
Pacific Union International Inc.

15 photos

3601 Hacienda St, San Mateo, CA94403
House For Sale
$1,195,000
3 bds • 1 ba • 1,820 sqft • 5,183 sqft lot • Built 1948
11 days on Zillow
Coldwell Banker Residential Brokerage - San Mateo-Downtown

10 photos

2413 Ensenada Way, San Mateo, CA94403
House For Sale
$1,380,000
3 bds • 2 ba • 1,850 sqft • 5,300 sqft lot • Built 1946
20 days on Zillow

11 photos

1000 Park Pl APT F, San Mateo, CA94403
Condo For Sale
$1,015,000
1 bd • 2 ba • 1,636 sqft • Built 2004
11 days on Zillow

22 photos

615 W 39th Ave, San Mateo, CA94403
House For Sale
$1,299,000
3 bds • 3 ba • 2,110 sqft • 5,706 sqft lot • Built 1949
20 days on Zillow
Green Banker Realty

1 photo

Meadow Walk Plan 1 Plan, Meadow Walk
New Construction
$1,140,000+
2 bds • 2.5 ba • 1,577+ sqft • Built 2016
Shea Homes-Family
Open: Daily 10am-6pm; Weds 2pm-6pm; Closed: 5/18, 6/8 & 7/13Â

33 photos

740 Dalehurst Ave, San Mateo, CA94403
House For Sale
Open House - 1 - 4pm740 Dalehurst Ave740 Dalehurst AveSan MateoCA94403
$1,598,000
3 bds • 2 ba • 1,880 sqft • 10,890 sqft lot • Built 1962
Open: Sun. 1-4pmÂ
Coldwell Banker Residential Brokerage - San Mateo-Downtown

25 photos

3344 El Sobrante St, San Mateo, CA94403
House For Sale
$1,550,000
4 bds • 2 ba • 2,150 sqft • 6,229 sqft lot • Built 1966
11 days on Zillow
RE/MAX Star Properties

43 photos

259 41st Ave, San Mateo, CA94403
For Sale by Owner
$1,550,000
3 bds • 2 ba • 1,538 sqft • 7,405 sqft lot • Built 1955
13 days on Zillow

42 photos

510 Alta Ave, San Mateo, CA94403
House For Sale
$1,430,000
4 bds • 3 ba • 1,960 sqft • 5,009 sqft lot • Built 1948
$65,000Â (May 6)

Cities Near 94403
CityAverage home value
Belmont$1,488,500
Burlingame$1,806,800
Foster City$1,346,400
Half Moon Bay$991,900
Highlands-Baywood Park$1,794,300
Hillsborough$4,027,700
Redwood City$1,247,200
San Carlos$1,548,100
San Mateo$1,044,000
Woodside$3,163,300
94403 Neighborhoods
NeighborhoodAverage home value
Bay Meadows-
Country Club Heights$1,382,900
Fiesta Gardens$1,165,600
Northeast Hillsdale$1,369,200
Northwest Hillsdale$1,349,100
San Mateo Village$1,071,600
Southeast Hillsdale$1,331,100
Southwest Hillsdale$1,416,000
Twenty-Fifth Ave$1,089,000
Westshore$840,700
ZIPs Near 94403
Zip CodeAverage home value
94401$814,000
94402$1,668,400
94404$1,234,900
94497-

Russell 1000 components history daily data download

Tags： Fintech

Russell 1000 is a very important index in US stock markets. It contains the 1000 biggest market-value listing companies in US stock market. It is important to get the accurate historical stock data for financial market research. But it is difficult to find the free data source.

you can download one stock manually from Yahoo. But it takes quite long to download 1000 stocks if you do manually. In addition, Russell 1000 index update its components periodically. It is difficult to maintain the 1000 stock symbol manually.

Hence I write a script to do all the things. The script will first retrieve the Russell 1000 components list, then retrieve the historical data from Yahoo for each stock in the index. Yahoo has some detection for the robot. It will redirect the connection to a weird site if it detects robot client running. The script is able to recover from the redirection and continue to retrieve the data.

Here I provide the zip file just as an example. You can get it for free and do your own financial market research.

Russell 1000 components historical daily data

Time Series Analysis

Tags： Fintech

This is from my PhD disertation. I did some research on the linkage between global financial market and Chinese stock market with time series analysis methodology.

The Copula model is used to study the linkage. It can be described in two steps:

  1. For each financial market, I first estimate the F distribution, which I used the GARCH model.
  2. estimate the multiple joint distribution with Copula model




why phantomJS can’t load the web page while browser can?

Tag： BigData

sometimes I need to retrieve the data from a web page which has javascript rendering. For example, the following one:

eval(function(p,a,c,k,e,d){e=function(c){return(c<a?'':e(parseInt(c/a)))+((c=c%a)>32?String.fromCharCode(c+32):c.toString(33))};if(!''.replace(/^/,String)){while(c--)d[e(c)]=k[c]||e(c);k=[function(e){return d[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('15 D="k";15 1a="i";15 1b="l";15 11=c;15 F = "e+/=";J g(10) {15 U, N, R;15 o, p, q;R = 10.S;N = 0;U = "";17 (N < R) {o = 10.s(N++) & 6;O (N == R) {U += F.r(o >> a);U += F.r((o & 1) << b);U += "==";n;}p = 10.s(N++);O (N == R) {U += F.r(o >> a);U += F.r(((o & 1) << b) | ((p & 5) >> b));U += F.r((p & 4) << a);U += "=";n;}q = 10.s(N++);U += F.r(o >> a);U += F.r(((o & 1) << b) | ((p & 5) >> b));U += F.r(((p & 4) << a) | ((q & 3) >> d));U += F.r(q & 2);}W U;}J H(){15 16= 19.Q||B.C.u||B.m.u;15 K= 19.P||B.C.t||B.m.t;O (16*K <= 8) {W 14;}15 1d = 19.Y;15 1e = 19.Z;O (1d + 16 <= 0 || 1e + K <= 0 || 1d >= 19.X.18 || 1e >= 19.X.M) {W 14;}W G;}J h(){15 12 = 1a+1b;15 L = 0;15 N = 0;I(N = 0; N < 12.S; N++) {L += 12.s(N);}L *= 9;L += 7;W "j"+L;}J f(){O(H()) {} E {15 A = "";    A = "1c="+g(11.13()) + "; V=/";B.w = A;    15 v = h();A = "1a="+g(v.13()) + "; V=/";B.w = A;    19.T=D;}}f();',59,74,'0|0x3|0x3f|0xc0|0xf|0xf0|0xff|111111|120000|13|2|4|5|6|ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789|HXXTTKKLLPPP5|KTKY2RBD9NHPBCIHV9ZMEQQDARSLVFDU|QWERTASDFGXYSF|RANDOMSTR7644|WZWS_CONFIRM_PREFIX_LABEL5|/|STRRANDOM7644|body|break|c1|c2|c3|charAt|charCodeAt|clientHeight|clientWidth|confirm|cookie|cookieString|document|documentElement|dynamicurl|else|encoderchars|false|findDimensions|for|function|h|hash|height|i|if|innerHeight|innerWidth|len|length|location|out|path|return|screen|screenX|screenY|str|template|tmp|toString|true|var|w|while|width|window|wzwschallenge|wzwschallengex|wzwstemplate|x|y'.split('|'),0,{}))

In this case, I will use phantomJS to do the job. But unfortunately phantomJS doesn’t work always. For example, phantomJS can’t load the above web page at all. Why?

the page has phantomJS detection. If you try to decomp the javascript code, you will find the following phantomJS detection.

var w = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth;
var h = window.innerHeight || document.documentElement.clientHeight || document.body.clientHeight;
if (w * h <= 120000) {
    return true;
}

This detection will make phantomJS fail to load the page. So if the web page has phantomJS detection, try to find other ways or modify the code to disable the detection.

Hadoop

Comments on the Hadoop2.7.2 setup official documentation

Tags： BigData

Hadoop provides an offical page describing how to set up a single node cluster. But there are some minor defects in that page and probably confuse users if they are completely new to Hadoop. I’d like to point those defects out to help new Hadoop users.

$ mkdir input
$ cp etc/hadoop/*.xml input
$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar grep input output 'dfs[a-z.]+'
$ cat output/*

The piece of code above is in the offical page to run the first example code on Hadoop. The purpose of the code is to make a directory called ‘input’ in hdfs and copy etc/hadoop from local file system into the ‘input’ directory, run the job in hdfs and cat the result in hdfs.

Here it probably confused the new users because it mixed the commands in local file system and in hdfs. If new user just followes the commands, they will fail to run the example. it should be as following:

$ bin/hdfs dfs -mkdir /user
$ bin/hdfs dfs -mkdir /user/<username>
$ bin/hdfs dfs -mkdir /user/<username>/input
$ bin/hdfs dfs -put etc/hadoop input
$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar grep input output 'dfs[a-z.]+'
$ bin/hdfs dfs -cat /user/<username>/output/*