Stepping into a pattern for handling asynchronous operations in Java Script called "JavaScript Promise"

I have been currently working on Parse.com and Automatic for my prototyping. I had a problem communication between Parse and Automatic web service through HTTP.

Here is my question on stackoverflow.

Parse.Cloud.httpRequest call with HTTP request header
The point is that I sent an HTTP request but any callback functions are not called.

According to the suggested solution, I found the following two mistakes in my code.

• Wrong callback function
• I did not respond to a response from a server 

Thanks to the solution, I could learn how to write a callback function for  asynchronous operations in JavaScript.

This is a pattern for handling asynchronous operations in JavaScript that you can see in JavaScript libraries like AngularJS. It is called "JavaScript Promises".

I stepped into the very beginning of asynchronous operations in JavaScript this time. If I found something interesting, I will write it on this blog.

Completed "Introduction to Databases" of Stanford Univeristy with "distinction"

I completed "Introduction to Databeses" of Stanford Univeristy with "Distinction", Yeah! This entry explains how this online course looks like.

Why did I participated in this course?

I was embedded software engineer and seldom use databases. Although, I was transferred to the Business Intelligence company as temporally trainee last October. Then, I often use several relational databases recently. So, I became to think I need to study database again. At that time, I found this course. Studying with book alone is a little bit hard, So I took this course.

Topics that this course covers

As I expected this course focuses on relational database theory, including SQL or table design, but they also cover non relational database including XML and JSON.

• Relational Database Overview
• XML Data
• JSON Data
• Relational Algebra
• SQL
• Relational Design Theory
• Querying XML
• Unified Modeling Language
• Indexes
• Transactions
• Constraints and Triggers
• Views
• Authorization
• Recursion
• Online Analytical Programming
• NoSQL Systems

How to participate in this course

In this course, every material became public from the beginning. You can start from any topic. Although, deadline of each assignment work is different. So, you should pick a topic that deadline is close first.

For each topic, you repeat the following procedure:

• Watch some short videos that explain topics
• Answer simple quizes（No score here）
• Try to basic problems
• multi-selection question（low score）
• Programming work（middle score）
• Try to challenge problems
• Advanced programming work（high score）

Certification

When you answer each question, it is automatically scored. You can see how much percentage of scores you already got. If you solve all basic problems, you get 50 % and can complete the course. If you also try challenge problems and get 75 %, you can get "State of Accomplishment with Distinction".

Oh, everything is English!

It is really hard point for Japanese people. I actually worked in the office where no one speaks in English but now my all coworkers speak in English. So, no problems with this for me now.

Although, I felt difficulty when a teacher talked really fast about a topic that I was not familiar with. At the time, I had to watch the videos again and again. As a result, I tried to understand by solving some problems. Or sometimes I research some Japanese materials related to the topics.

How hard it was?

It depends on your skill and knowledge. It seems that it takes 3 hours every week to complete the course. You need 7 to 8 hours every week to complete with distinction. I sometimes had to study all day on Saturday or Sunday. If you work full time, you cannot take more than one course at the same time.

The important point to take an online course

If every material becomes public from the beginning, you should consider your private schedule and make a concrete schedule for the course. You might be struggling with difficult problems, so you should have some time to spare.

You should not postpone challenge problems even though their deadlines are late because you might forget about details when you are back to the problems :)

--

Have fun with an online course!!

"Fluentd: Open Source Log Management" by Sadayuki Furuhashi, Treasure Data at SF Metric Meetup

Sadayuki Furuhashi talks about Fluentd: Open Source Log Management from Librato on Vimeo.

This is a participation report for the SF Meetup. This meetup is probably data analysis for server administration. A log management is really far from my current job but I have been interested in fluentd since I knew that it is good approach to make data collection easy. And this is a good chance to hear about it from the founder. So, I participated in this.

About SF Metric Meetup

In the 21st Century successful teams are data-driven. This monthly meetup provides a forum for monitoring geeks to gather and trade new ideas, data, and war stories. If you love data, this meetup is for you! (cited from meetup.com) 

About the speaker

Sadayuki Furuhashi is an architect and founder of Treasure Data. He is also a founder of some open source projects including fluentd.

About fluentd

Fluentd is extensive log management tool written by Ruby. Nowadays, the major web tech companes produce millions of lines of logs every day and their analysis sever operational status with the logs. Fluentd makes this collection/shape/analysis process easier. 

What they tried to do in this meetup

This time he came to the meetup with his coworkers of Treasure Data. I talked with some his coworkers about their business. They are trying to make his open source projects, including Fluentd, MessagePack popular through some meetups, before providing charged service. I realized that it is important to have this kind of networking opportunity and talk about own products to potential customers. I keep paying attention to their products and services.

Remote debugging for Tomcat and IntellJ on Max OS X

IntellJ supports remote debugging for Tomcat. It is really convenient for debugging. This entry shows how to adopt remote debugging with Tomcat and IntellJ.

My dev environment

• Max OS X 10.9.2
• Java 1.8.0
• Tomcat 8.0.3

Set up

Tomcat

1. Create a new file called <Tomcat install directory>/bin/setenv.sh. (This file will be automatically read by catalina.sh)
2. Add the following content to setenv.sh. 8080 is a port number for Tomcat. Please specify for your tomcat.
• CATALINA_OPTS="-agentlib:jdwp=transport=dt_socket,address=8080,server=y,suspend=n"
3. Start Tomcat normally using startup.sh.

IntellJ

1. Select "Run -> Edit Configuration" then "Run/Configuration" screen is open.
2. Specify "Name" like "Tomcat".
3. Specify "Port", which is the same number as Tomcat's port numer

Debug

1. If you want to start debugging on IntellJ, just select "Run -> debug 'Tomcat'". 
2. If IntellJ shows a message like below, you are already connected.
• Connected to the target VM, address: 'localhost:8080', transport: 'socket'
3. Now you can debug. Please refer here for more detail about debugging by IntellJ.

Apache Lucene: Then and Now, Java User Group meetup at Twitter HQ

Meetup.com - Apache Lucene: Then and Now

This time, Doug Cutting (@cutting) talked about the history of Apache Lucene and how Apache Lucene is used for the implementation of Internet search engines and local, single-site searching in major tech companies like Linkedin or Twitter. He also mentioned this project is integrated with Hadoop and still evolving.

Doug Cutting (@cutting) is the founder of numerous successful open source projects, including Lucene, Nutch, Avro, and Hadoop. Doug joined Cloudera in 2009 from Yahoo!, where he was a key member of the team that built and deployed a production Hadoop storage and analysis cluster for mission-critical business analytics. Doug holds a Bachelor’s degree from Stanford University and sits on the Board of the Apache Software Foundation. (cited from Meetup.com) 

His talk is based on the following blog entries.

• Cloudera Search: The Newest Hadoop Framework for CDH Users and Developers
• The Blur Project: Marrying Hadoop with Lucene

As you see, his roles in his company (Cloudara) is the implementation of the new open source project, Blur.

Blur is an Apache Incubator project that provides distributed search functionality on top of Apache Hadoop, Apache Lucene, Apache ZooKeeper, and Apache Thrift. When I started building Blur three years ago, there wasn’t a search solution that had a solid integration with the Hadoop ecosystem. Our initial needs were to be able to index our data using MapReduce, store indexes in HDFS, and serve those indexes from clusters of commodity servers while remaining fault tolerant. Blur was built specifically for Hadoop — taking scalability, redundancy, and performance into consideration from the very start — while leveraging all the great features that already exist in the Hadoop stack. (cited from blog.cloudera.com)

And Cloudera is providing a better way for non-programming users interact with Hadoop data.

In the context of our platform, CDH (Cloudera’s Distribution including Apache Hadoop), Cloudera Search is another framework much like MapReduce and Cloudera Impala. It’s another way for users to interact with Hadoop data and for developers to build Hadoop applications. Each framework in our platform is designed to cater to different families of applications and users (cited from blog.cloudera.com)

 See Cloudera blog for more details.

It seems that there are meetup of Java user group in SF once or twice a month. I am planning to continue joining meetup.

JSON Lint - validation tool for JSON

There are several validation tools for JSON. These tools help you to write and test JSON data or schema.

• JSONLint
• This tool validates that JSON data is valid or not.
• It does not deal with JSON schema.
• JSON Schema Lint
• This tool validates
• JSON schema is valid
• JSON data is valid
• JSON data is valid against JSON schema

INSERT VS COPY: The fastest way to do a bulk insert into PostgreSQL

It takes time to populate lots of data in database by many "INSERT" statements. The official PostgreSQL documentation mentions the topic.

PostgreSQL 9.3.2 Documentation Chapter 14. Performance Tips 14.4. Populating a Database

The chapter shows many options, but I am paying attention to first two.

Disable autocommit

PostgreSQL commits each statement automatically. That means if you run an INSERT statement, it runs like this for each statement:

1. Open a transaction
2. Insert data
3. Close the transaction

It is redundant. In this case, it becomes faster if you use autocommit like this:

BEGIN; -- the beginning of the transaction
INSERT xxxx
INSERT xxxx
END;    -- the end of the transaction

PostgreSQL does not commit the statements between BEGIN and END.

COPY

COPY is also to use populate data, but the different point is that COPY does not offer autocommit. It means that PostgreSQL commit after all data is populated on a database.
Please refer to the document about how to use COPY statement.