How to Create a User Home Directory in HDFS

sudo -u hdfs [hadoop-admin-user] fs -mkdir /user/[username]
sudo -u hdfs [hadoop-admin-user] fs –chown [username] : [username] /user/[username]

Loading static inner classes in Java

Use the following syntax when loading static inner classes. The key is to use $ (dollar) sign instead of . (dot) to separate outer and inner class names in your fully-qualified class name.

Class innerClass = Class.forName("com.foo.OuterClass$InnerClass");

Cherry Picking Commits in Subversion

I found this article quite useful in cherry picking commits in Subversion. Anyway I know for sure that merging in svn is not as fun as it is in other configuration management systems like git. Enjoy!.

My Google Summer of Code 2013 Proposal

My aim is to add support for Advanced Message Queuing Protocol (AMQP) in WS-Messenger module of Apache Airavata.

WS-Messenger is a message broker implementation that is based on publish/subscribe communication model. It is the core notification handler in Airavata that implements the WS-Eventing and WS-Notifications specifications. These mechanisms that are based on HTTP and Web services definitely have a lot to offer, but they are not the only ways to do things and most importantly, there are cases that these models cannot cater for. AMQP is an open standard application layer protocol for message-oriented middleware. It has an efficient binary wire protocol and a message routing framework based on point-to-point and publish-and-subscribe models. The objective of this project is to implement a robust and efficient publish/subscribe and store-and-forward messaging framework in WS-Messenger based on AMQP.

Read more here.

Google Summer of Code 2013 – Sri Lanka goes up 2 notches!

It is really exciting to see that Sri Lanka has climbed up the ladder this year with 56 acceptances. Its 14 more than last year’s regime and more importantly, I am one of those 14 .

After a long period of silence …

Its been more than a year since I wrote a blog post. I have been completely busy with my M.Sc. work including exams, continuous assessment, research work and more over family matters, etc … Not looking for excuses, but anyway it did not happen.

Hopefully from now on I will have blog posts at regular intervals. I think I have lot more things to write about the research work that I have been doing. That also includes the GSoC 2013 work that has turned out to be quite interesting.

Lets keep the fingers crossed!

Have you started to read papers?

If your answer is "not yet" then you are wasting your time I would say. Remember, the time is VERY limited.

I always believe that it is better to light a candle than curse the darkness.

Therefore I have decided to write reviews for the papers I read and post them here. Your comments on them are invaluable!.

Reading Research Literature

I have been reading (and trying to understand ;-) research papers for some time now. Luckily I now have reading papers as part of my M.Sc. curriculum. The good thing about that is that we are supposed to read one or more papers every week and submit a short summary for one of them. Also, each one of us has the chance of doing a presentation on a selected paper.

While doing that I happened to read following papers on reading papers which I found quite useful.

"How to Read a Paper" - http://www.cs.montana.edu/mwittie/other%20linked%20files/paper-reading.pdf

"Reading a Computer Science Research Paper" - http://pages.cpsc.ucalgary.ca/~pwlfong/Pub/inroads2009.pdf

In fact the "multi-pass" approach given in the former is quite effective.

Reading research papers is not that easy for a couple of reasons. Firstly if you are new to the domain the paper is based on, it requires substantial amount of effort to get background knowledge, without which you can not make sense of what the paper says. Secondly, the content in a paper is so "condensed" that you need to "expand" the content while reading to get the actual idea. Thirdly, research papers mostly include fair amount of complex mathematics. Another important thing is that you might have to read some of the citations in order to understand the paper thoroughly; at least certain parts of it.

Despite all that, once you get used to it, you will find reading research literature quite interesting. It is a best way to learn about things happening at forefront of your field(s) of interest. To be frank after reading a good paper, I always get the feeling that I already know something that most others don't which will have a major impact in say 5-10 years of time.

So, start reading research papers!

GCC Function Attributes

You can find the set of gcc function attribute options at [1]. If you are a smart C/C++ developer, you should always be concerned about these options.

[1] - http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html

Atomic operations in C/C++

The atomic operations provided by GCC [1] come in quite handy even though they are limited to a few data types and in fact Intel-specific. AFAIK SPARC has it's own set of variants.

[1] http://gcc.gnu.org/onlinedocs/gcc-4.4.0/gcc/Atomic-Builtins.html#Atomic-Builtins

SSH/SCP without entering password

It is quite cumbersome if you are to re-enter password each time you invoke scp/ssh. Especially when you have a build system managing hundreds of binaries and scp deploying each of them to a remote run environment.

I found [1] quite useful in that regard.

[1] - http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/comment-page-1/#comment-119367

_MSC_VER Values

VC6 - 1200
VC7 - 1300
VC7.1 - 1310
VC8 - 1400
VC9 - 1500

WINVER and _WIN32_WINNT Values

NT – 0X0400
2000 - 0x0500
XP - 0X0501
Server 2003 - 0X0502
Vista - 0X0600
Windows 7 - 0X0601

Two ways to proof of 0.999… = 1

Long Division

1/9 = 0.111...
9 x (1/9) = 9 x 0.111...
1 = 0.999...

Digit Manipulation

X = 0.999...
10X = 9.999...
10X - X = 9.999... - 0.999...
9X = 9
X = 1

Source : http://en.wikipedia.org/wiki/0.999...#Digit_manipulation

Hex/int conversion in C

int –> Hex

int i = 10;
void* h =  (void*)i;

Hex –> int

void* h = 0xa;
int i = (int)((const char*)h - (const char*)0);

GMail targeting under 1sec load time with HTML5

According to Google, many of the upcoming features of GMail are going to be  HTML5-ready. They hope HTML5 will speed up loading and execution time of GMail. Google’s goal is to get their Web-based email service to load in under 1 second. While going through the original article I noticed a couple of interesting things.

Speed is a feature! – Truly it is. I do believe speed is one of the most critical decision factor when it comes to selecting a software application. To be honest I am very much happy about the load and execution time of Google, but they are looking for more. Interesting!.

Currently, the Gmail program is comprised of 443,000 lines of JavaScript, with 978,000 lines if comments are included. – I did actually not know that it was written completely in JavaScript in the first place. The interesting point is roughly each line of JavaScript is accompanied by one line of comment. That implies how vital the comments are in a software programming and of course how complex the GMail logic is ;-).

Qpid Site Gets a Major Revamp!

Apache Qpid site gets a major revamp. It looks really awesome. Great work guys!

Airplanes punch holes in clouds and make it rain

This is a very rare scene in the sky. A recent study reveals that these punches are formed by airplanes. According to scientists droplets in clouds exist in a supercooled state, well below freezing, yet remain in liquid form. When planes go through these clouds, they cause a drop in temperature that freezes these droplets. As a result part of the cloud gets converted into ice crystals forming a punch hole. These ice crystals fall to earth in the form of snow or rain. So, this is another way airplanes mess around with the weather.

How to fix : DoModal inside a DLL Crashes with VS2005 SP1

ISSUE :

I had been experiencing a strange crash in my application since I installed VS2005 SP1.

It was an MFC-based add-in (i.e. a DLL) for MS Excel which did a DoModal internally to show up a Dialog. When I debugged I saw that the culprit was the following code segment in dlgcore.cpp.

Basically the AfxGetApp() returned NULL leading to an assertion fail.

FIX :

I inserted the code CWinApp theApp; in one of the source files in my DLL making a CWinApp-derived object available to the DLL. That solved my problem.

How to Debug Apache Qpid Broker with Intellij IDEA

Apache Qpid/Java no longer has a Apache Maven build as it’s build system was moved to Apache Ant some time back for historical reasons. Therefore you can not do usual mvn idea:idea gimmick to generate IDEA project files. You can create an IDEA project from the scratch but it is unlikely someone would bother to do all that. So, the only viable solution is to use Remote debugging facility in IDEA. This is how you could go about doing this business.

Basically you need to start Qpid broker in debug mode – JPDA mode to be precise - and have your IDE connect to the remote JVM through an IDEA Remote Debug Configuration so that you will be hooked into the remote JVM and get triggered for the calls that are being made.

You can could start the Qpid broker in JPDA mode by passing the command line argument –run:jpda to your startup script. On windows it is qpid-server.bat –run:jpda. The broker will start as a JPDA listener. You can set the wire protocol transport using the JPDA_TRANSPORT environment variable that defaults to dt_socket. You could also use dt_shmem on Windows platforms in case you need to use the shared memory transport. You can set the address (port) using the JPDA_ADDRESS environment variable that defaults to 8000.

Since we do not have an IDEA project for Qpid, we just create an empty project and attach source to it. Once that is done you could follow the steps given below to setup the debugger to use JPDA.

1. Go to Run –> Edit Configurations

2. Add a new configuration instance of type Remote

Set Transport depending on what you set in the broker startup script, Socket for the default case. Set Port accordingly, 8000 for the default case. You may also set the configuration instance name – Unnamed in the screen shot - and press OK.

3. Start your broker using the command qpid-server.bat -run:jpda

4. Launch debugger using Run –> Debug and you should see the following message in the IDEA console.

5. Set breakpoints as you wish and start debugging. Happy debugging! :-).