Hadoop 2.7 Installing on Ubuntu 14.04 (Pseudo-distributed mode)

Installing Java

Please read this artcile

Adding a dedicated Hadoop system user

For Hadoop, the accounts should have the same username on all of the nodes. This account is only for managing your Hadoop cluster. Once the
cluster daemons are up and running, you’ll be able to run your actual MapReduce jobs from other accounts.

bluething@ubuntu:~$ sudo addgroup hadoop
bluething@ubuntu:~$ sudo adduser --ingroup hadoop hadoop-user

Installing SSH

ssh has two main components:

ssh : The command we use to connect to remote machines – the client.
sshd : The daemon that is running on the server and allows clients to connect to the server.

The ssh is pre-enabled on Linux, but in order to start sshd daemon, we need to install ssh first.

Verify SSH installation

bluething@ubuntu:~$ which ssh
bluething@ubuntu:~$ which sshd
bluething@ubuntu:~$ which ssh-keygen

Install openssh-server

bluething@ubuntu:~$ sudo apt-get install openssh-server
bluething@ubuntu:~$ which sshd

Generate SSH key pair

bluething@ubuntu:~$ su - hadoop-user
hadoop-user@ubuntu:~$ ssh-keygen -t rsa -P ""
This command will create an RSA key pair with an empty password. Generally, using an empty password is not recommended, but in this case it is needed to unlock the key without your interaction (you don’t want to enter the passphrase every time Hadoop interacts with its nodes).
If you want to see public key form

hadoop-user@ubuntu:~$ more /home/hadoop-user/.ssh/id_rsa

Enable SSH access to your local machine with this newly created key.

hadoop-user@ubuntu:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Test the SSH setup by connecting to your local machine with the hadoop-user user. The step is also needed to save your local machine’s host key fingerprint to the hadoop-user user’s known_hosts file.

hadoop-user@ubuntu:~$ ssh localhost
The authenticity of host 'localhost (' can't be established.
ECDSA key fingerprint is 5b:5d:57:1a:08:34:51:9d:b2:26:3b:19:b3:84:eb:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

hadoop-user@ubuntu:~$ exit
Connection to localhost closed.


Installing Hadoop

Read more…

Categories: Hadoop

Downloading Java JDK on Linux via wget

Use this command for download Oracle JDK 8u60

wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.tar.gz

This wget essentially passes a cookies which pretends user has accepted the agreement

Categories: JAVA, Ubuntu

Play! Framework – Deployment using Jenkins [Installation and Configuration]

System requirement:

  • Java 7 or higher


  1. Jenkins installation
    sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
    sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
    sudo yum install jenkins
  2. Add jenkins to startup and then start
    sudo chkconfig jenkins on
    sudo service jenkins start
  3. Check if Jenkins had been running, go to http://your_ip:8080/


  4. Configure JDK
    Go to Manage Jenkins – Configure System. In JDK section set java home.
    Set JDK
  5. Add ssh configuration (for git)
    cd /var/lib/jenkins/
    sudo mkdir .ssh
    sudo nano config

    Fill config file with

    Host repo your_git_ip
        HostName your_git_ip
        IdentityFile /var/lib/jenkins/.ssh/your_git_key
        User your_git_username

    Tes ssh config

    sudo su -s /bin/bash jenkins
    git ls-remote -h ssh://your_git_username@your_git_ip/your_git.git HEAD

    If result is are “too open”, change your_git_key permission to 600
    Change owner of config and key

    sudo chown jenkins:jenkins your_git_key
    sudo chown jenkins:jenkins config
  6. Get activator
    download activator

    wget http://downloads.typesafe.com/typesafe-activator/1.3.5/typesafe-activator-1.3.5-minimal.zip

    unzip to /opt and change owner jenkins:jenkins

    chown jenkins:jenkins -R activator-1.3.5-minimal/

    Read more…

Categories: Jenkins, Play! Framework

Moving git repository to new server

1. Fetch all of the remote branches and tags from the existing repository to our local index

git fetch origin

2. Check all branch

git branch -a

3. Add new ssh config for new server. Create new file with text editor, I use notepad++, name with config. For windows save in C:\Users\sashihara\.ssh. After this you will have 3 file, config, id_rsa, id_rsa_aws.

Host host1 ip_host1
 HostName ip_host1
 IdentityFile C:\Users\sashihara\.ssh\id_rsa
 User gituser

Host host2 ip_host2
 HostName ip_host2
 IdentityFile C:\Users\sashihara\.ssh\id_rsa_aws
 User gituser

4. Add new remote origin, give different name. You must already have new git repository  (repo/app/new.git)

git remote add origin-new ssh://gituser@ip_host2/repo/app/new.git

5. Push to a new origin

git push --all origin-new

6. Push tags to a new origin

git push --tags new-origin

7. [optional] remove origin and rename new origin

git remote rm origin
git remote rename origin-new origin


Categories: Umum

Export PUTTY Sessions list

Type this command on command prompt

regedit /e "%userprofile%\desktop\putty-sessions.reg" HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\Sessions

That command will export putty session list to putty-sessions.req file locate on Desktop

If you want to export all setting, use this command

regedit /e "%userprofile%\desktop\putty.reg" HKEY_CURRENT_USER\Software\SimonTatham
Categories: Umum

Installing Oracle JDK on Ubuntu

March 8, 2015 1 comment

This tutorial will cover the installation of 64-bit Oracle Java 7 (7.0_75)

  1. Check to see if your Ubuntu system architecture is 32-bit or 64-bit, mine is 64 bit. Open up a terminal and run the following command below.
    bluething@ubuntu:~$ file /sbin/init
    /sbin/init: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=7a4c688d009fc1f06ffc692f5f42ab09e68582b2, stripped
  2. Check if you have Java installed on your system. Mine doesn’t have any Java installed.
    bluething@ubuntu:~$ java -version
    The program 'java' can be found in the following packages:
    * default-jre
    * gcj-4.8-jre-headless
    * openjdk-7-jre-headless
    * gcj-4.6-jre-headless
    * openjdk-6-jre-headless
    Try: sudo apt-get install <selected package>

    If you have Java installed on your system, the command should return like this, for Ubuntu default Java is OpenJDK.

    java version "1.7.0_15"
    OpenJDK Runtime Environment (IcedTea6 1.10pre) (7b15~pre1-0lucid1)
    OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

    If you have Java installed on your system, remove the OpenJDK/JRE with the following command.

    sudo apt-get purge openjdk-\*
  3. Create a directory to hold your Oracle Java JDK/JRE binaries.
    bluething@ubuntu:~$ sudo mkdir -p /usr/local/java
    [sudo] password for bluething:
    bluething@ubuntu:~$ ls -l /usr/local
    total 36
    drwxr-xr-x 2 root root 4096 Apr 16 2014 bin
    drwxr-xr-x 2 root root 4096 Apr 16 2014 etc
    drwxr-xr-x 2 root root 4096 Apr 16 2014 games
    drwxr-xr-x 2 root root 4096 Apr 16 2014 include
    drwxr-xr-x 2 root root 4096 Mar 8 05:44 java
    drwxr-xr-x 4 root root 4096 Apr 16 2014 lib
    lrwxrwxrwx 1 root root 9 Mar 8 01:56 man -> share/man
    drwxr-xr-x 2 root root 4096 Apr 16 2014 sbin
    drwxr-xr-x 7 root root 4096 Apr 16 2014 share
    drwxr-xr-x 2 root root 4096 Apr 16 2014 src
  4. Download the Oracle Java JDK/JRE for Linux, you can find here. Make sure you select the correct compressed binaries for your system architecture 32-bit or 64-bit. I put on home directory.
  5. Unpack the compressed Java binaries to the directory /usr/local/java.
    bluething@ubuntu:~$ sudo tar xvzf jdk-7u75-linux-x64.tar.gz -C /usr/local/java
    bluething@ubuntu:~$ sudo tar xvzf jdk-8u40-linux-x64.tar.gz -C /usr/local/java
    bluething@ubuntu:~$ ls -l /usr/local/java
    total 8
    drwxr-xr-x 8 uucp 143 4096 Dec 18 17:04 jdk1.7.0_75
    drwxr-xr-x 8 uucp 143 4096 Feb 10 21:42 jdk1.8.0_40
  6. Edit the system PATH file /etc/profile and add the following system variables to your system path. I’m using gedit.
    export JAVA_HOME
    export JRE_HOME
    export PATH
  7. Inform your Ubuntu Linux system where your Oracle Java JDK/JRE is located.
    bluething@ubuntu:~$ sudo update-alternatives --install "/usr/bin/java" "java" "/usr/local/java/jdk1.7.0_75/bin/java" 1
    update-alternatives: using /usr/local/java/jdk1.7.0_75/bin/java to provide /usr/bin/java (java) in auto mode
    bluething@ubuntu:~$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/local/java/jdk1.7.0_75/bin/javac" 1
    update-alternatives: using /usr/local/java/jdk1.7.0_75/bin/javac to provide /usr/bin/javac (javac) in auto mode
    bluething@ubuntu:~$ sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/local/java/jdk1.7.0_75/bin/javaws" 1
    update-alternatives: using /usr/local/java/jdk1.7.0_75/bin/javaws to provide /usr/bin/javaws (javaws) in auto mode
  8. Inform your Ubuntu Linux system that Oracle Java JDK/JRE must be the default Java.
    bluething@ubuntu:~$ sudo update-alternatives --set java /usr/local/java/jdk1.7.0_75/bin/java
    bluething@ubuntu:~$ sudo update-alternatives --set javac /usr/local/java/jdk1.7.0_75/bin/javac
    bluething@ubuntu:~$ sudo update-alternatives --set javaws /usr/local/java/jdk1.7.0_75/bin/javaws
  9. Reload your system wide PATH /etc/profile.
    bluething@ubuntu:~$ . /etc/profile

    Reboot Ubuntu

  10. Test to see if Oracle Java was installed correctly on your system.
    bluething@ubuntu:~$ java -version
    java version "1.7.0_75"
    Java(TM) SE Runtime Environment (build 1.7.0_75-b13)
    Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode)


Categories: JAVA, Ubuntu


February 22, 2015 Leave a comment

Problem Statement

At the annual meeting of Board of Directors of Acme Inc, every one starts shaking hands with everyone else in the room. Given the fact that any two persons shake hand exactly once, Can you tell the total count of handshakes?

Input Format
The first line contains the number of test cases T, T lines follow.
Each line then contains an integer N, the total number of Board of Directors of Acme.

Output Format

Print the number of handshakes for each test-case in a new line.


1 <= T <= 1000
0 < N < 106

This problem is about combinations. The formula is

For factorial calculation I’m using recursive with memoization to optimization process. Read more…