Pages

Enable Tomcat Debug with Intellij Idea

Nutshell guide to enable tomcat debug.

Prerequisites
1. Java & Tomcat

Step 1
Navigate to Tomcat Home directory bin folder

Step 2
Check setenv.sh is exists , if not create a file called setenv.sh and add following line.

 CATALINA_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"  

Step 3
Start tomcat. Your tomcat debug port is not 8000.

Step 4
Open up Intellij Idea.
Run --> Edit Configuration --> select + --> Select Remote
Change port to 8000 and save.

Setup Jenkins with Nginx reverse proxy

Setting up jenkins is something pretty straight forward. But configuring with Nginx is not that easy. So in this post I'm trying to ease this up for you.

Before setting up jenkins I would recommend oracle java installation. So here how it begins.

Step 1
1.Download and install java.

 sudo wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u31-b13/jdk-8u31-linux-x64.tar.gz"  

2.untar jdk and create JAVA_HOME in ~/.bashrc

Step 2
Jenkins setup
1.
 wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -  
 sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'  
 sudo apt-get update  
 sudo apt-get install jenkins  

To start jenkins fire up following command.

 sudo /etc/init.d/jenkins start  

To check logs

 tail -f /var/log/jenkins/jenkins.log  

To check your jenkins is up and running.
 http://<host>:8080/  


Step 3
In order to config nginx with jenkins we have do some customization.
1. Stop jenkins

 sudo /etc/init.d/jenkins stop  

2. Open jenkins script and add --prefix=/jenkins into JENKINS_ARGS

 JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT --prefix=/jenkins"  

3. Install nginx

 sudo aptitude -y install nginx  

4. Remove default config and add following configuration

 cd /etc/nginx/sites-available  
 sudo rm default ../sites-enabled/default  


 server {  
  listen     80;    # Listen on port 80 for IPv4 requests  
  server_name   54.194.149.122;  
  location ~ "^/static/[0-9a-fA-F]{8}\/(.*)$" {  
   #rewrite all static files into requests to the root  
   #E.g /static/12345678/css/something.css will become /css/something.css  
   rewrite "^/static/[0-9a-fA-F]{8}\/(.*)" /$1 last;  
  }  
  location /userContent {  
     #have nginx handle all the static requests to the userContent folder files  
     #note : This is the $JENKINS_HOME dir  
     root /var/lib/jenkins/;  
     if (!-f $request_filename){  
       #this file does not exist, might be a directory or a /**view** url  
       rewrite (.*) /$1 last;  
       break;  
     }  
     sendfile on;  
  }  
  location /jenkins {  
    sendfile off;  
    proxy_pass     http://127.0.0.1:8080;  
    proxy_redirect   default;  
    proxy_set_header  Host       $host;  
    proxy_set_header  X-Real-IP    $remote_addr;  
    proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;  
    proxy_max_temp_file_size 0;  
    #this is the maximum upload size  
    client_max_body_size    10m;  
    client_body_buffer_size  128k;  
    proxy_connect_timeout   90;  
    proxy_send_timeout     90;  
    proxy_read_timeout     90;  
    proxy_buffer_size     4k;  
    proxy_buffers       4 32k;  
    proxy_busy_buffers_size  64k;  
    proxy_temp_file_write_size 64k;  
 }  
 }  

5. http://<host>/jenkins

Hive + MongoDB

In this article I'll show you how to connect mongoDB and hive using MongoStorageHandler.

Suppose you want to load mongoDB data into hive. So that you can run any hive query (HQL) using hadoop map reduce.

First you have to setup hadoop , hive and mongoDB.

Then download the mongo-hadoop release based on your hadoop version.

extract it and copy all 3 jar files to hadoop/lib and hive/lib folders.

First we create a sample records in mongoDB.


 use hive-test  
 db.books.insert({ "_id": 1, "name": "Java 7", "author": "author1" });  
 db.books.insert({ "_id": 2, "name": "Hadoop", "author": "author2" });  
 db.books.insert({ "_id": 3, "name": "Hive", "author": "author3" });  

Then start hive console and run following commands.


 create schema books;  
 use books;  
 CREATE TABLE book (id int, name string, author string)  
 STORED BY "com.mongodb.hadoop.hive.MongoStorageHandler"  
 WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","name":"name","author":"author"}')  
 TBLPROPERTIES ( "mongo.uri" = "mongodb://localhost:27017/hive-test.books");  

Make sure data loaded into hive using

 select * from book;   

Troubleshoot

 Failed with exception org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.com.mongodb.hadoop.hive.MongoStorageHandler  
 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask  

Make sure you added above 3 jars into hadoop/lib and hive/lib