Pages

What is Zookeeper ?

Zookeeper is use to build general distributed application so its a coordination service.
In this tutorials we will look how to build a sample distributed application.
First look at the definition
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

Source : http://hadoop.apache.org/zookeeper/

In Zookeeper we can handle partial failure. Partial failure means something like this.
Suppose we send a message across the network to one node to another node.If the network fails sender does not know whether the receiver get the message. So the only way to find this is reconnect to the receiver and ask it. This is called the partial failure.

Zookeeper Characteristics
1.Simple
             Zookeeper has some few valuable operations, such as ordering and notifications.
2.Expressive
             Can build large data structures and protocols.
3.Highly Available
             Runs on a collection of machines and designed to be highly available. 
4.Loosely  coupled interaction
             Zookeeper participants do not need to know about one another.

Installing Zookeeper

Here are the steps to install zookeeeper.
1. To install zookeeper require java 6 or later version. You can find the java latest version in http://www.oracle.com/technetwork/java/javase/downloads/index.html 

2. After installing the jdk set the path.
   Windows : Set path in environment variable
   Unix : Open a terminal type vi ~/.bash_profile -> export JAVA_HOME=/path   /to/java/dir -> export PATH=$PATH:$JAVA_HOME/bin

3. Download Zookeeper http://hadoop.apache.org/zookeeper/releases.html

4.Unzip it and set zookeeper variable

5.Before running zookeeper service you have to make the zoo.cfg file inside the /conf/zoo.cfg as
   tickTime=2000
   dataDir=/path/zookeeper/dir 
   clientPort=2181

6. Now all are ready to start the Zookeeper server
    zkServer.sh start

7. zkServer status is use to check zookeeper is running or not

Group Membership



Create a Group

import java.io.IOException;
import java.util.concurrent.CountDownLatch;

import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.Watcher.Event.KeeperState;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;

public class CreateGroup implements Watcher {
private static final int SESSION_TIMEOUT = 5000;
private ZooKeeper zk;
private CountDownLatch connectedSignal = new CountDownLatch(1);
public void connect(String hosts) throws IOException, InterruptedException {
zk = new ZooKeeper(hosts, SESSION_TIMEOUT, this);
connectedSignal.await();
}
@Override
public void process(WatchedEvent event) { // Watcher interface
if (event.getState() == KeeperState.SyncConnected) {
connectedSignal.countDown();
}
}
public void create(String groupName) throws KeeperException,
InterruptedException {
String path = "/" + groupName;
String createdPath = zk.create(path, null/*data*/, Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);
System.out.println("Created " + createdPath);
}
public void close() throws InterruptedException {
zk.close();
}
public static void main(String[] args) throws Exception {
CreateGroup createGroup = new CreateGroup();
createGroup.connect(args[0]);
createGroup.create(args[1]);
createGroup.close();
}
}

Zookeeper API : http://hadoop.apache.org/zookeeper/docs/r3.2.1/api/index.html

Join The Group

public class ConnectionWatcher implements Watcher {
private static final int SESSION_TIMEOUT = 5000;
protected ZooKeeper zk;
private CountDownLatch connectedSignal = new CountDownLatch(1);
public void connect(String hosts) throws IOException, InterruptedException {
zk = new ZooKeeper(hosts, SESSION_TIMEOUT, this);
connectedSignal.await();
}
@Override
public void process(WatchedEvent event) {
if (event.getState() == KeeperState.SyncConnected) {
connectedSignal.countDown();
}
}
public void close() throws InterruptedException {
zk.close();
}
}


public class JoinGroup extends ConnectionWatcher {
public void join(String groupName, String memberName) throws KeeperException,
InterruptedException {
String path = "/" + groupName + "/" + memberName;
String createdPath = zk.create(path, null/*data*/, Ids.OPEN_ACL_UNSAFE,
CreateMode.EPHEMERAL);
System.out.println("Created " + createdPath);
}
public static void main(String[] args) throws Exception {
JoinGroup joinGroup = new JoinGroup();
joinGroup.connect(args[0]);
joinGroup.join(args[1], args[2]);
// stay alive until process is killed or thread is interrupted
Thread.sleep(Long.MAX_VALUE);
}
}

Retrive members in a group


public class ListGroup extends ConnectionWatcher {
public void list(String groupName) throws KeeperException,
InterruptedException {
String path = "/" + groupName;
try {
List children = zk.getChildren(path, false);
if (children.isEmpty()) {
System.out.printf("No members in group %s\n", groupName);
System.exit(1);
}
for (String child : children) {
System.out.println(child);
}
} catch (KeeperException.NoNodeException e) {
System.out.printf("Group %s does not exist\n", groupName);
System.exit(1);
}
}
public static void main(String[] args) throws Exception {
ListGroup listGroup = new ListGroup();
listGroup.connect(args[0]);
listGroup.list(args[1]);
listGroup.close();
}
}

Znodes can be two types. Persistent or Ephemeral. This type is set at creation time and may not be change later. When the client's session ends or client exit the application Ephemeral node will deleted. Ephemeral nodes not have children. But the persistent node will not be deleted when client's session ends or client exit the application.

Watches

Watches allow client to get notification when a node changes in a some way. Watches are set by a zookeeper service.

Useful Links
http://hadoop.apache.org/zookeeper/docs/current/
http://hadoop.apache.org/zookeeper/docs/r3.2.1/api/index.html







            

8 comments:

  1. There are lots of information about hadoop have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get to the next level in big data. Thanks for sharing this.


    Hadoop Training Chennai
    Hadoop Training in Chennai
    Big Data Training in Chennai

    ReplyDelete
  2. This is one such interesting and useful article that i have ever read. The way you have structured the content is so realistic and meaningful. Thank you so much for sharing this in here. Keep up this good work and I'm expecting more contents like this from you in future.

    Big Data Training Chennai | Best hadoop training institute in chennai | hadoop course in Chennai

    ReplyDelete
  3. Thanks for sharing detailed information of unified functional testing automation tool. QTP Course in Chennai | QTP training

    ReplyDelete
  4. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    Web designing Course in Chennai | Hadoop Training in Chennai

    ReplyDelete
  5. Informative content by intelligent experts.It is really useful to me.Keep on posting more article like this.
    Java course in chennai | Selenium Course in Chennai | Software Testing courses in chennai

    ReplyDelete
  6. Thanks for Sharing the valuable information and thanks for sharing the wonderful article..We are glad to see such a wonderful article..
    QTP Training in Chennai | QTP Training Institute in Chennai | QTP Training

    ReplyDelete