In this tutorials we will look how to build a sample distributed application.
First look at the definition
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them ,which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.
Source : http://hadoop.apache.org/zookeeper/
In Zookeeper we can handle partial failure. Partial failure means something like this.
Suppose we send a message across the network to one node to another node.If the network fails sender does not know whether the receiver get the message. So the only way to find this is reconnect to the receiver and ask it. This is called the partial failure.
Zookeeper Characteristics
1.Simple
Zookeeper has some few valuable operations, such as ordering and notifications.
2.ExpressiveCan build large data structures and protocols.
3.Highly Available
Runs on a collection of machines and designed to be highly available.
4.Loosely coupled interaction
Zookeeper participants do not need to know about one another.
Installing Zookeeper
Here are the steps to install zookeeeper.
1. To install zookeeper require java 6 or later version. You can find the java latest version in http://www.oracle.com/technetwork/java/javase/downloads/index.html
2. After installing the jdk set the path.
Windows : Set path in environment variable
Unix : Open a terminal type vi ~/.bash_profile -> export JAVA_HOME=/path /to/java/dir -> export PATH=$PATH:$JAVA_HOME/bin
3. Download Zookeeper http://hadoop.apache.org/zookeeper/releases.html
4.Unzip it and set zookeeper variable
5.Before running zookeeper service you have to make the zoo.cfg file inside the /conf/zoo.cfg as
tickTime=2000
dataDir=/path/zookeeper/dir
clientPort=2181
6. Now all are ready to start the Zookeeper server
zkServer.sh start
7. zkServer status is use to check zookeeper is running or not
Group Membership
Create a Group
import java.io.IOException;Zookeeper API : http://hadoop.apache.org/zookeeper/docs/r3.2.1/api/index.html
import java.util.concurrent.CountDownLatch;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.Watcher.Event.KeeperState;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
public class CreateGroup implements Watcher {
private static final int SESSION_TIMEOUT = 5000;
private ZooKeeper zk;
private CountDownLatch connectedSignal = new CountDownLatch(1);
public void connect(String hosts) throws IOException, InterruptedException {
zk = new ZooKeeper(hosts, SESSION_TIMEOUT, this);
connectedSignal.await();
}
@Override
public void process(WatchedEvent event) { // Watcher interface
if (event.getState() == KeeperState.SyncConnected) {
connectedSignal.countDown();
}
}
public void create(String groupName) throws KeeperException,
InterruptedException {
String path = "/" + groupName;
String createdPath = zk.create(path, null/*data*/, Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);
System.out.println("Created " + createdPath);
}
public void close() throws InterruptedException {
zk.close();
}
public static void main(String[] args) throws Exception {
CreateGroup createGroup = new CreateGroup();
createGroup.connect(args[0]);
createGroup.create(args[1]);
createGroup.close();
}
}
Join The Group
public class ConnectionWatcher implements Watcher {
private static final int SESSION_TIMEOUT = 5000;
protected ZooKeeper zk;
private CountDownLatch connectedSignal = new CountDownLatch(1);
public void connect(String hosts) throws IOException, InterruptedException {
zk = new ZooKeeper(hosts, SESSION_TIMEOUT, this);
connectedSignal.await();
}
@Override
public void process(WatchedEvent event) {
if (event.getState() == KeeperState.SyncConnected) {
connectedSignal.countDown();
}
}
public void close() throws InterruptedException {
zk.close();
}
}
public class JoinGroup extends ConnectionWatcher {
public void join(String groupName, String memberName) throws KeeperException,
InterruptedException {
String path = "/" + groupName + "/" + memberName;
String createdPath = zk.create(path, null/*data*/, Ids.OPEN_ACL_UNSAFE,
CreateMode.EPHEMERAL);
System.out.println("Created " + createdPath);
}
public static void main(String[] args) throws Exception {
JoinGroup joinGroup = new JoinGroup();
joinGroup.connect(args[0]);
joinGroup.join(args[1], args[2]);
// stay alive until process is killed or thread is interrupted
Thread.sleep(Long.MAX_VALUE);
}
}
Retrive members in a group
public class ListGroup extends ConnectionWatcher {
public void list(String groupName) throws KeeperException,
InterruptedException {
String path = "/" + groupName;
try {
Listchildren = zk.getChildren(path, false);
if (children.isEmpty()) {
System.out.printf("No members in group %s\n", groupName);
System.exit(1);
}
for (String child : children) {
System.out.println(child);
}
} catch (KeeperException.NoNodeException e) {
System.out.printf("Group %s does not exist\n", groupName);
System.exit(1);
}
}
public static void main(String[] args) throws Exception {
ListGroup listGroup = new ListGroup();
listGroup.connect(args[0]);
listGroup.list(args[1]);
listGroup.close();
}
}
Znodes can be two types. Persistent or Ephemeral. This type is set at creation time and may not be change later. When the client's session ends or client exit the application Ephemeral node will deleted. Ephemeral nodes not have children. But the persistent node will not be deleted when client's session ends or client exit the application.
Watches
Watches allow client to get notification when a node changes in a some way. Watches are set by a zookeeper service.
Useful Links
http://hadoop.apache.org/zookeeper/docs/current/
http://hadoop.apache.org/zookeeper/docs/r3.2.1/api/index.html
There are lots of information about hadoop have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get to the next level in big data. Thanks for sharing this.
ReplyDeleteHadoop Training Chennai
Hadoop Training in Chennai
Big Data Training in Chennai
This is one such interesting and useful article that i have ever read. The way you have structured the content is so realistic and meaningful. Thank you so much for sharing this in here. Keep up this good work and I'm expecting more contents like this from you in future.
ReplyDeleteBig Data Training Chennai | Best hadoop training institute in chennai | hadoop course in Chennai
Thanks for sharing detailed information of unified functional testing automation tool. QTP Course in Chennai | QTP training
ReplyDeleteNice and informative post..QTP Training in Chennai | QTP Training Institute in Chennai | QTP Course in Chennai
ReplyDeleteExcellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
ReplyDeleteWeb designing Course in Chennai | Hadoop Training in Chennai
Informative content by intelligent experts.It is really useful to me.Keep on posting more article like this.
ReplyDeleteJava course in chennai | Selenium Course in Chennai | Software Testing courses in chennai
Thanks for Sharing the valuable information and thanks for sharing the wonderful article..We are glad to see such a wonderful article..
ReplyDeleteQTP Training in Chennai | QTP Training Institute in Chennai | QTP Training
ReplyDeletevery nice article.Thanks for sharing the Post...!
Testing Training with Live Project
This blog was very interesting with good guidance, it is very helpful for developing my skill. I eagerly for your more posts, keep sharing...!
ReplyDeleteOracle DBA Training in Chennai
Oracle DBA Course in Chennai
Spark Training in Chennai
Oracle Training in Chennai
Linux Training in Chennai
Social Media Marketing Courses in Chennai
Primavera Training in Chennai
Unix Training in Chennai
Power BI Training in Chennai
Tableau Training in Chennai
Very Good Blog. Highly valuable information have been shared. Highly useful blog..Great information has been shared. We expect many more blogs from the author. Special thanks for sharing..
ReplyDeleteSAP Training in Chennai | AWS Training in Chennai | QTP Training in Chennai | Selenium Training in Chennai | Networking Training in Chennai