Pages

Hive + MongoDB

In this article I'll show you how to connect mongoDB and hive using MongoStorageHandler.

Suppose you want to load mongoDB data into hive. So that you can run any hive query (HQL) using hadoop map reduce.

First you have to setup hadoop , hive and mongoDB.

Then download the mongo-hadoop release based on your hadoop version.

extract it and copy all 3 jar files to hadoop/lib and hive/lib folders.

First we create a sample records in mongoDB.


 use hive-test  
 db.books.insert({ "_id": 1, "name": "Java 7", "author": "author1" });  
 db.books.insert({ "_id": 2, "name": "Hadoop", "author": "author2" });  
 db.books.insert({ "_id": 3, "name": "Hive", "author": "author3" });  

Then start hive console and run following commands.


 create schema books;  
 use books;  
 CREATE TABLE book (id int, name string, author string)  
 STORED BY "com.mongodb.hadoop.hive.MongoStorageHandler"  
 WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","name":"name","author":"author"}')  
 TBLPROPERTIES ( "mongo.uri" = "mongodb://localhost:27017/hive-test.books");  

Make sure data loaded into hive using

 select * from book;   

Troubleshoot

 Failed with exception org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.com.mongodb.hadoop.hive.MongoStorageHandler  
 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask  

Make sure you added above 3 jars into hadoop/lib and hive/lib

57 comments:

  1. This is very informative post. I really enjoy it and this is a great artical.I hope I will use this for our next business website.

    This is great job!

    Thanks for sharing this post.
    web development company uae

    ReplyDelete
  2. I get a lot of great information from this blog. Thank you for your sharing this informative blog. Just now I have completed hadoop certification course at a leading academy. If you are interested to learn Hadoop Training Chennai visit FITA IT training and placement academy.

    ReplyDelete
  3. You have certainly explained that Big data analytics is the process of examining big data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions..The big data analytics is the major part to be understood regarding Hadoop Training in Chennai program. Via your quality content i get to know about that in deep.Thanks for sharing this here.

    ReplyDelete
  4. This is the exact piece of information that I was searching for a long time(Hadoop Training in Chennai). Processing data is the biggest issue that every cloud based companies are facing worldwide(Big Data Hadoop Training in Chennai). Handling this problem made easy with the introduction of big data. Thank you so much for your worth able content here. Keep Posting article like this(Best Hadoop Training in Chennai).

    ReplyDelete
  5. Thanks for sharing informative post. Big data is a term that portrays the substantial volume of information; both organized and unstructured that immerses a business on an everyday premise. To know more details please visit Big Data Training in Chennai | Primavera Training in Chennai

    ReplyDelete
  6. Nice sharing. MongoDB is a widely used open source database system. Scop for mongodb is increasing day by day. Learn MongoDB Training from expert reach gangboard. Selenium Training

    ReplyDelete
  7. Hi, I am really happy to found such a helpful and fascinating post that is written in well manner. Thanks for sharing such an informative post.R Programming Online Training | Hadoop Online Training

    ReplyDelete
  8. Nice article updates. We've to post this blog informative it amazing. I like this our content update details. I read that our all post it's useful.
    Selenium Training in Chennai

    ReplyDelete
  9. Hey, I am truly upbeat to establish such an accommodating and intriguing post that is composed in well way. A debt of gratitude is in order for sharing such an instructive post...
    Selenium Training in Chennai
    Dot Net Training in Chennai
    Android Training in Chennai

    ReplyDelete
  10. This blog is having the general information. Got a creative work and this is very different one.We have to develop our creativity mind.This blog helps for this. Hadoop Online Training | Learn Hadoop Online Training | Hadoop Online Training in India



    ReplyDelete
  11. The scope of MongoDB and the Hive is increased day by day...
    Your blog is really much more informative and helpful to all people...
    Keep on blogging... Java Training in Chennai | Dot Net Training in Chennai

    ReplyDelete
  12. Hats off to your presence of mind..I really enjoyed reading your blog. I really appreciate your information which you shared with us.
    Data Science Online Training
    Hadoop Online Training

    ReplyDelete
  13. If anyone want to become an python certified professional reach besant Academy, which offers best python Training in bangalore with years of experienced professionals. For more details about courses…
    Selenium Training in Bangalore |
    Python Training in Bangalore |

    ReplyDelete
  14. It is really a great work and the way in which you are sharing the knowledge is excellent.Thanks for your informative article

    Hadoop Training in Marathahalli|
    Hadoop Training in Bangalore|
    Data science training in Marathahalli|
    Data science training in Bangalore|

    ReplyDelete
  15. This comment has been removed by the author.

    ReplyDelete
  16. Great and really helpful article! Adding to the conversation, you have provide more information, or express a new point of view.Nice information and updates. Really i like it and everyday am visiting your site... keep blogging.DevOps Training in Bangalore | DevOps Training in Bangalore

    ReplyDelete
  17. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.

    Data Science Training in Bangalore

    Datascience Training in Chennai

    ReplyDelete
  18. Informative post about hadoop, i am looking forward for realtime hadoop online training institute.

    ReplyDelete
  19. It has been simply incredibly generous with you to provide openly what exactly many individuals would’ve marketed for an eBook to end up making some cash for their end, primarily given that you could have tried it in the event you wanted.

    Data Science Training in Bangalore


    DataScience Training in Chennai


    ReplyDelete
  20. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    hadoop-training-institute-in-chennai

    ReplyDelete
  21. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    hadoop-training-institute-in-chennai

    ReplyDelete
  22. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.

    big data training in chennai


    ReplyDelete
  23. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful. calendar

    ReplyDelete


  24. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Selenium Training in chennai

    ReplyDelete

  25. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
    RPA Training in Bangalore



    ReplyDelete
  26. I would recommend my profile is important to me, I invite you to discuss this topic... clipping path service company

    ReplyDelete
  27. I'm happy to see the considerable subtle element here!. National Anisette Day

    ReplyDelete
  28. Two full thumbs up for this magneficent article of yours. I've really enjoyed reading this article today and I think this might be one of the best article that I've read yet. Please, keep this work going on in the same quality. Web Solutions company

    ReplyDelete
  29. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Data Science Training in Bangalore

    ReplyDelete
  30. Keeping the quality of a job expands its efficacy and helps to boost it. I’m pleased to get the info about the importance of ‘Blog commenting’ and some precious clues to improve it. Thanks for the article. Best AWS Training in Bangalore

    ReplyDelete
  31. I'm happy to see the considerable subtle element here!. private label manufacturers

    ReplyDelete
  32. This is also a very good post which I really enjoyed reading. It is not every day that I have the possibility to see something like this.. PhenQ

    ReplyDelete
  33. Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area.

    Data Science Training in Bangalore
    RPA Training in Bangalore
    Automation Anywhere Training in Bangalore

    ReplyDelete
  34. informative blog thanks for providing such a great information.
    Data Science Training in Hyderabad

    ReplyDelete
  35. I am always searching online for articles that can help me. There is obviously a lot to know about this. I think you made some good points in Features also. Keep working, great job autopilot automated autoblogging

    ReplyDelete