Pages

Hive + MongoDB

In this article I'll show you how to connect mongoDB and hive using MongoStorageHandler.

Suppose you want to load mongoDB data into hive. So that you can run any hive query (HQL) using hadoop map reduce.

First you have to setup hadoop , hive and mongoDB.

Then download the mongo-hadoop release based on your hadoop version.

extract it and copy all 3 jar files to hadoop/lib and hive/lib folders.

First we create a sample records in mongoDB.


 use hive-test  
 db.books.insert({ "_id": 1, "name": "Java 7", "author": "author1" });  
 db.books.insert({ "_id": 2, "name": "Hadoop", "author": "author2" });  
 db.books.insert({ "_id": 3, "name": "Hive", "author": "author3" });  

Then start hive console and run following commands.


 create schema books;  
 use books;  
 CREATE TABLE book (id int, name string, author string)  
 STORED BY "com.mongodb.hadoop.hive.MongoStorageHandler"  
 WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","name":"name","author":"author"}')  
 TBLPROPERTIES ( "mongo.uri" = "mongodb://localhost:27017/hive-test.books");  

Make sure data loaded into hive using

 select * from book;   

Troubleshoot

 Failed with exception org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.com.mongodb.hadoop.hive.MongoStorageHandler  
 FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask  

Make sure you added above 3 jars into hadoop/lib and hive/lib

92 comments:

  1. This is very informative post. I really enjoy it and this is a great artical.I hope I will use this for our next business website.

    This is great job!

    Thanks for sharing this post.
    web development company uae

    ReplyDelete
  2. I get a lot of great information from this blog. Thank you for your sharing this informative blog. Just now I have completed hadoop certification course at a leading academy. If you are interested to learn Hadoop Training Chennai visit FITA IT training and placement academy.

    ReplyDelete
  3. You have certainly explained that Big data analytics is the process of examining big data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions..The big data analytics is the major part to be understood regarding Hadoop Training in Chennai program. Via your quality content i get to know about that in deep.Thanks for sharing this here.

    ReplyDelete
  4. This is the exact piece of information that I was searching for a long time(Hadoop Training in Chennai). Processing data is the biggest issue that every cloud based companies are facing worldwide(Big Data Hadoop Training in Chennai). Handling this problem made easy with the introduction of big data. Thank you so much for your worth able content here. Keep Posting article like this(Best Hadoop Training in Chennai).

    ReplyDelete
  5. Thanks for sharing informative post. Big data is a term that portrays the substantial volume of information; both organized and unstructured that immerses a business on an everyday premise. To know more details please visit Big Data Training in Chennai | Primavera Training in Chennai

    ReplyDelete
  6. Nice sharing. MongoDB is a widely used open source database system. Scop for mongodb is increasing day by day. Learn MongoDB Training from expert reach gangboard. Selenium Training

    ReplyDelete
  7. Hi, I am really happy to found such a helpful and fascinating post that is written in well manner. Thanks for sharing such an informative post.R Programming Online Training | Hadoop Online Training

    ReplyDelete
  8. Nice article updates. We've to post this blog informative it amazing. I like this our content update details. I read that our all post it's useful.
    Selenium Training in Chennai

    ReplyDelete
  9. Hey, I am truly upbeat to establish such an accommodating and intriguing post that is composed in well way. A debt of gratitude is in order for sharing such an instructive post...
    Selenium Training in Chennai
    Dot Net Training in Chennai
    Android Training in Chennai

    ReplyDelete
  10. This blog is having the general information. Got a creative work and this is very different one.We have to develop our creativity mind.This blog helps for this. Hadoop Online Training | Learn Hadoop Online Training | Hadoop Online Training in India



    ReplyDelete
  11. The scope of MongoDB and the Hive is increased day by day...
    Your blog is really much more informative and helpful to all people...
    Keep on blogging... Java Training in Chennai | Dot Net Training in Chennai

    ReplyDelete
  12. Hats off to your presence of mind..I really enjoyed reading your blog. I really appreciate your information which you shared with us.
    Data Science Online Training
    Hadoop Online Training

    ReplyDelete
  13. If anyone want to become an python certified professional reach besant Academy, which offers best python Training in bangalore with years of experienced professionals. For more details about courses…
    Selenium Training in Bangalore |
    Python Training in Bangalore |

    ReplyDelete
  14. It is really a great work and the way in which you are sharing the knowledge is excellent.Thanks for your informative article

    Hadoop Training in Marathahalli|
    Hadoop Training in Bangalore|
    Data science training in Marathahalli|
    Data science training in Bangalore|

    ReplyDelete
  15. This comment has been removed by the author.

    ReplyDelete
  16. Great and really helpful article! Adding to the conversation, you have provide more information, or express a new point of view.Nice information and updates. Really i like it and everyday am visiting your site... keep blogging.DevOps Training in Bangalore | DevOps Training in Bangalore

    ReplyDelete
  17. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.

    Data Science Training in Bangalore

    Datascience Training in Chennai

    ReplyDelete
  18. Informative post about hadoop, i am looking forward for realtime hadoop online training institute.

    ReplyDelete
  19. It has been simply incredibly generous with you to provide openly what exactly many individuals would’ve marketed for an eBook to end up making some cash for their end, primarily given that you could have tried it in the event you wanted.

    Data Science Training in Bangalore


    DataScience Training in Chennai


    ReplyDelete
  20. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    hadoop-training-institute-in-chennai

    ReplyDelete
  21. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    hadoop-training-institute-in-chennai

    ReplyDelete
  22. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.

    big data training in chennai


    ReplyDelete
  23. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful. calendar

    ReplyDelete


  24. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Selenium Training in chennai

    ReplyDelete

  25. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
    RPA Training in Bangalore



    ReplyDelete
  26. I would recommend my profile is important to me, I invite you to discuss this topic... clipping path service company

    ReplyDelete
  27. I'm happy to see the considerable subtle element here!. National Anisette Day

    ReplyDelete
  28. Two full thumbs up for this magneficent article of yours. I've really enjoyed reading this article today and I think this might be one of the best article that I've read yet. Please, keep this work going on in the same quality. Web Solutions company

    ReplyDelete
  29. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Data Science Training in Bangalore

    ReplyDelete
  30. Keeping the quality of a job expands its efficacy and helps to boost it. I’m pleased to get the info about the importance of ‘Blog commenting’ and some precious clues to improve it. Thanks for the article. Best AWS Training in Bangalore

    ReplyDelete
  31. I'm happy to see the considerable subtle element here!. private label manufacturers

    ReplyDelete
  32. This is also a very good post which I really enjoyed reading. It is not every day that I have the possibility to see something like this.. PhenQ

    ReplyDelete
  33. Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area.

    Data Science Training in Bangalore
    RPA Training in Bangalore
    Automation Anywhere Training in Bangalore

    ReplyDelete
  34. informative blog thanks for providing such a great information.
    Data Science Training in Hyderabad

    ReplyDelete
  35. I am always searching online for articles that can help me. There is obviously a lot to know about this. I think you made some good points in Features also. Keep working, great job autopilot automated autoblogging

    ReplyDelete
  36. A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one. Key Feature Of Roblox Game

    ReplyDelete
  37. I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you. www.weightwatcherspointscalculator.com

    ReplyDelete
  38. This post is very simple to read and appreciate without leaving any details out. Great work! https://fnafworlddownloadfree.com/

    ReplyDelete
  39. A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one. best soccer prediction site.

    ReplyDelete
  40. Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I've been searching for. You have a clever yet attractive way of writing. top soccer prediction sites

    ReplyDelete
  41. I am a new user of this site so here i saw multiple articles and posts posted by this site,I curious more interest in some of them hope you will give more information on this topics in your next articles. full time draw prediction site

    ReplyDelete
  42. You actually make it look so easy with your performance but I find this matter to be actually something which I think I would never comprehend. It seems too complicated and extremely broad for me. I'm looking forward for your next post, I’ll try to get the hang of it! Photo retouching jobs

    ReplyDelete
  43. I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you. Photo Retouching Services

    ReplyDelete
  44. Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I've been searching for. You have a clever yet attractive way of writing. retouch photo online

    ReplyDelete
  45. Cool stuff you have and you keep overhaul every one of us
    National Cousins Day

    ReplyDelete
  46. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own Blog Engine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it. best gaming keyboard budget

    ReplyDelete
  47. The Breach is taking its place in my life, this is still one of the best space-set Ocean of Games around.

    ReplyDelete
  48. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us. Do check Six Sigma Training in Bangalore | Six Sigma Training in Dubai
    & Get trained by an expert who will enrich you with the latest trends.

    ReplyDelete
  49. Great post! This is very useful for me and gain more information, Thanks for sharing with us.

    Article submission sites

    Education

    ReplyDelete
  50. The blog which you are shared is very much helpful for us to knew about the web designing. thanks for your information.
    Web Designing Institute
    Best Web Design Courses
    Web Design Training Courses
    Learn Website Design
    Best Way to Learn Web Design

    ReplyDelete
  51. Awwsome informative blog ,Very good information thanks for sharing such wonderful blog with us ,after long time came across such knowlegeble blog. keep sharing such informative blog with us.
    Airport Ground Staff Training Courses in Chennai | Airport Ground Staff Training in Chennai | Ground Staff Training in Chennai

    ReplyDelete
  52. Appreciate Your Work... Thanks for Sharing Useful Information. I Just want to Share Some information related to RPA Training in Chennai hope it is useful for the Community Here.
    Robotic Process Automation Certification
    RPA Training
    RPA Training Institute in Chennai
    Robotic Process Automation Courses

    ReplyDelete
  53. This is a good post. This post give truly quality information. I’m definitely going to look into it. Really very useful tips are provided here. thank you so much. Keep up the good works.
    German Classes in Chennai
    German language training in chennai
    German Training in Chennai
    German classes chennai
    German teaching institutes in chennai
    German Training Institutes in Chennai

    ReplyDelete
  54. Thanks first of all for the useful info.
    the idea in this article is quite different and innovative please update more.
    Java Training center in Bangalore
    Java Training in Nolambur
    Java Training in Nungambakkam
    Java Training in Navalur

    ReplyDelete
  55. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us. Do check | Get trained by an expert who will enrich you with the latest updates.
    Java Training in Chennai
    PHP Training in Chennai
    Java Classes in Chennai
    Core Java Training in Chennai
    PHP course
    PHP Training

    ReplyDelete
  56. Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, because you have explained the concepts very well. It was crystal clear, keep sharing..
    Selenium Training in Chennai
    Selenium Course in Chennai
    iOS Course in Chennai
    Digital Marketing Training in Chennai
    J2EE Training in Chennai
    Different functions in testing
    Loadrunner course in Chennai
    Best Loadrunner training institute in chennai

    ReplyDelete