Trend Analysis with ElasticSearch and Couchbase

 As I have already said in a previous post of mine (  Problems with Heroku, Geodjango and Couchbase? Me too! ), recently i have started using couchbase a lot.

It offers a variety of cool features that you can read about on their site and make their product attractive.

What is more, there are far more interesting stuff for a developer to start using a technology than just a flexible data model.

If you like this article, read more at my updated blog

Personally, what i love in couchbase is the super cool python client, which makes all the everything so simple. There is also tones of documentation and examples, but my experience with the company is deeper than just that.
Every time i send them an email they respond immediately, offering great support (even though for the open source community).

Enough is enough. Let's talk a bit about coding here.


I wanted something much more sophisticated this time. I luckily run into a nice presentation on elastic search with couchbase and that inspired me that this would be a nice solution!

I needed to make some trend analysis on millions on text entries per day. As you can see, such a solution seems native with the specific architecture.

It was pretty easy to set up a new couchbase node and an elasticsearch one in an Ubuntu Server.

What I needed, was a replication node of the couchbase in the elasticsearch. Documentation was there, and google returned a lot of results.
But everything seemed to work for elasticsearch 2.0.

What i needed was the 0.92 version, because it uses all the latest features of Apache Lucene which by the way are AWESOME!

After a lot of searching, in order for this to work you need the elasticsearch transport couchbase plugin version 1.0.1 beta!

This is just a draft post and i hope i will be able to update it soon with more details.

To sum up,
I have a cluster with couchbase and replicating my data in elasticsearch, doing trend analysis on my data seamlessly!


Σχόλια

Δημοσίευση σχολίου

Δημοφιλείς αναρτήσεις από αυτό το ιστολόγιο

One day at the WS-REST workshop in Florence

Εικόνα
I am really excited to write this article. I indeed had a wonderful time participating both as a presenter as well as an attendee for my first ever  ws-rest  workshop which is collocated each year with the famous WWW conference . The program was full of interesting topics, regarding (what else) discussions about REST practices, research and new ideas. I am borrowing its description here! Web APIs for a Web in evolution The Web is changing at a tremendous speed, and Web APIs play an important part in that. In fact, the number of Web APIs is growing so quickly that we face many challenges. The WS-REST workshop series aim to connect Web researchers and engineers to tackle the issues we are facing. Many of them are not solved by far: How to dynamically integrate Web APIs? How to create intelligent clients for Web APIs? How can we deal with the enormous growth and diversity? Session 1 First, to kick start the session,  Erik Wilde  had the floor for a super interesting k
Read more

PyAPI: A python library to play with various API Standards, like Hydra, Swagger, RAML and more

Εικόνα
Ever been in a situation where your manager asks you to write documentation for the APIs that you developed? Let’s face it, if you’re like most developers, you love to code and hate to write. Furthermore, writing takes time away from critical tasks you need to do, such as feature development and bug fixing. It’s no surprise that API documentation often ends up being frustrating and confusing for the reader—it rarely gets the attention it deserves. Hopefully nowadays there are many nice tools to document your APIs .  Why Document Your APIs? Let’s start with the non-technical side of the issue. API documentation has been around ever since the first programming languages were created. There’s been plenty of time to develop effective processes for creating quality documentation, yet well-written API documentation is still quite rare. Why doesn’t it happen? First, documentation is seldom prioritized. Even though it has a large impact on how much a software platform is adopted,
Read more

What is the Semantic Web?

The Semantic Web is an idea of World Wide Web inventor  Tim Berners-Lee  that the Web as a whole can be made more intelligent and perhaps even intuitive about how to serve a user's needs. Berners-Lee observes that although  search engine s index much of the Web's content, they have little ability to select the pages that a user really wants or needs. He foresees a number of ways in which developers and authors, singly or in collaborations, can use self-descriptions and other techniques so that context-understanding programs can selectively find what users want. Web 2.0 Is Not the Semantic Web Web 2.0 is all about people. It's a social thing. The second generation of the World Wide Web is focused on the ability for people to collaborate and share information online. Where the Web contains static  HTML  pages, Web 2.0 is  dynamic , in that it serves applications to users and offers open communications with an emphasis on Web-based communities. Web 2.0, because it focu
Read more