Information on different Key Value stores

2010-03-18T12:55:12Z

In my previous blog on “Key Value stores: Usefulness in Cloud environment” I have mentioned the advantages and the scenarios where Key value stores can be useful. There are various Key value stores available over internet which differs from each other in some or other ways. So you need to decide on which to go for depending on your requirement.]]> Below are the few details that I came to know about some of the Key value stores. You can use these details to select the database that suits your need.

Database Name	Database Type	Indexing	Schema required	Accessible via Protocol
Simple DB	Key - value	Yes	No	REST and SOAP
Couch DB	Document Oriented	Yes	No	RESTful HTTP/JSON API
Cassandra	Column Orineted	Yes	No	RESTful HTTP/JSON API
Voldemort	Key - value	No	I had a very tough time in one of my projects where we were using the Azure management API from a web application (deployed in Azure cloud). Before this project, we had successfully used these APIs in a windows based application but this is first time we were using in a web application. And apparently had a good amount of learning in a very new dimension. Continue reading "Handling certificates for Azure management API calls" » Accessing CouchDB for Data Storage and Retrieval 2010-02-23T12:43:18Z Cloud Databases are an emerging type of non relational databases which do not follow relational algebra and are generally key-value oriented systems which are used for storing internet scale data and provide easy programmatic access. Databases like Amazon SimpleDB, Cassandra from Facebook and Apache CouchDB and many others falls in the category of non relational databases and has been widely used in large consumer based applications and are quite popular. Apache CouchDB is a document oriented, non-relational, schema less database used for storing internet scale data. Apache CouchDB provides a rich set of REST based JSON API’s to access the database to perform administrative as well as CRUD operations. I and Rajarshi Bhose have written an article which will demonstrate how to use CouchDB for CRUD and administrative operations using Java. This article is published at Dr. Dobb’s and it can be accessed at http://www.drdobbs.com/java/223100116. To learn in detail about some of the basics of CouchDB, and various other operations that can be performed like how to create database, how to insert data into database, how to retrieve data based on “keys” and also based on “field” values, please go through the tutorial. ]]> Accessing Hadoop DFS for Data Storage and Retrieval 2010-02-19T11:30:41Z Distributed File Systems (DFS) are a new type of file systems which provides some extra features over normal file systems and are used for storing and sharing files across wide area network and provide easy programmatic access. ]]> I have written a tutorial which will demonstrate how to use HDFS for basic distributed file system operations using Java. This tutorial is published at sys-con magazine and it can be accessed at http://opensource.sys-con.com/node/1287801. To learn in detail about how to use HDFS for data storage and retrieval using java, please go through the tutorial. SaaS makes financial accounting easier 2010-02-15T10:34:06Z It’s been so much talked and written about technical benefits of Cloud computing. I thought of looking at it from financial accounting point of view. Comparing the traditional IT expenses with Cloud computing expenses especially SaaS model, might give a different picture of the income statement. Before I go ahead and try putting forward my interpretation, want to make a disclaimer that I am no way finance or accounting expert, this is just a plain interpretation of what I understood. Comments and suggestions are most welcome. Continue reading "SaaS makes financial accounting easier" » Key Value stores: Usefulness in Cloud environment 2010-02-04T05:14:16Z Cloud Databases are a new type of non relational (key-value oriented) databases which are used for storing internet scale data and provide easy programmatic access. Databases like Amazon SimpleDB, Apache CouchDB, Project Voldemort and many others falls in the category of non relational databases and has been widely used and are quite popular. ]]> Features provided by some of these key value stores are: low latency, replication, scalability, distributed etc., which naturally suits them for use in cloud environment. Also the API’s provided by these key value stores are simple and easy to use. Here are some reasons for why you would choose Key Value database platform for your application: - The data store is cheap and integrates easily with your vendor's web services platform. - Your data is heavily document-oriented, making it a more natural fit with the key/value data model than the relational data model. - Your data does not need strict transactional and consistency like data related to social communities. - Your application deal with large amount of mostly read data. - Your foremost concern is on-demand, high-end scalability - that is, large-scale, distributed scalability, the kind that can't be achieved simply by scaling up. Security Issues with the Cloud 2010-01-21T06:27:30Z Despite all the hype surrounding the cloud, enterprise customers are still reluctant to get their hands wet. The reason… security. Industry verticals like Banking and capital markets, Defense and other high risk projects give far more weight age to security and fail safety than marginal reduction in capital expenditure. Undiscovered security holes, which lie hidden inherently in the OS for decades, may cause much higher damage when the whole system is exposed to the brute force hacking power of the public internet. ]]> Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 A recent published security hole in Microsoft Windows allows users with restricted access to escalate their privileges to system level. The most disturbing part in this expose is that this hole lay hidden in the OS from 32bit versions of Windows NT 3.1 upto and including Windows 7. That’s a hole that lay undetected for 17 years! And it’s not only Microsoft that’s facing the music. High-risk, kernel level security flaws continue to be reported in RHEL, SUSE and Debian. The philosophy followed, when encountered with a security flaw, by both of them are quite different and must also be considered when deciding on an environment prone to attacks. Microsoft follows the traditional approach, security through obscurity. The belief here is that if inherent flaws both theoretical and actual are not disclosed to the public, the chances of finding and exploiting them are drastically reduced. Debian follows a much more radical and much debated philosophy, full disclosure. The logic here is that even though blackhats (a hacker who breaks into networks or computers) may misuse this disclosure, whitehats (ethical hacker or penetration tester who focuses on securing and protecting IT systems) will be able to obtain more info and generate patches quickly. But at the end of the day, whatever be the excuses, high risk enterprises will always value security over marginal cost benefits. A thorough understanding of the underlying technologies, adequate security audits and risk analysis needs to be done before suggesting cloud based solutions to enterprises. An excellent article on Linux vs. Windows Security can be found here. MYOC - Offload compute intensive tasks on Azure using the Offline Processing pattern 2009-12-14T12:59:41Z In this post on the MYOC cloud development series, I will share an offline processing design pattern where certain computation tasks are offloaded to another execution task using queues and that can help reducing the overall processing time of online transactions. This is a very useful pattern to use, if you plan to build highly scalable and compute intensive application on the web today. This patterns is also used by many popular websites. Here I will demonstrate how we've used this pattern to help reduce the poll creation time. Continue reading "MYOC - Offload compute intensive tasks on Azure using the Offline Processing pattern" » Hybrid Clouds: Business Imperatives 2009-12-09T17:25:23Z What could possibly be some of the business imperatives for hybrid cloud adoption by enterprises? In my opinion one of foremost imperatives is the taxation laws to be enacted by governmnets that are based on corporate emissions. Considering the fact that a significant percentage of the carbon footprint includes emissions from the datacenters, government regulations to bring down corporate emissions could act as a trigger for multiple organizations to share hosting resources within the same data-center. It is in these contexts that we may see more of hybrid cloud model perhaps taking shape. Cloud Bursts: Reality versus Hype!!I 2009-12-09T16:37:04Z The taxonomy of cloud computing consisting of private clouds, public clouds and hybrid clouds is by now well established in the cloud computing literature and discussion forums. Inspite of the fact that proponents of cloud computing are gung-ho about associated technologies such as cloud burst -- the technology that ultimately helps realize hybrid clouds, Gartner predicts that businesses will use private clouds as of now. Even Forrestor backed private cloud in its report earlier this year. Does that mean the end of technologies associated with cloud burst? In my opinion it is not!!! While it is true that we may be witnessing a slowdown in hybrid cloud adoption, yet I believe that enterprises will be amenable towards adopting the cloud burst technology in the coming days. As different lines of business within an organization continues to invest in building their own private clouds, we will witness a scenario where an enterprise has multiple private clouds. The private clouds could belong to different lines of business or it could belong to one line of business possibly spread across multiple geographies. Typically, the private cloud of a particular business line on one hand can remain under-utilized for a major fraction of the time and on the other hand a short surge in demand could result in over-utilized servers and reduced quality of services even though spare capacity is available in the private cloud belonging to the other business line. Using cloud burst technologies, it possible to create a shared and yet private cloud. Each of the different LOBs has a dedicated cloud so that the applications belonging to a particular LOB runs only in its cloud under normal circumstances. Though discrete, these clouds form a shared virtual infrastructure so that an application in an overloaded cloud can procure the computing resources from an under-loaded cloud. That means, during overload situations, an application requiring more computing resources can procure these resources from a cloud different from its current host cloud. Both these cloud, however belong to the same organization; albeit different lines of business. Scenarios such as this are likely to be more common in future and can be termed as internal cloud bursts. As enterprises adopt cloud technologies organizations are going to witness internal cloud burst more frequently than external cloud burst where applications move to an external cloud provider’s environment such as Amazon, Mosso or Rackspace when faced with scarcity of resources. Shifting Gears: ISV Cloud On-boarding 2009-12-02T05:29:02Z Last couple of months; I had several exciting opportunities to interact with large ISVs in formulating the business strategies, technology strategies & GTM strategies to align and adopt cloud computing trends. I noticed that the ISVs are shifting the gears: the drivers are many, major one being the recent economic slowdown puts lot of growth & price pressures for ISVs as enterprises are conservative on IT spend. Also, the recent technology innovation trend – cloud computing – brings value in terms of cost efficiency and business agility.]]> Cloud On-boarding Roadmap The large ISVs are changing the business strategies and business models – expanding the offerings from large enterprise customers to SMB & retail customers to drive growth, increase market penetration and offer competitive & cost-effective IT solutions out-of-the-box (managed). They are considering Cloud & SaaS technology strategies in realizing these business goals and analyzing the impact on the existing product & service lines to size the cloud adoption effort. They are also looking for innovative platform accelerators that can help reduce the time-to-market for cloud / SaaS on-boarding. In some of the more strategic discussions, I also noticed that the ISVs are asking for sharing the risk of cloud adoption – it is somewhat new venture from many of them from both business & technology perspective. By the way it was not new for me; I have seen the latest business trends around ISV sector, wherein the innovation in product engineering is accelerated through trusted partnerships, alliances and joint go-to-market solutions with large system integrators. Industry Trends Some of the large ISVs are leading this change and driving the industry trends for e.g. Microsoft, SalesForce.com etc, many of them are still catching-up some-of-them in more strategic-way & some-of-them in more tactical-way forced by the industry. I also feel that there is a sense of fear – the cloud computing infrastructure-as-a-service wave had a significant impact on hosting providers, anyone who has not aligned it with this trend, is almost going to go out-of-business very soon. Similarly, the ISVs are the next players who will be hit by cloud computing platform-as-a-service and software-as-a-service waves. Of course, the system integrators are the next in terms of changes in the outsourcing models largely driven by ADM solutions & services to managed solutions & services based outsourcing, the recent leading analyst firm reports that 8% of the outsourcing models will run on SaaS based models in next 1 year. From business ecosystem perspective, ISVs On-boarding on Cloud creates a new, sustainable and economically scalable business environment for all stakeholders – customers, ISVs and system integrators. ISVs achieve growth, Customers get more cost effective solution options and System Integrators find new revenue channels. The economy, customers & technology is driving the change: the hosting providers, ISVs and Outsourcing providers have to shift the gears to survive & grow! Dallas – Information as a Service 2009-11-30T10:24:02Z We are witnessing information explosion over the internet, tons of information is getting accumulated in. However, we still struggle to get the “Accurate and Authentic Data”. Have you ever needed the zip code of a city, route to reach a place, dining menu of restaurants, weather forecast and history, crime rates in a specific area of city? This list just goes on. How do we get this data? Search this information on our favorite search engines and rest in peace when we find it!! But, do we really know whether the data which we got is really accurate?! It could be stale, misleading or just plain inaccurate!! Why can’t you get information as easily as you can get a size 40 Creamy white Louis Philippe shirt or maybe a striking green 8 GB IPOD shuffle; because INFORMATION is not commodity yet!! Continue reading "Dallas – Information as a Service" » MYOC - Telephony with Twilio to Vote 2009-11-20T13:23:05Z MYOC (Make Your Opinion Count) – an online poll application hosted on Microsoft Azure, uses Twilio to make it easier for people to participate in the online polls. Twilio is telephony in the cloud which exposes RESTful APIs to build scalable voice applications. It supports both inbound and outbound telephony calls. Pricing is developer friendly with pay-as-you-go model. MYOC uses Twilio in two ways – 1. Poll creator can place a call for participant to caste his/her vote 2. A participant can dial-in for a particular poll to caste his/her vote Let’s see what all it takes to use twilio in MYOC to call a participant and accept his/her vote or to handle an incoming call to cast a vote. Continue reading "MYOC - Telephony with Twilio to Vote" » Infosys on-boarding ISV's on the cloud - 2 2009-11-20T11:29:30Z In this blog, I will continue the discussion on a possible migration scenario of migrating existing ISV application to Azure. If you haven’t read my previous blog on this topic, then you can go here... Continue reading "Infosys on-boarding ISV's on the cloud - 2" » Points to notice while migrating data from traditional database to Azure SQL 2009-11-20T05:37:18Z Generally when it comes to migrating heavy data from one SQL server to another, among others following approaches are widely used: 1. Using SSIS package 2. And using BCP. One needs to keep in mind a few points for the fruitful migration without any anomaly. For one of our project we used SSIS package for migrating data but hit upon a few glitches in the course of our task. Find more info here

Cloud Computing

Information on different Key Value stores

Accessing CouchDB for Data Storage and Retrieval

Accessing Hadoop DFS for Data Storage and Retrieval

SaaS makes financial accounting easier

Key Value stores: Usefulness in Cloud environment

Security Issues with the Cloud

MYOC - Offload compute intensive tasks on Azure using the Offline Processing pattern

Hybrid Clouds: Business Imperatives

Cloud Bursts: Reality versus Hype!!I

Shifting Gears: ISV Cloud On-boarding

Industry Trends

Dallas – Information as a Service

MYOC - Telephony with Twilio to Vote

Infosys on-boarding ISV's on the cloud - 2

Points to notice while migrating data from traditional database to Azure SQL