Open Source has revolutionized IT sector as it harnesses the intelligence and efforts of the community to develop and maintain software faster. Our experts discuss the latest happenings and give their view points on how open source can be leveraged for your organization’s transformation.

Main | March 2018 »

October 30, 2017

Query-driven data modeling methodology for Apache Cassandra

This blog explains how to use query-driven data modeling in Apache Cassandra. NoSQL data modeling is a process that identifies entities as well as the relationships between them. It can be used to determine patterns when accessing data as well as the types of queries to be performed. In doing so, it reveals how data is organized and structured along with how database tables are designed and created. It is important to note that indexing the data can degrade the performance of queries. Hence, understanding indexing is essential in the data modeling process.

Data modeling in Cassandra focuses on the query-driven approach whereby specific queries are the key to organizing data. Let me first quickly explain these terms: Queries retrieve data from tables and schema defines how the data is arranged in the table. Thus, a query-driven database design facilitates faster reading and writing of data, i.e., the better the model design, the more rapid data is written and read.

Now, first, we must create a conceptual data model that will define all known entities, relationships, attribute types, keys, cardinality, and other constraints. This data model should be created in collaboration with business stakeholders and analysts. For example, a conceptual data model could be presented as ER-diagram.     

The next step is logical data modeling. Here, the conceptual data model is mapped to a logical data model based on queries that are defined in an application workflow. The logical data model corresponds to a keyspace where table schemas define columns as well as primary, partition and clustering keys. Thus, the query-driven approach provides a logical data model using data modeling principles, mapping rules, and mapping patterns.

Here are some rules for query predicates that ensure stability and efficiency:

    Only primary key columns should be used in the query predicate

    All partition key columns in the query predicate must have distinct values

    Clustering columns may be omitted in the query predicate

    All partition key(s) must be used in the predicate

Besides these query predicate rules, there are additional data modeling principles to map to logical data models. It is important to note that violating these principles and rules will affect the ability to support query requirements and may lead to loss of data and performance degradation.

Here are the fundamental principals of logical data modeling:

1.    Know your data, particularly entity and relationship type keys that are needed to be preserved and relied on to organize the data properly

2.    Know your queries such that all columns are preserved at the logical level

3.    Enable data nesting to merge multiple entities together based on a known criterion

4.    Minimize data duplication to ensure space and time efficiency

5.    Use equality search attributes to map to the prefix columns of the primary key

6.    Use inequality search attributes to map to the table clustering key column

7.    Use ordering attributes to map to clustering key columns with ascending or descending clustering order

8.    Use key attribute types to map to primary key columns and uniquely identify table rows

Finally, we must analyze and optimize this logical data model to create the physical data model. The above-mentioned modeling principles, mapping rules, and mapping patterns ensure correct and efficient logical schema. However, efficiency can still be impacted by database engine constraints or finite cluster resources such as typical table partition sizes and data duplication factors. There are some standard optimization techniques that can be used, including partition splitting, inverted indexes, data aggregation, and concurrent data access optimization. These methods can be used for optimizing the physical data model, although we will not be covering this topic in detail in this particular blog entry.

This is the way to go about enabling query-driven data modeling in Apache Cassandra.

Subscribe to this blog's feed

Follow us on

Blogger Profiles

Infosys on Twitter