Obsidian + 🪴 Quartz 4.0

DST Lezione 23

6 min read

DST Lezione 23

Wide Column Store and Cassandra

Source:

Cassandra

  • The store is not column wise
  • Cassandra is the most famous wide column store - Open Source project by Meta

Dictionary

  • Rows same as rows in relational systems, but they’re key value ofcourse;
  • Tables: are called wide columns, they are free of structure like mongodb documents;
  • Schema: in Cassandra is called key spaces, it’s a collection of wide columns (like schema are collections of table).

In Cassandra, a keyspace functions like a database, containing tables. Within a keyspace, you can set the consistency level and specify the number of data replicas. Replicas are manages like in DynamoDB with consistent hashing.

Gossip Protocol

One difference from DynamoDB is the GOSSIP protocol, used to manage failures. This protocol allows nodes to exchange messages to check on things, like whether nodes are up and running. Node failure is common, so how do you know which nodes are active at any moment without messaging all of them? GOSSIP protocol handles that.

Ofcourse, you can be certain if all nodes are active by connecting every node and wait until everyone answered. But this is an huge overhead

The Gossip protocol works in this way: you set a parameter called “Fan-out,” which is the number of random neighbors each node will message. For example, if Fan-out is 3, a random node sends messages to three other random nodes. Those three nodes then each send messages to three more nodes, and so on. It spreads like a virus: one infects three, those three infect three more, and so forth.

There will be duplicate messages and some wasted operations since a node might receive multiple messages, but the protocol doesn’t require any checking.

To figure out how many generations it takes to reach all nodes, use , where is the total number of nodes and is the Fan-out value. This gives you the height of the tree and is in the order of .

Addictionally, each node can have a role to confirm activity. For instance, if a node receives responses from at least two other nodes, it is considered active.This is a quick way to check for node failures compared to waiting for messages from all nodes.

Like Kafka, this system can be push-based, pull-based, or a mix of both.

You can have a node that sends messages (push), a node that check if the message was received (pull), or both (push-pull).

Nodes update each other about their availability roughly every second. There are many variations, but the main idea is to have a visiting tree with random choices at each generation. This helps enforce consistency, and you can choose how strict or permissive you want it to be.

While you can set the consistency level, it’s designed to be eventually consistent. If you need strict consistency for every copy, the system will be slow, and you might need to look for another system.

Cassandra Query Language

Cassandra mimics SQL with a language called CQL (Cassandra Query Language). It’s not a continuous query language but rather similar to SQL. However, there are some key differences:

  • No joins or nested queries.
  • Only count operations; no average, maximum, or minimum functions.
  • You have to take in mind that there is no schema so two rows could have different attributes.

GraphDB

  • Is a new technology and they are specifically useful to handle graph data.

The data is represented as a graph, which is a collection of vertices (nodes) and edges. Both nodes and edges can be labeled to indicate the types of entities and relationships they represent, and it is generally possible to store data associated with both individual nodes and individual edges

Let’s say you have a node, in an entity relational graph.

Could be represented as “graph” like this:

Why Use GraphDB Instead of Relational Systems?

Graph databases are optimized for managing graph data, unlike relational systems, which struggle with path algorithms like shortest path and graph diameter. If your queries involve complex graph traversals such as shortest path or minimum spanning tree, a relational database may be inadequate, especially for large graphs that don’t fit in memory.

GraphDB use cases

GraphDB is ideal for data involving relationships and connections, such as social networks, where you need to analyze data from connected nodes (e.g., friends of a user).

Neo4J

  • Docs: https://neo4j.com/docs/getting-started/cypher-intro/ Neo4j is a prominent graph database that uses a proprietary declarative language similar to SQL called cypher. It has built-in graph algorithms and employs an inverted indexing system at the physical level. The logical model used is the property graph model, where nodes and relationships can have attributes, which are flexible and schema-less.

  • At physical level, is all based on inverted indexing (backlink).

  • At logical level we use property graph model*. For example you have nodes and relations that both can have attribute and these are not simple value but are free records with no schema.

Neo4j is sometimes used on GPU because you can do parallel operations. It’s useful for doing analytics on highly connected data.

Future development of databases

There are two main development directions:

  • Polyglot Systems: No single system can understand all types of data, so multiple systems are used together.
  • Hybridization of SQL and NoSQL: Combining features of both, like MongoDB does.

As recap read: https://www.scylladb.com/learn/nosql/ in particular use cases: when to use a Nosql system and NoSQl vs SQL Examples and Use Cases. Combination of SQL and NoSQL databases is interesting.

The hard thing with these system is understanding when to use one.

Graph View

Backlinks