DST Lezione 21
Source:
- Fundamentals of Database Systems Seventh Edition | Chapter 24.3 See also:
- Lecture 19 on Key Value Store
NoSQL system
Document-based NoSQL systems, like MongoDB, typically store data as collections of similar documents, usually in JSON format. They’re optimized for JSON data.
However, there isn’t a database optimized for very large binary files. Big files are split into different blocks because of their huge size and footprint, making them heavy to retrieve and save. There’s not much that can be done to improve handling very large files.
MongoDB
MongoDB stores documents as BSON (Binary Json) format. The binary json format is a varion of JSON with some additional data types and is more efficient for storage than JSON. Json can also be nested, you could have a json document inside a json document. Each document is stored in a collection, which has a name (key) and an optional document (value).
MongoDB uses MQL (Mongo Query Language), which is a proprietary query language. Developers also claims that it supports ACID transaction. MQL offers features like restriction, projections, aggregations and even a kind of join.
Schemaless: A collection does not have a schema. The structure of data fields in documents is based on how the documents will be accessed and used. Users can choose a normalized design (like relational tuples) or a denormalized design (like XML documents or complex objects).
As MongoDB has developed, it has incorporated features from SQL, such as primary and secondary indexing, joins, and ACID transactions. However, adding these SQL-like functionalities can slow down the system. If you need frequent joins, it’s better to use another system, as they are in MongoDB just for special cases.
When to use MongoDB
- What is the advantage? It is convenient when you don’t have a fixed structure, when you don’t want to use a schema. It’s easier to insert values but you could have two tuples different from each other.
Example case use for MongoDB
Let’s do some practical example. Where do you get your data from? If it’s from automatic systems like sensors or cameras, the data has a consistent structure, so a relational DBMS makes sense.
But if you get data from APIs, like those from Facebook or X, the provider can change the structure without notifying you. If you can’t guarantee a fixed structure, using MongoDB makes sense.
Another consideration is the type of queries that are executed often. If aggregation and projection are enough, MongoDB is gucci. But for complex queries like nested queries and multiple joins, an SQL system is better.
JSON Schema
Sometime you could solve the variable structure problem with a JSON schema. It’s a JSON file that defines the structure your document should follow.
A question that could arise is why using a JSON schema when there is relational DBMS? People are lazy and like simpler things. From a business standpoint, MongoDB saves money because it’s simpler, unlike SQL which requires expertise. MongoDB doesn’t have the administrative burden of relational systems, making it easier and cheaper.
You can even have up to fifty copies geographically replicated with incredible throughput. The performance and quick reads it offers would cost a lot more with relational systems. So, these are the strengths of MongoDB.
Introduction to MongoDB query language
Source:
You do first the restriction with match and group for grouping by id.
The operations in MQL are:
- match is a restriction
- group is an aggregation
- projection is projection
- sort arranges things in order
- limit puts a cap on how much you see, like just showing 10 results.
- lookup: it’s the SQL join equivalent
- set is the update equivalent, unset is the opposite of set
- inc: self incrementing fields like artificial key in oracle, like add 1 or reduce 1
- push adds new elements or modifies existing ones.
- update: is similar to “Alter Table” in SQL.
- pull: removes elements from a field.
- bit: is for logical operations
In-Memory Database
An In-memory database is a non persistent database that stores everything in main memory. It is kind ironic that the first requirement to a databe is persistance. In fact the term database is misused, they are Data Structure Servers.
If everything is in memory, means that we have two order of magnitude speed up with respect to other databases. As we saw from previouis lessons, most of the cost comes from the I/O operations on the disk. With an in memory database you get a 10 to 100
You can flash the content of your main memory into disk and that gives some sort of persistancy, however you get less speedup.
Redis
Redis is probably the most famous data structure server. Data structures of Redis are based on hashing. Redis haw two persistence mechanisms: snapshot-based and log-based.
With snapshots, you flash everything from memory to disk every few minutes (default 5 minutes). You keep most of your data safe but you might lose up to 5 minutes of recent data. Flashing on the disk too often can slow down the system.
With log, you store in an heap file the list of operation. Same considerations of snapshots holds for log and discussed in previous lectures.
You could use either one or both at the same time. When asking what kind of mechanism and how to configure it you need to ask “how much data can you afford to lose?“. Writing too often doesn’t make it worth to use such system.
Redis data structures
For a complete list see documentation. We have basic key-value hases.
The query language only supports get and requires that you have a key. There are not fancy operations.
Some of the most useful data structure are:
- Lists: linked list of string values;
- Sets: unordered collection of unique strings;
- ZSet: “A Redis sorted set is a collection of unique strings (members) ordered by an associated score.”;
- Hashes: redis also supports dictionary
On the "Sorted Set"
A sorted set, it’s a kind of contradiction because in mathematics a set is not sorted. ZSet is very useful in use cases like real-time analytics, for example in an hackaton where the leaderboard is dynamic since solutions are continuely submitted. Another use case is like: when you are selling tickets, the associated score for a ticket id could be the timestamps. So using a ZSet could allow you to get i.e the first 100 sold tickets.
Redis Use Case
It is used as cache for hot data, lowering the latency. It is also useful when there is something that is hard to manage and handle in a programming language. (Like i don’t know, you want for some reason a cache with two separate process that access to the same data and you don’t want to manage in a programming language this stuff, it’s hard)
Losing data due to RAM failure is a worry on a home server, but in the cloud, there’s less risk. Cloud services have backup procedures and redundancy built-in, so if something goes wrong, it’s handled behind the scenes without affecting the user.
How to try redis + some hand on stuff
- try.redis.io (Note i only noted some features, see Redis docs for a complete reference)
Expire
- Expire: it is a feature that set a TTL (time to live) for data, like expire “my:resource” 120 means it will be deleted after 2 minutes
- For lists you have RPUSH/LPUSH for inserting on left or right items, and RPOP/LPOP for deleting.
Zset About the ZSET, in the TUTORIAL of try.redis.io:
ZADD hackers 1940 "Alan Kay"
Hashes Hashes are the analagous of a record and so on. with the difference that you have no schema.
HSET user:1000 name "John Smith"
HSET user:1000 mail "john.smit@example.com"
The “user:1000” is the primary key. This is another way to design the primary key, we saw in a previous lecture the fingerprinting with italian codice fiscale.