Your company (or the company you work for) certainly has data available. For example:
- Sales amount
- Fixed and variable costs
- Customer data (name, contact, products purchased, purchase date, etc.).
What can we do with this data?
We can use BI (Business Intelligence) techniques to extract relevant information and answer important questions such as:
- Is the company financially healthy?
- Is the price of the products within the ideal range?
- Will customer X benefit from any product other than what he currently consumes?
- What is the sales trend for the coming years or months?
Among many other questions ...
The secret to extracting relevant information is asking the right questions.
Once you have defined what you want to know and with the data in hands, we start the Data Engineering processes.
Data Engineering is defined (in a very simplistic way) in turning "raw data" into information that is useful.
For that it is necessary to know how to Extract, Transform and last, but not least, Load this data in safe and easily accessible places. These procedures are known by the acronym ETL.
Before starting the data extraction we have to identify whether the database is Relational or Non-relational.
Relational Database
Briefly, the relational database stores the data in tables. There is an intuitive relationship between the data in the table and that between the tables themselves.
Let's look at an example:
In the example above, we see a table containing, on each line, information about one particular customer. This format allows queries that return answers to simpler questions such as:
Which customers have a Silver subscription? Which customer is located in Los Angeles? etc.
This type of database is also known as SQL, which stands for Structured Query Language. SQL is a language with which relational databases are programmed.
With specific commands it is possible to make queries, delete, update, edit and add new data.
Seems practical doesn't it?
But let's put this relational database to the test of day-to-day activities ...
What if we wanted to add a second phone number for a specific customer? For example, a cell phone number? We will also add the position of the people that we know what position they hold and the number of employees in each company (from which we have this information).
Notice that we don't know all cell phone number or their position and the number of employees from all companies served.
This quickly causes the database become "bloated", with several empty spaces in different tables and makes the processes of query, extraction etc. extremely complex.😓
But calm down! Not everything is lost!
Let's take a look at the Non-Relational Database ...
The Non Relational Database, stores data in an optimized way, not only im simples tables.
Also known as NoSQL, which stands for "Not Only SQL", it allows for slightly different queries.
There are four types of non-relational databases:
1. Document-oriented databases - also known as document storage, this database is designed to store, retrieve and manage document-oriented information. Document databases often combine each key with a complex data structure (called a document).
2. Key-value - This is a database that uses different keys, where each key is associated with only one value in a collection. Think of it as a dictionary. This is one of the simplest database types among NoSQL databases.
3. Wide column - this database uses tables, rows and columns, but unlike a relational database, the names and format of the columns can vary from line to line in the same table.
4. Charts - a chart database uses chart structures for semantic queries with "nodes", "borders" and "properties" to represent and store data.
"Okay, so what? How are we going to solve the problem of our bloated and inefficient database?"
One option is the document-oriented database. So a customer record can have how many phone number we want in a document with its particular features. Storage is optimized for every need. 😀
Cool huh?
