Tiered Storage Setups - Hot and Cold Data
You can manage your data to ensure cost efficiency and Return on Investment (ROI).
The life cycle of your data starts from the running application (hot) and ends with the archiving of the data (cold). The temperature of your data is defined by its proximity with the running application and its storage type.
3DS OUTSCALE gives you the ability to manipulate different types of storage with diverse performance properties, each having a specific purpose set out below.
Overview
All in all, data management is a balance between Finances and Operations.
Storing the right type of data on the right type of storage is essential to your business and can help you achieve, for example, faster software solutions or better data-driven decisions.
To get started, try to determine the 20% of data that is most important for your business.
You should aim to distribute your data temperature according to the diagram below:
Background
Nowadays, most companies have to deal with the rise of large data collections, a phenomenon that proves to be challenging.
This leads us to potential issues that should be addressed appropriately, such as:
-
Handling growth
-
Filtering out irrelevant data
-
Managing costs by storing data in the right place
Generally, companies possess more cold data than hot data, and fail to optimize the different storage types according to data temperature.
Storage Types
The following storage types are available within the OUTSCALE Cloud:
-
BSU (Block Storage Unit) is a block storage solution, that is, a virtual hard drive disk you can attach to your virtual machines (VMs). You can also create snapshots from BSU volumes, which can help manage data. For more information, see About Volumes.
-
OOS (OUTSCALE Object Storage) is an autonomous service, not related to your VMs, and where data can be accessed via HTTP. For more information, see About OOS.
Data Journey
Depending on your current needs and usage, your data can go through temperature changes that determine which storage type is appropriate.
Daytime Data
You start the business day with your application installed on a VM. You also store your data and operating system on different disk partitions.
You create a 100 GiB BSU volume, attach it to your VM, and store your database on it. Because the database needs a quick access to the disk, the recommended storage type here is a classic disk mounted on your VM. That way, your data is "hot", in live production, close to your application and instantly available.
|
The integrity of such data is not covered by service level agreements (SLAs). You cannot roll back in case an issue occurs on the disk. |
End-of-day Data
At the end of the business day, you want to store the current state of your data and create a backup of it. You do not want this data available for your application, but you want it available in case you need to recover the disk.
To do so, you can create a snapshot of the current volume. Data stored on the snapshot is colder than the data running on the attached volume: you cannot access it directly as a volume disk for your application.
A snapshot only takes a few seconds to create and has a 99.9999% integrity guarantee. You can set up a rolling backup strategy where snapshots older than seven days are automatically deleted and replaced with the current-day snapshot.
At that point, BSU and snapshots are on the same IaaS platform as your business resources. The data is available in just a few seconds.
Night-time Data
Your rolling backup strategy is set up, and you are about to delete the seven-day-old snapshot. However, you are legally obligated to keep this data for one year.
This data is cold: it currently has no value for your business and might never go into live production. You do not need it to be available instantly, but you might need to access it eventually.
Here, the recommended and cheaper solution is to archive the data via OOS. Because it is network-based, storing the data is slower than with BSU, while still ensuring reliability.
Related Pages