Hello Cloud Enthusiasts! Today I will be showing you quick ways about how you can set up LMPs (Lifecycle Management Policies) for your data stored in Azure Blob Storage to save costs.

What are LMPs?

There could be scenarios where you are required to store a huge amount of data in an Azure Storage Account as Blobs. During the initial days, the data is accessed frequently by multiple systems. But as the data ages, its usage frequency decreases exponentially. This could pose a serious costing issue as you are paying for the data to be available immediately but nobody is using it and has no plan to use it in the new future. And when asked by the stakeholders regarding huge storage costs, all you can say is, you were trying to keep the data immediately available.

To help you with these situations, Azure has introduced LMPs, that have been available for storage options like S3 in AWS for quite some time. With the help of LMPs, you can –

• Transition blobs from cool to hot immediately when they are accessed, to optimize for performance.
• Transition blobs, blob versions, and blob snapshots to a cooler storage tier if these objects have not been accessed or modified for a period of time, to optimize for cost. In this scenario, the lifecycle management policy can move objects from hot to cool, from hot to archive, or from cool to archive.
• Delete blobs, blob versions, and blob snapshots at the end of their lifecycles.
• Define rules to be run once per day at the storage account level.
• Apply rules to containers or to a subset of blobs, using name prefixes or blob index tags as filters.

So basically, LMPs act as rule-based policies which can help you to move your data across different tiers based on their usage and help you in saving costs.

Data Storing Costs => 

Hot Tier > Cool Tier > Archive Tier

While Data retrieval costs => 

Hot Tier < Cool Tier < Archive Tier

So what this means is, you are good to store the data which is frequently accessed in Hot Tier, while it is beneficial to move the data to Archive tier if it is not gonna be used for at least 180 days, or 30 days for Cool tier.

What does an LMP look like?

Lifecycle Management Policy is a JSON based Rule ( also available in UI). To create one, you need to go to your Storage Account ( v2, premium, and Blob storage type supported only). On the left pane, under Data Management, there is an option called Lifecycle Management.

There you can create a new rule either using UI or in the code-view. The Basic LMP definition looks like this-

  "rules":[
    {
      "name": "rule1",
      "enabled": true,
      "type": "Lifecycle",
      "definition": {...}
    },
    {
      "name": "rule2",
      "type": "Lifecycle",
      "definition": {...}
    }
  ]

Rules- An Array of rules. You can define up to 100 rules in a policy.

Definition of a rule could look like this –

    "definition": {
            "actions": {
              "version": {
                "delete": {
                  "daysAfterCreationGreaterThan": 90
                }
              },
              "baseBlob": {
                "tierToCool": {
                  "daysAfterModificationGreaterThan": 30
                },
                "tierToArchive": {
                  "daysAfterModificationGreaterThan": 90
                },
                "delete": {
                  "daysAfterModificationGreaterThan": 2555
                }
              }
            },
            "filters": {
              "blobTypes": [
                "blockBlob"
              ],
              "prefixMatch": [
                "sample-container/blob1"
              ]
            }
          }

This rule basically utilizes LastModified property. And based on this property, it moves the data to Cool tier after 30 days, to Archive tier after 90 days, and deletes the blob after 2555 days ( 7 years). It also applies filters on the Blobs on which this policy is applied. Like in this example, it is looking for Block blobs only within ‘sample-container’ named Container and all those blobs which start with ‘blob1’.

You can also use LastAccessedTime property which has been made generally available by Microsoft last week. More on that here.

Pricing for LMPs

I know you are most interested in this part but to your surprise-

Lifecycle management policies are free of charge. Customers are billed for standard operation costs for the Set Blob Tier API calls. Delete operations are free. Each update to a blob’s last access time is billed under the other operations category. For more info on this, look here.

Note: All policies run once a day. So if you are creating a new policy, it may take 24 hours to run and perform the actions that you have mentioned in the definition.

So that’s it! Hope it was helpful.

• Azure
• CostOptimisation
• LifecycleManagement
• StorageAccount