The data retention section allows you manage how long your default high resolution data is retained for a given measurement type. It also allows you to add rules that will cause the high resolution data to be aggregated on some defined interval as well as defining when this aggregate data should expire.
The first thing you will need to do is select the measurement type whose retention and aggregation policies you wish to manage. You can do so by selecting the relevant measurement type from the selector in the top right corner of the page.
Each measurement type will have a default rule automatically added upon creation of the measurement type. The default rule applies to the high resolution data associated with the measurment type and lets the system know how long it should retain the data. The default time frame high resolution data is retained in the system is 7300 days or roughly 20 years. The only field that can be edited is the Max Age field. This rule can not be deleted.
With the exception of the default data retention rule and the interval field, which can not be edited, each data retention has the following fields when adding and editing a rule.
-
Name - The unique name that will be associated with the retention rule.
-
Max Age - The amount of time, in days, that the applicable data will be retained in the system before it is purged.
-
Interval - The size of the buckets, in minutes, that the data should be aggregated together within. For example, choosing an interval of 60 would cause the rule to aggregate your data into hours choosing the appropriate min, average, max, and creating a histogram representing the appropriate distribution of data points with the specified resolution.
-
Aggregation Histogram Configuration - This section allows configuring Histogram Resolution and Histogram Min Width for each measurement type value. Histogram Resolution describes the amount of data, as a percentage, that will be stored in a single histogram bucket. For example, choosing 0.1 for this field would result in 1,000 data buckets each storing 0.1% percent of the data. This rule applied to a 100 gigabit capable interface would allow for percentile calculations that were accurate within approximately 100 megabits. Histogram Min Width is the bucket width used in the histogram when its unable to determine a width to use based upon the resolution and min/max value defined in the measurement. This is especially applicable to the VLAN interface data whose max value is determined by stored data. For example, the bucket size of a VLAN interface data could be overly small (e.g., 10) if the max value is very low, which results in wasteful storage. The Histogram Min Width will be used instead if the calculated bucket size is smaller.
-
It's very important to understand the nature of your data since Aggregation Histogram Configuration has huge implication on storage size requirement. For example it's not sensible to set the Histogram Min Width to be 10 for any interface types
-
Meta Fields - Leaving this blank will cause the rule to be applied to all data within the measurement type. Choosing fields here will cause the rule to only be applied to data whose meta fields match ALL of the rows.
To add a new retention Policy click the "Add Retention Policy" link located on the upper right side of the table. A panel will be displayed with a form containing the data retention fields. Once, the form is complete click the "Done" button to add the retention Policy.
To edit an existing data retention Policy click the icon located at the far right of the row you wish to edit. A menu will be displayed. Click the "Edit" option to open the edit panel. Modify the fields you wish to change, then click the "Done" button to apply the changes.
To delete an existing data retention Policy click the icon located at the far right of the row you wish to delete. A menu will be displayed. Click the "Delete" option to remove the rule.
