Configuring data retention settings

Flow Monitor is designed to serve two primary purposes:

To give a minute-by-minute view of recent network traffic.
To give an overview of historical network traffic.

To accomplish these goals while keeping the size of its database reasonable, Flow Monitor uses a process of summarizing data at certain time intervals.

By default, Flow Monitor rolls up data on this schedule:

Complete raw data (which is collected every other minute and provides the detailed view of recent traffic) is kept for 4 hours.
After 4 hours, raw data is summarized into hourly averages.
After 1 days, hourly averages are summarized into daily averages.
After 3 days, daily data is archived.
After 7 days, archive data is purged from the archive database.

To configure the data collection interval:

Navigate to the Flow Monitor Settings dialog (Flow Monitor > Settings).
In the Data collection interval box, select how often you want Flow Monitor to write raw data from its sources to the Flow Monitor Active database. You may select 1, 2, 3, 4, 5, or 10 minutes. By default, raw data is written to the database every 2 minutes.
Click OK. The setting is saved and the Flow Monitor Settings dialog closes.

To configure address resolution intervals:

Navigate to the Flow Monitor Settings dialog (Flow Monitor > Settings).
In the Resolve private address interval box, enter the interval (in hours) that you want Flow Monitor to wait before it checks a private IP address to resolve information that may have changed for the address since the last private address lookup. By default, private addresses are resolved every 48 hours.
In the Resolve public address interval box, enter the interval (in hours) that you want Flow Monitor to wait before it checks a public IP address to resolve information that may have changed for the address since the last public address lookup. By default, public addresses are resolved every 720 hours.
Click OK. The setting is saved and the Flow Monitor Settings dialog closes.

To configure unclassified traffic collection:

Navigate to the Flow Monitor Settings dialog (Flow Monitor > Settings).
In the Expire unclassified traffic after box, enter the number of hours after which Flow Monitor should purge unclassified traffic. Unclassified traffic is traffic transmitted over ports that are currently not monitored by Flow Monitor. By default, this option is set to 0 (zero), which causes Flow Monitor to aggregate and retain data for all unclassified ports as a single value; detailed information about the individual unclassified ports over which traffic was transmitted is immediately discarded.
Click OK. The setting is saved and the Flow Monitor Settings dialog closes.

To configure flow data retention:

Navigate to the Flow Monitor Settings dialog (Flow Monitor > Settings).
If you want to allow Flow Monitor to automatically manage your data retention settings, select Auto tune flow data retention to automatically tune the flow data cleanup settings to manage database size and system performance. This option is selected by default. For more information on tuning flow data retention, see Data retention strategy and tuning.
If you want to manually manage your data retention settings, clear Auto tune flow data retention and set the following settings:
Note: When you manually adjust the Data Retention settings (Auto tune flow data retention option is cleared), you are presented with guidance in the message area at the bottom of the dialog as you adjust each setting. This feedback provides you with information about how the current, or proposed setting will affect the database size with respect to the maximum recommended database size (10 million records). For the raw data, hourly data, and daily data, the maximum recommended database size is compared against all of the data in these categories and is based on the size of the Flow Monitor Active database. For the Archive daily data after setting, the guidance is based on the size of the Flow Monitor Archive database.
- Percentage of traffic to retain. Use this option to determine the amount of the total hourly data you want to maintain during the hourly roll-up. This option is enabled when you clear the Auto tune flow data retention check box.
  Caution: While the default settings for data cleanup are conservative, when you modify the roll-up settings it can directly affect the size of the Flow Monitor databases and the performance of the application. We recommend that you modify these settings cautiously, and monitor the effects of changes to these settings on database size and application performance.
- Retain raw data for. Enter the number of hours of raw flow data you would like to maintain. This setting establishes a sliding time window of raw data that spans the specified period. Raw data that reaches the end of the period is rolled up. The roll up of raw data happens every hour on the hour. After data has been rolled up, Flow Monitor can only report using the hourly summations. By default, raw data is rolled up after 4 hours.
- Retain hourly data for. Enter the number of days you would like to maintain hourly data. This setting establishes a sliding time window of hourly data that spans the specified number of days. As hourly data ages beyond this period it is rolled up. The roll up of hourly data takes place daily. After hourly data is rolled up, Flow Monitor can only report aggregated totals for the entire 24-hour block of time. By default, hourly data is maintained for 1 day.
- Retain daily data for. Enter the number of days of daily data you would like to maintain before archiving. This setting establishes a sliding time window of daily data that spans the specified number of days. As daily data ages beyond this period, it is archived. Flow Monitor continues to have visibility into archived data with some restrictions. By default, daily data is archived after 3 days.
- Retain archive data for. Enter the number of days of daily data you would like to maintain in the archive database. This setting establishes a sliding time window of archived daily data that spans the specified number of days. As the archived daily data ages beyond this period it is purged from the database. After archived data is purged, Flow Monitor can no longer report on the data. By default, archive data is purged from the database after 7 days.
Click OK. The setting is saved and the Flow Monitor Settings dialog closes.

To configure interface data retention:

Navigate to the Flow Monitor Settings dialog (Flow Monitor > Settings).
Set the following settings to tune your interface data retention:
Note: Raw interface data is used to represent total interface traffic for the period and to calculate 95th percentile values for the Interface Overview and Interface Usage reports. Because of data compaction, interface data has a smaller impact on data storage, so it can be maintained for longer periods of time.
- Retain raw data for. Enter the number of days of raw interface data you would like to maintain. This setting establishes a sliding time window of raw interface data that spans the specified number of days. As raw interface data ages beyond this point it is rolled up. After data has been rolled up, Flow Monitor can only report using the summations produced in the roll-up process. By default, raw interface data is rolled up after 8 days.
  Caution: While the default settings for data cleanup are conservative, when you modify the roll-up settings it can directly affect the size of the Flow Monitor databases and the performance of the application. We recommend that you modify these settings cautiously, and monitor the effects of changes to these settings on database size and application performance.
  
  Important: If 95th percentile values are going to be used for billing purposes, you should maintain a set of raw interface data that matches the billing period to ensure accurate results. To gather the data needed to calculate the 95th percentile values for the interface, set the Roll up raw data after setting for Interface Data to match or exceed the billing period.
- Retain hourly data for. Enter the number of days you would like to maintain hourly interface data. This setting establishes a sliding time window of hourly interface data that spans the specified number of days. As hourly data ages beyond this period it is rolled up. The roll up of hourly interface data takes place daily. After hourly interface data is rolled up, Flow Monitor can only report aggregated totals for the entire 24-hour block of time. By default, hourly interface data is maintained for 35 days.
- Retain daily data for. Enter the number of days of daily interface data you would like to maintain before archiving. This setting establishes a sliding time window of daily interface data that spans the specified number of days. As daily interface data ages beyond this period, it is archived. Flow Monitor continues to have visibility into archived interface data. By default, daily interface data is archived after 180 days.
- Retain archive data for. Enter the number of days of daily interface data you would like to maintain in the archive database. This setting establishes a sliding time window of archived daily interface data that spans the specified number of days. As the archived daily interface data ages beyond this period it is purged from the database. After archived interface data is purged, Flow Monitor can no longer report on the data. By default, archive interface data is purged from the database after 365 days.
  Important: Any changes made to data roll up intervals are not enforced until the Flow Monitor collector service is restarted. For more information, see Stopping or restarting the collector.