Configuring data roll-up intervals

Flow Monitor is designed to serve two primary purposes:

• To give a minute-by-minute view of recent network traffic.
• To give an overview of historical network traffic.

To accomplish these goals while keeping the size of its database reasonable, Flow Monitor uses a process of summarizing data at certain time intervals.

By default, Flow Monitor rolls up data on this schedule:

• Complete raw data (which is collected every other minute and provides the detailed view of recent traffic) is kept for 4 hours.
• After 4 hours, raw data is summarized into hourly averages.
• After 1 days, hourly averages are summarized into daily averages.
• After 3 days, daily data is archived.
• After 7 days, archive data is purged from the archive database.

You can configure the intervals to roll up data more or less frequently depending on your network's size and traffic volume.

To set data roll up time intervals:

1. From any workspace view or report in the web interface, select GO. The GO menu appears.
2. If the Flow Monitor section is not visible, click Flow Monitor. The Flow section of the GO menu appears.
3. Select Configure > Flow Settings. The Flow Settings dialog appears.
4. Under Report Data, customize the roll up intervals to meet your requirements.

   Important: For recommended settings based on your network's size and characteristics, see the WhatsUp Flow Monitor Performance Tuning Guide.

   • Data collection interval. Select how often Flow Monitor writes raw data from its sources to the database. You may select 1, 2, 3, 4, 5, or 10 minutes. By default, raw data is written to the database every 2 minutes.

   Note: Modifying collection interval settings affects the granularity you see in Flow Monitor reports. If the interval is set to 5 minutes, you cannot distinguish traffic collected during the first minute from traffic collected during the fourth minute.

   • Resolve private address interval. When the Flow Monitor collector service encounters an IP address, it tries to determine information about the host attached to the IP address. After this information is resolved, it is stored in the Flow Monitor database. Enter the interval (in hours) that you want Flow Monitor to wait, before it checks the private IP address again, to resolve information that may have changed for the address. By default, private addresses are resolved every 48 hours.
   • Resolve public address interval. When the Flow Monitor collector service encounters an IP address, it tries to determine information about the host attached to the IP address. After this information is resolved, it is stored in the Flow Monitor database. Enter the interval (in hours) that you want Flow Monitor to wait, before it checks the public IP address again, to resolve information that may have changed on the address. By default, public addresses are resolved every 720 hours (30 days).

   Tip: Because public IP addresses are less likely to be changed, you may want to use longer intervals than used for the Resolve private address interval option.

   • Expire unclassified traffic after. Enter the number of hours after which Flow Monitor should purge unclassified traffic. Unclassified traffic is traffic transmitted over ports that are currently not monitored by Flow Monitor. By default, this option is set to 0 (zero), which causes Flow Monitor to aggregate and retain data for all unclassified ports as a single value; detailed information about the individual unclassified ports over which traffic was transmitted is immediately discarded.

   Note: The collector will purge any unclassified data that has no activity after the Expire unclassified traffic after value is satisfied.

Data Cleanup

You can use the data cleanup section of the Flow Monitor Settings dialog to set data cleanup parameters for both flow data and interface data. Periodic roll-up and archival of flow data minimizes system resources needed for data storage.

Flow Data Cleanup Settings

Flow data includes many parameters (input and output interfaces, source and destination IP addresses, port numbers, byte rates, flow end times, etc.) which while useful in providing information, may quickly fill available storage. While rolling up the data makes for efficient storage, you may lose time related information about individual flows. The following parameters are used to control the cleanup of flow data.

• Roll up raw data after. Enter the number of hours of raw flow data you would like to maintain. This setting establishes a sliding time window of raw data that spans the specified period. Raw data that reaches the end of the period is rolled up. The roll up of raw data happens every hour on the hour. After data has been rolled up, Flow Monitor can only report using the hourly summations. By default, raw data is rolled up after 4 hours.

  Caution: While the default settings for data cleanup are conservative, when you modify the roll-up settings it can directly affect the size of the Flow Monitor databases and the performance of the application. We recommend that you modify these settings cautiously, and monitor the effects of changes to these settings on database size and application performance.

• Roll up hourly data after. Enter the number of days you would like to maintain hourly data. This setting establishes a sliding time window of hourly data that spans the specified number of days. As hourly data ages beyond this period it is rolled up. The roll up of hourly data takes place daily. After hourly data is rolled up, Flow Monitor can only report aggregated totals for the entire 24-hour block of time. By default, hourly data is maintained for 1 day.
• Archive daily data after. Enter the number of days of daily data you would like to maintain before archiving. This setting establishes a sliding time window of daily data that spans the specified number of days. As daily data ages beyond this period, it is archived. Flow Monitor continues to have visibility into archived data with some restrictions. By default, daily data is archived after 3 days.
• Expire archive data after. Enter the number of days of daily data you would like to maintain in the archive database. This setting establishes a sliding time window of archived daily data that spans the specified number of days. As the archived daily data ages beyond this period it is purged from the database. After archived data is purged, Flow Monitor can no longer report on the data. By default, archive data is purged from the database after 7 days.

Interface Data Cleanup Settings

Raw interface data is provided by the flow collector, or the collector can be configured to collect raw interface data directly from the network device when the collector is receiving sampled flow data. This raw interface data is used to represent total interface traffic for the period and to calculate 95th percentile values for the Interface Overview and Interface Usage reports. Because of the data compaction, interface data has a smaller impact on data storage, so it can be maintained for longer periods of time.

The following parameters are used to control the clean up of interface data.

• Roll up raw data after. Enter the number of days of raw interface data you would like to maintain. This setting establishes a sliding time window of raw interface data that spans the specified number of days. As raw interface data ages beyond this point it is rolled up. After data has been rolled up, Flow Monitor can only report using the summations produced in the roll-up process. By default, raw interface data is rolled up after 8 days.

  Caution: While the default settings for data cleanup are conservative, when you modify the roll-up settings it can directly affect the size of the Flow Monitor databases and the performance of the application. We recommend that you modify these settings cautiously, and monitor the effects of changes to these settings on database size and application performance.

  Important: If 95th percentile values are going to be used for billing purposes, you should maintain a set of raw interface data that matches the billing period to ensure accurate results. To gather the data needed to calculate the 95th percentile values for the interface, set the Roll up raw data after setting for Interface Data to match or exceed the billing period.

• Roll up hourly data after. Enter the number of days you would like to maintain hourly interface data. This setting establishes a sliding time window of hourly interface data that spans the specified number of days. As hourly data ages beyond this period it is rolled up. The roll up of hourly interface data takes place daily. After hourly interface data is rolled up, Flow Monitor can only report aggregated totals for the entire 24-hour block of time. By default, hourly interface data is maintained for 35 days.
• Archive daily data after. Enter the number of days of daily interface data you would like to maintain before archiving. This setting establishes a sliding time window of daily interface data that spans the specified number of days. As daily interface data ages beyond this period, it is archived. Flow Monitor continues to have visibility into archived interface data. By default, daily interface data is archived after 180 days.
• Expire archive data after. Enter the number of days of daily interface data you would like to maintain in the archive database. This setting establishes a sliding time window of archived daily interface data that spans the specified number of days. As the archived daily interface data ages beyond this period it is purged from the database. After archived interface data is purged, Flow Monitor can no longer report on the data. By default, archive interface data is purged from the database after 365 days.

1. Click OK to save changes.

   Important: Any changes made to data roll up intervals are not enforced until the Flow Monitor collector service is restarted. For more information, see Stopping or restarting the collector.