Configuring Tasks - Hosts - File Notifications

File notifications are a mechanism that MOVEit Central can use to start a task almost immediately, based on the arrival of a file in a directory. This feature is available only for MOVEit DMZ and filesystem sources, and is also known as "event-driven transfers".

File notifications vs. scheduler

Prior to the introduction of file notifications in MOVEit Central 3.1, in order for a task to notice a file, the task had to be scheduled to run repeatedly. Each time the task was run, it would check for files matching a given mask. Oftentimes, no such files had arrived, so no action was taken, and the task would be run again later by the scheduler. In order for a task to see a file shortly after its arrival, the task had to be scheduled to run often, causing extra overhead for both the remote server and MOVEit Central itself.

By contrast, file notifications allow you to run a task only when files it's interested in are actually available. And with file notifications, the task runs almost immediately after the arrival of a file.

The MOVEit Central component that implements file notifications is called the "notifier".

Enabling file notifications

File notifications are enabled on a host-by-host basis.

To enable file notifications for all tasks accessing a given MOVEit DMZ host, use MOVEit Central Admin to edit that host and check the "Use Notifications" checkbox.

Note: The MOVEit DMZ server must be running MOVEit DMZ 3.1 or later. If it is not, MOVEit Central will figure it out during operation, and will disable notifications for that host. If you later upgrade that MOVEit DMZ server, you will have to stop and start MOVEit Central for it to notice the new capability on that server.

To enable file notifications for a Windows directory or remote share, you must edit a Windows File System and Shares host. To enable file notifications for all tasks accessing a remote mapped drive (share), edit that share and choose "Use Notifications". To enable file notifications for local drives, edit the "(Local)" FileSystem entry and choose "Use Notifications".

Note: On remote filesystems, file notifications work only for remote servers running modern Windows operating systems. Remote servers running Windows 9x or a non-Microsoft operating system will usually not provide file notifications even though they may still be able to share files.

Scheduler ramifications

Like the scheduler, the MOVEit Central file notifier requires a task to have a schedule in order to run that task. As with the scheduler, the notifier will run the task only during the scheduled intervals.

However, tasks for which all sources use notifications are run much less frequently than when file notifications are not used. Those tasks will not be run periodically as specified by the task interval (e.g., "every 15 minutes"). Instead, with a few exceptions, the task will be run only when files that match one of that task's sources actually arrive. Also, the "Repeat only until first success" option on a schedule is ignored if all sources are subject to file notifications.

The exceptions are these:

If, for instance, a task is scheduled to run every 15 minutes between 08:00 and 17:00, and all of its sources are marked to use notifications, then the "every 15 minutes" portion of the scheduling information is ignored. Instead, the task is run at 08:00 (in order to catch any files that may have arrived overnight), and thereafter the task is run only when a file arrives.

If only some of a task's sources correspond to hosts for which notifications have been enabled, then the task will be run both by the normal scheduler and by the notifier.

The arrival of a given file may cause several tasks to run, if those tasks are all watching the same directory. The notifier respects the "Include Subdirectories" option.

The simultaneous arrival of multiple files that are being looked for by a task will not necessarily cause that task to be run multiple times. However, if a task is already running when files for that task arrive, the notifier will queue a request to run the task again when the task completes.

Implementation

MOVEit Central learns of files arriving on MOVEit DMZ by periodically contacting MOVEit DMZ servers and asking for a list of recently-arrived files. The frequency of this polling is governed by the Global Settings option "Notification Polling Interval". The default is 5 seconds; the minimum is 1 second and the maximum is 3600 seconds. One request per MOVEit DMZ server is made during each interval.

Because only one request is made per interval, regardless of the number of tasks affected, and because MOVEit Central does not need to signon and signoff each for each request, this process is much more efficient than having many tasks each run and check for themselves.

Note: The MOVEit Central notifier uses the username and password associated directly with the host. Tasks with sources that override this username may not be notified properly.

If this username cannot signon, MOVEit Central will wait for a period of 60 times the notification interval (up to a maximum of 3600 seconds) before trying again. This extra delay is intended to prevent MOVEit DMZ from locking out the user or IP address.

For filesystem notifications, the notifier uses Windows directory change notifications (using ReadDirectoryChangesW). When a file arrives, MOVEit Central waits until the file is no longer locked by another process before it runs the corresponding task(s). Generally, this prevents problems in which MOVEit Central tries to read an incomplete file being created by a program on the remote computer.