Previous Topic

Next Topic

Book Contents

Book Index

Backup/Restore Utility

Backing up the MOVEit Transfer system can be done in either of the following ways:

Capabilities

The utilities can back up and restore the following:

The backup and restore utilities automatically determine where the critical MOVEit Transfer files are located on the server, as well as how to connect to the MySQL database to perform lock operations. Administrators can override these automatic detections.

Backups and restorations can be done across differing types of MOVEit Transfer installations (but NOT differing versions), such as web farm and standalone installations, and installations with differing path structures. This capability requires the backup and restore utilities to NOT backup MOVEit Transfer installation path information, database access information, or email server settings. These are typically configured on a per-installation basis and should not be replicated across distinct servers.

Limitations

The backup utility's ability to back up the MOVEit Transfer database tables applies only to MySQL databases. When using Microsoft SQL Server as the database for MOVEit Transfer, the utility must be run with the --without-database option, and you must back up the MOVEit Transfer database using other tools. If the utility detects that SQL Server is being used, and the --without-database option has not been specified, it will display an error message and exit.

The backup utility obtains a read lock on all MOVEit Transfer database tables before backing up the MySQL database information. This is to ensure that the data does not change during the backup process. As a result, any requests that come into MOVEit Transfer while the tables are locked will wait until the tables are available again before continuing (the read locks are released after the database tables have been backed up). For this reason, it is best if the backup process is run during off-hours.

The backup and restore utilities do NOT back up NTFS permissions, NT users/groups, or IIS settings. If you are using these utilities to maintain a "hot standby" —you must set up these items on the second machine ahead of time. (Frequently, people create a hot standby by restoring a full tape backup onto a new machine, then use periodic runs of the backup utility to keep data on the hot standby fresh.)

The backup utility cannot override the do not export private key setting you might have set when you imported your SSL server certificates. If the utility encounters an SSL server certificate export error related to private keys, it will immediately print the error encountered and then exit. You can use the --ignore-cert-export-errors option to prevent the utility from exiting like this; however, the problem certificates will still not be backed up.

important Important: The MOVEit Transfer Backup/Restore utility cannot backup MOVEit Transfer filestores that use Azure Blob storage. (For backing up Azure Blob storage, the AzCopy can be a good option.)

The backup and restore utilities are installed by the MOVEit Transfer setup program, and are placed in the MOVEitDMZ\Scheduler folder. Filenames are DMZBackup.exe and DMZRestore.exe. The programs must be in this location in order to use some requisite libraries provided by MOVEit Transfer.

In addition to the backup and restore utilities, two additional programs are installed for use by the utilities:

Using the Backup Utility

Open a command prompt and cd to the Scheduler subdirectory of your MOVEit Transfer non-web directory. Execute the following command.

dmzbackup

This creates a file named MOVEitDMZ_Backup_xxxxxxxx.7z in your Scheduler directory, where xxxxxxxx is the current date in YYYYMMDD format. This file will contain all the necessary files and information to reconstruct your MOVEit Transfer configuration. It will not backup the at-rest encrypted files in your MOVEit Transfer configuration, however, it will backup the existing folder structure. To backup the encrypted files as well, add the --with-files command-line option to the above command.

Because doing a complete backup with files requires a large amount of free disk space on the server (approximately twice as much free disk space as the size of your \MOVEitDMZ\Files directory), many customers opt to run a config-only backup, and then backup the encrypted files in a different way (NTBackup, read-only FTP server, etc.). Customers who use this method should add the --with-file-tables option to force the backup utility to back up the file-related database tables, which are normally skipped when a config-only backup is performed.

If you would like to see what the utility is doing while it runs, add the --debug command-line option to the above command. For a complete list of the options available in the MOVEit Transfer backup utility, see the Backup Utility Commands section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitDMZ\Scheduler>dmzbackup --help

Backup Utility Commands

The MOVEit Transfer Backup Utility is a .NET console application that runs in the Scheduler directory on a MOVEit Transfer system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command dmzbackup --help. The default values listed for the database location, DSN, and DMZ directories are gathered from the registry on the MOVEit Transfer system. The values may be overridden by using the command-line options.

C:\MOVEitTransfer\Scheduler>DMZBackup.exe --help

Commandline Help for DMZBackup (help for utility shipped with 2020 release shown)

MOVEit Transfer Backup Utility
Version 12.0.0.0
Copyright c 2020 Progress Software Corporation

Usage: dmzbackup [options]
Options:
  --backup-database-files
    Backup the database by copying the table files instead of
    using the mysqldump utility. This option should not be used
    unless specifically required by the circumstances.

  --backup-database-settings
    By default the database connection settings are NOT backed up.
    It is assumed that the system being restored to already has
    its desired database connection configured. However, the
    --backup-database-settings option can be used to include the
    database connection settings in the backup file.

  --dbdatalocation=<directory>
    Location of database data files (default C:\MySQL\data\)

  --dbdsn=<dsn>
    Database connection DSN name (default MOVEitDMZ)

  --debug
    Turn debug mode on

  --dmznonwebdir=<directory>
    DMZ non-web directory path (default C:\MOVEitTransfer)

  --dmzprogramfilesdir=<directory>
    DMZ Program Files path (default 'C:\Program Files\MOVEit\MOVEitTransfer')

  --dmzwebdir=<directory>
    DMZ web directory path (default C:\MOVEitTransfer\wwwroot)

  --exit-on-cert-export-errors
    Exit immediately if an error is detected while backing up SSL
    client and server certificates. This option has no effect if
    --without-certs is enabled.

  -h, --help
    Display this help screen and exit

  --ignore-cert-export-errors
    Do not exit immediately if an error is detected while backing
    up SSL client and server certificates. This option has no effect
    if --without-certs is enabled. (default)

  --output=<output_filename>
    Relative or absolute output filepath
    (default MOVEitDMZ_Backup_YYYYMMDD.[zip|7z])
    The date macros [YYYY], [MM], and [DD] may be used and will
    be replaced by the current year, month, and day, respectively.
    The time macros [HH], [TT], and [SS] may also be used and will
    be replaced by the current hour, minute, and second respectively.

  --tempdir=<directory>
    Base temporary directory to use 
    (default C:\Users\Administrator\AppData\Local\Temp\2\)

  --use-zip
    Use Zip.exe to create the backup file instead of 7z.exe. This will 
    generally result in a slightly larger backup file, and will fail if 
    the size of the backup file exceeds roughly 2GB, but is faster than 
    7z. Use if your file collection is small (<2GB).

  --use-7z
    Use 7z.exe to create the backup file instead of zip.exe. This will 
    generally result in a slightly smaller backup file, and supports 
    backup files larger than 2GB, but is slower than zip. Use if your 
    file collection is large (>2GB). (default)

  --with-certs
    Back up SSL client and server certificates from the Microsoft
    Certificate Store. Certificate backup files will be encrypted. (default)

  --without-certs
    Do not back up SSL server and client certificates

  --with-files
    Back up encrypted files along with DMZ filesystem structure

  --without-files
    Do not back up encrypted files along with DMZ filesystem structure (default)

  --with-filesystem
    Back up encrypted DMZ filesystem structure (default)

  --without-filesystem
    Do not back up encrypted DMZ filesystem structure

  --with-file-tables
    Back up the file tables from the database. This is implied when using
    --with-files, but is not on by default when --without-files is in effect.
    Since --without-files is on by default, file tables are not backed up by
    default.

  --without-database
    Do not back up database (use if managing database backup by a
    different method)

  --without-ftpnatmap
    Do not back up FTP NAT mappings (use if backup DMZ has different
    IP address)

  --without-ipbindings
    Do not back up IP bindings for FTP and SSH services (use if backup
    DMZ has different IP address)

  --7z-no-compression
    Force 7z.exe to use no compression when creating the backup file. Will 
    generally result in a larger backup file, but is much faster. Use if 
    your file collection has many large, incompressible files (compressed 
    video files, disk images, etc). This option has no effect if 
    --use-zip is enabled.


NOTE: Paths with spaces can be entered by surrounding the entire argument 
with quotes. For example:
    "--tempdir=D:\Temp Dir\Sub Folder"

Backup Utility Specifics

The backup utility creates a backup file by copying all desired files, folder structures, and settings into a temporary directory, and then archiving the contents of that directory using either Zip or 7-Zip. The temporary directory is then removed at the end of the procedure. The backup utility performs the following actions:

Using the Restore Utility

important Important: Running a MOVEit Automation Restore will overwrite all existing configuration information with the information found in the backup file. DO NOT run a restoration on a system that has information you want to keep.

Prerequisites:

  1. In a command prompt, cd to the Scheduler subdirectory of the MOVEit Transfer non-web directory.
  2. Issue the following command:

    dmzrestore path-to-your-backup-file --debug

    where

    The restore utility stops your webserver and database, decompress the backup file to a temporary location, copies the backed up files to their appropriate locations, and then restarts the database and webservers.

For a complete list of the options available in the DMZ restore utility, see the MOVEit Transfer Restore Utility section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitTransfer\Scheduler>DMZRestore.exe --help

Restore Utility Commands

The DMZ Restore Utility is a .NET console application that runs in the Scheduler directory on a MOVEit Transfer system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command dmzrestore --help. The default values listed for the database location, DSN, and DMZ directories are gathered from the registry on the DMZ system. The values may be overridden by using the command-line options.

C:\MOVEitDMZ\Scheduler>dmzrestore --help

Commandline Help for DMZRestore (help for utility shipped with 2020 release shown)

MOVEit Transfer Restore Utility
Version 12.0.0.0
Copyright c 2020 Progress Software Corporation by Progress Software Corporation

Usage: dmzrestore [options] <dmzbackupfile>
Options:
  --dbdatalocation=<directory>
    Location of database data files (default C:\MySQL\data\)

  --dbdsn=<dsn>
    Database connection DSN name (default MOVEitDMZ)

  --debug
    Turn debug mode on

  --dmznonwebdir=<directory>
    DMZ non-web directory path (default C:\MOVEitTransfer)

  --dmzwebdir=<directory>
    DMZ web directory path (default C:\MOVEitTransfer\wwwroot)

  -h, --help
    Display this help screen and exit

  --restore-database-settings
    By default the database connection settings that were saved to the backup
    file (if any) are NOT restored. It is assumed that the target system
    already has its desired database connection configured. However, the
    --restore-database-settings option can be used to restore the database
    connection settings from the backup file.

  --tempdir=<directory>
    Base temporary directory to use (default C:\Users\Administrator\AppData\Local\Temp\2\)

  --use-zip
    Use Zip.exe to extract the backup file instead of 7z.exe. Use if you used
    the --use-zip option when creating the backup file. If neither this, or
    the --use-7z option is selected, the decompresser will be selected
    automatically, based on the backup file extension (.zip will cause zip.exe
    to be used, all other extensions will cause 7z.exe to be used).

  --use-7z
    Use 7z.exe to extract the backup file instead of zip.exe. Use if you used
    the --use-7z option when creating the backup file, or allowed the default
    action to occur. If neither this, or the --use-zip option is selected, the
    decompresser will be selected automatically, based on the backup file
    extension (.zip will cause zip.exe to be used, all other extensions will
    cause 7z.exe to be used). (default)

  --with-files
    Restore encrypted files along with DMZ filesystem structure if they exist
    in the backup file (default)

  --without-files
    Do not restore encrypted files along with DMZ filesystem structure even if
    they exist in the backup file


NOTE: Paths with spaces can be entered by surrounding the entire argument with quotes.
For Example:
  "--tempdir=D:\Temp Dir\Sub Folder"

Restore Utility Specifics

The restore utility, like the backup utility, first creates a temporary directory. It then extracts all files from the backup file into the temporary directory, and then copies the backed up files to their appropriate locations, overwriting any existing files. In order to correctly perform the restore, it first stops the webserver and database server. The services are restarted once the restore is complete, and the temporary directory is removed. The restore utility performs the following actions

  1. The temporary directory is randomly generated and then created. The backup file is extracted to this location.
  2. The IIS webserver is stopped to allow proper restoration of appropriate files.
  3. The keys contained in the registry backup file are restored into the window registry.
  4. The database export is read back into the database, restoring the table structure and data entries.
  5. The encrypted filesystem folder structure, and any files backup up, are copied into the appropriate encrypted filesystem directory.
  6. The contents of the backed up images directory are copied into the appropriate images directory.
  7. Any backed up custom stylesheets are copied to the appropriate templates directory.
  8. Any backed up custom templates are copied to the appropriate templates directory, automatically creating any necessary custom subdirectories.
  9. Any backed up SSL certificates are copied to the Certs\C00X subdirectory of the DMZ non-web directory, where X is 0 (meaning the server is standalone). From here, the MOVEit Transfer Helper service will process the certificate files and load them into the Microsoft Certificate Store.
  10. The IIS webserver is restarted, and the temporary directory is removed.

Creating an Automated Backup Process

It is best practice to back up the configuration structure of a MOVEit Transfer server every night, preferably during off-peak hours. The following steps show how to create a backup process using the DMZ Backup Utility, the MOVEit Xfer command-line secure transfer client and the Windows Scheduled Tasks system. This backup process requires the MOVEit Transfer web service to be up. The process will consist of a batch file that stops the web server, backs up the DMZ system, restarts the web server, and finally uploads the backup file to a location on the DMZ server itself. From there, the backup file should be pulled down and stored in a safe location (the MOVEit Automation super-client is recommended for this step).

Preparing for the Backup Process

Do the following before you create the backup process.

  1. Obtain and install all necessary software:

    On the MOVEit Transfer server:

  2. Create a backup user to handle uploads, temporary storage, and downloads of backup files. See below for instructions. This user should only be allowed to access MOVEit Automation from the internal machine that will be handling the downloading of backup files from the system.
  1. Log on to MOVEit Transfer as an administrator.
  2. Switch to the Users page and click Add User.
  3. Enter a username such as backup_user and fill in the rest of the user account information. If you do not want this user to receive emailed notifications, leave the email address field blank. Set the permission level to User. Click Add User.
  4. In the left pane, click Users, and in the list, click the username to open the user profile. In the User Authentication section, under the Remote Access line, click Select Ruleset.
  5. On the Remote Access Settings page, under the Select Ruleset line, select Use Custom Rules and click Change Remote Access Settings.
  6. Next to the Use Custom Rules radio option, click the View Custom Rules link. .Add a rule to allow access from the host that will be handling the downloading of backup files from the system.
  7. You might also want to restrict this user to the HTTP Clients interface only and/or exempt this user from password expiration.

Backup Process Batch File

The batch file commands that will execute the backup process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols. You may also remove or add any additional -- options you want to be part of the process.

rem *** AUTOMATED MOVEit DMZ Backup Procedure *** 
SET Task=DMZBackup
del %Task%.err

rem Execute the backup process. This will create 
rem a file with the name MOVEitDMZ_Backup_YYYYMMDD.7z,
rem where YYYYMMDD is the current year, month, and day.
rem A log of the results and another containing just the errors will also be saved
dmzbackup > %Task%.Log 2>%Task%.Err --tempdir=<temp directory> --without-files
rem If there were any errors in the logs, notify the administrators 
for /F %%A in ("%Task%.Err") do If %%~zA equ 0 del %Task%.Err 
IF EXIST %Task%.Err GOTO errors 
GOTO ok 
:errors 
echo ERRORS 
ReportErrors.exe %Task% %Task%.err %Task%.Log 
GOTO done 
:ok echo OK 

rem If all OK... 
rem Upload the file to the DMZ server
xfer -quiterror -z -user:<backup account username> 
-password:<backup account password> -s:dmz_bkup_commands.txt localhost 

rem If there were transfer errors, notify the administrators 
IF NOT %ERRORLEVEL%==0 GOTO xferbad 
:xferok 
rem Delete backup file 
del /Q MOVEitDMZ_Backup_* 
GOTO done 
:xferbad 
ReportErrors.exe %Task%_During_Xfer %Task%xfer.Log %Task%xfer.Err 
GOTO done 

:done

MOVEit Xfer Commands File

The command file that MOVEit Xfer will execute to upload the backup file should have the name dmz_bkup_commands.txt. It is shown below.

<backup-user-name>
<backup-user-password>
put MOVEitDMZ_Backup_*
quit

Scheduling the Backup Process

The Windows Scheduled Tasks service is used to schedule the running of the backup process. The process should be scheduled during off hours. Follow these steps to schedule the backup process:

  1. Click Start | Programs | Accessories | System Tools | Scheduled Tasks. This will bring up the Scheduled Tasks window.
  2. Double-click Add Scheduled Task. This will start the Schedule Task wizard. Click Next to start the wizard.
  3. Click Browse to locate the backup process batch file. Click Next.
  4. Select how often to run the backup process, and at what times. It is best practice to run the backup process nightly.
  5. Select an appropriate Windows user account to run the backup process as. The account should have permissions to create directories and files in the temp directory, and to read files from the various MOVEit Transfer directories. It should also have permissions to run the MOVEit Xfer program, as well as the ZIP program found in the MOVEitDMZ\Utilities directory.
  6. Click Finish to complete the setting up of the schedule task.

Finished

The automated backup process should now be set up and ready to run. You may test the process by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting Run.

Handling the Backup File

At this point, a client on the internal network can be used to pull the backup file off of the MOVEit Transfer system and store it to a safe location, such as a fileserver location that gets regularly backed up to tape. MOVEit Automation makes automating this task simple and effective. A MOVEit Automation system on the internal network can be used to pull down the backup file at some time after the file is created, and then forward that file off to a safe location. Follow the steps below to configure MOVEit Automation to perform this task.

  1. Make sure MOVEit Automation has the target MOVEit Transfer server configured as a DMZ host. Also make sure that whatever destination system you choose for the file is also configured as a host.
  2. Create a new Task for the backup file gathering task.
  3. Create a Source element and set the source as the target MOVEit Transfer server. If the DMZ server is already being used in other MOVEit Automation tasks, you should probably set the source to use a non-default user account, and then enter in the backup account username and password. Browse the DMZ server and select the backup user's home folder. Turn on the option to delete the source file after successful transfer.
  4. Create a Destination element and set it to the location you wish to forward the file to. The name the file will end up with at the destination can be configured at this point using the Filename setting. If the destination is where the backup file will be stored, it is a good idea to include the date of the backup in the filename, with the four-digit year first for easier sorting. In that case, Macros can be used to generate the correct filename: moveitdmz_backup_[yyyy][mm][dd].7z. If the destination is a hot-standby machine that will be restoring from the backup file, the filename should be a fixed name, to allow an automated restore process to function correctly. In this case, a name such as moveitdmz_backup.7z is recommended. Also, in this case, the Overwrite Existing Files option should be turned on.
  5. Create a schedule element and set it to run approximately one hour after the backup process is scheduled to run on the DMZ server.
  6. If you wish to test the Task, you can do so by right-clicking on it and selecting Run Now. You should place a test file in the correct location to make sure the file transfers are working correctly.

Creating an Automated Restore Process

An automated restore process can be useful for keeping an up-to-date hot-standby copy of MOVEit Transfer for use during loss of the primary DMZ system. The process relies on one or more clients such as MOVEit Automation to transfer the backup file from the primary DMZ to the backup DMZ. Once the backup file has been transferred to the backup DMZ, the restore process will download the file and restore it to the local system.

A copy of all the needed software should be obtained and installed. This should include the DMZ Restore Utility and a copy of MOVEit Xfer. No user configuration should be necessary, assuming the hot-standby machine has previously been manually synced to the primary machine. This process will use the same user account that the backup process used.

In order to allow the restore process to download and process the backup file correctly, it should always be put on the server with a constant name. Using MOVEit Automation, this can be accomplished by configuring the Filename section of the Destination element to a fixed name, such as moveitdmz_backup.7z.

Restore Process Batch File

The batch file commands that will execute the restore process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols.

rem First, download the backup file from the 
rem DMZ server
xfer -z -user:<backup account username> -password:<backup account password> -s:dmz_rstr_commands.txt localhost
rem Next, execute the restore process.. 
dmzrestore --tempdir=<temp directory> <DMZ backup filename>
rem Delete backup file 
del /Q <DMZ backup filename>

MOVEit Xfer Commands File

The command file that MOVEit Xfer will execute to download the backup file and then remove it from the DMZ server should have the name dmz_rstr_commands.txt. It is shown below.

prompt 
mget moveitdmz_backup_*.7z 
mdel moveitdmz_backup_*.7z

As with the backup process, the Windows Scheduled Tasks service is used to schedule the running of the restore process. The process can be scheduled for any time after the file is scheduled to arrive on the DMZ server, since this DMZ server should be a hot-standby, and not currently in production use. Follow the directions in the Creating an Automated Backup Process section to create a scheduled process.

Finished

The automated restore process should now be set up and ready to run. You may test the process (assuming you have a file to download) by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting Run.

Disaster Recovery

With backup and restore processes properly configured to keep a hot-standby machine synced with a primary machine, as shown above, an organization can easily provide redundant DMZ services. If the primary machine goes down for any reason, the hot-standby should be able to jump in without any further configuration, since it should always be running and synced to the primary. The only configuration required to change over would be in the routing and DNS systems, to point customers at the standby instead of the primary.

Batch Files to Start and Stop MOVEit Services

Two batch files in the Scheduler folder allow administrators to easily start and stop all MOVEit Transfer services in a safe order with a single command.