System Configuration - Backup/Restore Utility

Overview

Backing up the MOVEit DMZ system can be done in one of two ways. First, you may elect to use existing backup procedures at your organization to handle backing up the system. If this is the case, you will want to consult the Technical Reference for a complete list of the files, folders, and registry settings that MOVEit DMZ uses, and that need to be backed up.

An alternative to a traditional backup-to-tape method is to use the MOVEit DMZ backup and restore utilities provided by Ipswitch. These command-line utilities perform complete backups and restorations of all the files, folders, certificates and registry settings necessary to replicate your current MOVEit DMZ configuration onto this or another platform.

Capabilities

The backup and restore utilities are capable of backing up and restoring the MOVEit DMZ configuration database (MySQL only), configuration registry keys, folder structure, and custom color schemes, logos, and templates. Encrypted files can also be optionally backed up (they are not by default), as can both server and client SSL certificates (they are by default) to create a complete backup of the MOVEit DMZ state. Both utilities automatically determine where the critical MOVEit DMZ files are located on the server, as well as how to connect to the MySQL database to perform lock operations. Both utilities also provide options to allow administrators to override these automatic detections.

Backups and restorations can be done across differing types of DMZ installations (but NOT differing versions), such as resilient and standalone installations, and installations with differing path structures. This capability requires the backup and restore utilities to NOT backup MOVEit DMZ installation path information, database access information, email server settings, or resiliency settings. These are typically configured on a per-installation basis, and should not be replicated across differing servers.

Limitations

The backup utility's ability to back up the MOVEit DMZ database tables only applies to MySQL databases. When using Microsoft SQL Server as the database for MOVEit DMZ, the utility must be run with the "--without-database" option, and the MOVEit DMZ database should be backed up using other tools. If the utility detects that SQL Server is being used, and the "--without-database" option has not been specified, it will display an error message and exit.

The backup utility obtains a read lock on all MOVEit DMZ database tables before backing up the MySQL database information. This is to ensure that the data does not change during the backup process. As a result, any requests that come in to MOVEit DMZ while the tables are locked will wait until the tables are available again before continuing (the read locks are released once the database tables have been backed up). For this reason, it is best if the backup process is run during off hours.

The backup and restore utilities do NOT back up NTFS permissions, NT users/groups, or IIS settings. If you are using these utilities to maintain a "hot standby", you will need to set these items up on the second box ahead of time. (Frequently people create a hot standby by restoring a full tape backup onto a new machine, then use periodic runs of the backup utility to keep data on the hot standby fresh.)

The backup utility is also not able to override the "do not export private key" setting you may have set when you imported your SSL server certificates. If the utility encounters an SSL server certificate export error related to private keys it will immediately print the error encountered and then exit. You can use the "--ignore-cert-export-errors" option to prevent the utility from exiting like this, however the problem certificates will still not be backed up.

Installation

Starting with MOVEit DMZ 3.1.5, the backup and restore utilities are installed by the MOVEit DMZ setup program, and are placed in the MOVEitDMZ\Scheduler folder. The two application filenames are DMZBackup.exe and DMZRestore.exe. The programs need to be in this location in order to use some requisite libraries provided by MOVEit DMZ. These utilities may not work correctly if run from any path other than MOVEit DMZ's Scheduler folder. If the utilities are run from outside the Scheduler folder, an error message will be displayed and the utility will exit.

In addition to the backup and restore utilities themselves, two additional programs are installed for use by the utilities. The first is an archiving application which supports the creation of files larger than 2GB. This application is called 7-Zip and is run using the 7z.exe program file. The second is an SSL certificate extracting application which dumps the various Microsoft SSL certificate stores into a format which is usable by the backup and restore utilities. This application is called ExportCerts.exe.

Using the Backup Utility

To perform a simple backup, open a command prompt and cd to the Scheduler subdirectory of your MOVEit DMZ non-web directory. Next, execute the following command.

C:\MOVEitDMZ\Scheduler>dmzbackup

This will create a file called MOVEitDMZ_Backup_xxxxxxxx.7z in your Scheduler directory, where xxxxxxxx is the current date in YYYYMMDD format. This file will contain all the necessary files and information to reconstruct your DMZ configuration. It will not backup the actual encrypted files in your DMZ configuration, however, it will backup the existing folder structure. To backup the encrypted files as well, add the "--with-files" command-line option to the above command.

Since doing a complete backup with files requires a large amount of free disk space on the server (roughly twice as much free disk space as the size of your \MOVEitDMZ\Files directory), many customers opt to run a config-only backup, and then backup the encrypted files in a different way (NTBackup, read-only FTP server, etc.). Customers who use this method should add the "--with-file-tables" option, to force the backup utility to back up the file-related database tables, which are normally skipped when a config-only backup is performed.

If you would like to see what the utility is doing while it runs, add the "--debug" command-line option to the above command. For a complete list of the options available in the DMZ backup utility, see the Backup Utility Commands section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitDMZ\Scheduler>dmzbackup --help

Backup Utility Commands

The DMZ Backup Utility is a .NET console application which runs in the Scheduler directory on a MOVEit DMZ system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command "dmzbackup --help". The default values listed for the database location, DSN, and DMZ directories are gathered from the registry on the DMZ system. The values may be overridden by using the command-line options.

C:\MOVEitDMZ\Scheduler>dmzbackup --help
MOVEit DMZ Backup Utility v5.5.0.0
Copyright (c) 2006-2008 by Ipswitch, Inc.

Usage: dmzbackup [options]
Options:
  --backup-database-files
    Backup the database by copying the table files instead of
    using the mysqldump utility. This option should not be used
    unless specifically required by the circumstances.

  --dbdatalocation=<directory>
    Location of database data files (default C:\MySQL\data\)

  --dbdsn=<dsn>
    Database connection DSN name (default moveitdmz)

  --debug
    Turn debug mode on

  --dmznonwebdir=<directory>
    DMZ non-web directory path (default C:\MOVEitDMZ)

  --dmzprogramfilesdir=<directory>
    DMZ Program Files path (default 'C:\PROGRA~1\MOVEit')

  --dmzwebdir=<directory>
    DMZ web directory path (default C:\MOVEitDMZ\wwwroot)

  --exit-on-cert-export-errors
    Exit immediately if an error is detected while backing up SSL
    client and server certificates. This option has no effect if
    --without-certs is enabled.

  -h, --help
    Display this help screen and exit

  --ignore-cert-export-errors
    Do not exit immediately if an error is detected while backing
    up SSL client and server certificates. This option has no effect
    if --without-certs is enabled. (default)

  --output=<output_filename>
    Relative or absolute output filepath
    (default MOVEitDMZ_Backup_YYYYMMDD.[zip|7z])
    The date macros [YYYY], [MM], and [DD] may be used and will
    be replaced by the current year, month, and day, respectively.
    The time macros [HH], [TT], and [SS] may also be used and will
    be replaced by the current hour, minute, and second respectively.

  --tempdir=<directory>
    Base temporary directory to use
    (default C:\DOCUME~1\sdn\LOCALS~1\Temp\)

  --use-zip
    Use Zip.exe to create the backup file instead of 7z.exe. This will
    generally result in a slightly larger backup file, and will fail if
    the size of the backup file exceeds roughly 2GB, but is faster than
    7z. Use if your file collection is small (<2GB).

  --use-7z
    Use 7z.exe to create the backup file instead of zip.exe. This will
    generally result in a slightly smaller backup file, and supports
    backup files larger than 2GB, but is slower than zip. Use if your
    file collection is large (>2GB). (default)

  --with-certs
    Back up SSL client and server certificates from the Microsoft
    Certificate Store. Certificate backup files will be encrypted. (default)

  --without-certs
    Do not back up SSL server and client certificates

  --with-files
    Back up encrypted files along with DMZ filesystem structure

  --without-files
    Do not back up encrypted files along with DMZ filesystem structure (default)

  --with-file-tables
    Back up the file tables from the database. This is implied when using
    --with-files, but is not on by default when --without-files is in effect.
    Since --without-files is on by default, file tables are not backed up by
    default.

  --without-database
    Do not back up database (use if managing database backup by a
    different method)

  --without-ftpnatmap
    Do not back up FTP NAT mappings (use if backup DMZ has different
	IP address)

  --without-ipbindings
    Do not back up IP bindings for FTP and SSH services (use if backup
	DMZ has different IP address)

  --7z-no-compression
    Force 7z.exe to use no compression when creating the backup file. Will
    generally result in a larger backup file, but is much faster. Use if
    your file collection has many large, incompressible files (compressed
    video files, disk images, etc). This option has no effect if
    --use-zip is enabled.


NOTE: Paths with spaces can be entered by surrounding the entire argument
with quotes. For example:
    "--tempdir=D:\Temp Dir\Sub Folder"

Backup Utility Specifics

The backup utility creates a backup file by copying all desired files, folder structures, and settings into a temporary directory, and then archiving the contents of that directory using either Zip or 7-Zip. The temporary directory is then removed at the end of the procedure. The following details the specific actions taken by the backup utility:

  1. The temporary directory is randomly generated and then created. All backup files will be copied to this location, and then archived.
  2. Most registry keys in the HKLM\SOFTWARE\Standard Networks\siLock key are read up and copied to a Windows Registry Editor compatible file, which is placed in the temporary directory. Registry keys containing paths, database information, and administrator email information are skipped to allow restoring to machines with different configurations than the original.
  3. If the MySQL database is being backed up, a read lock is obtained on all MOVEit DMZ database tables, and the appropriate table data is exported to the temporary directory using the "mysqldump" utility. If the "--with-files" or "--with-file-tables" options are specified, all table data will be exported. Otherwise, the "files", "newfiles", "folderfile", and "log" tables are skipped. Once all table data has been exported, the read locks are released.
  4. The complete encrypted filesystem folder structure is copied to the temporary directory. If the "--with-files" option is specified, the encrypted files themselves will also be copied, in their appropriate locations. If the "--with-files" or "--with-file-tables" options are specified, per-organization tamper checking hash information files will also be copied.
  5. The contents of the "images" subdirectory of the DMZ web directory is copied to the temporary directory to preserve any custom images provided by the server operator or organization administrators.
  6. Any custom CSS stylesheets, used to create MOVEit DMZ color schemes, are copied to the temporary directory.
  7. Any custom templates are copied to the temporary directory.
  8. The ExportCerts.exe utility program is run which exports all SSL certificates in the Microsoft Certificate Store and stores them as files in the temporary directory.
  9. The contents of the temporary directory are archived and optionally compressed into a single file. The temporary directory is then removed.

Using the Restore Utility

To perform a simple restoration, you must first have completely installed MOVEit DMZ on your target system. For best results, the version of DMZ should be the same as the version that you backed up. (If this is not the case, such as if you are migrating your DMZ server to a new platform, and are upgrading to a new version at the same time, you will need to re-run the MOVEit DMZ installation program after the restoration and choose the Repair option. Be sure to read our MOVEit DMZ Migration Guide if this is what you are doing.) You must also have transferred the backup file to the target system. Once you are ready to begin the restoration, open a command prompt and cd to the Scheduler subdirectory of your MOVEit DMZ non-web directory. Next, execute the following command, making sure to substitute your correct backup file path for the entry in the brackets below:

C:\MOVEitDMZ\Scheduler>dmzrestore <path to your backup file>

The restore utility will stop your webserver and database before actually restoring files. It will then decompress the backup file to a temporary location and then copy the backed up files to their appropriate locations. Finally, it will restart the database and webservers. If you would like to see what the utility is doing while it runs, add the "--debug" command-line option to the above command. For a complete list of the options available in the DMZ restore utility, see the MOVEit DMZ Restore Utility section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitDMZ\Scheduler>dmzrestore --help

NOTE: A DMZ restoration will overwrite all existing configuration information with the information found in the backup file. DO NOT run a restoration on a system that has information you want to keep.

Restore Utility Commands

Like the backup utility, the DMZ Restore Utility is a .NET console application which runs in the Scheduler directory on a MOVEit DMZ system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command "dmzrestore --help". The default values listed for the database location, DSN, and DMZ directories are gathered from the registry on the DMZ system. The values may be overridden by using the command-line options.

C:\MOVEitDMZ\Scheduler>dmzrestore --help
MOVEit DMZ Restore Utility v5.5.0.0
Copyright (c) 2006-2008 by Ipswitch, Inc.

Usage: dmzrestore [options] <dmzbackupfile>
Options:
  --dbdatalocation=<directory>
    Location of database data files (default D:\MySQL\data\)

  --dbdsn=<dsn>
    Database connection DSN name (default moveitdmz)

  --debug
    Turn debug mode on

  --dmznonwebdir=<directory>
    DMZ non-web directory path (default D:\MOVEitDMZ)

  --dmzwebdir=<directory>
    DMZ web directory path (default D:\MOVEitDMZ\wwwroot)

  -h, --help
    Display this help screen and exit

  --tempdir=<directory>
    Base temporary directory to use (default C:\DOCUME~1\Administrator\LOCALS~1\Temp\)

  --use-zip
    Use Zip.exe to extract the backup file instead of 7z.exe. Use if you used
    the --use-zip option when creating the backup file. If neither this, or
    the --use-7z option is selected, the decompresser will be selected
    automatically, based on the backup file extension (.zip will cause zip.exe
    to be used, all other extensions will cause 7z.exe to be used).

  --use-7z
    Use 7z.exe to extract the backup file instead of zip.exe. Use if you used
    the --use-7z option when creating the backup file, or allowed the default
    action to occur. If neither this, or the --use-zip option is selected, the
    decompresser will be selected automatically, based on the backup file
    extension (.zip will cause zip.exe to be used, all other extensions will
    cause 7z.exe to be used). (default)

  --with-files
    Restore encrypted files along with DMZ filesystem structure if they exist
    in the backup file (default)

  --without-files
    Do not restore encrypted files along with DMZ filesystem structure even if
    they exist in the backup file


NOTE: Paths with spaces can be entered by surrounding the entire argument with quotes.
For Example:
  "--tempdir=D:\Temp Dir\Sub Folder"

Restore Utility Specifics

The restore utility, like the backup utility, first creates a temporary directory. It then extracts all files from the backup file into the temporary directory, and then copies the backed up files to their appropriate locations, overwriting any existing files. In order to correctly perform the restore, it first stops the webserver and database server. The services are restarted once the restore is complete, and the temporary directory is removed. The following details the specific actions taken by the restore utility:

  1. The temporary directory is randomly generated and then created. The backup file is extracted to this location.
  2. The IIS webserver is stopped to allow proper restoration of appropriate files.
  3. The keys contained in the registry backup file are restored into the window registry.
  4. The database export is read back into the database, restoring the table structure and data entries.
  5. The encrypted filesystem folder structure, and any files backup up, are copied into the appropriate encrypted filesystem directory.
  6. The contents of the backed up images directory are copied into the appropriate images directory.
  7. Any backed up custom stylesheets are copied to the appropriate templates directory.
  8. Any backed up custom templates are copied to the appropriate templates directory, automatically creating any necessary "custom" subdirectories.
  9. Any backed up SSL certificates are copied to the Certs\C00X subdirectory of the DMZ non-web directory, where X is the resiliency node number, or 0 if the server is standalone. From here, the MOVEit DMZ Helper service will process the certificate files and load them into the Microsoft Certificate Store.
  10. The IIS webserver is restarted, and the temporary directory is removed.

Creating an Automated Backup Process

Overview

Ipswitch recommends backing up the configuration structure of a MOVEit DMZ server every night, preferably during off-peak hours. The following steps will show how to create such a backup process using the DMZ Backup Utility, the MOVEit Xfer command-line secure transfer client and the Windows Scheduled Tasks system. This backup process requires the MOVEit DMZ web service to be up. The process will consist of a batch file which stops the web server, backs up the DMZ system, restarts the web server, and finally uploads the backup file to a location on the DMZ server itself. From there, the backup file should be pulled down and stored in a safe location (the MOVEit Central super-client is recommended for this step).

Preparing for the Backup Process

Several things should be done before actually creating the backup process. First, a copy of all the needed software should be obtained and installed. On the MOVEit DMZ server, this should include the DMZ Backup Utility and a copy of MOVEit Xfer (MOVEit Xfer is a free command-line secure transfer client available from Ipswitch). Next, a user needs to be created to handle uploads, temporary storage, and downloads of backup files. This user should only be allowed to access DMZ from whatever internal machine will be handling the downloading of backup files from the system (access from the local machine is automatically allowed). Follow these steps to create a backup user:

  1. Log on to DMZ as an administrator.
  2. Switch to the Users page and click "Add New User".
  3. Enter a username such as "backup_user" and fill in the rest of the user account information. You may leave the email address blank if you do not want this user to receive emailed notifications. Set the permission level to "User".
  4. Open the user profile after creating the user and click "Select Ruleset" in the "Remote Access" line under the Security Information Section. Select "Use Custom Rules" and click the Change button.
  5. Click the "View Custom Rules" link that appears next to the "Use Custom Rules" radio option. Add a rule to allow access from the host that will be handling the downloading of backup files from the system.
  6. (You may also want to restrict this user to the "HTTP Clients" interface only and/or exempt this user from password expiration.)

Backup Process Batch File

The batch file commands that will execute the backup process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols. You may also remove or add any additional "--" options you want to be part of the process.

rem *** AUTOMATED MOVEit DMZ Backup Procedure ***
SET Task=DMZBackup
del %Task%.err

rem Execute the backup process. This will create
rem a file with the name MOVEitDMZ_Backup_YYYYMMDD.7z,
rem where YYYYMMDD is the current year, month, and day.
rem A log of the results and another containing just the errors will also be saved
dmzbackup > %Task%.Log 2>%Task%.Err --tempdir=<temp directory> --without-files

rem If there were any errors in the logs, notify the administrators
for /F %%A in ("%Task%.Err") do If %%~zA equ 0 del %Task%.Err
IF EXIST %Task%.Err GOTO errors
GOTO ok
:errors
echo ERRORS
ReportErrors.exe %Task% %Task%.err %Task%.Log
GOTO done
:ok
echo OK

rem If all OK...
rem Upload the file to the DMZ server
xfer > %Task%xfer.Log 2>%Task%xfer.Err -quiterror -z -user:<backup account username> -password:<backup account password> -s:dmz_bkup_commands.txt localhost

rem If there were transfer errors, notify the administrators

IF NOT %ERRORLEVEL%==0 GOTO xferbad
:xferok
rem Delete backup file
del /Q MOVEitDMZ_Backup_*
GOTO done
:xferbad
ReportErrors.exe %Task%_During_Xfer %Task%xfer.Log %Task%xfer.Err
GOTO done

:done

MOVEit Xfer Commands File

The command file which MOVEit Xfer will execute to upload the backup file is a very simple file and should have the name "dmz_bkup_commands.txt". It is shown below.

prompt
mput MOVEitDMZ_Backup_*

Scheduling the Backup Process

The Windows Scheduled Tasks service is used to schedule the running of the backup process. The process should be scheduled during off hours. Follow these steps to schedule the backup process:

  1. Click Start | Programs | Accessories | System Tools | Scheduled Tasks. This will bring up the Scheduled Tasks window.
  2. Double-click "Add Scheduled Task". This will start the Schedule Task wizard. Click "Next" to start the wizard.
  3. Click "Browse" to locate the backup process batch file. Click "Next".
  4. Select how often to run the backup process, and at what times. Ipswitch recommends running the backup process nightly.
  5. Select an appropriate Windows user account to run the backup process as. The account should have permissions to create directories and files in the temp directory, and to read files from the various MOVEit DMZ directories. It should also have permissions to run the MOVEit Xfer program, as well as the ZIP program found in the MOVEitDMZ\Utilities directory.
  6. Click "Finish" to complete the setting up of the schedule task.

Finished

The automated backup process should now be set up and ready to run. You may test the process by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting "Run."

Handling the Backup File

At this point, a client on the internal network can be used to pull the backup file off of the MOVEit DMZ system and store it to a safe location, such as a fileserver location that gets regularly backed up to tape. MOVEit Central makes automating this task simple and effective. A MOVEit Central system on the internal network can be used to pull down the backup file at some time after the file is created, and then forward that file off to a safe location. Follow the steps below to configure MOVEit Central to perform this task.

  1. Make sure Central is has the target MOVEit DMZ server configured as a DMZ host. Also make sure that whatever destination system you choose for the file is also configured as a host.
  2. Create a new Task for the backup file gathering task.
  3. Create a Source element and set the source as the target MOVEit DMZ server. If the DMZ server is already being used in other Central tasks, you should probably set the source to use a non-default user account, and then enter in the backup account username and password. Browse the DMZ server and select the backup user's home folder. Turn on the option to delete the source file after successful transfer.
  4. Create a Destination element and set it to the location you wish to forward the file to. The name the file will end up with at the destination can be configured at this point using the Filename setting. If the destination is where the backup file will be stored, it is a good idea to include the date of the backup in the filename, with the four-digit year first for easier sorting. In that case, Macros can be used the generate the correct filename: "moveitdmz_backup_[yyyy][mm][dd].7z". If the destination is a hot-standby machine which will be restoring from the backup file, the filename should be a fixed name, to allow an automated restore process to function correctly. In this case, a name such as "moveitdmz_backup.7z" is recommended. Also, in this case, the Overwrite Existing Files option should be turned on.
  5. Create a schedule element and set it to run approximately one hour after the backup process is scheduled to run on the DMZ server.
  6. If you wish to test the Task, you can do so by right-clicking on it and selecting "Run Now". You should place a test file in the correct location to make sure the file transfers are working correctly.

Creating an Automated Restore Process

Overview

An automated restore process can be useful for keeping an up-to-date hot-standby copy of MOVEit DMZ for use during loss of the primary DMZ system. The process relies on one or more clients such as MOVEit Central to transfer the backup file from the primary DMZ to the backup DMZ. Once the backup file has been transferred to the backup DMZ, the restore process will download the file and restore it to the local system.

Preparing for the Restore Process

A copy of all the needed software should be obtained and installed. This should include the DMZ Restore Utility and a copy of MOVEit Xfer. No user configuration should be necessary, assuming the hot-standby machine has previously been manually synced to the primary machine. This process will use the same user account that the backup process used.

In order to allow the restore process to download and process the backup file correctly, it should always be put on the server with a constant name. Using MOVEit Central, this can be accomplished by configuring the Filename section of the Destination element to a fixed name, such as "moveitdmz_backup.7z".

Restore Process Batch File

The batch file commands that will execute the restore process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols.

rem First, download the backup file from the
rem DMZ server
xfer -z -user:<backup account username> -password:<backup account password> -s:dmz_rstr_commands.txt localhost

rem Next, execute the restore process..
dmzrestore --tempdir=<temp directory> <DMZ backup filename>

rem Delete backup file
del /Q <DMZ backup filename>

MOVEit Xfer Commands File

The command file which MOVEit Xfer will execute to download the backup file and then remove it from the DMZ server is a very simple file and should have the name "dmz_rstr_commands.txt". It is shown below.

prompt
mget moveitdmz_backup_*.7z
mdel moveitdmz_backup_*.7z

Scheduling the Restore Process

As with the backup process, the Windows Scheduled Tasks service is used to schedule the running of the restore process. The process can be scheduled for any time after the file is scheduled to arrive on the DMZ server, since this DMZ server should be a hot-standby, and not currently in production use. Follow the directions in the Creating an Automated Backup Process section to create a scheduled process.

Finished

The automated restore process should now be set up and ready to run. You may test the process (assuming you have a file to download) by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting "Run."

Disaster Recovery

With backup and restore processes properly configured to keep a hot-standby machine synced with a primary machine, as shown above, an organization can easily provide redundant DMZ services. If the primary machine goes down for any reason, the hot-standby should be able to jump in without any further configuration, since it should always be running and synced to the primary. The only configuration required to change over would be in the routing and DNS systems, to point customers at the standby instead of the primary.

Batch Files to Start and Stop MOVEit Services

Two batch files in the "Scheduler" folder allow administrators to easily start and stop all MOVEit DMZ services in a safe order with a single command.