BigFix Performance

The BigFix Server installer includes an installer for MSDE 2000 SP3a. Due to restrictions built into MSDE 2000 by Microsoft, when too many database connections are open, MSDE will slow itself down. The BigFix Server components use several database connections and each BigFix Console operator uses a database connection. When multiple BigFix Console operators are using BigFix at the same time, you will most likely see performance degradation due to the MSDE restrictions. This can corrected by using SQL Server 2000 instead of MSDE 2000.

By default, Microsoft SQL Server 2000 uses "Full" transaction logging to allow for optimal data recovery. To get better performance from BigFix, you should change the transaction type to "Simple". Here are instructions on how to change the SQL Server 2000 transaction logging type. If you are using MSDE 2000, you will not need to change this setting because MSDE 2000 defaults to using "Simple" transaction logging.

By default, the BigFix Server downloads patches, upgrades, etc. from the Internet and then the BigFix Clients download the files from the BigFix Server. If the BigFix Server is in a separate geographic location from a group of BigFix Clients with a slower network connection, each BigFix Client must download the file over the slow link from the BigFix Server.

For example, imagine that there is a company with a main office in LA with the BigFix Server and a remote office in NY with 500 BigFix Clients connected using a T1. If Windows 2000 SP4 (120 MB download) is deployed to each of the BigFix Clients in NY, there will be a total of 60 GB of network traffic, which can easily overwhelm the T1 connection (this situation would be much worse if there is a slower connection of 256 kbps or even 56 kpbs), which is used for many other activities.

To avoid this unattractive use of bandwidth, a BigFix Relay should be installed on a computer in NY and the BigFix Clients in NY should be configured to download from the BigFix Relay. Using this configuration, the BigFix Relay will download and cache each file from the BigFix Server and the BigFix Clients will download directly from the BigFix Relay. In the previous example, the total network traffic across the link from LA to NY would be reduced from 6 GB to 120 MB. Note that the BigFix Relays will also group and compress communications other than downloads from the BigFix Clients to the BigFix Server providing even more bandwidth savings.

For more information about BigFix Relays, see http://support.bigfix.com/bes/misc/besrelays.html.

For all but the smallest BigFix deployments (< 500 BigFix Clients), a primary BigFix Relay should be set for each BigFix Client even if they are not in a remote location. The reason for this is that the BigFix Server performs a lot of tasks including gathering new Fixlet content from the BigFix servers, distributing new Fixlet content to the BigFix Clients, accepting and processing reports from the BigFix Clients, providing data for the BigFix Consoles, sending downloaded files (which can be quite large) to the BigFix Client, and much more. By using BigFix Relays, the burden of communicating directly with every BigFix Client is effectively moved to a different computer (the BigFix Relay computer), which frees up the BigFix Server to do other tasks. If BigFix Relays are not used, you may see that performance degrades significantly when an action with a download is sent to the BigFix Server and you may even see errors.

Setting up BigFix Relays in the appropriate places and correctly configuring the BigFix Clients to use them (either by letting the BigFix Client auto-select their closest BigFix Relay or by manually configuring the BigFix Clients to use a specific BigFix Relay) is the most important and highest impact performance change.

For more information about BigFix Relays, see http://support.bigfix.com/bes/misc/besrelays.html.

By default the BigFix Clients "check in" to the BigFix Server on a regular interval known as a "heartbeat". When the BigFix Clients send in a heartbeat to the BigFix Server, they will update their "Last Report Time" property along with any other properties that have changed since the last heartbeat. In medium to large BigFix deployments, processing the heartbeats can consume significant BigFix Server resources. To ensure optimal performance, the heartbeat should be raised from the default 15 minutes to 1 hour or even 2-6 hours for larger BigFix deployments. The heartbeat can be changed under the File > Preferences menu in the BigFix Console.

When the BigFix Console is being used, the BigFix Console will query the BigFix Server database and cache the results locally. The cache is updated according to the BigFix Console refresh period. The more BigFix Clients, the more data is transferred to the BigFix Console using database resources and network bandwidth. The BigFix Console refresh period should be raised to from its default of 15 seconds to 30 seconds, 60 seconds, or even 120 seconds for large deployments with lots of simultaneous BigFix Console users. The refresh rate can be changed under the File > Preferences menu in the BigFix Console (note that this setting is per BigFix Console.

The BigFix Server and BigFix Relay's normal operations involve creating and processing a lot of temporary files. This activity is essential for good performance of BigFix, but can be slowed down dramatically if a virus scanner is scanning each file. To address this issue, configure your virus scanner on the BigFix Server and BigFix Relay computers to exclude the BigFix Server folder and all subfolders (default is "C:\program files\bigfix enterprise\bes server") or the BigFix Relay and its subfolders (default is "C:\program files\bigfix enterprise\bes relay"). Refer to instructions from your virus scanner for more information on how to set this exclusion rule.

The BigFix Server and BigFix Relay's normal operations involve creating and processing a lot of temporary files. This activity is essential for good performance of BigFix, but can be slowed down dramatically if Windows file indexing is turned on or if the drive is set to use file compression. If these computers have indexing or compression enabled, you should disable them.

The hard drive performance of the BigFix Server is very important to the overall system performance. If improperly configured, RAID controllers can cause severe performance problems. Specifically, some on-board RAID controllers for hard drives come configured with the hard drive cache set to 100% Read/0% Write performance. This configuration will cause very poor performance because all hard drive writes will be very slow. The optimal configuration is 50% Read/50% Write performance. Refer to your RAID instructions for how to set this. If the on-board controller cannot be configured for 50% Read/50% Write performance, then you should purchase an off-board controller.

The hard drive performance of the BigFix Server is very important to the overall system performance. If improperly configured, RAID controllers can cause severe performance problems. The BigFix Server will both read and write lots of data to disk. For this reason, RAID 0 or RAID 0+1 are all good choices for RAID configurations (note that RAID 0 offers no data redundancy).

The BigFix Server performance is affected heavily by the write performance of the computer. The BigFix Server writes many files to disk and also writes data to the database at the same time. If the BigFix Server's program folder is on a separate physical drive than the SQL Server database, the hard drive writes can be done in parallel, which can lead to speed increases.

The BigFix Server and BigFix Relay's performance are partially bound by bandwidth -- especially for large file downloads. By using gigabit network connections or dual network cards on the BigFix Server and BigFix Relays, you can effectively increase the bandwidth and increase the speed of the BigFix Clients downloading and reporting.

When the BigFix Console connects to the BigFix Server computer, all of the data about the BigFix Clients, the Fixlet messages, actions, etc. are sent from the BigFix Server to the BigFix Console. The amount of information transferred grows as the number of BigFix Clients grows and can become quite large for medium to large deployments. By using the management rights feature to restrict the number of BigFix Clients managed by each BigFix Console operator, the bandwidth between the BigFix Server and BigFix Console can be reduced significantly because the BigFix Console will only receive information about a subset of the BigFix Clients.

When the BigFix Console operator takes an action, there is an option under the "Execution" tab to "Distribute the execution of this action over __ minutes to reduce network load". If this setting is not used, the targeted BigFix Clients will all simultaneously attempt to execute the action, which usually involves downloading a file from the BigFix Server (or BigFix Relay). Not only will this put a large load on the BigFix Server, but you may also use a significant load on total network bandwidth (this is especially a problem if BigFix Relays are not used). To avoid this problem, always set the "temporal distribution" setting. You should set the number higher (120 minutes) if many BigFix Clients are going to download a large file and set it lower (30 minutes) when targeting only a few BigFix Client computers or when deploying smaller patches.

The system requirements for the BigFix Console computer increase substantially for medium to large deployments. If you notice that the BigFix Console is running very slowly, using a lot of memory, or constantly causing HD access, you should use the BigFix Console on a more powerful computer. In particular, adding more memory can significantly improve the performance of the BigFix Console. The BigFix Console recommendations are available here.

Actions are either in the Open, Closed, or Stopped state. Open actions are sent to BigFix Clients so that the BigFix Client can deteremine if it is supposed to run the actions. Each open action puts a small additional load on the BigFix Client. If there are many open actions, the BigFix Client will be a little slower to respond to newly relevant Fixlets/actions. It is good practice to close open actions if there is no need to have them open. As a reference point, over 10,000 open actions is a very large number of open actions; 5,000 is pretty large; 1,000 is a little high; 200 should be no problem.

The BigFix Console will list every action that has been taken including every result from every BigFix Client. As the number of actions grows large, the BigFix Console will use more memory to load the actions and the cache load/write times increase. You can right click and delete old actions, which will mark them as deleted in the database. Once they are marked as deleted, the BigFix Console will no longer load them saving memory and load time. The actions will continue to be in the database after they are deleted, but they will not be accessible to the BigFix Console or to web reports.