Automatic Patchwork
NEW DOD POLICY REQUIRES AUTOMATED
TECHNOLOGY FOR MANAGING PATCHES
AGAINST SOFTWARE SECURITY FLAWS.
Faced with growing cyber attacks that target software security flaws, the Pentagon is mandating use of automated patch-management technology designed to quickly protect networks against fastmoving threats.
In the civilian and military sectors alike, cyber attacks are happening more frequently and more aggressively. Early this year, a hacker pleaded guilty to government intrusion after he was caught hacking into IT systems at the Weapons Division of the Naval Air Warfare Center in California, as well as into the Defense Information Systems Agency. According to news reports, the hacker orchestrated the intrusion by creating and selling “botnets”—armies of computers from which cyber attacks are launched.
There are as many as 20 flaws per 1,000 lines of software code, according to a Government Accountability Office (GAO) report, and hackers have become expert in exploiting the vulnerabilities that result. Between 1995 and the first half of 2003, there were 11,155 security vulnerabilities that resulted from software flaws, based on a CERT Coordination Center report.
Because of tools that automate cyber attacks, it may take only hours or minutes to attack thousands of computers, as opposed to weeks or months, as was the case in the past. The Slammer worm, for example, infected at least 75,000 systems in only minutes. Dozens of new attack tools are newly available on the Internet every month, the GAO report indicates.
Patches are developed and released by software vendors when vulnerabilities are discovered. Using older technology, IT managers commonly spend up to two hours per day managing patches, according to Gartner researchers. It can easily cost $300 per server to manually install a patch.
Patch management within the Department of Defense is set to get a significant boost. A new policy requires that as soon as patches are available, they be applied automatically to all computers on the Global Information Grid (GIG).
The Joint Task Force-Global Network Operations (JTFGNO), a component of U.S. Strategic Command, directs the operation and defense of the GIG. In a Communication Tasking Order (CTO) issued in November 2005, the JTF-GNO directed all DoD components that are connected directly to the GIG to immediately initiate automated patch-compliance vulnerability scanning and mitigation procedures as part of the Information Assurance Vulnerability Management program. Automated patch management software applications will carry out this process, shifting away from manual patch management processes.
“The CTO mandates use of specific tools that will enhance responsiveness and efficiency and further aid commanders in managing and defending the GIG,” said a JTF-GNO spokesman.
For security reasons, the JTF-GNO will not discuss timelines for implementation and compliance reporting, nor will it discuss any information relating to the specific tools recommended.
Computers on the GIG have not entirely lacked automated patch-management processes. Several vendors currently provide this service to various DoD entities. The CTO simply requires that all offices must automate their patch management.
TIMING IS EVERYTHING
When a new software vulnerability comes to light, attackers are often quicker to exploit it than the victims are to protect against or patch it. The time between patch availability and first attack can be as little as an hour or even minutes, compared with weeks or months in the past.
More than 90 percent of computer security breaches are said to involve a vulnerability caused by a missing patch that is available but had not been applied yet. Such was the case for the Slammer worm, for which a patch was reportedly available six months before the worm caused widespread problems.
Software patching has become unmanageable, contended Jack Danahy, founder and chief technology officer at Ounce Labs.
“The government’s Information Assurance Vulnerability Alert (IAVA) process for notifying agencies of new vulnerabilities and available patches is an extremely expensive and time-consuming method of remediating flaws that in most cases could have been fixed with relatively simple changes during software development,” Danahy said. “The former chief information officer of the Air Force, John Gilligan, even noted recently that the costs of patching software were more than the original purchase and maintenance price of the product.”
So the more quickly the software patches are installed, the less vulnerable an enterprise will be. That’s where automated patch management comes in. This enables the automatic detection of vulnerabilities, retrieval of patches or software updates from vendor sites and distribution and installation of patches and software updates throughout an enterprise on multiple computing platforms.
Ounce Labs does not offer automated patchmanagement solutions but rather products that scan source code to allow customers to identify and remediate security vulnerabilities and policy violations. The other companies mentioned in this article, however, do offer automated patch-management solutions.
Manual patch management can be costly and time-consuming. It involves being on the lookout for new vulnerabilities and patches, downloading the patches, testing them and distributing them to the computer systems. “Patching manually, you would first have to determine what state your computer was in, and that itself was a lot of work. Then you would go to the Microsoft site and search for any patches that were needed for that computer. It could take about three hours for just one computer,” said Chris Schwartzbauer, vice president for worldwide sales at Shavlik Technologies.
Automated patch-management applications, by contrast, typically carry out those functions on their own. The tasks include inventorying all hardware and software within a network, looking for vulnerabilities as well as unauthorized and/or outdated software; selecting and downloading the appropriate patches; testing the patches to verify their functionality and compatibility; installing them; and reporting on the status. Thousands of computers can be patched simultaneously by one administrator in a single centralized location.
“There’s been a profound change in the industry,” said Mark Shavlik, president and chief executive officer of Shavlik Technologies. “For instance, one administrator now can patch 1,000 computers in a few hours, whereas five years ago it would have taken nearly 3,000 hours.”
A team of five people could update 20,000 computers in three days, he said.
“Manual patch management generally involves many different products to maintain your network, while automated patch management brings it all into one integrated subscription across multiple platforms,” explained Chris Andrew, vice president of security technologies at Patchlink Corp. “So I can manage Sun boxes and Windows boxes and Macintoshes all from a central location rather than having to use one-of-a-kind solutions for every kind of product and operating system on the network. IT administrators can manage patches, updates, hard fixes and new versions of software, all from the central repository and within your network.
“In a large government agency, you may run a network scan once a week or some set schedule, and try to fix things based on a network scan,” Andrew continued. “With automated patch management the system is automatically scanned dynamically. So as you apply a series of patches, the automated patch system will then scan that box internally and make sure all the necessary files have been replaced and the registry keys have been set correctly. And if anything is wrong, patches can be redeployed. So you’re always up to date.”
“Altiris Inventory Solution helps you understand what types of machines you have in your environment and their condition. Our products can also help you prioritize the roll out of patches and new applications,” said Jim Barker, product line manager for security and compliance products at Altiris. “Generally speaking, if you have never had a centralized patch management system, you do not want to go into your environment, bring up the infrastructure, distribute the agent and flip all switches to the on position. This would create a tremendous amount of traffic across your network. Our Inventory Solution allows you to see and understand your environment and then make decisions about a phased implementation of patches to that environment.”
THE BIG PICTURE
In addition, administrators get a bird’s-eye view of the entire system. “Typically, patch management systems obtain this information from Active Directory or Microsoft Windows Network and display them in a tree structure,” said Vijay Adusumilli, product manager at St. Bernard Software. “Some patch management systems allow users to create groups of machines. For example, the administrator may want to patch servers differently than he patches desktops. He can deploy to multiple groups of machines from a central console. Patches are deployed from the patch server.”
Not having that big-picture view is like padlocking your door while leaving the windows open, said John Menkart, director of government sales at Opsware. “You feel secure because you’re patched, but in fact, unless you have visibility to all aspects of the configuration of the servers themselves, you’re still vulnerable.”
Shavlik pointed out that networks typically are in a constant state of flux. “Computers are coming and going, configurations are changing, users are coming and going. It’s very dynamic. So what the automation does is continually rebuild the bird’s-eye view and also keeps a history of past views. So you can see what changes are happening, where the hot spots are or the state of your system at some point in the past.”
With configuration visibility, patches can then be targeted or limited to certain areas, if warranted. “You may find out that you only need to apply a patch to, say, 10 of your 2,000 boxes. That’s a big increase in efficiency and certainly in speed,” said Mike Dunbar, regional vice president for federal sales at Configuresoft.
Traditional patching software places an “agent” in each computer to periodically check for new updates by interfacing with a patch database on a server.
“The agents wake up on a periodic basis and contact a patch server to figure out if they have any patches to deploy or other work that needs to be done. If the patch server indicates that there is some work for the agents, they perform it and report the results of the action back to the patch server,” said Adusumilli.
With an agent-less option, by contrast, nothing is installed on the individual computer. Instead, the patch server will query the managed computers to find out the patches installed and to deploy patches utilizing the patch database. The encrypted communication takes place using standard Windows protocols.
Some automated patch-management solutions allow large organizations to deploy multiple patch servers, a tactic that is conducive to patching large distributed networks. Other solutions require administrators to connect individually to each patch server in order to deploy patches.
“Once agents are deployed on all platforms, the agents start inventorying the server from the hardware perspective, from the software perspective and configuration perspective, and feeding that back to the core and building that model,” said Menkart, noting that the process may take a matter of hours. Then, once the agent is on the servers and the “core” is talking to the agent, a patch can be uploaded automatically.
“I can push a button and apply patches to those servers,” Menkart added. “The beauty of this solution is, because the agent is doing all this inventory, I know exactly how many of each kind of server I have, which have patches and which don’t.”
That capability is particularly important in a round-the-clock operating environment, where applications can’t be stopped just to patch a server. “What you want to be able to do is patch a subset of the servers, have the monitoring systems ignore those while they are being patched and reboot it, directing the traffic to the remaining servers, then switching the traffic to those servers that have been patched, and then move on to the remaining servers,” Menkart explained.
An integrated configuration management-based solution adds a configuration management database to provide enterprise configuration visibility, regardless of whether machines are powered off or disconnected, according to Dunbar. Configuration information also provides assurance that the patch was fully implemented, including whether the required reboot was executed, and it provides in-depth information that is crucial to automating analysis and optimizing the patch-management and verification process.
POLICY VIOLATIONS
Policy violations within software, such as logging sensitive data without encryption or not using proper access control on critical systems, are potentially much riskier than individual coding errors. “For security products in general, central control is important for pinpointing areas of greatest risk to maximize remediation efforts and document progress for regulatory compliance,” said Danahy. “And flexibility with various environments is a must for most organizations, so a consistent process can be established across different functional areas.”
If there are heterogeneous, stovepiped systems, patch-management software needs to accommodate multiple locations. Extensibility is therefore very important. “Nobody wants to have to get rid of every single stovepipe application there just to implement patch management,” Andrew said. “It’s much better to actually have a technology that can be applied to any kind of platform. Any system with a Java virtual machine, for example, would be able to scan for patches and be able to apply updates. So that extensibility is key.”
Many automated patch-management products also carry out enterprise-security-configuration duties, such as checking for and cleaning off spyware and other unauthorized software, as well as checking security settings such as password expirations and access privileges. ♦
