Zero Day

Table of Contents

- Zero Day

- Copyright

- Introduction

- Digital Universe Riddled with Holes

- Everyday machines vulnerable to hacking

- Marvel of connectivity illustrates cyber risks

- Cyberattacks hit targets with human touch

- Hacking kits fuel cyberspace arms race

- A virtual battleground for Pentagon’s cyberwarriors

- Health systems at risk of hacking

- About the Author

- More from The Washington Post

- Connect with Diversion Books

Zero Day

The Threat in Cyberspace

by Robert O’Harrow, Jr.,

The Washington Post

Copyright

Diversion Books
A Division of Diversion Publishing Corp.
443 Park Avenue South, Suite 1004
New York, New York 10016

www.DiversionBooks.com

Copyright © 2013 by The Washington Post

Cover map art by Sohail Al-Jamea/The Washington Post

Edited by Jeff Leen, Washington Post Investigations Editor

All rights reserved, including the right to reproduce this book or portions thereof in any form whatsoever.

For more information, email info@diversionbooks.com.

First Diversion Books edition January 2013.

ISBN: 978-1-938120-76-3

Introduction

The digital universe came to life on Oct. 29, 1969, when researchers at UCLA sent a single message to the Stanford Research Institute. It wasn’t much of a message, just one word: “login.” In fact, only the first two letters made it through before the system crashed. But a communications revolution had begun.

In the coming years, as networks spread and computer power exploded, hundreds of million of people found their way online. And then an idea that first emerged in science fiction became a vital reality: Cyberspace.

Cyberspace is now the most complex man-made environment on earth. It expands every moment, warping notions of time and space and diminishing the distinction between machines and people. The electronic impulses that carry the data online move at lightning speed. A roundtrip from Washington, D.C., to Beijing online occurs in less time than it takes a major league fastball to cross home plate.

An uncounted variety of devices have been linked in to cyberspace, offering almost magical features and services. Smart phones, industrial control computers, railroads, satellites, jet fighters, new cars, laptops, data routers, medical equipment, elevators, video cameras, ATMs, toll booths and GPS systems. The list grows by the hour. A researcher at Cisco estimates that more than 12 billion devices have connected so far, a number that will more than quadruple over the next decade.

Dr. David Clark, a senior research scientist at MIT, helped developed the architecture of the Internet in the 1970s. He and his colleagues had no idea what was coming.

“People say to me, ‘Did you imagine hooking every computer in the world together,’” Clark said. “And I said, ‘Oh yes, we thought there might be 10,000 of them.’”

Because security was not a priority, hacking has always been a problem. But something changed in 2012. The world became aware. It seemed as though everyone — the president, lawmakers, corporate executives and regular people — began acknowledging that the threats to all the systems in cyberspace were unprecedented, unrelenting and possibly existential.

Attacks on computers and networks now occur every hour of every day. Espionage of military communications. Theft of money and intellectual property. Disruption of water service and nuclear processing facilities. Invasions of privacy. And yet, because it is often impossible to know who is behind a hack, or where it originates, or even when it has occurred, the best that defenders can do is react quickly and well. Resilience became a mantra in 2012. Deterrence remained only a hope.

“The most destructive scenarios involve cyber actors launching several attacks on our critical infrastructure at one time, in combination with a physical attack on our country,” Defense Secretary Leon Panetta said in October 2012

“Attackers could also seek to disable or degrade critical military systems and communication networks. The collective result of these kinds of attacks could be a cyber Pearl Harbor.”

A panel of the nation’s top scientists told the Pentagon not long ago that “the cyber-universe is complex well beyond anyone’s understanding and exhibits behavior that no one predicted, and sometimes can’t even be explained well.”

The Washington Post decided to try. For more than a year, reporter Robert O’Harrow devoted himself to exploring and explaining the nature of cyberspace and the threats proliferating there. His stories spelled out the technical underpinnings of network vulnerabilities, the hacks used to exploit those systems and the consequences.

This e-book, Zero Day: The Threat in Cyberspace, is a compilation of that reporting. With narratives built around hackers, security researchers and corporate executives, O’Harrow illuminated the mind-bending challenges online. He had a simple goal: to help regular people, lawmakers and others to better understand one critical aspect of a fast-changing world.

All of cyberspace is driven at the most fundamental level by computer code, 1s and 0s in trillions of combinations, the simplest of man-made creations. Computer code has created unimagined benefits. But because of the world’s accelerating dependence on code, it also has brought pervasive vulnerabilities.

At last count, there were more than 53,000 different vulnerabilities in software systems that can be exploited to take control. Many of them have never been patched. But there are also uncounted number of unidentified vulnerabilities known as zero days — flaws in software that could enable intruders to infiltrate computer systems, take them over and steal information.

“And since no one knows except the attacker about [zero days], it’s really hard to defend against them. So you can’t just look for the attack because you’ve never seen one before,” said Charlie Miller, a former hacker for the National Security Agency and a leading security researcher.

“And so you figure software has millions and millions of lines of code and there could be lots of these flaws lurking in there deep somewhere and you can’t find them all,” he said. “So there’s always going to be flaws in software and it’s always a race to see, can the good guys or the vendors find these flaws first or can the bad guys?”

So much of the world’s activity takes place in cyberspace now – including commerce, banking and communications — the Pentagon has declared cyberspace a new domain of war. Battles in cyberspace are increasingly considered an inevitability. The cyberwarriors are already preparing — to both defend and attack.

“It’s going to happen,” said former National Defense University Professor Dan Kuehl. “And when that day comes, and in our opinion it will, if we’re not prepared for it, the problems and repercussions will be magnified.”

Part I

Digital Universe Riddled with Holes

(Matt McClain)

Sunday June 3, 2012

Charlie Miller prepared his cyberattack in a bedroom office at his Midwestern suburban home.

Brilliant and boyish-looking, Miller has a PhD in math from the University of Notre Dame and spent five years at the National Security Agency, where he secretly hacked into foreign computer systems for the U.S. government. Now, he was turning his attention to the Apple iPhone.

At just 5 ounces and 4 1/2 inches long, the iPhone is an elegant computing powerhouse. Its microscopic transistors and millions of lines of code enable owners to talk, send e-mail, take photos, listen to music, play games and conduct business, almost simultaneously. Nearly 200 million iPhones have been sold around the world.

The idea of a former cyberwarrior using his talents to hack a wildly popular consumer device might seem like a lark. But his campaign, aimed at winning a little-known hacker contest last year, points to a paradox of our digital age. The same code that unleashed a communications revolution has also created profound vulnerabilities for societies that depend on code for national security and economic survival.

Miller’s iPhone offensive showed how anything connected to networks these days can be a target.

He began by connecting his computer to another laptop holding the same software used by the iPhone. Then he typed a command to launch a program that randomly changed data in a file being processed by the software.

The alteration might be as mundane as inserting 58 for F0 in a string of data such as “0F 00 04 F0.” His plan was to constantly launch such random changes, cause the software to crash, then figure out why the substitutions triggered a problem. A software flaw could open a door and let him inside.

“I know I can do it,” Miller, now a cybersecurity consultant, told himself. “I can hack anything.”

After weeks of searching, he found what he was looking for: a “zero day,” a vulnerability in the software that has never been made public and for which there is no known fix.

The door was open, and Miller was about to walk through.

Holes in the system

The words “zero day” strike fear in military, intelligence and corporate leaders. The term is used by hackers and security specialists to describe a flaw discovered for the first time by a hacker that can be exploited to break into a system.

In recent years, there has been one stunning revelation after the next about how such unknown vulnerabilities were used to break into systems that were assumed to be secure.

One came in 2009, targeting Google, Northrop Grumman, Dow Chemicaland hundreds of other firms. Hackers from China took advantage of a flaw in Microsoft’s Internet Explorer browser and used it to penetrate the targeted computer systems. Over several months, the hackers siphoned off oceans of data, including the source code that runs Google’s systems.

Another attack last year took aim at cybersecurity giant RSA, which protects most of the Fortune 500 companies. That vulnerability involved Microsoft Excel, a spreadsheet program. The outcome was the same: A zero-day exploit enabled hackers to secretly infiltrate RSA’s computers and crack the security it sold. The firm had to pay $66 million in the following months to remediate client problems.

The most sensational zero-day attack became public in the summer of 2010. It occurred at Iran’s nuclear processing facility in Natanz. Known as Stuxnet, the attack involved a computer “worm” - a kind of code designed to move throughout the Internet while replicating itself. Last week, the New York Times reported that President Obama had approved the operation as part of a secret U.S.-Israeli cyberwar campaign against Iran begun under the Bush administration.

Among other things, the worm was built to infect thumb drives. Investigators think that when one of the infected drives was inserted into a computer at the Natanz plant, its code quickly found its target: It made hundreds of centrifuges designed to refine uranium run too fast and self-destruct, while sending signals to monitors that all was well.

To complete its mission, the Stuxnet worm relied on four zero days.

Just days ago, researchers released information about Flame, another cyberattack. It appears to be designed as a massive espionage and surveillance tool, also aimed at Iran, that can steal data and listen in on phone calls.

Some researchers believe it exploits zero-day vulnerabilities similar to those in Stuxnet.

The vastness of cyberspace

Miller and his kind are masters of code. At a fundamental level, there is almost nothing simpler than the stuff of their obsessions. There is software, which is written computer language. Computers transform software into machine code, which is simply 0′s and 1′s. Those “binary digits,” or bits, organized in trillions of combinations, serve as both the DNA and digital blood of our modern electronic world.

Bits guide the electrical impulses that tell the world’s computers what to do. They enable the seemingly magical applications that computer and smartphone users take for granted. Bits have also given life to the most dynamic man-made environment on Earth: cyberspace.

Not too long ago, “cyberspace” was pure fiction. The word appeared in “Neuromancer,” a 1984 novel that described a digital realm in which people, properly jacked in, could navigate with their minds. Author William Gibson described it as a “consensual hallucination experienced daily by billions of legitimate operators.”

Now cyberspace is a vital reality that includes billions of people, computers and machines. Almost anything that relies on code and has a link to a network could be a part of cyberspace. That includes smartphones, such as the iPhone and devices running Android, home computers and, of course, the Internet. Growing numbers of other kinds of machines and “smart” devices are also linked in: security cameras, elevators and CT scan machines; global positioning systems and satellites; jet fighters and global banking networks; commuter trains and the computers that control power grids and water systems.

So much of the world’s activity takes place in cyberspace - including military communications and operations - that the Pentagon last year declared it a domain of war.

All of it is shot through with zero days.

“We have built our future upon a capability that we have not learned how to protect,” former CIA director George J. Tenet has said.

Researchers and hackers, the good guys and bad, are racing to understand the fundamental nature of cyberspace. For clues about how to improve security - or to mount better attacks - they have turned to physics, mathematics, economics and even agriculture. Some researchers consider cyberspace akin to an organism, its security analogous to a public health issue.

One of the things they know for sure is that the problem begins with code and involves what “Neuromancer” described as the “unthinkable complexity” of humans and machines interacting online.

“The truth is that the cyber-universe is complex well beyond anyone’s understanding and exhibits behavior that no one predicted, and sometimes can’t even be explained well,” concluded JASON, an independent advisory group of the nation’s top scientists, in a November 2010 report to the Pentagon. “Our current security approaches have had limited success and have become an arms race with our adversaries.”

Hacker life

To picture the scale of cyberspace and the scope of the cybersecurity problem, think of the flow of electronic data around the world as filaments of light. Those virtual threads form a vast, brilliant cocoon around the globe.

The electronic impulses that carry the data move at lightning speed. A round-trip between Washington, D.C., and Beijing online typically occurs in less time than it takes for a major leaguer’s fastball to cross home plate. Blink, and you miss it.

It almost doesn’t matter where hackers work. In the physics governing cyberspace, hackers, terrorists and cyberwarriors can operate virtually next door to regular people browsing the World Wide Web or sending e-mails or phone texts.

Charlie Miller works in suburban St. Louis, in a room that has a small desk, a laptop, a large monitor and power cords that snake across the floor. A wooden bookshelf holds technical manuals alongside his kids’ plastic toys and stuffed animals.

The main clue about what he does for a living is a wall poster for the movie “Hackers.” “Their Crime Is Curiosity,” it says.

The 39-year-old Miller is regarded by some as among the best hackers in the world, but he does not fit the stereotype of an alienated outsider. For starters, he is one of the good guys, a white-hat hacker. He is a security consultant, and he hunts zero days as a hobby. A father of two, trim and balding, he is deceptively modest about his special talents. But his rsum entry about his NSA experience speaks volumes:

“Performed computer network scanning and reconnaissance. Identified weaknesses and vulnerabilities in computer networks. Executed numerous computer network exploitations against foreign targets.”

Apple would not be happy about his plan to attack the iPhone. Like other technology companies, Apple does not want questions about security to taint its products. The company has a well-deserved reputation for developing strong software systems. (Apple officials declined to comment for this story.)

But Miller wasn’t being malicious. He wanted to have fun, prove that it could be done and let the attack serve as a warning about the insecurity of the networked world.

Most of all, he wanted to win a prestigious annual contest where hackers convene to show off the skills that they generally keep to themselves. To win the contest, known as “Pwn2Own,” Miller had to discover a zero day and exploit it. (Pwn is hacker lingo for taking control of a computer.)

If he won, he would receive $15,000, the device he had pwned and a white blazer (modeled on the green jacket worn by winners of the Masters golf tournament). He had won the prize before for hacking Apple products, but it was getting harder.

As he settled into a large black swivel chair in his office, Miller knew he had a challenge on his hands. He did not doubt whether he would find a flaw. He only wondered how bad it would be.

Cracking the iPhone

In December 2010, Miller reached out to a friend and security colleague, Dionysus Blazakis.

Blazakis, 30, started hacking in 1994 and has been breaking code ever since. But instead of breaking the law, he decided to become a software developer. He and Miller worked for the same computer security firm in Baltimore, Independent Security Evaluators. He’s also a zero-day hunter.

In instant chat messages, the two bantered about the technical details of the iPhone’s software. Like hackers everywhere, they wanted to find the easiest route to a vulnerability that would let them take control. Unlike most hackers, they had a deadline: The contest began on March 9, 2011.

“Where do you start? . . . What do you focus on?” Miller recalled asking himself. “The hard part is figuring out the soft part to go after.”

Reading through all the software instructions was out of the question. That might have worked two decades ago, when computer systems were simpler and the Web was still a novelty. A desktop computer then might have a million lines of software. Today, the software in a desktop computer could have 80 million lines or more. Finding the zero days by hand would be like searching a beach for a grain of sand of a particular shade of tan.

Miller and Blazakis decided to rely on a hacker technique known as “fuzzing” - inserting random data into applications and trying to force them to crash.

Making systems crash is easier than it might seem. Software programs are miracles of human ingenuity, veritable cathedrals made of letters and digits. But unlike Notre Dame in Paris or Duomo in Milan - which took lifetimes to build and remain sturdy to this day - digital architecture is constantly evolving and can be made to crumble with the right push at the wrong spot.

Miller attributes that fragility to companies that place sales and novel applications over computer security.

“Companies want to make money,” he said. “They don’t want to sit around and make their software perfect.”

Many of those vulnerabilities are related to errors in code designed to parse, or sort through, data files sent over the Internet. A typical computer has hundreds of parser codes in its operating system. One good example is an image parser. It identifies the information that makes up a digital photo, processes it and then sends the file to the part of the machine designed to display the image.

Hackers will insert corrupted data in the photo’s code to disrupt the parser software, cause it to crash and open the way for it to be hijacked.

“If an application has never been fuzzed, any form of fuzzing is likely to find bugs,” said Microsoft researchers in a recent paper on the use of fuzzing to improve security.

No human being fuzzing by hand could cause a sufficient number of crashes to routinely allow a hacker to identify a zero day. So Miller and others write programs to do it. Miller’s fuzzing program enables him to connect to a variety of computers and keep track of thousands of crashes, including where in the software the crash took place.

“99.999 percent of the time, nothing bad happens,” Miller explained. “But I do it a billion times, and it happens enough times it’s interesting.”

The heart of his program is a function that randomly substitutes data in a targeted software program. He called the 200 lines of code that make up this function his “special sauce.”

To begin his iPhone hack, he took four Apple computers, one a laptop borrowed from his wife, and connected them to another computer holding the iPhone’s software, the entire amalgamation spread over the benchlike desks of his home office. The homey set-up, complete with an overstuffed bookcase crowned by a bowling pin, looked like the lair of a graduate student pursuing a science project.

Miller ran the mini-network 24 hours a day for weeks. One machine served as the quarterback, launching and coordinating the fuzz attacks, tracking the crashes and collecting the details. Before 7 most mornings, he woke up, went into the office, signed into the quarterback computer and checked on the progress, like a kid hoping for snow.

He was on the lookout in particular for failures that involved computer memory management - a serious flaw that could offer the way in.

“The memory manager keeps track of where things are, where new things should go, et cetera,” Miller recalled. “If a program crashes in the memory manager, it means the computer is confused about what things are located where. This is pretty serious because it means it is in a state where it might be persuaded to think my data is something it thinks is entirely something else.”

For now, most of the crashes were trivial. February was approaching, and time was short. Miller and Blazakis still did not have their zero day.

The hunt for flaws

Zero days have become the stuff of digital legend. In the 1996 science-fiction movie"Independence Day,” characters played by Will Smith and Jeff Goldblum launched a “virus” that took advantage of a zero-day vulnerability, crashed the computer system of an alien mothership and saved the world.

But they have always been more than just science fiction. For decades, hackers and security specialists have known about the existence of zero days. And as software proliferated, along with computers and networks, so have zero days. The researchers who found them often had no incentive to share their finds with the affected companies. Sometimes the researchers simply released the vulnerabilities publicly on the Internet to warn the public at large.

Government agencies that secretly engaged in hacking operations, along with some affected software makers, bought zero days from a thriving gray market, according to interviews with hackers and security specialists.

In 2005, a security firm called TippingPoint began offering bounties to researchers. Executives of the Austin-based firm reasoned that they could learn much for their own use while spurring the industry to fix threats by creating a master list. They called their effort the Zero Day Initiative.

Since then, more than 1,600 researchers have been paid for reporting almost 5,000 zero days. Starting at hundreds of dollars, the bounties soar into the tens of thousands. A hacker in Shanghai named Wu Shi has earned close to $300,000 for reporting more than 100 flaws in Web browsers.

The system seemed ideal, except for one thing: The software makers often failed to heed the warnings. Some vulnerabilities remained for two years or more.

In 2007, TippingPoint, now owned by Hewlett-Packard, decided to underscore the problem by holding a high-profile event. The Pwn2Own contest would not only require hackers to find zero days but to put them into action in what is known as an “exploit” or attack.

Getting closer

On Jan. 24, 2011, Miller and Blazakis saw a glimmer of hope. An especially promising crash appeared ripe for exploitation.

“Figuring out what to look at,” Miller wrote to his partner, “so we’re ready to rock.”

They had found it inside the part of the browser software that enables iPhone users to view PowerPoint presentations. It involved portions of the file that stored information about the location and size of shapes, such as a circle, square or triangle that would appear on a page of a presentation.

“Really it was just bytes in a file. It just happened that it had something to do with a shape. We didn’t really care,” Miller said later. “As long as it was doing something wrong with the data.”

This could be their zero day, but more testing was required to see if they could exploit it.

Both men dove back into the technical details of the iPhone’s PowerPoint software. It was hard labor, even for highly skilled hackers. Blazakis stopped shaving and grew a “hacker’s beard.” He put in 18-hour days as he tried to reverse engineer the PowerPoint application in order to take control of it without causing too much disruption.

Bit by bit, they began mastering the layout of the PowerPoint software. They developed an understanding of it that rivaled those who designed it.

Finally, they found a way to add their malicious code into the application and take control of a part of the iPhone.

“I think it’s under control now,” Miller wrote during an instant-message exchange on Jan. 27. “Sweet.”

Now they had to complete the exploit by figuring out a way to insert that code into an iPhone and ensuring that they could consistently hijack the device. Unlike the movies, where hackers are portrayed as breaking into computers as if they were cracking into digital safes, successful hacks often require deception and the unwitting complicity of the victim.

On Feb. 3, Miller joked to his friend about their struggle: “Looking for bugs fame money girls glory.”

Miller and Blazakis decided to create a way to lure an iPhone user to a bogus Web page. They would set up the page and trick a user into downloading a PowerPoint file. The file would appear normal, but it would contain their malicious code. (Known as “social engineering,” it’s the same technique used in the Google and RSA attacks.)

With the deadline looming, they began having video conference calls. They linked their computers in cyberspace and worked in tandem. They were a tired but formidable pair, cutting corners on their day jobs as security researchers as they closed in on the elusive exploit.

“The last two days were chaotic,” Blazakis said. “I stayed up most of the night doing this.”

On March 8, Miller flew to the contest, which was part of a security conference in Vancouver, B.C. But they still were not sure of the exploit. They continued fiddling with it right up to the eve of the event, including during Miller’s stopover in Seattle.

Their chance came on March 10. As he sat with judges and other hackers in a narrow conference room set up in the hotel, Miller had lingering fears that the hack still might not work on demand. He had just five tries to make it work under the contest rules.

When Miller’s turn arrived, he went behind a long table at one end of the room, where the judges sat with their own computers. Yellow cables snaked through the area (the hackers use cables instead of wireless to prevent other hackers from swiping the zero days in play). Miller connected his old white Apple laptop and looked out at other hackers, spectators and some reporters milling about.

A judge played the role of the unwitting iPhone user. The test phone was placed in an aluminum box to block unwanted wireless signals as an additional measure against any attempted theft of a zero-day exploit by other hackers. Miller told him to browse to the phony Web page holding a PowerPoint presentation that Miller had created. Hidden in the presentation’s data was the malicious code.

The image of the phone’s browser was projected onto a large screen. The judge typed in an address for the Web page, but the presentation never appeared. Instead, the image on the screen jumped back to the home page of the phone.

Miller, sitting with his own computer, knew just what had happened. In that moment, he had gained access to all the names and other information on the phone’s address book. He had found a way to strip privacy protections from a key part of the device.

He nudged one of the judges sitting near him and pointed to his screen, which was displaying the iPhone’s address book. He and Blazakis, who was looking on via a video feed to an iPhone he was holding in Baltimore, had won.

The next day, Miller received an oversize check worth $15,000 and beamed as he put on the white winner’s jacket.

Several weeks later, Apple acknowledged the exploit indirectly when the company issued a “patch.” As a result of the hackers’ work, the flaw they found and exploited was no longer a zero day.

Miller and Blazakis knew that behind the contest’s irreverent fun was a sobering reality.

“We’re smart and have skills and such, but we’re not that extraordinary,” Miller said later. “Imagine if you were a government or a Russian mob or a criminal syndicate and you could get 100 guys like us or a 1,000 guys?”

Extra Content

Fuzzing to find a zero day

Future of Cybersecurity

The Author on C-SPAN’s Communicators

Part II

Everyday machines vulnerable to hacking

(Matt McClain)

Monday June 4, 2012

It began as a hobby for a teenage computer programmer named John Matherly, who wondered how much he could learn about devices linked to the Internet.

After tinkering with code for nearly a decade, Matherly eventually developed a way to map and capture the specifications of everything from desktop computers to network printers to Web servers.

He called his fledgling search engine Shodan, and in late 2009 he began asking friends to try it out. He had no inkling it was about to alter the balance of security in cyberspace.

“I just thought it was cool,” said Matherly, now 28.

Matherly and other Shodan users quickly realized they were revealing an astonishing fact: Uncounted numbers of industrial control computers, the systems that automate such things as water plants and power grids, were linked in, and in some cases they were wide open to exploitation by even moderately talented hackers.

Control computers were built to run behind the safety of brick walls. But such security is rapidly eroded by links to the Internet. Recently, an unknown hacker broke into a water plant south of Houston using a default password he found in a user manual. A Shodan user found and accessed the cyclotron at the Lawrence Berkeley National Laboratory. Yet another user found thousands of unsecured Cisco routers, the computer systems that direct data on the networks.

“There’s no reason these systems should be exposed that way,” Matherly said. “It just seems ludicrous.”

The rise of Shodan illuminates the rapid convergence of the real world and cyberspace, and the degree to which machines that millions of people depend on every day are becoming vulnerable to intrusion and digital sabotage. It also shows that the online world is more interconnected and complex than anyone fully understands, leaving us more exposed than we previously imagined.

Over the past two years, Shodan has gathered data on nearly 100 million devices, recording their exact locations and the software systems that run them.

“Expose online devices,” the Web site says. “Webcams. Routers. Power Plants. iPhones. Wind Turbines. Refrigerators. VoIP Phones.”

Homeland security officials have warned that the obscurity that had protected many industrial control systems was fast disappearing in a flood of digital light.

“This means that these delicate [control computers] are potentially reachable from the Internet by malicious and skilled adversaries,” a Department of Homeland Security paper concluded in 2010.

The number of intrusions and attacks in the United States is rising fast. From October to April, the DHS received 120 incident reports, about the same as for all of 2011. But no one knows how often breaches have occurred or how serious they have been. Companies are under no obligation to report such intrusions to authorities.

A weak link in the system

Industrial control systems are the workhorses of the information age. Like other computers, they run on code and are programmable. Unlike laptops, smartphones and other consumer technology, they’re stripped down and have little style or glitz.

Costing as little as a few thousand dollars and up to $50,000, they’re often housed in plain metal boxes with few lights or switches. Control systems now open and shut water pipes, regulate the flow of natural gas, manage the production of chemicals, and run data centers, power-plant turbines and commuter trains.

The control computers collect data from electronic sensors, analyze it and send it on to desktop computers that serve as the “human-machine interface.” They afford managers precise and remote control of the machinery.

The most far-flung and powerful of these networked systems are called supervisory control and data acquisition, or SCADA. They give companies central control of large numbers of pumps, generators, oil rigs and other operations.

The allure of long-distance network control is hard to resist. Manufacturers of control computers have promised that such networks can cut costs by reducing the number of workers in the field. Siemens Industry Inc., a leader in the field, said in a recent marketing brochure that it is “more important than ever” to adopt control devices “to respond to the increasing international competitive pressure.”

The systems are often hardened against weather or tough conditions and can run nonstop for months or years. But many were designed for another era, before the mesh of networks reached into every corner of the globe, and some of the systems rely on outdated hardware and software.

A recent examination of major control systems by six hacker-researchers working with the security firm Digital Bond found that six of seven devices in the study were riddled with hardware and software flaws. Some included back doors that enabled the hackers to download passwords or sidestep security completely.

Researchers found that one machine made by General Electric, the D-20, uses the same microprocessor installed in Apple computers two decades ago. The company that made its operating software stopped updating it in 1999. It is often shipped to customers with no meaningful security. “Security is disabled by default,” the manual says. “To log in, enter any name; you do not need a password.”

In a statement to The Washington Post, General Electric said: “The D-20 was designed for deployment in a layered security environment, in which asset owners and operators employ a range of measures to prevent, detect and respond to intrusions. GE actively works with our customers to design and support those security measures.”

The company added that the software for the machine “is designed to be secure and includes a layer of password-protection, which can be activated if the customer chooses to do so.”

Other machines had flaws that enabled the researchers to take control through electronic back doors.

In January, Digital Bond said the results were “a bloodbath, mostly.”

“Most of the guys were able to hack their controllers in a single day,” said K. Reid Wightman, a Digital Bond security researcher and former Pentagon cyberwarrior. “It’s just too easy. If we can do it, imagine what a well-funded foreign power could do.”

The owners of control computers long assumed that few outsiders understood or cared how power plants and other facilities worked. They also figured the systems were safe within their facilities, disconnected from outside networks.

But like much of the rest of the world, the systems were rapidly being linked to global networks, often through indirect connections. Many of those connections came as executives sought more refined detail about their operations. With few exceptions, corporate networks used by executives are linked in some way to the Internet.

Because of the strange nature of cyberspace, even an employee passing through a plant with a wireless connection on a laptop can create a temporary data link that exposes control systems to intruders.

“They have sort of connected through osmosis,” said Marty Edwards, a senior cybersecurity official at the Department of Homeland Security. “What we have done is connect to everything.”

An accidental discovery

The idea for Shodan came to John Matherly in 2003, when he was a teenager attending community college in California. Obsessed with the digital world, he named his project after a malevolent character in a video game called System Shock II. The character, Sentient Hyper-Optimized Data Access Network, or Shodan, is an artificial intelligence entity that thinks it is a goddess and sets out to eradicate humans.

Matherly, who grew up in Switzerland, toyed with his system for years as he earned a degree in bioinformatics from the University of California at San Diego and built his career as a programmer, data miner and Web developer. His early Shodan versions found only hundreds of devices a day on the Web, and the information was not searchable. After devoting months to the project in 2009, he made a breakthrough, solving the search problem and locating many more devices.

When he launched his first live version of the program, in November of that year, he thought it might catch on with software makers who wanted to know about the systems being used by potential customers. On his Web site, Matherly described his program as “the world’s first computer search engine that lets you search the Internet for computers. . . . Find devices based on city, country, latitude/longitude, hostname, operating system and IP.”

The Shodan software runs 24 hours a day. It automatically reaches out to the World Wide Web and identifies digital locators, known as Internet protocol (IP) addresses, for computers and other devices. The program then attempts to connect to the machines. If a connection is made, Shodan “fingerprints” the machine, recording its software, geographic location and other data contained in the identification “banner” displayed by devices on the Internet.

Such identifying information is called “metadata” - and it’s far more common, useful and problematic than anyone had realized. Shodan compiles the information in Matherly’s servers - about 10 million devices every month now - and makes it almost as easy to query online as a Google search.

At first, the Shodan discoveries seemed trivial: devices commonly linked to networks such as printers and Web servers. But as queries became more sophisticated, troubling findings started emerging. One researcher using the system found that a nuclear particle accelerator at the University of California at Berkeley was linked to the Internet with virtually no security. Another identified thousands of data routers - the devices that make networks possible - open to anyone. Because they required no passwords, they could be taken over with ease.

“It was only after nearly a year that individual researchers began digging deeper through the Shodan data to locate devices that weren’t part of the known, discovered Internet,” Matherly said. “Water-treatment facilities, power plants, particle accelerators and other industrial control systems had been hidden from traditional search engines.”

As the dimensions of the challenge posed by Shodan became clear, the DHS Industrial Control Systems Cyber Emergency Response Team issued a stark warning in October 2010, noting “the increased risk” of brute-force attacks on “systems available on the Internet.”

The alert recommended placing all control system assets behind firewalls, using secure remote-access methods and disabling default passwords.

A researcher at Cambridge University, Eireann Leverett, used Shodan to identify more than 10,000 control computers linked to the Internet, many of them with known vulnerabilities. Leverett concluded that many operators had no idea how exposed they were - or even realized that their machines were online.

“This could be used to carry out remote attacks on selected devices or identify networks for further reconnaissance and exploitation,” Leverett wrote in a thesis, “Quantitatively Assessing and Visualising Industrial System Attack Surfaces,” published in June 2011. “Malicious actors might already be doing this.”

In the United States, security experts Billy Rios and Terry McCorkle said this spring that their research suggests the situation is worse than even Leverett demonstrated. Rios, who works for Google, and McCorkle, who works for Boeing, are both Shodan users who study control systems on their own time.

“The number of control systems on the Internet is far greater than anybody realizes,” said McCorkle, who along with Rios recently discussed control computer vulnerabilities at the National Defense University at Fort McNair. “These systems are insecure by their nature.”

Matherly said he wants his search engine used to improve security. But he said it can be used to shred it as well.

“Shodan has lifted the barrier. There’s no going back,” Matherly said. “Once you shed light on it, you can’t go back into hiding.”

A history of digital attacks

One story from the Cold War shows that cyberattacks on control systems have been in the imagination for a long time. Though some details are hard to confirm, it describes an attack that experts believe could happen today.

In 1981, a Soviet KGB colonel who became a spy for France, code name Farewell, shared Soviet plans to use a Canadian front company to secretly acquire technology to automate the Trans-Siberian gas pipeline, according to “At the Abyss: An Insider’s History of the Cold War,” by Thomas Reed, a former Pentagon official. Tipped off by the French, U.S. officials set up a front company to sell the technology, but only after they made some undetectable alterations to the computer code.

The alterations eventually “reset pump speeds and valve settings to produce pressures far beyond those acceptable to the pipeline joints and welds,” Reed wrote two decades later. “The result was the most monumental non-nuclear explosion and fire ever seen from space.”

A KGB veteran later disputed the account. A document on the CIA’s Web site confirmed only that “contrived computer chips” were provided to the Soviets and “flawed turbines were installed on a gas pipeline.”

Evidence of the threat to control computers mounted.

In 1997, a teenage hacker using a personal computer and a dial-up connection shut down part of a telephone network in Worcester, Mass., cutting off the local airport’s air-traffic-control communications.

In 2000, Vitek Boden, a supervisor at a technology firm in Australia, was bitter that he did not get a job with the Maroochy Shire Council, according to Joseph Weiss, author of “Protecting Industrial Control Systems From Electronic Threats."Using a radio transmitter, Boden launched an attack against a wastewater-treatment system in Queensland, remotely accessing the control systems and releasing hundreds of thousands of gallons of raw sewage into local streams and parks. He was sentenced to two years in jail.

“Marine life died, the creek water turned black and the stench was unbearable for residents,” an Australian Environmental Protection Agency official said later.

In 2007, skeptics still claimed that the threat of cyberattacks on real-world machinery was theoretical. In a demonstration called Project Aurora, the Department of Homeland Security along with power industry officials decided to test the theory themselves.

In the end, many doubters were silenced.

The target was a 5,000-horsepower diesel engine, the kind of machine that often serves as a backup generator for manufacturers and large organizations. Engineers at the Idaho National Laboratory hacked into the generator’s embedded control computer through a network. By repeatedly triggering circuit breakers, they created massive torque on the machinery, which eventually started to shake, smoke and tear itself to pieces.

Mark Zeller, who specializes in industrial power systems at Schweitzer Engineering Laboratories Inc., said the Aurora Project set off a scramble in the power industry to identify links to cyberspace and improve “electronic” perimeter security.

Those efforts include assessing the links between control systems and networks and creating layers of defenses against intruders. In some cases, that means creating “air gaps” - physical separations that cannot be breached by wireless connections - between networks and control systems along with stronger password protection.

“They have really taken this electronic security perimeter thing seriously,” Zeller said. “It’s a big issue now.”

At the same time, the DHS has stepped up its efforts, including providing advice and assistance to industries to reduce cyber-risks.

The government now routinely issues alerts about new threats to control systems. Alerts are also issued by a private industry group, the North American Electric Reliability Corp., or NERC, the organization of electrical grid operators in the United States.

Three weeks ago, NERC said that control computers on the Internet “face increased exposure” because of Shodan and hacking tools. The NERC alert said that “it is possible that hackers or hacktivist groups may cause sporadic component failures as they identify and interact with these devices.”

A sophisticated new virus called Flame, apparently aimed at intelligence collection against Iran, was revealed last week, underscoring anew the threats in cyberspace. But the most powerful and ingenious cyberattack ever publicly disclosed involved industrial control systems in Iran. Called Stuxnet when its code was discovered on the Internet in the summer of 2010, the attack alerted the world to the true potential for attacks on critical infrastructure.

Last week, the New York Times reported that Stuxnet was part of a U.S.-Israeli covert operation against Iran approved by President Obama. Stuxnet targeted a control computer called an S7 produced by Siemens and used by the Iranian government to operate centrifuges in the process of enriching uranium.

The malicious code designed to attack the machines was included as payload in a package of software called a computer “worm.”

The worm was launched into the Internet and spread rapidly around much of the world, like a virus during flu season. But most of the computers and systems infected were in Iran.

The worm code was designed to self-replicate. Investigators said it apparently infected flash drives in Iran, helping it jump from networks to unconnected computers at the Iranian nuclear processing facility in Natanz.

Stuxnet took advantage of four unknown software flaws, or zero days, to crack through security in a variety of computer systems. The attack code eventually directed the S7s to operate uranium-refining centrifuges at speeds beyond their tolerances while sending misleading data to monitors showing that all was well.

It was brilliant and devastating. Analysts believe that hundreds of centrifuges were damaged, though no one outside the operation knows for sure.

“The real-world implications of Stuxnet are beyond any threat we have seen in the past,” said the authors of an analysis of the worm issued by Symantec, a computer security firm. “Stuxnet is the type of threat we hope to never see again.”

Among those shaken to the core was Siemens.

“Stuxnet marked a turning point for the entire automation industry, turning theoretical problems into headlines,” Raj Batra, president of the industry automation division at Siemens, told The Post.

Exploiting flaws

News of Stuxnet jolted hackers around the world like a double shot of espresso, waking them up to the once-obscure world of industrial control systems.

One of them was Dillon Beresford, an energetic hacker and security consultant in Texas. He read an article about the attack in Wired magazine.

“It inspired me,” Beresford said. “I wanted to disprove that it would take a nation-state to pull this off.”

“I’m like, no, I’m going to do this in my living room.”

Beresford wasn’t just being brash. He had found zero-day vulnerabilities over the years. “At the end of the day, it’s all just code,” he said.

Starting in January 2011, Beresford worked almost nonstop for two months. He focused on the Siemens S7 line of controllers.

Like any good hack, it started with research. Beresford found an online “coding library” run by a German researcher. It contained source code for a wide variety of computers, including the S7s. Night after night he studied, focusing in particular on what is known as the machine’s communications protocol.

He discovered the protocol was designed to make it easier for machines to communicate with the Internet. Security was an afterthought.

Beresford persuaded his boss at the time - a manager at NSS Labs, a security firm - to buy him four of the industrial control systems for thousands of dollars each. “If you do find something, let people know you’re from NSS,” his boss told him.

The devices came mounted on heavy boards, ready for testing. The S7 is a plain rectangular metal container with heat vents and ports for cables, about the size of a large shoe box.

Beresford set them up on his workbench in the bedroom of his apartment in suburban Austin. He connected them to his laptop and began to hunt.

“I was up every night until 5 a.m.,” he said. “I love to write code.”

Several weeks into his experiments, Beresford made the first of several discoveries of flaws in the S7s. One of them took advantage of the fact that the protocol did not encrypt its communication with other networks, allowing a hacker to easily read and steal the “plain text” passwords.

Beresford said the protocol was created by designers who assumed the machines would operate behind the safety of an “air gap” between them and open networks. At the time, no one anticipated the use of thumb drives to close such gaps, as in the Stuxnet attack.

He also found a digital back door that enabled him to read the device’s internal memory, including the password stored on the device.

In May 2011, Beresford sent his findings to the DHS. The feds studied his work and confirmed it. In an alert issued on July 5, the agency announced it was working with Siemens on the S7 vulnerabilities.

“I crushed it,” he said. “All average guys, your typical hacker, could very easily replicate this.”

Since then, using his Shodan account, Beresford has found more than 100 S7s online, all of them potential targets.

Batra of Siemens acknowledged the vulnerabilities and said the company is working hard to address them. The company last week announced it is offering new security enhancements for its industrial control systems.

“Siemens’s automation products are rigorously tested with regard to industrial security and yet must be designed to also balance the requirements of open industrial solutions, which drive productivity,” he said. “There will never be an endpoint when it comes to industrial security threats, but companies can better protect their systems by staying up to date with the research community, following the guidance of governmental agencies, and by working with responsible, technologically innovative vendors like Siemens.”

Something to prove

Other hackers also began turning their attention to industrial control computers after hearing about Stuxnet.

One of them, an anonymous hacker who calls himself pr0f, is a bright, unemployed 22-year-old who favors hoodie sweatshirts and lives in his parents’ home somewhere overseas. He is among the growing numbers of Shodan users.

After studying control systems in the wake of Stuxnet, he thought the insecurity of the devices seemed crazy and irresponsible.

“Eventually, somebody will get access to a major system and people will be hurt,” he later said. “It’s just a matter of time.”

He vowed to prove how easy it was to get in. On Nov. 17, he saw an article online about an apparent industrial control system attack in the United States. The article said a hacker in Russia had apparently destroyed a pump in a water utility in Springfield, Ill.

Pr0f had been expecting something like this, but he was incredulous when he read a statement in the story from a DHS official.

“At this time there is no credible corroborated data that indicates a risk to critical infrastructure entities or a threat to public safety,” the statement said.

The hacker fumed: How could Homeland Security play down something so important?

“It was the final straw,” pr0f said. “I was angry. I said, ‘Yep, let’s do something.’ “

The Springfield episode turned out to be an accident not connected to Russia, but he did not learn that until later. Impulsively, he began programming his computer to search the Internet for a Siemens S7 controller. The first one he found just happened to be an S7 in South Houston, a small town thousands of miles and an ocean away from where he sat.

The hacker navigated to the machine’s Internet address. When prompted to identify himself as an approved operator, he knew just what to do, because he had read the manual. He typed in the default password: three simple digits. A moment later, he was at the controls of a water plant that serves 16,000 Texans.

“This required almost no skill,” the young man wrote online a short time later, using an e-mail address in Romania to cloak his identity.

The S7 was installed when the town upgraded its water plant more than a decade ago. That was long before most people thought of industrial control systems as targets. “Nobody gave it a second thought,” Mayor Joe Soto said. “When it was put in, we didn’t have terrorists.”

The intrusion took all of 10 minutes. The hacker did not cause any damage. Instead, he recorded images of the control system as proof of how easy it was for him to get in.

“I didn’t actually know what the machine was going to control when I started, but I logged in, and well, saw the stuff I took screen shots of,” he said in an e-mail exchange. “I was just amazed.”

So was Soto, after he saw images of the plant’s control panels on the Internet. He and other town officials ordered the gap closed immediately and then considered the implications.

“We’re probably not the only one who is wide open,” Soto said later. “He caught everyone with our pants down.”

Extra Content

How the Stuxnet worm worked

Poll: How vulnerable are you?
The Author Answers

The Author speaks with Diane Rehm

Part III

Marvel of connectivity illustrates cyber risks

(Matt McClain)

Thursday July 12, 2012

John Sublett and his colleagues had an audacious, digital-age plan. They wanted to use the Internet to enable businesses to manage any kind of electronic device, anywhere on the planet, through the computer equivalent of a universal remote control. In 1996, nothing like it had been seen before.

“We said, ‘Hey, there’s this cheap network, ready to use” Sublett recalled.

Their company, Richmond-based Tridium, would succeed - but with far-reaching implications for the security of the online universe known as cyberspace.

Tridium’s driving technology, 4 million lines of software code called the Niagara Framework, is a marvel of innovation. With the click of a mouse, Niagara enables plant managers to view video streams, high-rise superintendents to operate air conditioners and elevators, security officials to track personnel inside U.S. military facilities, and nurses to monitor medical devices in hospitals.

At least 11 million devices and machines in 52 countries, including security and surveillance systems in homes, have been linked to the Internet through Niagara, most of them in the past two years. But behind that success is a looming threat: an unknown number of Niagara-run networks are vulnerable to attacks from hackers, an examination by The Washington Post has found.

Last week, after more than a month of conversations with The Post, the company in a confidential security bulletin warned customers about the vulnerabilities and described ways to mitigate them.

“We’re not going to say Niagara is secure,” Sublett said in an interview. “We try to soften it and say we’re trying to make it as secure as possible.”

Tridium’s story illustrates the unintended consequences of the world’s rush to connect machines and devices in cyberspace. It also demonstrates how even small missteps in writing software or configuring systems can have huge implications. In cyberspace, determined hackers routinely transform obscure gaps into major security holes.

Over the past two years, hackers and cyberwarriors who once focused primarily on traditional computers and networks have put control systems in their crosshairs, damaging machinery, stealing information from networks and spying on facilities. Warnings from the Department of Homeland Security about the threats have become a drumbeat, while officials at the Pentagon and the White House consider them a national security priority.

After discussing Tridium with a Post reporter, a pair of security researchers decided on their own to zero in on Niagara and discovered gaps that would enable hackers to download and decrypt user names and passwords. The researchers, Billy Rios and Terry McCorkle, shared their findings with The Post and reported them to cybersecurity officials at the Department of Homeland Security, who recommended several measures to Tridium, including better security training for customers.

“There are hundreds of thousands of installations on networks, including [Defense Department] installations and Fortune 500 firms,” said Rios, a 34-year-old security researcher and a co-author of “Hacking: The Next Generation,” a handbook for security experts. “These customers have no idea they are exposed.”

In interviews, Sublett defended Niagara’s security, saying it follows industry “best practices” and “is basically secure.” He said Tridium has long recommended, to customers who asked, that users protect against intrusions by placing Niagara behind more-secure “virtual private networks.”

Sublett said executives learned about the vulnerabilities almost a year ago, when a Niagara customer that uses the software to manage Pentagon facilities turned up issues in an audit. He said Tridium is working on fixes. The firm also is doing more to train customers about security than it has in the past, he said.

“We’re committed to making our framework more secure,” Sublett said. “And we know it’s our responsibility to educate our community.”

No longer off the radar

For more than a decade, few people gave much thought to the security of commercial control systems on the Internet.

Tridium executives said attacks seemed unlikely, because hackers had not traditionally targeted such systems. In interviews, the executives said they and their customers generally assumed that control systems were buffered somewhat by their obscurity.

Rios and other “white hat” hackers - those who seek to improve security by exposing flaws - noticed Niagara systems popping up online in recent years, as office buildings, apartment complexes and other facilities automated heating systems, security and other operations. One researcher documented thousands of “portals” online. Others shared details about Niagara locations and speculated about their security.

But the interest did not take hold beyond a small cadre of security specialists, because few people grasped the implications of Tridium’s business and its expanding reach.

Commercial control systems rely on computer software, microprocessors and networks. They do not have to be as quick or as finely calibrated as the industrial control systems that run power generators, manufacturing equipment and other heavy machinery. But they are far more numerous.

In 1996, Sublett and five colleagues wanted to give customers options to connect a wide variety of devices into a single network. At the time, devices had to be controlled separately by each manufacturer’s software.

The team formed Tridium and designed Niagara to leverage the technology that is at the core of the Web. The Web uses the universal Hypertext Transfer Protocol, or HTTP, to facilitate computer-to-computer communication over the Internet.

In 1999, Tridium introduced the software framework to the market. Niagara permitted other companies to tailor the software to incorporate any device. At its core, Niagara serves as a kind of middleman that transforms the electronic babble of every network-connected device into a single manageable language.

Like the Web, Niagara was easy to use, and word spread in the world of control systems. Over the next few years, so many smaller companies began licensing Niagara to use in building automation projects that the big makers of commercial control systems began to follow suit.

In 2005, as the notion of connecting machines to cyberspace was gaining momentum, Honeywell International bought the company for an undisclosed amount and allowed Tridium to operate independently. Sales of the system soared, and Tridium’s promotional materials include examples of Niagara’s growing presence around the world.

In Chicago, the Niagara software manages heating, lighting, security and more for two federal government buildings that house the FBI, the Drug Enforcement Administration, the U.S. Attorney’s Office, the Internal Revenue Service and other agencies.

In Dubai, in the United Arab Emirates, managers of the 53-story 21st Century Tower apartment complex use the software to control fire detection, security sensors, air conditioning and myriad other operations.

At Singapore’s Changi Airport, Niagara helps manage more than 110,000 devices and sensors. The James Cook University Hospital in Middlesbrough, England, relies on Niagara to manage everything from “critical medical systems” to elevators, security, lighting and kitchen refrigeration.

Some Defense Department facilities in the United States also depend on Niagara. T hat includes the giant Tobyhanna Army Depot in Pennsylvania, which uses Niagara to control boilers. Some military installations use Niagara to provide surveillance and access control at “high security” facilities, said Marc Petock, Tridium’s vice president for global marketing and communications.

Growing numbers of home-automation companies are using Niagara to enable homeowners to control lighting and security systems.

One of the most widespread uses of Niagara involves about 575 Wawa convenience stores nationwide, where the software connects oven doors, gasoline pumps, exterior lights, freezers and security cameras, all controlled or monitored from a Wawa command center.

Wawa embodies Tridium’s lofty ambitions, as Petock describes them: “Any device, any system, any network, any protocol from anywhere at any time.”

Inside Niagara

Rios first noticed Niagara more than a year ago, while working on a security project. He became intrigued about the framework while attending a security conference in January, as he drank beers and smoked a cigar on a veranda at a conference center in Miami.

Rios is a former Marine captain who served in Iraq with a signals intelligence unit and later as an information assurance analyst at the Defense Department, helping to protect networks. Since then, he has held senior security positions at Microsoft and Google.

On their own time, Rios and his research partner, McCorkle, also a corporate security specialist, have made a specialty of finding vulnerabilities in industrial and commercial control systems. The two have been credited with reporting at least 25 serious vulnerabilities to cybersecurity officials at the Department of Homeland Security and vendors.

Rios was troubled by what he was hearing about Niagara in Miami. Another researcher described how he had mapped thousands of Niagara-driven networks. He said they were linked directly to the Internet even though many apparently required only user names and passwords for access. To a hacker such as Rios, that was virtually no security at all.

Following the conference, after a Post reporter called and shared new details about Tridium - including the use of Niagara by the Pentagon and other federal agencies - Rios decided to take a closer look.

On Jan. 26, sitting in his family’s San Jose living room and working on a laptop, Rios began to work. He created an account on a Tridium Web site devoted to helping users, lurking quietly as he read complaints and recommendations. The comments from users helped him to construct a “mental map” of how the software worked.

“I’m learning about its weaknesses, and I’m learning about the common configurations that I’ll likely see in the wild,” he said later.

A key moment came when someone in the online forum referred to technical manuals and a Web address where they were located. Rios and McCorkle downloaded the manuals and pored over them for clues. Rios saw a reference to a Niagara demonstration site online and turned his attention there.

As he toyed with the demo and hunted for an exploit, his insights gleaned from the manuals crystallized. “Within five minutes, I’ve found what I’m looking for,” Rios later wrote in an e-mail to The Post. “I find a flaw that gives remote attackers the ability to download all the user names and passwords for all the users on the Niagara server. I test it against the demo server . . . it works. I test it against a couple of other places . . . it works. The attack is trivial and very reliable.”

The exploit, well known in the hacker world, is called a directory traversal attack. It enabled Rios to turn the Web’s core function - the communication protocol that is intended to make everything easier - to his advantage. With some deft alterations to the Niagara Framework’s Web address, he was able to order the framework to perform certain tasks. One of them was to electronically hand over a “configuration file,” which happens to contain user names, passwords and other sensitive material.

By the time he was finished, it was 3 a.m. on Saturday, Jan. 28. Rios crafted a technical note about his exploit to cybersecurity officials at DHS and, after encrypting the message for security, sent it off. “All total, it took me 2 days to go from zero knowledge to remote password theft,” Rios wrote.

The passwords Rios had grabbed were scrambled for security by a mathematical formula called a “hash.” But that offered limited protection. Automated computer tools can crack the hashed passwords with relative ease. (In a recent attack on LinkedIn, a social-networking site, hackers made off with 6.5 million hashed passwords and immediately began cracking them.) “Once the passwords are decrypted, you can simply log in to the Niagara Framework as any user you desire,” Rios said.

A senior cybersecurity official at DHS acknowledged that the department had received Rios’s information and had talked to Tridium.

Tridium officials attributed the demo vulnerability to an employee who they said set it up incorrectly. They said some Niagara Framework users have also misconfigured their systems. Other users have never bothered to take secure measures, such as using a hard-to-hack virtual private network.

Sublett said the company intends to change the location of the configuration file to make it harder for hackers to find. Tridium also is trying to figure out the best way to change the framework’s default security settings “so it’s not as easy to make a mistake.” And it is going to improve the hash that scrambles passwords because “it’s not as strong as it should be,” Sublett said.

The company, still in talks with DHS officials, has begun a push to better communicate to Niagara users about the security risks in cyberspace. The security bulletin last week was part of that process.

“We’re not out there claiming we are bulletproof secure,” Sublett said. “Nobody is 100 percent secure.”

Extra Content

Hackers exploit ‘guest user’ account of software that allows remote operation

Homeland Security alert for Niagara

Confidential security bulletin to users

Tridium’s Niagara framework

Timeline: Key events in cyber history

Exploring a new domain of war

Part IV

Cyberattacks hit targets with human touch

(Jeffrey MacMillan)

Thursday September 27, 2012

The e-mails arrived like poison darts from cyberspace.

Some went to the Chertoff Group, a national security consulting firm in Washington. Others targeted intelligence contractors, gas pipeline executives and industrial-control security specialists. Each note came with the personal touches of a friend or colleague.

“Attach[ed] is a quote for the Social Media training we discussed,” said one message sent on July 3 to the vice president of EnergySec, a federally funded group in Oregon that focuses on the cybersecurity of the nation’s power grid.

But like much of the digital universe, the e-mails were not what they seemed. They were cyberweapons, part of a devastating kind of attack known as “social engineering.”

Emerging details about the e-mails show how social engineering - long favored by con artists, identity thieves and spammers - has become one of the leading threats to government and corporate networks in cyberspace.

The technique involves tricking people to subvert a network’s security. It often relies on well-known scams involving e-mail, known as “spear phishing,” or phony Web pages. But such ploys now serve as the pointed tips of far more sophisticated efforts by cyberwarriors to penetrate networks and steal military and trade secrets.

The e-mails this spring and summer appear to be part of a long-running espionage campaign by a hacker group in China, according to interviews with security researchers and documents obtained by The Washington Post. Some of the e-mails, including those sent to the Chertoff Group and EnergySec, were caught by suspicious employees. Others hit home.

“Multiple natural gas pipeline sector organizations have reported either attempted or successful network intrusions related to this campaign,” officials at the Department of Homeland Security said in a confidential alert obtained by The Post.

The May 15 alert, by the department’s specialists in industrial control systems, said “the number of persons targeted appears to be tightly focused. In addition, the email messages have been convincingly crafted to appear as though they were sent from a trusted member internal to the organization.”

Social-engineering attacks revolve around an instant when a computer user decides whether to click on a link, open a document or visit a Web page. But the preparation can take weeks or longer.

Serious hackers investigate their targets online and draw on troves of personal information people share about themselves, their friends and their social networks. Facebook, Twitter and other social media have become prime sources for the hackers, specialists said.

“Everybody has their trigger,” said Bruce M. Snell, director of technical marketing at McAfee Security Systems. “A good social engineer will find that trigger.”

Once malicious software code is delivered, it burrows in and hides in a targeted network. That code, known as malware, can lurk for years in intelligence or attack schemes that are sometimes known as “advanced persistent threats.” Eventually, the code reaches back out to the hackers for instructions, often cloaking the communication through encryption or masking it to seem like innocuous Web browsing by an employee.

Over the past three years, most major cyberattacks on U.S. corporations have included social engineering, specialists said. That includes hacks of Google and security giant RSA. Researchers think that scores of attacks were designed by the same Chinese hackers who appear to be involved in the current e-mail campaign. Some U.S. officials think the hackers may have links to the Chinese military.

The Chinese are not the only ones using the technique. Cyberwarriors at the Pentagon receive social-engineering training for offensive and defensive missions, knowledgeable specialists said.

David Kennedy, a security consultant and former National Security Agency analyst, said he is amazed at the effectiveness of the techniques.

“I have done hundreds of these, and I have never been stopped,” said Kennedy, who teaches social engineering to other security specialists. “It sounds horrible, but it works every single time.”

The human factor

Social engineering works because it targets a vulnerable part of cyberspace that cannot be patched with technical fixes: human beings. People want to believe that their communication is safe.

“Because it goes at the human level, not at the technological level, we’re all vulnerable,” said Joseph Nye Jr., a distinguished service professor at Harvard University who is on the board of advisers to the Chertoff Group. Nye said he has received at least six spear-phishing e-mails purporting to be from the Chertoff Group. He said he deleted them all, but he added, “Every once in awhile, one of these will get by you.”

The explosive growth of cyberspace has created a fertile environment for hackers. Facing the flood of e-mail, instant messages and other digital communication, many people have a hard time judging whether notes or messages from friends, family or colleagues are real. Many don’t even try. Hackers are so confident about such permissiveness that they sometimes begin their attacks in social media three or four steps removed from their actual targets. The hackers count on the malicious code spreading to the proper company or government agency - passed along in photos, documents or Web pages.

“This is the next evolution of social engineering, where victims are researched in advance and specifically targeted,” said a recent Internet threat report by Symantec, a computer security firm. “The very nature of social networks makes users feel that they are amongst friends and perhaps not at risk. Unfortunately, it’s exactly the opposite and attackers are turning to these sites to target new victims.”

At the same time, technology is transforming social engineering. One online data-mining service favored by hackers - as well as by security researchers and law enforcement - works much like a laser-focused Google. The automated system, called Maltego, enables users to quickly bring together and analyze disparate details about people from all corners of cyberspace, showing an individual’s links to friends, family, work associates and personal interests.

“None of these steps are particularly difficult to code or do by hand. But doing it by hand is painful,” said Roelof Temmingh, founder and managing director of Paterva, the small South African company that sells the service. “Maltego can do all of this in a flash.”

Temmingh demonstrated Maltego’s utility not long ago by looking for a person to target at Fort Meade, home to the super-secret NSA. He typed in Fort Meade’s latitude and longitude and searched for Twitter users. In a couple of steps, Maltego quickly delivered the name of a person who tweeted at the Fort Meade location.

With that, Maltego searched MySpace, a dating Web site and other resources to build a rich profile: a young Army private who served in South Korea, likes to smoke and drink, divorced and looking for a “serious relationship.” She likes Harry Potter movies and “The Cosby Show.” Maltego also turned up her name, address and birthdate.

Building a tool kit

In 2009, David Kennedy began digging deep into corporate security for a Fortune 1000 company as a penetration tester, identifying flaws that hackers could exploit. He wanted to know whether employees could be duped into clicking on unknown documents or handing over confidential information over the phone. Most of them could.

Kennedy concluded that social engineering was “the next biggest attack vector.” He teamed up with a nonprofit organization called Social-Engineer.org to develop products that would make security testing more effective.

The result is known as the Social Engineering Toolkit. Its many applications can identify targets and deliver attack payloads, secretly, like digital stealth missiles. The tool kit also provides ready-made code for attacks.

In an irony of the digital age, the same tools are available for free to attackers.

“Can a bad guy take all this and get better? Sure. . . . But that is not the intended goal,” said Chris Hadnagy a founder of Social-Engineer.org and author of the book “Social Engineering: The Art of Human Hacking.” “What we are doing is trying to weaponize people to be protected against this threat.”

Kennedy described one effective approach involving a tool that creates instant copies of real Web pages and embeds them with malicious code. In an e-mail, the attacker could pose as an executive of a company, seeking help from an IT department employee. The attacker has studied the employee and knows he is new to the company and probably eager to please his superiors.

In a phone call that appears to be coming from within the company, the attacker asks the IT staffer why a certain Web page won’t open. The attacker directs the staffer to the bogus Web page. The intrusion occurs the moment the IT staffer visits the page.

“I find that leveraging human compassion is generally the best way to gain what I want,” Kennedy, now president of TrustedSec, said in a recent seminar.

A tailor-made attack

The current intrusion campaign began in December, possibly earlier. That’s when analysts think the attacks first started against gas pipeline company executives. With some study, it became clear that the e-mails were part of a sophisticated campaign. Only certain executives were singled out for attention. The e-mails were tailored to them.

The attackers were relentless, launching e-mails on at least 13 days. They also were creative. Attached to the e-mails were documents covering a variety of subjects that might be of interest to the executives: the U.S. debt crisis, Adobe updates, iTunes help and an analysis of the presidential election.

The true scope of the campaign started to become apparent in June, after cybersecurity researchers at a security firm called Digital Bond received one of the phony e-mails. The Digital Bond researchers specialize in industrial-control computers that operate power plants and other critical infrastructure.

The note, which appeared to be coming from their boss, Dale Peterson, was filled with technical jargon. “Details are available at: Leveraging_Ethernet_Card_ Vulnerabilities_ in_Field_Devices.pdf Download it and have a look,” the e-mail said.

The intrusion failed in part because the attackers slipped up and because a Digital Bond researcher was alert. The e-mail was signed “Peterson.” But the security researchers knew that their boss uses his first name on e-mail, not his last.

Other security researchers asked to review the situation found the attachments were not actually .pdf documents but “executable files” that deposited “Trojan horse” code when a computer user clicked on them, said Jaime Blasco, security lab manager at AlienVault, who reviewed the attack.

Blasco and his partner, Ruben Santamarta, found the hackers had used multiple Web server computers to give instructions to the malicious code. The electronic trail on those servers led to other victims, including the Chertoff Group; NJVC of Vienna, Va., a contractor for the National Geospatial-Intelligence Agency; and the National Electrical Manufacturers Association (NEMA) in Arlington County, which represents companies that make components for power grids.

Officials at NJVC and NEMA acknowledged the attacks but said employees prevented network intrusions. Department of Homeland Security officials declined to discuss the episodes.

The scope of the attacks expanded in July, when the cybersecurity group EnergySec was hit. EnergySec President Patrick Miller also reached out to Blasco for help. Based on evidence, it appeared to be the same attackers: a group of Chinese hackers that had been using social engineering for nearly a decade to break into systems across the globe with impunity.

Cyber-researchers have dubbed them the Comment Crew or Comment Group. The name stems from the fact that hackers include attack commands in the comments that programmers typically include in HTML code to document their goals or make notes of changes.

The Comment Crew has become notorious for using simple social-engineering techniques, including well-crafted e-mails, in elaborate hacks that breach security, load “remote access tools,” or RATs, and siphon off oceans of data from victims.

Though it is sometimes impossible to definitively identify hackers, because of the hall-of-mirrors nature of cyberspace, they often leave behind compelling digital evidence. Researchers said the IP address of a Web server and a particular method of writing HTML comments links the attacks on the gas pipeline executives to those against the Chertoff Group and others. It also links the current campaign to a series of earlier devastating attacks by the Comment Crew, dubbed Operation Shady RAT.

Those intrusions compromised hundreds of systems over at least five years, including federal agencies, defense contractors and the United Nations, according to studies by McAfee and the Dell SecureWorks Counter Threat Unit.

“The above patterns of attack are very similar to attacks carried out by the actors responsible for the Shady RAT campaign,” said Ned Moran, a researcher at the nonprofit Shadowserver Foundation who also analyzed the attack on Digital Bond.

Joe Stewart, director of malware research at the Dell SecureWorks Counter Threat Unit, estimates the group has at least 100 members who work at specific tasks such as social-engineering research, malware development and the processing of stolen information. In essence, the Comment Crew has made a business of cyber-espionage. Their activity online shows they typically work 9 to 5 - Shanghai time - and take off Chinese holidays.

Stewart and others say Chinese hackers have been using a troubling variation of social engineering called a “watering hole” attack. Instead of sending e-mails with links - something that some security-conscious computer users now avoid - they try to entice wary victims to visit familiar, authentic Web sites that have been secretly loaded with attack code. Think of a lion near a watering hole.

One ploy involves an e-mail announcement of online coupons for half-price drinks or food at a favorite bar. The attack comes when the victims visit the Web site seeking the coupons and unwittingly download the malware.

In a new report, Symantec researchers said some hackers are simply co-opting Web pages popular in certain industries, such as the energy sector, and waiting for victims to arrive.

With enough money, focus, malware and social-engineering skills, “anybody can get into anyplace,” Stewart said. “The most careful person is not going to have a defense against it.”

Extra Content

Hackers break into energy technology company

Using Social Media for cyberattack

Part V

Hacking kits fuel cyberspace arms race

(Sohail Al-Jamea)

Tuesday November 13, 2012

Ryan Linn’s hacks into corporate networks have become almost a matter of routine. On one recent morning, he woke up at his home near the Research Triangle in eastern North Carolina and walked down to an extra bedroom that he uses as an office.

He sat at a workbench laden with computers, signed on to one of them and loaded a program called Metasploit. Sipping a Diet Coke, Linn typed out a few commands and casually launched an attack on a network thousands of miles away. A few seconds later, a report came back: The network had been penetrated. How would he like to proceed?

Chalk up another one for Metasploit, an automated tool kit that makes breaking into networks almost as easy for experienced hackers as ordering food off an online menu.

Metasploit and a host of similar tools are becoming as commonplace for many hackers as Firefox and Microsoft Office are for regular computer users.

They are part of an escalating arms race in cyberspace, where millions of attacks and intrusions occur every day. By prepackaging the myriad computer commands that penetrate and exploit target networks, hackers have dramatically eased the process.

Security researchers and consultants, including Linn, use such hacking tools to identify vulnerabilities and help organizations patch them. Bad-guy hackers, known as black hats, and cyberwarriors use similar illicit kits to spy on, steal from and wreak havoc in corporate and government computers.

Metasploit and many other hacker tool kits are available free to anyone who has an Internet connection.

Linn acknowledges the irony. But he likened Metasploit and other tool kits to a “Swiss army knife” and said the positive features “far outweigh the negatives.”

“Metasploit is a tool designed for researchers and security professionals, but just like many tools there are uses for it that are illegal,” said Linn, a security consultant at Trustwave’s SpiderLabs. “We don’t outlaw screwdrivers and hammers because someone might use them for murder, though. We prosecute those people who use them illegally.”

A researcher named H.D. Moore began working on Metasploit in 2002. Moore, now 31, is the chief security officer with Rapid7, a security firm that sells a commercial version of Metasploit and helps offset the cost of maintaining the free system. A computer researcher and hacker based in Austin, Moore wanted to simplify the development of computer hacks known as exploits. To keep pace with growing numbers of criminal cyberattacks, he wanted to make security hacking, or “penetration testing,” more systematic.

Metasploit works by creating ready-made packages of computer code, known as “modules,” that can be downloaded from metasploit.com. Once they are launched, the tools can find network vulnerabilities and take control of the systems.

Metasploit also serves as something of a global clearinghouse of hacker knowledge, tools and practices. Because it is an “open source” system, it relies on contributions from experienced hackers. Its popularity has soared during the past several years. Starting with 11 exploits in 2003, Metasploit now has close to 1,000.

About 300 people in at least 20 countries have donated exploits so far. The contributors also collaboratively review the offerings to be sure they work effectively. Moore estimated that about 1 million people downloaded the free version during the past year, with about 5 million since its inception. It appears that about 200,000 penetration testers, including the U.S. military’s cyberwarriors, use it regularly, he said.

No one knows how many bad guys employ Metasploit and similar tools. Fears about that potential have been raised in Germany and elsewhere. But Moore said black-hat hackers typically rely on other tool kits that are less focused on research and more focused on swift, illegal break-ins.

Moore said the fact that criminals, spies and others with ill intent can access Metasploit is a necessary trade-off. To keep Metasploit up to date, hackers have to be able to contribute details about the newest vulnerabilities and attack methods.

An organization that keeps track of known vulnerabilities said it has documented more than 53,000, a number that rises every day.

“All we’re trying to do is put everyone on a level playing field,” Moore said.

‘A taste of things to come’

When Metasploit emerged, even veteran hackers marveled at its design and simplicity. A 2004 presentation about it at Black Hat Las Vegas, the annual hacker conference, was titled “Hacking Like in the Movies,” according to a 290-page online book called “Metasploit Toolkit” by David Maynor and several other security researchers.

“The hall was packed to the gills. People stood in the aisles, and the crowd was spilling over to the main corridor,” the authors wrote. “Applause flowed freely throughout the session, and the consensus was clear, ‘Metasploit had come of age.’ But we should have known better. That was only a taste of things to come.”

The extraordinary thing about Metasploit is the digital architecture that streamlined what had been a laborious process of exploit development. That process invariably involved several steps for anyone, good or bad: the discovery of a software vulnerability; the analysis of the code to see whether the vulnerability could be exploited; the writing of the exploit itself, including the commands that tell a target system to open up to an intruder; and testing to ensure the exploit worked.

With Metasploit, all those steps are already done and packaged together with still other features, including tailor-made “payloads” that take effect and hand over control of a system after a hacker gets in.

Other systems have been created to ride on top of Metasploit and make it even easier to use. One called Armitage was created by Raphael Mudge, who was recently hired under contract by the Defense Advanced Research Projects Agency to develop new cybertools.

“Armitage recommends exploits and will optionally run active checks to tell you which exploits will work,” Mudge said in an Armitage tutorial. “If these options fail, use the Hail Mary attack to unleash Armitage’s smart automatic exploitation against your targets.”

In some cases, Moore said, researchers use the Metasploit framework to apply pressure on software vendors to improve the security of their products. If the vendors neglect to fix a known bug, the researchers write an attack module to spur them to act.

That happened this year when a group of researchers created attack modules for six industrial control systems, the computers that operate the power grid, water plants and other critical infrastructure.

“It forces the security vendors to take that vulnerability seriously,” Moore said. “And it forces the vendors responsible for that software to provide a patch or a work-around.”

Alan Paller, director of research at the Sans Institute, one of the world’s leading cybersecurity training organizations, said Metasploit contributors are playing a crucial role in highlighting the pervasive vulnerabilities in systems throughout cyberspace.

“They solve a critical problem for us,” Paller said. “They are necessary tools right now when much of the world is still in denial.”

No one knows how many illicit attack kits are sold to black-hat hackers. Offers appear every day across the Internet. Moore said exploit kits that employ “botnets” in criminal schemes often sell for up to $10,000.

A botnet is a network of computers that have been infected by malicious software and are controlled by bad guys. They often send spam, but they are also used to send malicious code, or malware, in coordinated attacks on networks.

Moore said that in several cases, the bad guys have used botnets to attack Metasploit as punishment for spurring fixes to widely attacked vulnerabilities.

“We do a good job killing bugs,” Moore said. “When the Metasploit adds a new attack, it instantly raises the visibility of that vulnerability.”

Many options

Robin Jackson sat in his Helena, Mont., office and prepared to launch his next hack. The target: a Chinese company’s Web site.

Jackson is a security researcher for a firm called WT Forensics. He said he also participates in informal networks of hacker-intelligence specialists who try to keep watch on the black hats and cyberwarriors across the globe.

He described his China effort as an exploratory “gray hat” hack to see if the target company’s Web page was vulnerable. He decided he would use a set of commands to make his attack seem as though it were coming from a computer in London. To penetrate the Web server, he would turn to the collection of tool kits he keeps on his computer.

In addition to Metasploit, Jackson relies on a number of other automated attack kits almost every day to do his job. There are many of them: Nmap scans the configuration of networks. John the Ripper and Hashcat crack passwords. The Social Engineering Toolkit combines automation with manipulation techniques to help hackers trick people into giving them access to networks.

A host of commercial systems, including a premium version of Metasploit, make it possible to attack multiple client machines at a time. A firm called Immunity, maker of a security tool kit called Canvas, recently released a related commercial system called Swarm. It enables security researchers to scan and attack up to a million servers an hour.

For this exploratory mission, Jackson decided to use a more focused free tool called Havij. With a few clicks on his keyboard, he directed Havij at the targeted Internet address in China. He typed “%Inject Here%” to launch the program.

Havij has been built to send thousands of permutations of commands to implement something known as an SQL Injection attack. Havij would keep hammering the targeted Web server until it sent a command that slipped by the server’s security.

A few years ago, Jackson would have had to type each attack command by hand. With Havij, he can launch the attack, sip his coffee and wait. “Unlike the manual process, Havij automatically does everything seamlessly and much more quickly,” he said.

For all their benefits, Jackson said, the kits are lowering the barriers to entry for inexperienced hackers. Criminal hackers and “hacktivists” can simply download the tool kit and then watch an instructional video on YouTube to get started.

Members of the hacktivists group Anonymous have used the system to target police and military networks. A group called Team GhostShell relied on it to compromise hundreds of Chinese Web sites.

“The Internet not only enables the distribution of hacking tools, but it also offers the hands-on instruction and training on how to use these,” Jackson said. “There are literally thousands upon thousands of videos . . . which show the neophyte how to install and use these tools.”

Extra Content

Hacker Menu
Washington Post live cybersecurity conference

Part VI

A virtual battleground for Pentagon’s cyberwarriors

(Patterson Clark and Robert O’Harrow Jr.)

November 27, 2012 Tuesday

CyberCity has all the makings of a regular town. There’s a bank, a hospital and a power plant. A train station operates near a water tower. The coffee shop offers free WiFi.

But only certain people can get in: government hackers preparing for battles in cyberspace.

The town is a virtual place that exists only on computer networks run by a New Jersey-based security firm working under contract with the Air Force. Computers simulate communications and operations, including e-mail, heating systems, a railroad and an online social networking site dubbed FaceSpace.

Think of it as something like the mock desert towns that were constructed at military facilities to help American soldiers train for the war in Iraq. But here, the soldier-hackers from the Air Force and other branches of the military will practice attacking and defending the computers and networks that run the theoretical town. In one scenario, they will attempt to take control of a speeding train containing weapons of mass destruction.

To those who participate in the practice missions, the digital activity will look and feel real. The “city” will have more than 15,000 “people” who have e-mail accounts, work passwords and bank deposits. The power plant has employees. The hospital has patients. The coffee shop’s customers will come and go, using the insecure WiFi system, just as in real life.

To reinforce the real-world consequences of cyberattacks, CyberCity will have a tabletop scale model of the town, including an electric train, a water tower and a miniature traffic light that will show when they have been attacked.

“It might look to some people like a toy or game,” Ed Skoudis, founder of Counter Hack, the security firm in central New Jersey that is developing the project, said recently while giving a reporter a tour of the fledgling system. “But cyberwarriors will learn from it.”

CyberCity provides insight into some of the Pentagon’s closely guarded plans for cyberwar. It also reflects the government’s growing fears about the vulnerabilities of the computers that run the nation’s critical infrastructure. Last month, Defense Secretary Leon E. Panetta said that digital attacks “could be as destructive as the terrorist attack on 9/11″ and virtually paralyze the country.

“If a crippling cyberattack were launched against our nation, the American people must be protected,” he said. “And if the commander in chief orders a response, the Defense Department must be ready to obey that order and to act.”

Behind those fears is an unsettling reality: Networks in the United States will remain vulnerable to attacks for some time to come because no one understands cyberspace well enough to ensure security.

In the four decades since the Internet began, most cybersecurity research has been conducted on the fly or as an afterthought, according to interviews with security specialists and computer scientists. Now, with the world linking up its communications, infrastructure, military, banking, medical and other systems at a lightning pace, the dynamic of cyberspace has grown too complex. Rigorous scientific experimentation that might lead to security breakthroughs is only beginning.

In the meantime, attackers hold a huge advantage. They can choose the time, place and method of strikes. Defenders almost always have to settle for reacting, making fixes after the damage has been done.

CyberCity aims to prepare government hackers to hold their own until long-term solutions can be found.

“The problem is the bad guys are getting better much faster than we are,” Skoudis said. “We don’t want to fall further behind on this.”

Realistic virtual environments

CyberCity is one of hundreds of virtual environments - often known as cyber ranges or test beds - launched in recent years by military, corporate and academic researchers to confront the mind-bending security challenges posed by cyberspace, where millions of attacks or intrusions occur every day.

Some small ranges study the effects of malicious software and viruses. Some hope to emulate the Internet itself and become scientific instruments of sorts, akin to mountaintop telescopes or particle accelerators, that will enable researchers to seek out the elusive fundamentals of cyberspace. The most ambitious of these, the National Cyber Range, was developed by the Defense Advanced Research Projects Agency. It has cost about $130 million since 2008. The agency said seven large-scale experiments have been conducted by Pentagon researchers.

Creating realistic virtual environments is extraordinarily challenging. In cyberspace, a global network of networks, more than 2 billion people interact with at least 12 billion computers and devices, including global positioning systems, mobile phones, satellites, data routers, ordinary desktop computers and industrial control computers that run power plants, water systems and more.

In many cyber ranges, the simulated Web servers, routers, mobile phones and other network devices operate essentially as they do in the real world, but they have few if any physical components. The virtual devices simply exist as computer code.

Merit Network Inc., a nonprofit technology group in Michigan, just launched a cyber range at Eastern Michigan University that promises to conduct “live fire” exercises. The Defense Department runs the Information Assurance Range in Stafford County, Va. It gives cyberwarriors a safe, closed environment to practice intrusions and security testing.

In Hampshire, England, and Millersville, Md., Northrop Grumman runs cyber ranges that allow corporate and government clients in Britain and the United States to create models of their own networks and employee activity. Northrop officials liken their systems to flight simulators.

Christopher Valentino, a research and development director in the cyberintelligence division of Northrop Grumman Information Systems, said one key to a successful range is closely approximating the way human psychology plays out on real networks.

“It’s very hard to find ‘normal,’ “ he said.

The University of Southern California’s Information Sciences Institute operates the Defense Technology Experimental Research (DETER) project, one of the most ambitious research ranges in the world. It is driven by 500 computers and funded by the Department of Homeland Security and the National Science Foundation. It aspires to become a leader in “cyber-security experimental science.”

“The development of a science of cybersecurity could take decades,” Fred B. Schneider, the Samuel B. Eckert professor of computer science at Cornell University and a Pentagon adviser, wrote recently in “The Next Wave,” a nonclassified publication of the National Security Agency. “The sooner we get started, the sooner we will have the basis for a principled set of solutions to the cybersecurity challenge before us.”

Network activity of 90 million

Two hundred miles south of Counter Hack and its CyberCity, computer researcher Pat McGarry demonstrated how some powerful cyber ranges attempt to approximate the mix of physics and psychology that rises out of the interaction of billions of people and machines online.

One day this fall, McGarry worked his way through menus on his computer at his home office in Arlington, making a series of choices, like a kid preparing to play a video game. He was setting up a test of a corporate network, using a cyber range in a “box.”

The box, made by his company, Ixia BreakingPoint, is a digital powerhouse seven inches high and 19 inches wide that strings together the equivalent of 200 computer processors. The boxes start at $100,000, and top-of-the-line machines go for $1.2 million. Customers include the National Security Agency, the Defense Department’s Information Assurance Range, the DETER project and others.

With a click of his mouse, McGarry decided there would be 2 million “people” communicating on his cyber range. With another click, he ordered up much Web browsing and directed those computer users to send e-mail and download videos. (For verisimilitude, the machine generates some e-mails by drawing on real-life sources, such as the short stories of P.G. Wodehouse.) “How the deuce could Jeeves know anything about it?” one passage in a made-up e-mail said.

The firm says that by connecting multiple boxes, it can emulate the network activity of up to 90 million people.

“Think about how cool that is,” McGarry said.

To run his test, McGarry employed a digital model of a corporation’s network. And he selected some standard hacker methods to disable Web sites, steal passwords and find flaws that open the way for intrusions. When he hit enter, the network came alive and the automated attacks began.

As a torrent of traffic began flowing on the network McGarry monitored the attacks. Ten minutes into the test, he saw that the virtual hackers he had unleashed were able to break through firewalls and take control of the network.

Can the hacks be blocked entirely?

“The answer is a clear no,” he said.

Real-world effect of hacking

The idea for CyberCity grew out of conversations that Skoudis had two years ago with senior Air Force officials eager to convey to cyberwarriors the impact that hacking can have on real-world operations such as water plants and power grids.

At the time, the Pentagon had recently declared cyberspace the newest domain of war. U.S. forces also had secretly launched cyberattacks against Iran’s nuclear enrichment facilities, disabling almost 1,000 uranium centrifuges in 2009 and 2010. That attack, disclosed this year, involved a malicious computer “worm” known as Stuxnet. It is the most notable attack on critical infrastructure that has come to light.

Skoudis ran a network-hacking training program called NetWars through the SANS Institute, a leading security organization that has trained thousands of government and civilian employees. Working through SANS, he agreed to create CyberCity for less than $1 million. It would be a modest range with an urgent, focused goal.

“We’re not trying to do a lot of theoretical work here,” Skoudis said. “Our focus is on very practical applications, training cyberwarriors.”

The Air Force believes that training on cyber ranges is a key to keeping pace with changing threats from criminals, terrorists or even nation-states. The practice missions in CyberCity are expected to begin in the next few weeks.

“We are growing our abilities to use cyberspace to our advantage through training and trials in systems such as cyber ranges,” said Maj. Gen. Suzanne Vautrinot, commander of the 24th Air Force and overseer of the service’s cyber operations. “We posture ourselves to move at the ever-changing speed of technology. We are able to do this successfully by providing operationally relevant ranges for operator training and operational test activities.”

This fall, Skoudis and a small team of hackers built CyberCity using the equivalent power of about 50 computers - along with servers in a data center south of Washington that maintains records for the military and intelligence agencies.

The team also is constructing a “kinetic space” - an old-fashioned scale model of the town - to allow the cyberwarriors see evidence of their attacks. They bought the scale-model buildings, trains and other supplies from a hobby shop. With its water tower, train station and low-rise factory building, it resembles towns across New Jersey.

Skoudis stood over the unfinished scale model of CyberCity, picked up the train station and grinned.

“In the future, nearly all military missions will have a cyber component,” he said. “Fingers-on-keyboard experience is vital.”

Five cameras will be mounted around the scale models, providing streaming video of flashing lights and other indicators that the attacks have occurred. Some of the training scenarios sound like movie scripts. Skoudis said they are all plausible.

One scenario requires U.S. government hackers to raise a railroad drawbridge to prevent a train carrying a weapon of mass destruction from entering the city. Another involves a hijacked Navy vessel and plotters who have been communicating on FaceSpace. The mission of the good guys is to hack into FaceSpace and pinpoint the location of the hijackers through WiFi.

Tim Medin, a researcher on the project, recently demonstrated a third attack in the offices of Counter Hack. He typed commands on his laptop and entered CyberCity. His screen showed lines of code and commands. But he may as well have been sitting in the fictive city’s coffee shop. He was about to attack the hospital, using the shop’s free WiFi system.

Medin, playing the role of a foreign special forces operative, was intent on exploiting several computer systems, with the aim of assassinating a VIP at the hospital. In the scenario, cyberwarriors would try to prevent that by gaining control of the network and blocking the attacks.

Foreign intelligence operatives had been following a senior hospital doctor online and in person to prepare for the assault. The attack began when the doctor “entered” his favorite coffee shop and typed his user name and password into the hospital network. Because the WiFi system was open and unprotected, Medin was able to record the password and used it to get into the hospital’s electronic medical records system. Then he launched a ready-made attack - called an SQL Injection - that gave him control of the in-house Web-based prescription system.

Medin discovered that the target was highly allergic to a certain medication. Medin inserted the lethal drug into the target’s daily prescriptions. He said vulnerable software in the health-care industry makes such cyberattacks possible.

“It’s too insecure,” he said.

Extra Content

Practicing for cyberwar

Part VII

Health systems at risk of hacking

(Leah Nash)

Wednesday December 26, 2012

As the health-care industry rushed onto the Internet in search of efficiencies and improved care in recent years, it has exposed a wide array of vulnerable hospital computers and medical devices to hacking, according to documents and interviews.

Security researchers warn that intruders could exploit known gaps to steal patients’ records for use in identity theft schemes and even launch disruptive attacks that could shut down critical hospital systems.

A year-long examination of cybersecurity by The Washington Post has found that health care is among the most vulnerable industries in the country, in part because it lags behind in addressing known problems.

“I have never seen an industry with more gaping security holes,” said Avi Rubin, a computer scientist and technical director of the Information Security Institute at Johns Hopkins University. “If our financial industry regarded security the way the health-care sector does, I would stuff my cash in a mattress under my bed.”

Compared with financial, corporate and military networks, relatively few hacks have been directed at hospitals and other medical facilities. But in recent months, officials with the Department of Homeland Security have expressed growing fear that health care presents an inviting target to activist hackers, cyberwarriors, criminals and terrorists.

“These vulnerabilities may result in possible risks to patient safety and theft or loss of medical information,” a DHS intelligence bulletin said in May.

Security researchers are starting to turn up the same kinds of trivial-seeming flaws that earlier opened the way for hackers to penetrate financial services networks, Pentagon systems and computers at firms such as Google.

Rubin has documented the routine failure to fix known software flaws in aging technology and a culture in which physicians, nurses and other health-care workers sidestep basic security measures, such as passwords, in favor of convenience.

Another researcher found that a system used to operate an electronic medicine cabinet for hospital prescriptions in Oklahoma could be easily taken over by unauthorized users because of weaknesses in the software interface.

OpenEMR, an open-source electronic medical records management system that is about to be adopted worldwide by the Peace Corps, has scores of security flaws that make it easy prey for hackers.

The University of Chicago medical center operated an unsecure Dropbox site for new residents managing patient care through their iPads, using a single user name and password published in a manual online.

After a Post reporter called about the vulnerabilities, officials at the cabinet manufacturer and the medical center took steps to close the gaps. The Peace Corps said it was considering changes.

Government oversight and industry practices have not kept pace with the changing technology. The Food and Drug Administration, which is responsible for overseeing medical devices, most recently published guidance on cybersecurity in 2005.

The agency has urged hospitals to allow vendors to guide them on security of sophisticated devices. But the vendors sometimes tell hospitals that they cannot update FDA-approved systems, leaving those systems open to potential attacks. In fact, the agency encourages such updates.

“A lot of people are very confused about FDA’s position on this,” said John Murray Jr., a software compliance expert at the agency.

A Government Accountability Office report in August noted that defibrillators and insulin pumps are vulnerable to hacks. In July, one researcher-hacker was able for the first time to use a specialized search engine called Shodan to discover a medical device, a wireless patient glucose monitor in Wisconsin, linked to the Internet and open to hacking.

The Department of Health and Human Services is overseeing the move to electronic health records systems, some of which have documented security vulnerabilities.

John Halamka, a physician and Harvard University professor who is co-chairman of the HHS health information technology standards committee, said security in the health-care industry is “not as good” as in other industries. But he added that the industry is aware of the problems and is scrambling to make improvements.

“It’s completely headed in the right direction,” he said.

But Laurie Williams, a computer scientist at North Carolina State University, said health care remains widely vulnerable.

“There are basic, basic, Security 101 vulnerabilities we identified,” said Williams, who was among a team of researchers that identified numerous security flaws in several electronic heath records systems two years ago. “I’m concerned that at some point the hackers are really going to begin exploiting them. And that’s going to be a scary day.”

A lingering issue

Questions about the cybersecurity of medical systems have been simmering for more than a decade. But the issue has intensified as hospitals embrace wireless devices and electronic records. Some health-care officials assumed that their networks were too obscure, or offered too few financial enticements, to be of interest to hackers.

Information technology executive Peter Tippett, the chief medical officer for Verizon, said the threat from cyberspace should not be overstated. Simple theft of laptops and other devices make up the bulk of incidents.

“The fact is, there aren’t many attacks,” said Tippett, who oversees ICSA Labs, an independent division of Verizon that tests electronic health records systems and other security products for government certification. “The bad guys so far at least have been looking for money.”

Still, Tippett acknowledged that health care ranks near “the bottom of the list” of industries in terms of cybersecurity. “It’s about like retail,” he said.

In July, a consortium of hospitals, health plans, pharmacies, drug companies and government agencies called the Health Information Trust Alliance launched a cybersecurity incident response and coordination center to defend against “cyber crime, cyber espionage and cyber activism.”

No one knows exactly how many intrusions have occurred, but anecdotes are mounting. Medical devices at Veterans Affairs facilities were infected by malicious viruses at least 181 times from 2009 to 2011, according to the DHS intelligence report that surfaced in May.

On March 30, a hacker broke into a network server at the Utah Health Department, gained access to Medicaid data about 780,000 people and stole an undetermined number of records. Authorities traced attackers to computers in Eastern Europe. Utah officials acknowledged the breach and said they are taking extensive measures to protect patients against identity theft.

HHS officials said health-care providers must combine cultural, practical and technological solutions to defend against theft and hacking. The officials also said that they have ramped up enforcement efforts against organizations that failed to protect patient information.

“While there is always more work to do, we have reached record settlements against companies who violated privacy laws and sent a message to everyone that privacy violations will not be tolerated,” said Leon Rodriguez, director of the HHS Office for Civil Rights.

‘A pipeline for attackers’

Three years ago, Rubin, the Johns Hopkins researcher, began assessing systems at major hospitals and clinics, making visits to operating rooms and intensive-care units.

He found that doctors and medical workers used the same computers to connect to both the Internet and internal networks. Rubin said doctors become “a pipeline for attackers into the sensitive networks.”

One nurse told Rubin that she had the job of typing in a physician’s password constantly so that the doctor would not have to, leaving the unattended machine unprotected. “She literally walked around the room logging the doctor into every machine, every hour,” Rubin said. “Unbelievable.”

He declined to name the institutions he studied because to do so would violate his research agreements.

“The doctors and technicians I spoke with seemed mostly well aware that their systems are vulnerable,” said Rubin, who has previously found security problems in voting machines. He said that health care “is an industry with the least regard, understanding and respect for IT security of any I’ve seen, and they have some of the most personal and sensitive information of anyone.”

Another researcher, Tim Elrod, a consultant at FishNet Security, found vulnerabilities in a system that enables care providers using a Web browser to automatically dispense drugs from a secure cabinet produced by Omnicell.

Working with Stefan Morris, Elrod discovered that unauthorized users could sidestep the login and password page and gain control of a cabinet at a hospital run by Integris Health, the largest health organization in Oklahoma. They used a well-known hacking technique called a “forced browsing” attack.

“At that point, we had full administrative control,” Elrod said. “We could do anything.”

After being contacted by The Post, Peter Fisher, vice president of engineering at Omnicell, said he “is launching an immediate investigation into this reported vulnerability.” The same day, the company issued a software fix to customers around the globe.

“Omnicell is committed to delivering the highest level of data security to our customers as demonstrated by our regular release of software updates, which include security enhancements,” Fisher said.

John Delano, chief information officer for Integris, confirmed the Omnicell flaw and said his company last year disconnected it from any networks that might link to the Internet.

“Unfortunately, a lot of times you run into vendors who have poorly coded software,” Delano said. “That’s the case here.”

After an inquiry by The Post, a researcher at the University of Florida, Shawn Merdinger, found flaws in the use of wireless iPads by new medical residents at the University of Chicago medical center.

Merdinger found a manual for the iPad initiative posted online, publishing a single user name and password for all the residents to use a shared Dropbox account. The idea was to promote collaboration.

But the arrangement opened the medical center to “social engineering” attacks, where hackers plant documents, such as PDFs, that are loaded with malicious code. Once the documents are uploaded, the iPads could become infected, handing over control of hospital networks to hackers.

After The Post alerted the medical center, officials closed the gap.

“This Dropbox account was intended to be used only to share educational material among residents,” Cindy Kitching-Pena, director of the Department of Medicine Education Programs, said in a statement. “Nevertheless, the username and password to the account have been changed, and the account will be terminated.”

Known vulnerabilities

In February 2009, Congress mandated the widespread adoption of electronic health records (EHR) computer systems as part of the stimulus legislation known as the American Recovery and Reinvestment Act. The law included as much as $36 billion in stimulus funding to promote the “meaningful use” of such systems. It was the Obama administration’s first big step toward health-care reform.

Since then, tens of thousands of doctors, hospitals and other health-care operations have received more than $8.1 billion in government payments, and they have begun using the systems to digitize and share millions of patients’ records in ways that proponents say will save billions of dollars and improve care.

The law required electronic health records systems to be certified by independent labs to meet an array of standards established by HHS, but those standards include few security provisions, according to documents and interviews.

Officials have known for years about vulnerabilities in the systems. In 2007, the eHealth Vulnerability Reporting Program, a group that included senior health-care officials, concluded that “commercial EHR systems are vulnerable to exploitation given existing industry practices” and that the “skill level required to exploit is low.”

Two years ago, Williams, the North Carolina State researcher, and her colleagues found common flaws in four systems that would expose users’ login information and enable outsiders to access patients’ records.

The group’s report urged rigorous security testing before electronic health record vendors could be certified for stimulus funding.

Federal officials have not gone that far, but Farzad Mostashari, the national coordinator for health information technology at HHS, said they “have taken important steps with vendors to make electronic health records more secure,” such as requiring encryption of data on laptops.

Among the systems that HHS has certified is OpenEMR, an open-source software developed by a nonprofit charitable group called OEMR. The software can be downloaded for free.

Williams’s group - along with several white-hat hackers - has found hundreds of vulnerabilities in the system.

OEMR’s leaders acknowledged the flaws but said it would take an experienced hacker to exploit them. Chief technology officer Kevin Yeh said his group fixes problems as soon as it learns about them and that other Web-based systems probably have the same weaknesses.

He added that federal certification standards “are not sufficient.”

About the Author

Robert O’Harrow Jr. is a reporter on the Investigative Unit of the Washington Post. During two decades at The Post, he has won multiple journalism awards, including the top prize from Investigative Reporters and Editors for a series about government contracting fraud, waste and abuse. O’Harrow has twice been a Pulitzer Prize finalist, most recently in 2009 for a series of explanatory stories about the global financial crisis. He is author of the 2005 book, “No Place to Hide,” about data profiling, surveillance and national security. He was co-producer of a radio documentary by the same name. In 2003, he won the Carnegie Mellon Cybersecurity Award.

More from The Washington Post

Get the latest news and essential information directly in your inbox from The Washington Post.
Sign up for free e-newsletters and alerts.

Connect with The Washington Post on Twitter @washingtonpost and Facebook.

Learn more about the Washington Post Series:

Connect with Diversion Books

Connect with us for information on new titles and authors from Diversion Books, free excerpts, special promotions and contests, and more:

Twitter: @DiversionBooks

Facebook: www.Facebook.com/DiversionBooks

eNewsletter: Diversion Books eNewsletter