The Hollywood Presbyterian Medical Center (HPMC) in Los Angeles, California was affected by a malware attack on 05 February. What followed was a series of misquoted ransom demands, conflicting response messages, and jumbled reports of the sequence of events. By 15 February, the hospital reported that they had restored services and were back in operation with their electronic medical records system. What was the most striking fact to come out of the story was the acknowledgement that the hospital paid $17,000 in bitcoin to the attacker. We will look deeper into this story. There are many lessons to be learned from what happened.
Related: More from Sophos
Wired magazine had an early report that misquoted the ransom demanded at $3.4 million. The Los Angeles Times also reported that the amount was paid before contacting authorities. After a week of developments, the amount turned out to be lower, and the letter from the Hospital CEO appears to suggest that law enforcement was notified immediately upon the realization that their system was not communicating. There appears to be a problem with the messaging coming from hospital officials and the details briefed by law enforcement. The simple fact that the FBI took over the case signaled how serious the incident became.
Related: More from the LA Times
Local NBC affiliate KNBC quoted the hospital CEO as saying the cyber attack was “clearly not a malicious attack; it was just a random attack”. This has a detrimental effect on the messaging coming from the affected organization. During the response phase of Emergency Management, the public (and affected victims) are looking for truthful, consistent messages from leadership. They will keep in mind the reality that a prominent hospital in an affluent part of the country is likely to be targeted by criminal actors. Trying to stand behind the statement that the attack is “random” strains credulity and runs the risk of making the senior leadership appear to be both unprepared and incapable of dealing with a modern day attack on the infrastructure.
It is worth noting that the hospital staff made the correct call to revert to paper copies and continue to do as much as they could with the “old system” as the response plan was moved through each step. Station KNBC reported around 911 patients were sent to other (unaffected) medical facilities to properly provide the critical services the HPMC systems could not provide. Functionally, the hospital had a good response solution for ensuring there was no loss of life, limb, or eyesight.
“…the payers list is a prime target for additional attacks, since those organizations have already demonstrated their willingness to accede to an attacker’s demands”
– Bob Shaker, Symantec Cyber Security Service
The obvious question that comes from the events that unfolded is what next? If even one hacker has figured out how to shut down a prominent hospital because the criminal knows they can pay, what institution is next? Will they try a larger facility in the future? Will they try something that potentially effects even more lives? The sad reality is that they will. The next question is easier to manage: who? With a serious approach to IT system security, and a dedication to a real Emergency Management Plan, institutions can confidently say “not me”.
The 4 phases of Disaster Response (Emergency Management):
The preparedness phase is very much about planning. Planning key scenarios, planning roles and responsibilities, planning routes for movement, timing how long each process takes, and estimating equipment weight and transportation needs. Preparedness also includes documenting what normal looks like. In the case of HPMC, it also included the ability to document without electronic means. In looking at what HPMC went through regarding the hacker attack on their network, we see clear signs that the hospital recognized and had done some work on a disaster response plan. The hospital’s ability to quickly revert to paper versions of the medical records process shows they had pre-printed reserves of the appropriate forms and documents. Sending patients to other facilities for tests and other care related tasks showed that they had identified related and similar care facilities in the area and pre-planned the routes. In short, they demonstrated that they were very good at the physical care of the patient.
The electronic side of the emergency response plan appears to have been less well developed. As mentioned earlier, there are indications from law enforcement sources that the ransom paid to the hacker was completed before the hospital notified police. Information security company TrustedSec was interviewed by CBS News in the reporting of this story. Their CEO observed that the quick payment is an indication the backup process for electronic medical records was not properly managed. Hospital leadership has been insistent that no medical records were compromised. If even one record was encrypted without the express consent of the hospital staff or leadership, this is in fact a compromise of medical records. There is almost a sense of having “dodged the bullet” with this incident. That is incorrect. There is a $17,000 loss that will undoubtedly come from an insurance settlement. That still constitutes a real cost and loss to the overall system.
Related: CBS News reporting
Electronic records are recommended to be backed up on a regular schedule. Obviously the details of when and how that schedule is carried out must remain confidential. Backup operations need to be appropriately budgeted for and reviewed on a regular basis. This helps to insure the integrity of the data being backed up and to prevent “storing a copy of the problem”, should a virus or other malware be present in the system. These processes and procedures need to be fully documented and detailed plans developed in the Preparedness Phase of Emergency Management planning. Both the IT department and the staff must have confidence in the system, or it will fail to be used and allow a window of opportunity for attack. A point of vulnerability.
The response phase of Emergency Management is where all the plans, rehearsals, drills, and alert rosters become a reality. This is the phase that ends up being discussed in the media, and can often serve as a major benchmark for future development. Ultimately, success is gauged on managing the human cost, but loss of information and capital assets also effect humans. If all losses are able to be minimized, the response can be assessed to have been adequate. In looking at the HPMC incident, there were some missteps that indicate the response phase may not have been as clearly understood by all levels of leadership. Of particular interest is the management of communications with the public. The hospital clearly has a relatively prestigious location in the US and quickly became newsworthy once the details of the attack began coming out. Journalists often provide a valuable public service in “keeping them honest”, as CNN likes to say. They will fact check public statements against multiple sources and point out any discrepancies. This creates a unique burden on both the legal department (or advisers) and the PR department. Synchronizing the release of details during the response phase will help insure that messaging is accurate, consistent, and timely.
The public statement by the HPMC CEO is admirable in the straightforward way it describes what happened and what responses were taken as a result. It even describes why the hospital chose to pay the ransomware demand of 40 bitcoin ($17,000 USD). The messaging is a little confusing, however when compared to reports citing law enforcement sources familiar with the case. They indicate the hospital paid the ransom amount before notifying law enforcement authorities. This can have repercussions on insurance settlement and incur additional litigation costs as inevitable lawsuits follow in the wake of the incident. In short, every word matters, and the timing of those words matter as well.
Christoper Watson is an Information Technology professional with success in both public and private sector businesses. Technical and business experience is complimented with operational and strategic planning, international contracting work, leadership development and team building competency in challenging physical and political environments. Experience leverages strengths with technology services and providing timely and relevant information to senior executive leadership. Christopher is based out of the Oklahoma City office.