In order to determine how encryption fits into server data protection, consider 4 encryption components on the server side: passwords, tables, partitions and  inter-tier socket communications.

In these 4 components of a application / database server encryption policy, note that some countermeasures are required (for example one-way hashes of passwords, while other such as encrypting specify table columns may or may not be relevant to a particular application).

1. Encrypted password storage

You must encrypt passwords. It’s surprising to me how many Web sites don’t bother encrypting user passwords – See cases Universal Music Portugal where e-mail addresses and clear-text passwords are dumped on Internet.

What is more surprising is the confusion between encryption and hashing.

Don’t use AES for encrypting passwords in your MySQL or Oracle or MS SQL database.  You’ll end up storing the AES key somewhere in the code and an attacker or malicious insider can read the key by opening up one of your application DLLs in Notepad++ and read that key in a jiffy and breach your entire MySQL database with a single SELECT statement.

Database user passwords should be stored as MD5 hashes, so that a user  (such as a DBA) who has been granted SELECT access to the table (typically called ‘users’)  cannot determine the actual password. Make sure that different instances have different salts and include some additional information in the hash.

If you use MD5 encryption for client authentication, make sure that  the client hashes the password with MD5 before sending the data on the network.

2. Encrypt specific database table columns

The PostgreSQL 9.1 pgcrypto module allows certain fields to be stored encrypted. This is especially useful if some of the data is sensitive for example in the case of ePHI where the Web application needs to comply with the CFR 45 Appendix A Security rule. The client software provides the decryption key and the data is decrypted on the server and then sent to the client.  In most cases the client (a database driver in an MVC application such as Ruby on Rails or CakePHP or ASP.NET MVC is also a server side resource and often lives on the same physical server as the database server. This is not a bad thing.

3. Encrypt entire data partitions

Encrypting entire data partitions has its place.

On Linux, encryption can be layered on top of a file system using a ”loopback device”. This allows an entire file system partition to be encrypted on disk, and decrypted by the operating system. Many operating systems support this functionality, including Windows.

Encrypting entire partitions is a security countermeasure for physical attacks, where the entire computer is stolen. Research we did in 2007 indicated that almost 50% of large volume data breaches employed a physical attack vector (stealing a notebook at a hotel checkin desk, hijacking a truck transporting backup tapes to Iron Mountain and smash and grab jobs where thieves know the rent-a-cop walkaround schedule and break in and steal desktop computers.

On the other hand, once the volume is mounted,  the data is visible.

4. Encrypt socket communications between server tiers

SSL has it’s place, although SSL is not a silver bullet countermeasure for Microsoft Windows vulnerabilities and mobile medical devices vulnerabilities as I wrote here, here and here.

SSL connections encrypt all data sent across the network: the password, the queries, and the data returned. In database client-server connections,  relational database systems such as PostgreSQL allow administrators to specify which hosts can use non-encrypted connections (host) and which require SSL-encrypted connections (hostssl). Also, clients can specify that they connect to servers only via SSL. Stunnel or SSH can also be used to encrypt transmissions.

In principle I shut down as many operating services as I can – especially those that call out and/or listen on the Internet but this is supposed to be a development machine with access to our private git repository and sending out email via a Postfix relay.

On our own  small scale of a lab with 6-7 machines for testing network and software security of customer applications, I  got  thinking that most system vulnerabilities live in the cracks of system integration of components and packaged software while most of the industry’s efforts in software security are directed towards new software implementations.

Last year I gave a talk on quantitative methods for estimating operational risk of information systems in the annual European GRC meeting in Lisbon – you can see the presentation below.

As a I noted in my talk, one of the crucial phases in estimating operational risk is data collection: understanding what threats, vulnerabilities you have and understanding not only what assets you have (digital, human, physical, reputational) but also how much they’re worth in dollars.

Many technology people interpret data collection as some automatic process that reads/scans/sniffs/profiles/processes/analyzes/compresses log files, learning and analyzing the data using automated  algorithms like ANN (adaptive neural networks).

The automated log profiling tool will then automagically tell you where you have vulnerabilities and using “an industry best practice database of security countermeasures”,  build you a risk mediation plan. Just throw in a dash of pie charts and you’re good to go with the CFO.

This was in fashion about 10 years ago (Google automated audit log analysis and you’ll see what I mean) for example this reference on automated audit trail analysis,  Automated tools are good for getting a quick indication of trends, and  tend to suffer from poor precision and recall that  improve rapidly when combined with human eyeballs.

The PCI DSS council in Europe (private communication) says that over 80% of the merchants/payment processors with data breaches  discovered their data breach  3 months or more after the event. Yikes.

So why does maintaining 3 years of log files make sense – quoted from PCI DSS 2.0

10.7 Retain audit trail history for at least
one year, with a minimum of three
months immediately available for
analysis (for example, online, archived,
or restorable from back-up).
10.7.a Obtain and examine security policies and procedures and
verify that they include audit log retention policies and require
audit log retention for at least one year.
10.7.b Verify that audit logs are available for at least one year and
processes are in place to immediately restore at least the last
three months’ logs for analysis

Wouldn’t it be a lot smarter to say -

10.1 Maintain a 4 week revolving log with real-time exception reports as measured by no more than 5 exceptional events/day.

10.2 Estimate the financial damage of the 5 exceptional events in a weekly 1/2 meeting between the IT manager, finance manager and security officer.

10.3 Mitigate the most severe threat as measured by implementing 1 new security countermeasure/month (including the DLP and SIEM systems you bought last year but haven’t implemented yet)

I’m a great fan of technology, but the human eye and brain does it best.

It is convenient for a customer to buy a product and feel “secure” but, as businesses become more and more interconnected, as cloud services percolate deeper and deeper into organizations, and as  government compliance regulation becomes more complex and pervasive; the security “problem” becomes more difficult to solve and even harder to sell.

I believe that there are 3 reasons why it’s hard to sell security:

The first is that it’s complex stuff, hard to explain and even harder to build a cost-justified security countermeasure plan and measure security ROI.  The nonsense propagated by security vendors like Symantec and Websense do little to improve the situation and only exacerbate the low level of credibility for security product effectiveness with  pseudo science and ROI calculations written by wet-behind-the-ears English majors marcom people who freelance for security vendors – as I’ve noted in previous posts here, here, here and here.

The second is related to prospect theory. A CEO is risk hungry for a high impact, low probability event (like an attack on his message queuing transaction processing systems) or theft of IP by a competitior and risk averse to low impact, high probability events like malware and garden variety dictionary attacks on every ssh service on the Net.

The third is related to psychology.   Why is it a good idea to cold call a CIO and tell him that the multi-million dollar application his business developed is highly vulnerable?    Admitting that his software is vulnerable and going to the board to ask for big bucks to fix the problem is tantamount to admitting that he didn’t do his job and that someone else should pay the price.  Very bad idea.

This is why cloud services are a hit.

Security is baked into the service. You pay for the computing/storage/messaging resource like you buy electricity. The security is “someone else’s problem”  and let’s face it, the security professionals at Rackspace or Amazon or Google App Engine are better at security than we are. It’s part of their core business.

The next step after cloud services is the security industry evolving into a B2B industry like the automotive or energy industry.  You don’t buy brakes from a McAfee and a car from Checkpoint – you buy a car from GM and brakes are part of the system.

That’s where we need to go – building the security into the product instead of bolting it on as an after-sale extra

Certainly, all the DLP vendors think so.  Only problem is, the PCI DSS Council doesn’t even have DLP in their standard which pretty much guarantees zero regulatory tail wind for DLP sales to payment card industry players.

I’m actually impressed that Symantec didn’t manage to influence the PCI DSS council to include DLP in the standard. An impressive display of professional integrity and technology blindness.

A while back, we did a software security assessment for a player in the online transaction space.

When I asked the client and auditor what kind of real time data loss monitoring they have in place, just in case, they have a bug in their application and/or one of their business partners or trusted insiders steals data, the answers where like “umm, sounds like a good idea but it is not required by PCI DSS 2.0″

And indeed the client is correct.

PCI DSS 2.0 does not require outbound, real time or any other kind of data loss monitoring.

The phrases “real time” and “data loss” don’t appear in the standard. The authors of the standard like file-integrity monitoring but in an informal conversation with a PCI DSS official in the region, he confessed to not being familiar with DLP.

Here are a few PCI  monitoring requirements.

None of these controls directly protect the the payment card from being breached. They are all indirect controls and very focused on external attackers – not on trusted insiders nor business partners.

1. Use file-integrity monitoring or change-detection software on logs to ensure that existing log data cannot be changed without generating alerts (although new data being added should not cause an alert).
2. If automated monitoring of wireless networks is utilized (for example, wireless IDS/IPS, NAC, etc.), verify the configuration will generate alerts to personnel.
3. Deploy file-integrity monitoring tools to alert personnel to unauthorized modification of critical system files, configuration files, or content files; and configure the software to perform critical file comparisons at least weekly.
4. Monitor and analyze security alerts and information, and distribute to appropriate personnel.
5. Verify through observation and review of policies, that designated personnel are available for 24/7 incident response and monitoring coverage for any evidence of unauthorized activity, detection of unauthorized wireless access points, critical IDS alerts, and/or reports of unauthorized critical system or content file changes.

Oh man.

Scott Caldwell tells us that the FDA doesn’t rule “out” or “in” any particular technology, including Windows Embedded.

Having said that, Microsoft has very clear language in their EULA regarding the use of Windows Embedded products:

“The Products are not fault-tolerant and are not designed, manufactured or intended for any use requiring fail-safe performance in which the failure of a Product could lead to death, serious personal injury, severe physical or environmental damage (“High Risk Activities”).”

Medical device vendors  that  use Windows operating systems for less critical devices, or for the user interface are actually increasing the threat surface for a hospital, since any Windows host can be a carrier of malware that can take down the entire hospital network, regardless of it’s primary mission function, be it user-friend UI at a nursing station or intensive care monitor at the bedside.

Medical device vendors that use Microsoft IT systems management “best-practices” often  take the approach of “bolting-on” third party solutions for anti-virus and software distribution instead of developing robust, secure software, “from the ground up” with a secure design, threat analysis, software security assessment and secure software implementation.

Installing third-party security solutions that need to be updated in the field, may be inapplicable to an embedded medical device as the MDA (Medical Device Amendments of 1976) clearly states:

These devices may enter the market only if the FDA reviews their design, labeling, and manufacturing specifications and determines that those specifications provide a reasonable assurance of safety and effectiveness. Manufacturers may not make changes to such devices that would affect safety or effectiveness unless they first seek and obtain permission from the FDA.

It’s common knowledge that medical device technicians use USB flash drives and notebook computers to update medical devices in the hospital. Given that USB devices and Windows computers are notoriously vulnerable to viruses and malware, there is a reasonable threat that a field update may infect the Windows-based medical device. If the medical device is isolated from the rest of hospital network, then the damage is  localized, but if the medical device is networked to an entire segment, then all other Windows based computers on that segment may be infected as well – propagating to the rest of the hospital in a cascade attack.

It’s better to get the software security right than to try and bolt in security after the implementation.Imagine that you had to buy the brakes for a new car and install them yourself after you got that bright new Lexus.

It is not unusual for medical device vendors to fall victim to the same Microsoft marketing messages used with enterprise IT customers – “lower development costs, and faster time to market” when in fact, Windows is so complex and vulnerable that the smallest issue may take a vendor months to solve. For example – try and get Windows XP to load the wireless driver without the shell.   Things that may take months to research and resolve in Windows are often easily solved in Linux with some expertise and a few days work. That’s why you have professional medical device  software security specialists like Software Associates.

With Windows, you get an application up and running quickly, but it is never as reliable and secure as you need.

With Linux, you need expertise to get up and running, and once it works, it will be as reliable and secure as you want.

Yves Rutschle says that outlawing Microsoft Windows from medical devices in hospitatls  sounds too vendor-dependant to be healthy (sic) (Seems to me that this would make the medical device industry LESS vendor-dependent, not more vendor-dependent, considering the number of embedded Linux options out there.)

Yves suggests that instead, the FDA should create a “proper medical device certification cycle. If you lack of inspiration, ask the FAA how they do it, and maybe make the manufacturers financially responsible for any software failure impact, including death of a patient“. (The FDA does not certify medical devices, they grant pre-market approval).

I like a free market approach but consider this:

• Bruce Schneier proposed adopting Federal legislation to make companies pay for security breaches – the proposal never got traction. Today, the Obama administration might adopt the idea as being in the same spirit of penalizing financial service providers for non-compliance.
• If the FDA has premarket approved a medical device, common-law claims for negligence are not an option for consumers. See the Supreme Court ruling “Riegel v. Medtronic “, 2008 -

(Held)The MDA’s pre-emption clause bars common-law claims challenging the safety or effectiveness of a medical device marketed in a form that received premarket approval from the FDA. Pp. 8–17.

Maybe the FDA should learn from the FAA but in the meantime, it seems to me if the FDA pre-market validation process had an item requiring a suitable operating system EULA, that would pretty much solve the problem.

I was standing in line at Ben Gurion airport, waiting for my bag to be x-rayed. A conversation started with a woman standing next to me in line. The usual sort – “Where are you traveling and what kind of work do you do?”. I replied that I was traveling to Warsaw and that I specialize in data security and compliance – helping companies prevent trusted insider theft and abuse of sensitive data.

She said, “well sure, I understand exactly what you mean – you help enforce ethical behavior of people in the organization”.

I stopped for a moment and asked her, hold on – “what kind of business are you in”? She said – “well, I worked in the GSS for years training teams tasked with protecting high echelon politicians and diplomats. I understand totally the notion of enforcing ethical behavior”. And now? I asked. Now, she said, ” I do the same thing, but on my own”.

Let’s call my new friend “Sarah”.

Sarah’s ethical approach was for me, a breath of fresh air. Until that point, I had defined our data security practice as an exercise in data collection, risk analysis and implementation of the appropriate technical security countermeasures to reduce the risk of data breach and abuse. Employees, competitors and malicious attackers are all potential attackers.  The objective is to implement a cost-effective portfolio of data security countermeasures – policies and procedures, software security assessments, network surveillance, data loss prevention (DLP) and encryption at various levels in the network and applications.

I define security as protecting information assets.

Sarah defines security as protecting ethical behavior.

In my approach to data security, employee behavior is an independent variable, something that might be observed but certainly, not something that can be controlled. Since employees, contractors and business partners tend to have their own weaknesses and problems that are not reported on the balanced score card of the company, my strategy for data security posits that it is more effective to monitor data than to monitor employees and prevent unauthorized transfer or modification of data instead of trying to prevent irrational or criminal behavior of people who work in the extended enterprise.

In Sarah’s approach to data security, if you make a set of rules and train and enforce ethical behavior with good management, sensing and a dosage of fear in the workplace; you have cracked the data security problem.

So – who is right here?

Well – we’re both right, I suppose.

The answer is that without asset valuation and analysis of asset vulnerabilities, protecting a single asset class (human resources, data, systems or network) while ignoring others, may be a mistake.

Let’s examine two specific examples in order to test the truth of this statement.

Consider a call center with 500 customer service representatives. They use a centralized CRM application, they have telephones and email connectivity. Each customer service representative has a set of accounts that she handles. A key threat scenario is leaking customer account information to unauthorized people – private investigators, reporters, paparazzi etc… The key asset is customer data but the key vulnerability is the people that breach ethical behavior on the way to breaching customer data.

In the case of customer service representatives breaching customer privacy, Sarah’s strategy of protecting ethical behavior is the best security countermeasure.

Now, consider a medical device company with technology that performs imaging analysis and visualization. The company deploys MRI machines in rural areas and uses the Internet to provided remote expert diagnosis for doctors and patients who do not have access to big city hospitals. The key asset transmitted from the systems for remote diagnosis is PHI (protected health information), and the key vulnerabilities are in the network interfaces, the applications software and operating systems that the medical device company uses.

In  the case of remote data transfer and distributed/integrated systems, a combined strategy of software security, judicious network design and operating system selection (don’t use Microsoft Windows…) is the correct way to protect the data.

My conversation with Sarah at the airport gave me a lot of food for thought.

Data loss prevention (DLP technology) is great  and  ethical employee behavior is crucial but they need to work hand in glove.

Where there are people, there is a need to mandate, monitor and reinforce ethical behavior using  a clearly communicated corporate strategy with employees and contractors. In an environment where users require freedom and flexibility in using applications such as email and search, the ethical behavior for protecting company assets starts with company executives who show from personal example that IT infrastructure is to be used to further the company’s business and improving customer service and not for personal entertainment, gain or gratification.

It’s the simple things in life that count.

As you may have read recently, Cybercrime is now costing the UK $43.5 billion and around $1 trillion globally.

Sponsored by security and compliance auditing vendor nCircle, the Webinar pitch didn’t cite any sources for the $1 trillion number nor the $43.5 billion number.

A little googling revealed the UK government report UK Cyber crime costs UKP 27BN/year. Published on the BBC’s website, the report offers a top-level breakdown of the costs of cybercrime to Britain using a fairly detailed scheme of classification and models. Regardless of how badly UK businesses are hit by cybercrime, there are several extremely weak points in the work done by Detica for the UK government.

a) First  - they don’t have any empirical data on actual cybercrime events.

Given the number of variables and lack of ‘official’ data, our methodology uses a scenario- based approach.

Which is a nice way of saying

the UK government gave us some money to do a study so we put together a fancy model, put our fingers in the air and picked a number.

b) Second – reading through the report, there is a great deal of information relating to fraud of all kinds, including Stuxnet which has nothing to do with the UK cyber crime space. Stuxnet does not seem to have put much of a dent in the Iranian nuclear weapons program although, it has given the American President even more time to hem and haw about Iranian nuclear threats.

What this tells me is that Stuxnet  has become a wakeup call for politicians to the malware threat that has existed for several years. This may be a good thing.

c) Third – the UK study did not interview a single CEO in any of the sectors they covered. This is shoddy research work, no matter how well packaged. I do not know a single CEO and CFO that cannot quantify their potential damage due to cyber crime – given a practical threat model and coached by an expert not a marketing person.

So – who pays the cost of cyber crime?

The consumer (just ask your friends, you’ll get plenty of empirical data).

Retail companies that have a credit card breach incur costs of management attention, legal and PR which can always to leveraged into marketing activities. This is rarely reported in the balance sheet as extraordinary expenses so one may assume that it is part of the cost of doing business.

Tech companies that have an IP breach is a different story and I’ve spoken about that at length on the blog. I believe that small to mid size companies are the hardest hit contrary to the claims made in the UK government study.

I would not venture a guess on total global cost of cyber crime without empirical data.

What gives me confidence that the 1 Trillion number is questionable is that it just happens to be the same number that President Obama and other leaders have used for the cost of IP theft – one could easily blame an Obama staffer for not doing her homework….

If one takes a parallel look at the world of software piracy and product counterfeiting, one sees a similar phenomenon where political and commercial organizations like the OECD and Microsoft have marketing agendas and axes to grind leading to number inflation.

I have written on the problems associated with guessing and rounding up in the area of counterfeiting here  and software piracy.

Getting back to cyber crime, using counterfeiting as a paradigm, one sees clearly that the consumer bears the brunt of the damage – whether it’s having her identity stolen and having to spend the next 6 months rebuilding her life or whether you crash on a mountain bike with fake parts and get killed.

If consumers bear the brunt of the damage, what is the best way to improve consumer data security and safety?

Certainly – not by hyping the numbers of the damage of cyber crime to big business and government. That doesn’t help the consumer.

Then – considering that rapid rollout of new and even sexier consumer devices like the iPad 2, probably not by security awareness campaigns. When one buys an iPhone or iPad, one assumes that the security is built in.

My most practical and cheapest countermeasure to cyber crime (and I will distinctly separate civilian crime from terror ) would be education starting in first grade. Just like they told you how to cross the street, we should be educating our children on open, critical thinking and not talking to strangers anywhere, not on the street and not on FB.

Regarding cyber terror – I have written at length how the Obama administration is clueless on cyber terror

One would hope that in defense of liberty – the Americans and their allies will soon implement more offensive and more creative measures against Islamic and Iranian sponsored cyber terror than stock answers like installing host based intrusion detection on DoD PCs

In a burst of wikipanic, Bank of America has dived into full-on counterespionage mode…15 to 20 bank officials, along with consulting firm Booz Allen Hamilton, will be “scouring thousands of documents in the event that they become public, reviewing every case where a computer has gone missing and hunting for any sign that its systems might have been compromised.”

Interesting that they needed Booz and Hamilton.  I thought Bank of America was a Vontu DLP (now Symantec) customer.  It says something about the technology either not working, being discarded or simply not implemented properly because the Wikileaks announcement was made in October 2009. So it took BoA over a year to respond.  Good luck finding forensics over a year after the leak happened.

This is a good thing for information security consultants and solution providers, especially if it drives companies to invest in DLP. There are some good technologies out there and companies that implement DLP thoughtfully (even if for dubious reasons) will be profiting from the improved visibility into transactions on their network and better protection of IP and customer data.

Ethics of the bank executive aside, it is conceivable (albeit totally speculative), that the Obama administration is behind the Wikileaks disclosures on US banking. It is consistent with the Obama policy that required banks to accept TARP funds and stress testing in order to make the financial institutions more beholden to the Federal government. This is consistent with the State Department cables leak, which also appears (from my vantage point in the Middle East) to be deliberately disclosed to Wikileaks in order further the agenda against the Iranians without coming out and saying so specifically.

• Information Security is Warfare.
• Most of your information security strategy is reactionary with “Penetrate and Patch” methods
• Few implementations address the collection of information about attackers.

The key Elements in Information Security Strategy

I propose to stop reacting and go back on the offensive, with a proactive security strategy based on control, collection, capture and change:

Control: Managing the access of information to and from the network and systems.
Collection: Gathering information about user habits and systems behavior.
Capture: The capture of information from anomalous events on the network.
Change: Adapt the security posture to meet new situations.

By basing both defensive and offensive tactics on these four strategic elements, you can poractively control who accesses your network, collect information about abnormal transactions, capture anomalous events, and adapt your security posture to meet changing situations.

Defensive Information Security Tactics

• Network Access Control.
• Host Access Control.
• Intrusion Prevention Systems
• Data loss prevention (DLP)
• Application firewalls
• Backups

Offensive Information Security Tactics

• Honey Pots and Honey Nets.
• Attacking and auditing your own systems.
• Proactive response to attacks.

Acknowledgement: Christopher Neitzert (Chris@Neitzert.com) who was the first to delve into how to improve information security with a combination of both offensive and defensive tactics.