Category Archives: Security

Doing Your Due Diligence on Security Scanning and Penetration Testing Vendors

All too often, development shops and IT professionals become complacent with depending on packaged scanning solutions or a utility belt of tools to provide security assurance testing of a hosted software solution.  In the past five years, a number of new entrants to the security evaluation and penetration testing market have created some compelling cloud-based solutions to perimeter testing.  These tools, while exceptionally useful for a sanity check of firewall rules, load balancer configurations, and even certain industry best practices in web application development, are starting to create a false sense of security in a number of ways.  As these tools proliferate, infrastructure professionals are becoming increasingly dependent upon their handsomely-crafted reporting about PCI, GLBA, SOX, HIPPA, and all the other regulatory buzzwords that apply to certain industries.  If you’re using these tools, have you considered:

Do you use more than one tool?  If not, and you should, is there any actual overlap between their testing criteria?

There is a certain incestuous phenomenon that develops in any SaaS industry that sees high profit margins: entrepreneurs perceive cloud-based solutions as having a low barrier to entry.  This perception drives new market entrants to cobble together solutions to compete for share in the space.  But are these fly-by-night competitors competitively differentiated from their peers?

Sadly, I have found in practical experience this not to be the case.  Too many times have I have enrolled in a free trial of a tool or actually shelled out for some hot new cloud-based scanning solution to find at best only existing known vulnerabilities are duplicatively reported by this new ‘solution’, with only false positives appearing as the ‘net new’ items to bring to my attention.  Here in lies the rub — when new entrants to this market create competing products, there is an iterative reverse engineering that goes on — they run existing scanning products on the market against websites, check to see those results, and make sure they develop a solution that at least identifies the same issues.

That’s not good at all.  In any given security scan, you may see, perhaps, 20% of the total vulnerabilities a product is capable of finding show up as a problem in a scan target.  Even if you were to scan multiple targets, you may only be seeing mostly the same kinds of issues in each subsequent scan.  Those using this as a methodology to build quick-to-market security scanning solutions are delivering sub-par offerings that may only identify 70% of the vulnerabilities other scanning solutions do.  eEye has put together similar findings in an intriguing report I highly recommend reading.  Investigating the research and development activities of a security scanning provider is an important due diligence step to make sure when you get an “all clear” clean report from a scanning tool, that report actually means something.

How do you judge your security vendor in this regard?  Ask for a listing of all specific vulnerabilities they scan for.  Excellent players in this market will not flinch at giving you this kind of data for two reasons: (1) a list of what they check for isn’t as important as how well and how thoroughly they actually assess each item, and (2) worthwhile vendors are constantly adding new items to the list, so it doesn’t represent any static master blueprint for their product.

Does your tool test more than OWASP vulnerabilities?

The problem with developing security testing tools is in part the over-reliance on the standardization of vulnerability definition and classifications.  While it is helpful to categorize vulnerabilities into conceptually similar groups to create common mitigation strategies and mitigation techniques, too often security vendors focus on OWASP attack classifications as the definitive scope for probative activities.  Don’t get me wrong, these are excellent guides for ensuring the most common types of attacks are covered, but they do not provide a comprehensive test of application security.  Too often the types of testing such as incremental information disclosure, where various pieces of the system provide information that can be used to discern how to attack the system further, are relegated to manual penetration testing instead of codified into scanning criteria.  Path disclosure and path traversal vulnerabilities are a class of incremental information disclosures that are routinely tested for by scanning tools, but they represent only a file-system basis test for this kind of security problem instead of part of a larger approach to the problem through systematic scanning.

Moreover, SaaS providers should consider DoS/DDoS weaknesses as security problems, not just customer relationship or business continuity problems.  These types of attacks can cripple a provider and draw their technical talent to the problem at hand, mitigating the denial of service attack.  During those periods, this can and has recently been used in high-profile fake-outs to either generate so much trash traffic that other attacks and penetrations are difficult to perceive or react to, or to create opportunities for social engineering attacks to succeed with less sophisticated personnel while the big-guns are trying to tackle the bigger attacks.  Until weaknesses that can allow for high-load to easily take down a SaaS application are included as part of vulnerability scanning, this will remain a serious hole in the testing methodology of a security scanning vendor.

So, seeing CVE identifiers and OWASP classifications for reported items is nice from a reporting perspective, and it gives a certain credence to mitigation reports to auditors, but don’t let those lull you into a false sense of security coverage.  Ask your vendor what other types of weaknesses and application vulnerabilities they test for outside of the prescribed standard vulnerability classifications.  Otherwise, you will potentially shield yourself from “script kiddies”, but leave yourself open to targeted attacks and advanced persistent threats that have created embarrassing situations for a number of large institutions in the past year.

What is your mobile strategy?

Native mobile applications are the hot-stuff right now.  Purists tout the HTML5-only route to mobile application development, but mobile web development alone isn’t enough to satisfy Apple to get access to the iOS platform, (since 2008) and consumers still can detect a web app that is merely a browser window and prefer the feature set that comes from native applications, including camera access, accelerometer data, and usage of the physical phone buttons into application navigation.  The native experience is still too nice to pass up to be at the head-of-the-class in your industry.

If you’re a serious player in the SaaS market, you have or will soon have a native mobile application or hybrid-native deliverable. If you’re like most other software development shops, mobile isn’t your forte, but you’ve probably hired specific talent with a mobile skill set to realize whatever your native strategy is.  Are your architects and in-house security professionals giving the same critical eye to native architecture, development, and code review as they are to your web offering?  If you’re honest, the answer is: probably not.

The reason your answer is ‘probably not’ is because it is a whole different technology stack, set of development languages, and testing methodology where the tools you invested in to secure your web application do not apply to your native application development.  This doesn’t mean your native applications are not vulnerable, it means they’re vulnerable in different ways that you don’t even know or are testing for yet.  This should be a wake-up call for enterprise software shops: because a vulnerability exists only on a native platform does not mitigate its seriousness.  It is trivial to spin up a mobile emulator to host a native application and use the power of a desktop or server to exploit that vulnerability on a scale that could cripple a business through disclosure or denial of service.

Your native mobile security scanning strategy should minimally cover two important surface areas:

1. Vulnerabilities in the way the application stores data on the device in memory and on any removable media

2. Vulnerabilities in the underlying API serving the native application

If you’re not considering these, then you probably have not selected a native application security scanning tool checking for these either.

In Conclusion

Security is always a moving target, as fluid as the adaptiveness of the techniques of attackers and the rapid pace of change in technologies they attack.  Don’t treat security scanning and penetration testing as a checklist item for RFP’s or to address auditor’s concerns — understand the surface areas, and understanding the failings of security vendors’ products.  Understand your assessments are valid only in the short-term, and re-evaluation of your vendor mix and their offerings on a continual basis is crucial.  Only then will you be informed and able to make the right decisions to be proactive, instead of reactive, regarding the sustainability of your business.

Leave a comment

Posted by on May 29, 2013 in Security


Thwarting SSL Inspection Proxies

A disturbing trend in corporate IT departments everywhere is the introduction of SSL inspection proxies.  This blog post explores some of the ethical concerns about such proxies and proposes a provider-side technology solution to allow clients to detect their presence and alert end-users.  If you’re well-versed in concepts about HTTPS, SSL/TLS, and PKI, please skip down to the section entitled ‘Proposal’.

For starters, e-commerce and many other uses of the public Internet are only possible because the capability for encryption of messages to exist.  The encryption of information across the World Wide Web is possible through a suite of cryptography technologies and practices known as Public Key Infrastructure (PKI).  Using PKI, servers can offer a “secure” variant of the HTTP protocol, abbreviated as HTTPS.  This variant itself encapsulates other application level protocols, like HTTP, using a transport-layer protocol called Secure Socket Layer (SSL), which as since been superseded by a similar, more secure version, Transport Layer Security (TLS).  Most users of the Internet are familiar with the symbolism common with such secure connections: when a user browses a webpage over HTTPS, usually some visual iconography (usually a padlock) as well as a stark change in the presentation of the page’s location (usually a green indicator) show the end-user that the page was transmitted over HTTPS.

SSL/TLS connections are protected in part by a server certificate stored on the web server.  Website operators purchase these server certificates from a small number of competing companies, called Certificate Authorities (CA’s), that can generate them.  The web browsers we all use are preconfigured to trust certificates that are “signed” by a CA.  The way certificates work in PKI allows certain certificates to sign, or vouch for, other certificates.  For example, when you visit, you see your connection is secure, and if you inspect the message, you can see the server certificate Facebook presents is trusted because it is signed by VeriSign, and VeriSign is a CA that your browser trusts to sign certificates.

So… what is an SSL Inspection Proxy?  Well, there is a long history of employers and other entities using technology to do surveillance of the networks they own.  Most workplace Internet Acceptable Use Policies state clearly that the use of the Internet using company-owned machine and company-paid bandwidth is permitted only for business use, and that the company reserves the right to enforce this policy by monitoring this use.  While employers can easily review and log all unencrypted that flows over their networks, that is any request for a webpage and the returned rendered output, the increasing prevalence of HTTPS as a default has frustrated employers in recent years.  Instead of being able to easily monitor the traffic that traverses their networks, they have had to resort to less-specific ways to infer usage of secure sites, such as DNS recording.

(For those unaware and curious, the domain-name system (DNS) allows client computers to resolve a URL’s name, such as, to its IP address,  DNS traffic is not encrypted, so a network operator can review the requests of any computers to translate these names to IP addresses to infer where they are going.  This is a poor way to survey user activity, however, because many applications and web browsers do something called “DNS pre-caching”, where they will look up name-to-number translations in advance to quickly service user requests, even if the user hasn’t visited the site before.  For instance, if I visited a page that had a link to, even if I never click the link, Google Chrome may look up that IP address translation just in case I ever do in order to look up the page faster.)

So, employers and other network operators are turning to technologies that are ethically questionable, such as Deep Packet Inspection (DPI), which looks into all the application traffic you send to determine what you might be doing, to down right unethical practices of using SSL Inspection Proxies.  Now, I concede I have an opinion here, that SSL Inspection Proxies are evil.  I justify that assertion because an SSL Inspection Proxy causes your web browser to lie to it’s end-user, giving them a false assertion of security.

What exactly are SSL Inspection Proxies?  SSL Inspection Proxies are servers setup to execute a Man-In-The-Middle (MITM) attack on a secure connection, on behalf of your ISP or corporate IT department snoops.  When such a proxy exists on your network, when you make a secure request for, the network redirects your request to the proxy.  The proxy then makes a request to for you, returns the results, and then does something very dirty — it creates a lie in the form of a bogus server certificate.  The proxy will create a false certificate for, sign it with a different CA it has in its software, and hand the response back.  This “lie” happens in two manners:

  1. The proxy presents itself as the server you request, instead of the actual server you requested.
  2. The proxy states the certificate handed back with the page response is a different one than what was actually handed back by that provider, in this case.

This interchange would look like this:

It sounds strange to phrase the activities of your own network as an “attack”, but this type of interaction is precisely that, and it is widely known in the network security industry as a MITM attack.  As you can see, a different certificate is handed back to the end-user’s browser than what in the above image.  Why?  Well, each server certificate that is presented with a response is used to encrypt that data.  Server certificates have what is called a “public key” that everyone knows which unique identifies the certificate, and they also have a “private key”, known only by the web server in this example.  A public key can be used to encrypt information, but only a private key can decrypt it.  Without an SSL Inspection Proxy, that is, what normally happens, when you make a request to, first sends back the public key of the server certificate for its server to your browser.  Your browser uses that public key to encrypt the request for a specific webpage as well as a ‘password’ of sorts, and sends that back to  Then, the server would use its private key to decrypt the request, process it, then use that ‘password’ (called a session key) to send back an encrypted response.  That doesn’t work so well for an inspection proxy, because this SSL/TLS interchange is designed to thwart any interloper from being able to intercept or see the data transmitted back and forth.

The reason an SSL Inspection Proxy sends a different certificate back is so it can see the request the end-user’s browser is making so it knows what to pass on to the actual server as it injects itself as a proxy to this interchange.  Otherwise, once the request came to the proxy, the proxy could not read it, because the proxy wouldn’t have’s private key.  So, instead, it generates a public/private key and makes it appear like it is’s server certificate so it can act on its behalf, and then uses the actual public key of the real server certificate to broker the request on.


The reason an SSL Inspection Proxy can even work is because it signs a fake certificate it creates on-the-fly using a CA certificate trusted by the end user’s browser.  This, sadly, could be a legitimate certificate (called a SubCA certificate), which would allow anyone who purchases a SubCA certificate to create any server certificate they wanted to, and it would appear valid to the end-user’s browser.  Why?  A SubCA certificate is like a regular server certificate, except it can also be used to sign OTHER certificates.  Any system that trusts the CA that created and signed the SubCA certificate would also trust any certificate the SubCA signs.  Because the SubCA certificate is signed by, let’s say, the Diginotar CA, and your web browser is preconfigured to trust that CA, your browser would accept a forged certificate for signed by the SubCA.  Thankfully, SubCA’s are frowned upon and increasingly difficult for any organization to obtain because they do present a real and present danger to the entire certificate-based security ecosystem.

However, as long as the MITM attacker (or, your corporate IT department, in the case of an SSL Inspection Proxy scenario) can coerce your browser to trust the CA used by the proxy, then the proxy can create all the false certificates it wants, sign it with the CA certificate they coerced your computer to trust, and most users would never notice the difference.  All the same visual elements of a secure connection — the green coloration, the padlock icon, and any other indicators made by the browser, would be present.  My proposal to thwart this:

Website operators should publish a hash of the public key of their server certificates (the certificate thumbprint) as a DNS record.  For DNS top-level domains (TLD’s) that are protected with DNSSEC, as long as this DNS record that contains the has for is cryptographically signed, the corporate IT department of local clients nor a network operator could forge a certificate without creating a verifiable breach that clients could check for and then warn to end users.  Of course, browsers would need to be updated to do this kind of verification in the form of a DNS lookup in conjunction with the TLS handshake, but provided their resolvers checked for an additional certificate thumbprint DNS record anyway, this would be a relatively trivial enhancement to make.

EDIT: (April 15, 2013): There is in fact an IETF working group now addressing this proposal, very close to my original proposal! Check out the work of the DNS-based Authentication of Named Entities (DANE) group here: — on February 25, they published a working draft of this proposed resolution as the new “TLSA” record.  Great minds think alike. 🙂

1 Comment

Posted by on September 15, 2012 in Ethical Concerns, Open Standards, Privacy, Security



The Long Overdue Case for Signed Emails

A technology more than a decade-old is routinely ignored by online banking vendors despite a sustained push to find technology that counteract fraud and phishing: S/MIME.  For the unaware, S/MIME is a set of standards that define a way to sign and encrypt e-mail messages using a public key infrastructure (PKI) to either provide a way to prove the identity of the message sender (signing), to encrypt the contents of the message so that only the recipient can view the message (encryption), or both (signing and encrypting).  The use of a PKI scheme to create secure communications is generally implemented with asymmetric sets of public and private keys, where in a signing scenario, the sender of messages makes their public key available to the world which can be used to validate that only the corresponding private key was used to craft a message.

This secure messaging scheme offers a way for financial institutions to digitally prove any communication dressed up to look like it came from the institution in fact was crafted by them.  This technology can both thwart the falsification of the ‘from address’ from which a message appears to be sent as well as ensures the content of the message, it’s integrity, is not compromised by any changing of facts or figures or the introduction of other language, links, or malware by any of the various third-parties that are involved with transferring an e-mail from the origin to the recipient.  The application for financial institutions is obvious in a world where over 95% of all e-mail sent worldwide is spam or a phishing scam.  Such gross abuse of the system threatens to undermine the long-term credibly medium, which, in a “green” or paperless world, is the only cost-effective way many financial institutions have to maintain contact with their customers.

So, if the technology is readily available and the potential benefits are so readily apparent, why hasn’t digital e-mail signatures caught on in the financial services industry?  I believe there are several culprits here:

1. Education. End-users are generally unaware of the concept of “secure e-mail”, since implementing digital signatures from a sender’s perspective requires quite a bit of elbow grease, today colleagues don’t send secure messages between each other.  Moreover, most financial institution employees are equally untrained in the concept of secure e-mail, how it works, and much less, how to explain it to their customers to make it understandable as well as a competitive advantage.  Financial institutions have an opportunity to take a leadership role with digital e-mail signatures, since as one of the most trusted vendors any retail customer will ever have, creating a norm of secure e-mail communications across the industry can drive both education and technology adoption.  Even elderly users and young children understand the importance of the “lock icon” in web broswers before typing in sensitive information such as a social security number, a credit card number, or a password — with proper education, users can learn to demand the same protection afforded by secure e-mail.

2. Lack of Client Support.  Unfortunately, as more users shift from desktop e-mail clients to web-based e-mail clients like Gmail and Yahoo Mail, they lose a number of features in these stripped down, advertising-laden SaaS apps, one of which is often the ability to parse a secure e-mail.  The reasons of this are partially technological (it does take a while to re-invent the same wheel desktop client applications like Outlook and Thunderbird have mastered long ago), partially a lack of demand due to the aforementioend ‘education’ reason, and partially unscrupulous motives of SaaS e-mail providers.  The last point I want to call special attention to because of the significance of the problem:  Providers of SaaS applications “for free” are targeted advertising systems, which have increasingly used not just the profile and behavior of end-users to develop a targeted promotional experience, but the content of their e-mails themselves to understand a user’s preferences.  Supporting S/MIME encryption is counter to the aim of scanning the body of e-mails to determine context, when in a secure e-mail platform, Hotmail for instance, would be unable to peek into messages.  Unfortunately, this deliberate ommission of encryption support in online e-mail clients has meant that digital signatures, the second part of the S/MIME technology, is often also omitted.  In early 2009, Google experimented with adding digital signature functionality to Gmail; however, it was quickly removed after it was implemented.,   If users came to demand secure e-mail communications from their financial institutions, these providers would need to play along.

3. Lack of Provider Support.  It’s no secret most online banking providers have a software offering nearly a decade old, which is increasingly a mishmash of legacy technologies, stitched together with antiquated components and outdated user interfaces to create a fragile, minimally working fabric for an online experience.  Most have never gone back to add functionality to core components, like e-mail dispatch systems to incorporate technologies like S/MIME.  Unfortunately, because their customers who are technologically savvy enough to request such functionality represents a small percentage of their customer base, even over ten years later, other online banking offerings still neglect to incorporate emerging security technologies.  While a bolt-on internet banking system has moved from a “nicety” to a “must have” for large financial services software providers, the continued lack of innovation and continuous improvement in their offers is highly incongruent with the needs of financial institutions in an increasingly connected world where security is paramount.

S/MIME digital e-mail signatures is long over-due in the fight against financial account phishing; however, as a larger theme, financial institutions either need to become better drivers of innovation in stalwart online banking companies to ensure their needs are met in a quickly changing world, or they need to identify the next generation of online banking software providers, who embrace today’s technology climate and incorporate it into their offerings as part of a continual improvement process.

Leave a comment

Posted by on June 16, 2010 in Open Standards, Security