![\"\"](\"https://seanmcelroy.micro.blog/uploads/2026/865e2059d9.jpg\")
Today I passed the GIAC Security Essentials Certification, also known as the GSEC. I passed with a 95% on my first certification attempt, so I thought it might be useful to decompose my thoughts on this one for any who attempt it in the future.Today I passed the GIAC Security Essentials Certification, also known as the GSEC. I passed with a 95% on my first certification attempt, so I thought it might be useful to decompose my thoughts on this one for any who attempt it in the future.
My background is technical - I started my career in software engineering and database performance tuning, moved into engineering leadership roles, and eventually ended up pursuing my interests in cybersecurity, where I have been a CISO at two financial services firms. Yet, I’m still very hands-on during the day, and I recently wrote a QUIC userspace implementation to learn the spec in the evenings. I have previously earned the CISSP and CISM certifications, although these are more leadership and risk management focused credentials that don’t speak much to technical aptitude as it relates to security. For that reason, as well as my personal desire to keep my technical skills sharp while also working at the executive level and leading a team, I decided to apply to, and was accepted into, the SANS Technology Institute’s (STI) Masters of Science in Information Security Engineering program. The first stop along the MSISE journey is the GSEC.
\nAs part of the MSISE program, I pay tuition for a graduate class to gain access to SANS training and the associated GIAC exam which provides me with a grade for my course. This was my very first SANS training and my first GIAC exam. There was an option provided for me to directly challenge the exam since I do have and did recently earn my CISSP, but my STI student advisor kindly recommended I take in the full training experience. I was admittedly reluctant both because I feel I am pretty strong technically and because it would have been slightly cheaper and faster for me to just go straight to the GSEC exam, but the advice was well founded.
\nThe SANS SEC 401 class by Dr. Eric Cole was outstanding. Dr. Cole’s presentation style feels genuine and engaging over the self-paced OnDemand modality I chose. I walked into this content with the preconceived notion that much of this would be review for me, and honestly, a lot of it was for me. This isn’t to state the course is remedial, simply that as a builder of security programs, the concepts and advice aren’t new to me, but some of the technical pieces were. I learned new and useful tools as part of this course, and I could see this as an excellent foundational course for current and aspiring security team members in any organization. Finding high quality training content is exceptionally valuable to me in my day job, and of course for me personally taking this course.
\nAs other GIAC alumni will tell you, since the GSEC is an open-book exam, developing good indexing skills, as others who have recounted their experiences state, is critical. I followed Josh Armentrout’s index format, and I walked into the exam with about 4 pages of indexes I developed throughout the course. Admittedly, the way I learn best is by reading, so I spent my time in SEC 401’s OnDemand video with Dr. Cole on x2 speed and scanning pages in the book as I went along for index-worthy concepts or terms. I did not spend any time highlighting the books or listening to MP3’s, just focusing on the audio and what I was reading. I would finish a ‘day’ at 2x in about 2 nights of my time, devoting about 5 hours a night for a couple of weeks to get through it all with a worthwhile index. There’s no specific tips or tricks to the content – the course syllabus plainly states what will be covered, and that’s the reality of what OnDemand provided. I will say read your entire book. Sometimes key concepts have interesting nuances that end on the back of a page on a trailing paragraph. Don’t skip those.
\nWith my course, in addition to the self-study quizzes in the OnDemand portal - which test the content of SEC401, not the GSEC - I received two GIAC practice tests and the final GIAC exam test ready to schedule. While everything in the OnDemand portal is self-paced, repeatable, and not timed (other than the overall subscription access), the GIAC practice tests are delivered in the same format as the exam - timed, but they also provide explanations for any incorrectly answered questions. The MSISE program has a learning community portal where generous souls who do not use both GIAC practice tests give away their tests to others who want extra shots. While that’s awfully nice of them, and I was tempted to do the same, I found value in taking both practice tests to test and refine the quality of my index. I’m glad I did, and would suggest never to give away a practice test if you feel you could use it to benefit your index or your comprehension of the breadth of the training topics. (Hey, you paid for these practice tests, so you come first.) I took my first practice test as an ‘open internet’ variant where I would quickly Google something to answer the test, but then make sure my notes were fully fleshed out from what external sources could add. My last practice test was ‘closed internet, open book’ to mimic the actual exam experience, and this was a last test of my index for completeness, since that’s all I would have on the test day. Obviously, I carefully read the explanations to anything I answered incorrectly and tuned my notes and did additional readings to make sure I did not repeat any misfires.
\n![\"April](\"https://proxy.duckduckgo.com/iu/?u=https%3A%2F%2Fgeekandbooknerdsite.files.wordpress.com%2F2014%2F04%2Fkid20carrying20books.jpg&f=1\")
Finally, exam day came today! I’m no stranger to these types of tests or Pearson Vue, so the experience was predictable and suitable. It is interesting walking into a Pearson Vue with an armful of books since most exams they test for allow no notes or books. I came in with all six course books, the lab workbook, the network quick reference guide, my index, and a separate page of notes I made about common ports and protocols that were not on the network quick reference guide but were mentioned elsewhere in the course material. I used everything I brought in, if only to take the exam at a ‘leisurely’ pace and spend adequate time double checking my answers.
Unlike the CISSP or CISM which are based on practical experience (with the exception of the CISSP’s strange obsession with fire suppression controls…), the GSEC was much more knowledge-based, specifically on the SEC401 training materials. So, the right answer is less likely to come from things you already know (come on, you don’t really know ALL those nmap switches), but from what you have learned and can recall or find. Arguably, this is a bit more realistic, as aren’t all technical folks somewhat depending on their navigation of StackOverflow or Google-fu? :)
\nIt’s hard to know from the outside whether SEC401 is custom tailored to the GSEC, or whether the GSEC is really testing SEC401, but they fit together like pieces of a puzzle. Answers to questions often came nearly verbatim from the slides, or more often, the narrative, in the SEC401 books I had in tow. That’s not a knock on the SANS content or the GIAC exam - I call this out simply to advise those studying for the GSEC to intimately know the SEC401 material as it is presented in the books. Treat the high-quality OnDemand video as a wonderful supplement, but don’t go light on your reading and indexing of your spiral bound friends. Also, do the labs, and repeat them until you could recognize a screenshot of output to a tool you covered in the curriculum or in a lab. If you couldn’t recognize a screenshot or command well enough by sight, you probably aren’t soaking in the technical material at the level you need to demonstrate competency at the higher end of the spectrum.
\nThis process got me from a 89% on my cavalier run through the first practice test, to a 92% on my second practice test, to a 95% on exam day. There’s really no tricks to doing well on the GSEC or tricks the exam will try to play on you. It is plainly written, very technical, and you would be a fool not to be prepared with the associated SANS training and a well-crafted index before sitting down to make an attempt. (Check out Lesley Carhart’s great post on studying and indexing too, if you have not already.) Even if you might think ‘I know all this’, you probably don’t have the GSEC cinched unless you give it serious attention and a good study.
\nI hope this helps someone out there!
\n", "date_published": "2019-01-30T23:40:28+00:00", "url": "https://blog.seanmcelroy.com/2019/01/30/thoughts-on-passing-the-giac.html", "tags": ["Uncategorized"] }, { "id": "http://seanmcelroy.micro.blog/2019/01/05/despite-doh-and-esni-with.html", "title": "Despite DoH and ESNI, with OCSP, web activity is insecure and not private", "content_html": "Policy fixes like network neutrality are still in play, but these threats aren’t unlikely one-offs that target individuals, they are systemic abuses by technology providers. Technology fixes, though, are seeking to limit the visibility of web activity, such as the names of websites one visits or the content they download, indiscernible to anyone except the requester and the actual website operator.
\nA couple of standards have gained traction to address these weaknesses in DNS and TLS, with proposals termed DNS over HTTPS (DoH) and encrypted SNI (ESNI), respectively.
\nHowever, as the web matured, methods for hosting many different sites on the same server or set of servers took off and there was no longer a 1:1 match for a domain name and a web server. SNI was an extension that lets a client, like a web browser, specify “I want Yahoo.com” so the web site provider could return the correct, unique certificate to setup the channel for Yahoo.com, even though it could also be serving lots of other sites too. However, the “I want Yahoo.com” is exchanged in plain-text before the certificate is provided and before an encrypted channel is established.
\nThat means savvy technology providers could just look here instead of logging DNS requests for similar data on what host names to which a customer is attempting to connect. This is becoming far more viable as HTTPS Everywhere, user agent changes, and free certificate authorities like Lets Encrypt are making ‘secure by default’ the new reality for the web. More TLS means more encryption, but also more consistency in finding hostnames in SNI fields.
\n![\"oscprequest\"](\"https://seanmcelroy.micro.blog/uploads/2026/497f09c087.jpg\")
\nThe request has a one-way hash of the distinguished name and public key in the certificate as well as the serial number of the certificate. Unsalted hashes mean anyone could poll CT logs for all distinguished names, build their own hash lookup dictionary, and then compare this value to their dictionary. However, the unhashed serial number makes this far easier, as many CT logs support direct lookup of certificates by their serial number. In the following screenshot, you can see a trivial lookup to find out my lab virtual machine was connecting out to support.mozilla.org.
\n![\"ctlookup\"](\"https://seanmcelroy.micro.blog/uploads/2026/e2216ec257.jpg\")
\nWhere privacy is a requirement, OCSP transactions exchanged using HTTP MAY be protected using either Transport Layer Security/Secure Socket Layer (TLS/SSL) or some other lower-layer protocol.\nIncorrectly, some presume OCSP must be performed over insecure HTTP to address a address a 'chicken and egg' problem that would arise from trying to validate the certificate of a secure OCSP site to validate the certificate of another secure site. While implementation details could be non-trivial, solutions like pinning the TLS certificates of well-known OCSP responders could address that challenge.\n
It is important, though, to consider that in the cat-and-mouse game of threats to privacy and privacy-protecting technologies, OCSP is a more readily available source of metadata on users as HTTPS adoption increases, CT logs become mandatory and pervasive, and insecure OCSP communications dominate the responder implementations. As other privacy holes are addressed, such as DoH and ESNI, to keep users' Internet activity private, OCSP is a challenge at scale to address as well.
\n", "date_published": "2019-01-05T18:51:28+00:00", "url": "https://blog.seanmcelroy.com/2019/01/05/despite-doh-and-esni-with.html", "tags": ["Uncategorized"] }, { "id": "http://seanmcelroy.micro.blog/2017/06/24/powershell-oneliner-to-find-outbound.html", "title": "PowerShell one-liner to find outbound connectivity via WinRM", "content_html": "In controlled environments, it’s useful to know when outbound connectivity is not restricted to a predefined list of required hosts, as many standards like PCI require. Here’s a helpful one-liner that will query your Active Directory instance for computer accounts that are enabled, and then for each of them try to connect to a site from that machine, as orchestrated by WinRM. If you use this script, just know that you will probably see a sea of errors for machines that connect be reached from your source host via WinRM. My go-to site for testing non-secure HTTP is asdf.com, but you could use anything target and port you desire based on what should not be allowed in your environment. I have changed the snippet below to example.com (which will not work) so I don’t spam the poor soul who runs asdf.com, but you should replace that with google.com or whatever host to which you wish to verify connectivity.
\nInvoke-Command -ComputerName (Get-ADComputer -Filter {Enabled -eq \"True\"}\n -Property Name,Enabled | foreach { $_.Name }) -ScriptBlock\n { Test-NetConnection -Port 80 \"example.com\" | Select TcpTestSucceeded }\nThe output will be dropped into look something like this:
\nTcpTestSucceeded PSComputerName RunspaceId \n ---------------- -------------- ---------- \n True YOUR-HOST-1 d5fd044c-c268-460e-a274-d3253adc8ce2 \n True YOUR-HOST-2 98206f71-80c1-4e7e-a467-fec489c542ee \n False YOUR-HOST-3 d0b6cf57-e833-44a6-a7bb-aebd4d854b5c \n True YOUR-HOST-4 14af618b-1ca7-4c1f-bb56-ce58dbd4af94\n
It’s a great sanity check before an audit or after major changes to your network architecture or security controls. Enjoy!
\n\n
\n
\n", "date_published": "2017-06-24T23:35:28+00:00", "url": "https://blog.seanmcelroy.com/2017/06/24/powershell-oneliner-to-find-outbound.html", "tags": ["Security","Programming","Aside"] }, { "id": "http://seanmcelroy.micro.blog/2017/05/27/sql-injection-with-new-relic.html", "title": "SQL Injection with New Relic [PATCHED]", "content_html": "
![](\"https://docs.newrelic.com/sites/default/files/thumbnails/image/APM-Databases-Slow-query-trace.png\")
\nWhat’s not so awesome is when your APM’s method for retrieving this data creates a SQL injection flaw in your application that wasn’t there before. In October 2016, I became aware of some strange errors when a DBA was trying to load SQL Server trace files into PSSDiag, due to a formatting problem in the trace file itself. Our DBA discovered that unclosed quotation marks were causing problems with PSSDiag loading trace files. So, how could an unclosed quotation mark even be happening? It’s a hallmark of a SQL injection exploit, and so I began digging.
\nIt appeared our ORM (NHibernate at the time) was sending unparameterized queries, and one of the field values had an unescaped quotation mark, which was causing the error in PSSDiag. However, in other cases the same query, unique to an area of our code, would be issued with parameters. Upon further digging, it actually appeared our application was submitting the same query twice, first with the parameterized query version, and a second with parameter values replaced into the query string, sandwiched with a SET SHOWPLAN_ALL. It looked a bit like this:
\nexec sp_executesql N'INSERT INTO dbo.Table (A, B, C) \nVALUES (@p0, @p1, @p2);select SCOPE_IDENTITY()'\n,N'@p0 uniqueidentifier,@p1 uniqueidentifier, @p2 nvarchar(50)'\n,@p0='{Snipped}',@p1='{Snipped}',@p2=N'I don''t even'\nFollowed by:
\nSET SHOWPLAN_ALL ON\nINSERT INTO dbo.Table (A, B, C)\nVALUES ('{Snipped}', '{Snipped}', 'I don't even');select SCOPE_IDENTITY()\nAs you can see in the first example created by NHibernate, the word “don’t” was properly escaped; however, in the subsequent execution, it was not. This second statement is sent by our very same application process, which New Relic will instrument using the ICorProfilerCallback2 profiler hook to retrieve application performance statistics. But it doesn’t just snoop on the process, it actually hijacks database connections to periodically piggyback on their ‘echo’ of requests to retrieve metrics used to populate their slow queries feature. The SET SHOWPLAN_ALL directive causes the subsequent request not actually to return data, but to just return the execution plan.
\n(DBA’s will note this is actually not a reliable way retrieve this data at all, as parameterized queries can and often do have very different query execution plans when parameter sniffing and lopsided column statistics are in play. But that’s how New Relic does it.)
\nThis is pretty bad, because now virtually every user-provided input that is sent to your database, even if programmed using secure programming practices to avoid SQL injection flaws, becomes vulnerable with New Relic is installed with the Slow Queries feature enabled. That being said, New Relic does not send this second ‘show plan’ and repeated statement set for every query. It samples, appending it only onto some executions of any given statement. An attacker attempting to exploit this would not be able to do so consistently; although, repeated attempts on something like the username field of a login screen, which in many systems is likely log to a database table that stores usernames of failed login attempts, would occasionally succeed when the subsequent SHOWPLAN_ALL and unparamaterized version of the original query is injected at the end of the request by New Relic.
\nEven if I don’t agree with it, I understand the position companies take about not proactively issuing CVE’s. However, I do believe software creators must clearly indicate when action is needed by their users to update software they provide to resolve security vulnerabilities. Many IT administrators take the ‘if it’s not broken, don’t update it’ approach to components like the New Relic .NET Agent, and if no security urgency is communicated for an update, it could take months to years for it to be updated in some environments. While some companies may be worried about competitors' narratives or market reactions to self-disclosing, the truth is vulnerabilities will eventually be disclosed anyway, and providing an appropriate amount of disclosure and timely communications for security fixes is a sign of a mature vulnerability management program within a software company.
\nAlso, be sure if you put any mitigation techniques in place that they actually work. We stumbled upon another bug in working around the issue that was subsequently fixed in 6.11.613 where trying to turn off the ‘slow query’ analysis feature per the New Relic documentation did not consistently work.
\nGiven the potential gravity of this issue, I have quietly sat on this for almost 7 months to allow for old versions of this agent to be upgraded by New Relic customers, in the name of responsible disclosure. I have not done any testing on versions of New Relic agents other than the .NET one, but I would implore security researchers to test agents from any APM vendor that collects execution plans as part of their solution for this or similar weaknesses.
\n", "date_published": "2017-05-27T01:10:31+00:00", "url": "https://blog.seanmcelroy.com/2017/05/27/sql-injection-with-new-relic.html", "tags": ["Security"] }, { "id": "http://seanmcelroy.micro.blog/2015/08/25/first-impressions-matter.html", "title": "First Impressions Matter", "content_html": "When it comes to researching vendors, first impressions matter so much. I tend to judge any potential vendor by its sales apparatus, not just because it is the first impression, but because that positioning and interaction will tell you so much more than any press release, executive ‘corporate culture’ communication, or other third-party source of information on financial or industry strength. Things I notice right off the bat that influence my decision to continue engagement or build trust:
\nIn the summer of 2008, I was preparing a large strategic product shift within Myriad Systems, Inc. to unify a suite of ancillary banking productions I had built and managed: remote deposit capture, merchant capture, expedited payments, e-Statements, e-Notices, check imaging, and a one-to-one marketing solution among many others. A key opportunity presented itself in that we had several large and progressive financial institution clients that were interested in what an MSI online banking offering could look like, particularly given the relatively poor user experience in the online banking offerings at the time. This would have completed a big piece of the end-user product portfolio for MSI, and while as daunting as online banking from the ground-up is, it stood to provide substantial strategic value to our whole suite.
\nComputer Services, Inc. began courting MSI and started a full acquisition in August of 2009. It was clear CSI’s intent was to maximize the value of the print and mail operational assets of MSI, but it had little interest in its online banking products other than to preserve existing revenue streams. This disinterest in the strategic vision of the online web applications as a product portfolio was the impetus for me to pursue my personal career interests of building a best-in-breed online banking solution outside of the MSI umbrella.
\nJeff Vetterick and Richard Owens, two industry colleagues that had previously had stints at MSI, reached out when they heard of my desire to continue to build online banking and move on and encouraged me to reach out to Gary Nelson, an acquaintance who was part of the very successful build and sale of Advanced Financial Services to Metavente (an interesting and great story in of itself), who had interest in this as well. After AFS, Gary had many interests and projects, a significant one being part of an idea to build a learning management system that provided tools for schools to impart educational content in an online tool where students would have a fictitious bank account balance and through different learning modules, understand concepts of spending, budgeting, and the time-value of money.
\nWhen I spoke to Gary in September, I found this initiative was in wind-down: the project had exceeded its funding, and only an IT manager had been retained as a temporary contractor to document and turn over all the company’s assets. Gary engaged me as a consultant to perform an analysis of the source code developed by that team to determine if there was any value in it as an asset for sale as the company was closed up. I reviewed the company’s source and patents, but when I started looking at the few cloud VM’s and pulled open the Subversion repository where the source code was to be, I found a shocking lack of value: what did exist were some architectural documents and some demoware in the form of static screens coded into a .NET MVC ‘shell project’ that had no actual implementation or integration of the key concepts around educational content delivery and assessment. Looking back at the Finnovate presentation the team from this company did, I found only that minimal proof of concept presented on stage, but little more complete.
\nThe internal company documentation in the form of ‘wikis’, agile storyboards, and some unorganized developer notes showed no cohesive technical direction or architectural plan. When I began reviewing invoices for consultants and local contractors, a sad picture materialized: I felt Gary and other investors had been somewhat duped by a mixture of technical ineptitude and probably some overbilling greed by people and local development ‘firms’. I delivered the news that what assets I could find and review had little fire-sale value, other than perhaps one patent that had some intrinsic value, but no implementation. I exemplified this situation by opening the source code for the portion of the system that purported to calculate a ‘relationship score’ about how much an end-user understood financial literacy content and how their behavior in their accounts, transactions, and progress in meeting their financial goals; the source code simply ran in an endless empty loop, doing nothing. Demoware.
\nAfter delivering the news to Gary and preparing for whatever my next endeavor would end up being, Gary suggested I reach out to Stephen Bohanon, a consultant with Catalyst Consulting Group who had previous been a high-performing salesperson with AFS. After several discussions, it became clear Gary had an appetite to try a pivot in the financial technology web application space, and both Stephen and I were interested in building a world-class online banking solution - he as a formidably talented sales executive to build relationships and grow the organization, and myself to grow a technical team that would architect and build our next-generation online banking user experience.
\nAnd with no pre-existing source code, and only great ideas, tremendous perseverance, and some money (thanks, Gary!), we founded Alkami.
\n", "date_published": "2015-06-25T19:28:25+00:00", "url": "https://blog.seanmcelroy.com/2015/06/25/alkami-genesis.html", "tags": ["Uncategorized"] }, { "id": "http://seanmcelroy.micro.blog/2014/10/15/security-advisory-for-financial-institutions.html", "title": "Security Advisory for Financial Institutions: POODLE", "content_html": "Yesterday evening, Google made public a new form of attack on encrypted connections between end-users and secure web servers using an old form of encryption technology called SSL 3.0. This attack could permit an attacker who has the ability to physically disrupt or intercept an end-user’s browser communications to execute a “downgrade attack” that would could cause an end-user’s web browser to attempt to use the older SSL 3.0 encryption protocol rather than the newer TLS 1.0 or higher protocols. Once an attacker successfully executed a downgrade attack on an end-user, a “padded oracle” attack could then be attempted to steal user session information such as cookies or security tokens, which could be further used to gain illicit access to an active secure website sessions. This particular flaw is termed the POODLE (Padding Oracle On Downgraded Legacy Encryption) attack. At this time this advisory was authored, US-CERT had not yet published a vulnerability document for release yet, but has reserved advisory number CVE-2014-3566 for its publication, expected today.
\nIt is important to know this is not an attack on the secure server environments that host online banking and other end-user services, but is a form of attack on end-users themselves who are using web browsers that support the older SSL 3.0 encryption protocol. For an attacker to target an end-user, they would need to be able to capture or reliably disrupt the end-user’s web browser connection in specific ways, which would generally limit the scope of this capability to end-user malware or attackers on the user’s local network or that controlled significant portions of the networking infrastructure an end-user was using. Unlike previous security scares in 2014 such as Heartbleed or Shellshock, this attack targets the technology and connection of end-users. The nature of this attack is one of many classes of attacks that exist that target end-users, and is not the only such risk posed to end-users who have an active network attacker specifically targeting them from their local network.
\nThe proper resolution for end-users will be to update their web browsers to versions that have not yet been released that completely disable this older and susceptible SSL 3.0 technology. In the interim, service providers can disable SSL 3.0 support, with the caveat that IE 6 users will no longer be able to access sites with SSL 3.0 without making special settings adjustments in their browser configuration. (But honestly, if you are keeping IE 6 a viable option for your end-users, this is one of many security flaws those issues are subject to). Institutions that run on-premises software systems for their end-users may wish to perform their own analysis of the POODLE SSL 3.0 security advisory and evaluate what, if any, server-side mitigations are available to them as part of their respective network technology stacks.
\nDefense-in-depth is the key to a comprehensive security strategy in today’s fast-developing threat environment. Because of the targeted nature of this type of attack, and its prerequisites for a privileged vantage point to interact with an end-user’s network connection, it does not appear to be a significant threat to online banking and other end-user services, and this information is therefore provided as a precaution and for informational purposes only.
\nAll financial institutions should subscribe to US-CERT security advisories and to monitor the publication of CVE-2014-3566 once released for any further recommendations and best practices. The resolution for end-users of updated versions of Chrome, Firefox, Internet Explorer, and Safari which remove all support for the older SSL 3.0 protocol will be made through their respective vendor release notification channels. For more information from US-CERT once published, refer to the Google whitepaper directly at https://www.openssl.org/~bodo/ssl-poodle.pdf
\n", "date_published": "2014-10-15T14:24:41+00:00", "url": "https://blog.seanmcelroy.com/2014/10/15/security-advisory-for-financial-institutions.html", "tags": ["Security"] }, { "id": "http://seanmcelroy.micro.blog/2014/10/02/alkami-a-retrospective.html", "title": "Alkami: A Retrospective", "content_html": "What a wild and crazy journey the last five years have been.
\nWhen I started this blog in 2009, it was shortly after I had inked a deal with an angel investor and journeyed down the road with him and my other co-founder and established Alkami Technology. Against significant odds, this October marks the five year anniversary of a roller-coaster ride on up, which galvanized Alkami as the clear leader in the online banking space. Before jumping into this endeavor, I was no stranger to walking products from idealization to realization or running enterprise services in a SaaS model. But doing all that against the tremendous downside risks of the start-up world, as the new kid on the block among a world of established, very-well funded competitors has been challenging. Actually, it’s been brutal.
\nReflecting on the past sixty months, I’ve started to pull together my notes from the early days, both before and after founding Alkami, and I will be commemorating this milestone with a series of blog posts on some company history - the why and how, as well as some valuable and hard-learned lessons along the way. No one, no company finds tremendous success spontaneously. While a Inc 500 splash piece on a company might portray success like a serendipitous fairy tale, only through a voracious appetite for risk, an iron stomach for failure, and a committed and skilled team does any great company find its footing. It’s a great feeling to walk into the office every week and see new, fantastic talent we’ve added to our team and forward-leaning designs and concepts in our flagship solution. It’s also a very satisfying one to know your personal efforts and sacrifices made that team and that company possible.
\nThis series of posts will not be a beating of the chest or self-congratulatory account of our accolades. Our work is far from over, and I judge success on a much longer time horizon. But it will be a real account of our origin story, entrepreneurship, missteps and course correction, and moving from start-up to scale-out in a slow sales cycle, highly-regulated industry. It’s one thing to have a hip product idea you incubate through an accelerator and debut on a demo day. It’s a very different thing to bootstrap a firm and an entire platform where you have to answer a few hundred RFP questions to get a prospect to even talk with you, many other steps to get just one sale, and many sales to get that kind of investor attention.
\nThose pieces are now in place and solidifying every day as we take an aggressive product and technical vision to its successful conclusion. I’m honored to have found great working partners, worked (and still mostly continue to work) with some of the most committed and skilled people across a variety of disciplines along the way. As we look back in retrospect at five formative years, I’m eager to chronicle our story and to add others who will extend and craft our bright future. Stay tuned.
\n", "date_published": "2014-10-02T03:00:31+00:00", "url": "https://blog.seanmcelroy.com/2014/10/02/alkami-a-retrospective.html", "tags": ["Uncategorized"] }, { "id": "http://seanmcelroy.micro.blog/2014/09/25/security-advisory-for-financial-institutions.html", "title": "Security Advisory for Financial Institutions: Shell Shock", "content_html": "Please be specifically aware that a patch was provided to close the issue for the original CVE-2014-6271; however, this patch did not sufficiently close the vulnerability. The current iteration of the vulnerability is CVE-2014-7169, and any patches applied to resolve the issue should specifically state they close the issue for CVE-2014-7169. Any devices that are vulnerable and exposed to any untrusted network, such as a vendor-accessible extranet or the public Internet should be considered suspect and isolated and reviewed by a security team due to the ability for “worms”, or automated infect-and-spread scripts that exploit this vulnerability, to quickly affect vulnerable systems in an unattended manner. Any affected devices that contain private keys should have those keys treated as compromised and have those keys reissued per your company’s information security policies regarding key management procedures.
\nFor further reading on this issue:
\n\n", "date_published": "2014-09-25T16:15:34+00:00", "url": "https://blog.seanmcelroy.com/2014/09/25/security-advisory-for-financial-institutions.html", "tags": ["Security"] }, { "id": "http://seanmcelroy.micro.blog/2014/08/07/enduser-credential-security.html", "title": "End-User Credential Security", "content_html": "This week’s announcement that a Russian crime syndicate has amassed 1.2 billion unique usernames and passwords across 420,000 websites would seem like startling news in 72-point font on the front of major newspapers, if it wasn’t sad it was such a commonplace announcement these days. With four more months to go and still higher than the estimated 823 million compromised credentials part of 2013 breaches affecting Adobe to Target, it’s from Black Hat 2014 I find myself thinking about what we as ISV’s, SaaS providers, and security professionals can do to protect users in the wake of advanced persistent threats and organized, well-funded thieves wreaking havoc on the digital identities and real assets of our clients and customers.
\nUnlike Heartbleed or other server-side vulnerabilities, this particular credential siphoning technique obviously targeted users themselves to affect so many sites and at least 542 unique addresses affecting at least half that many unique users. Why are users so vulnerable to credential-stealing malware? To explore this issue, let’s immediately discard a tired refrain inside software houses everywhere: users aren’t dumb. All too often, good application security is watered down to its least secure but most useful denominator for an overabundance of concern that secure applications may frustrate users, lower adoption, and reduce retention and usage. While it is true that the more accessible the Internet becomes, the wider the spectrum the audience that uses it, from the most expertly capable to the ‘last-mile’ of great grandparents, young children, and the technologically unsophisticated. However, this should neither be grounds to dismiss end-user credential security as a concern squarely in service provider’s court to address nor should it be an excuse to fail to provide adequately secure systems. End-user education is our mutual responsibility, even if that means three more screens, additional prompts to confirm identity or action, or an out-of-band verification process. Keeping processes as stupefying simple as possible because our SEO metrics show that’s the way to marginally improve adoption, reduce cart abandonment, or improve site usage times breeds complacency that ends up hurting us all in the long-run.
\nCan we agree that 1FA needs to end? In an isolated world of controlled systems, a username and password combination might have been a fair assertion of identity. Today’s systems, however, are neither controlled or isolated - the same tablets that log into online banking also run Fruit Ninja for our children, and we pass them over without switching out any concept of identity on a device that can save our passwords and represent them without any authentication. Small-business laptops often run without real-time malware scanning software, easily harvesting credentials through keystroke logging, MitM attacks, cookie stealing, and a variety of other commonplace techniques. Username and passwords fail us because they can be saved and cached just as easily as they can be collected and forwarded to command and control servers is Russia or elsewhere. I’m not one of those anarchists advocating ‘death to the password’ (remember Vidoop?), but using knowledge-based challenges (password, out-of-wallet questions, or otherwise) as the sole factor of authentication needs to end. And it needs to end smartly: sending an e-mail ‘out of band’ to an inbox loaded in another tab on the same machine, or an SMS message read by Google Voice in another tab means your ‘2FA’ is really just one factor layered twice instead of two-factor authentication. A few more calls into the call center to help users cope with 2FA will be far cheaper in the long-run than the fallout of a major credential breach that affects your sites users.
\nWe need to also discourage poor password management: allowing users to choose short or non-complex passwords and warning them about their poor choices is no excuse - we should just flatly reject them. At the same time, we need to recognize that forcing users to establish too complex of a password will encourage them to establish a small number of complex passwords and reuse them across more sites. This is one of the largest Achilles’s Heels for end-users: when a compromise of one site does occur, and especially if you have removed the option for users to establish a username not tied to their identity (name, e-mail address, or otherwise), you have made it tremendously easier for those who have gathered credentials from one site to have a much higher likelihood of exploiting them on your site. Instead, we should consider nuances to each of complexity requirements that would make it likely a user would have to generate a different knowledge-based credential for each site. While that in of itself may increase the chance a user would ‘write a password down’, a user who stores all their passwords in a password manager is still arguably more secure than the user who users one password for all websites and never writes it anywhere.
\nFinally, when lists of affected user accounts become available in uploaded databases of raw credentials that are leaked or testable on sites such as https://haveibeenpwned.com/ - ACT. Find out your users that have overlap with compromised credentials on other sites, and proactively flag or lock their accounts or at least message to them to educate and encourage good end-user credential security. We cannot unilaterally force users to improve the security of their credentials, but we can educate them, and we can make certain their eventual folly through our inaction.
\n", "date_published": "2014-08-07T22:30:28+00:00", "url": "https://blog.seanmcelroy.com/2014/08/07/enduser-credential-security.html", "tags": ["Social Responsibility","Security"] }, { "id": "http://seanmcelroy.micro.blog/2014/04/02/when-to-ride-the-service.html", "title": "When to Ride the Service Bus", "content_html": "One of the great things about adding new, senior talent to a storied team working on a large, complex, and successful enterprise application solution is the critical technical review that results in a lot of “why did/didn’t you do it this way?” questions. You have two options for responding to those questions - ignore or passively dismissing them, or taking the questions seriously as a challenge to prove out why you would make a decision you and your team made 5 years ago the same if you had to consider for the first time today, in today’s frameworks, development methodologies, and the current team makeup and skills inventory. If you choose to dismiss these opportunities to critically review your prior decisions, it says a lot about your management style, general appreciation of technology and response to its change, and positions your team to take a reactionary, defensive posture to architecture rather than create a team that honors a proactive, continuous improvement perspective. Far more interesting too are those questions that ask why the system is architected in a general way, rather than a theological debate on whether a particular technology component choice is superior to all over or one’s preferred/familiar choice.
\nThe particular question the new engineer asked was, “Why aren’t we using a service bus?” Instead of answering him directly, I figured this as a good opportunity to explore the previous decision we made that not only did not include an enterprise service bus (ESB) in the original design, but rejected its inclusion when it was strongly suggested by our first customer because they were standardizing on a service bus-centric architecture themselves. The primary advantage of a service bus is to layer an abstraction across heterogeneous systems by implementing a centralized communication mechanism across components. By applying this architectural model, you can get some key benefits including orchestration, queuing to handle intermittent component availability, and extensibility points for message routing to alter dispatch logic or transform messages. Implementing the service bus pattern requires some kind of adapter to be written for each component of the system, either as a local modification to each component or by choosing to standardize on a communication channel provided by the ESB. Even in the latter, usually some minor accommodation is required to allow the ESB to receive and encapsulate the native message for delivery to the destination component. Our first customer was a notable player in the community banking market, and was productizing multiple new SaaS-based web applications that depended on data feeds coming from many different customers. In their scenarios, data was consumed by one application, parsed, and delivered to other applications, which in turn may have created additional data feeds for other products, in a cyclic communication/dependency non-directed graph. Each application was developed by different teams, and there was no unified technology stack adoption - some teams were developing on EJB and Flex, others were pure .NET, and teams generally had the discretion to choose whatever they could argue would solve the job, without a strong technology leader looking to unify the stack for similar applications that delivered CMS and pseudo-online banking functionality using a common input data set.
\nFor this customer, ESB was a solution to a problem - their choices lead to a highly concurrent development process with multiple independent teams - but also supported connecting a heterogeneous environment of interdependent components, each of which accomplished limited objectives. This organization was running red-hot - developing ancillary products to a highly engaged and fanatic client base of community banks, where their limiting factor was their speed of innovation and delivery. By agreeing on a common communication mechanism that ESB could provide, there was something, albeit low-level, to which all teams agreed. In the ‘controlled agile chaos’ they found themselves in, the abstraction bought them flexibility to adapt changing business requirements using orchestration. In theory anyway - they ended up moving much slower than they anticipated, but this wasn’t the fault of ESB. ESB solves two classes of problems. The first is the common use case of large, disparate enterprises looking to marry systems established from the dawn of client-server architectures to the newest Node.js hotness, without having to bend the will of any particular system to the communication conventions of any other, which may prove impossible if both systems are proprietary. This is a common use case for BizTalk, especially in the financial sector. All the other benefits you can name off from a service bus architecture are really secondary advantages to this key objective. The second is the use case that any layer of indirection provides: an abstraction you can use to increase the speed of development when requirements are incomplete or prone to pivot. In each case, you invest in a layer to reduce the cost of future change. This particular customer chose NServiceBus as their message-oriented middleware. We seriously evaluated both the general architectural concepts ESB as well as the particular technology they suggested and came up with a definitive ‘no’ to that choice. While it made a lot of sense for our customer, it did not make sense for us because:
\nThe central design decision we made was that ESB’s provide some great features and that ties you into an ESB, but if we could get those features another way that was just as convenient or more so, we’d prefer the plug-and-play flexibility of leveraging existing solutions for components such as caching and load balancing in the environment our solution operates, or pick those pieces ad-hoc for those concerns rather than pick the best omnibus solution and work around any specific shortcomings for any one of them. In reviewing the current industry literature and blog posts and looking at general trends, it would seem our decision not to marry our solution is generally the path many take when not required to integrate legacy systems as part of an orchestration chain or when using non-HTTP based transport mechanisms. If you’re using one, hopefully it’s for a good and necessary reason! For us, though, we decided not to hop on a service bus that could take us somewhere we already arrived.
\nIn the continuing revelations about the depth to which governments have gone to subjugate global communications in terms of privacy, anonymity, and security on the Internet, one thing is very clear: nothing can be trusted anymore.
\nBefore you wipe this post off as smacking of ‘conspiracy theorist’, take the Snowden revelations disclosed since Christmas, particularly regarding the NSA’s Tailored Access Operations catalog that demonstrates the ways they can violate implicit trust in local hardware by infecting firmware at a level where even reboots and factory ‘resets’ cannot remove the implanted malware, or their “interdiction” of new computers that allow them to install spyware between the time it leaves the factory and arrives at your house. At a broader level, because of the trend in global data movement towards centralizing data transit through a diminishing number of top tier carriers - a trend is eerily similar to wealth inequality in the digital era - governments and pseudo-governmental bodies have found it trivial to exact control with quantum insert attacks. In these sophisticated attacks, malicious entities (which I define for these purposes as those who exploit trust to gain illicit access to a protected system) like the NSA or GCHQ can slipstream rogue servers that mimic trusted public systems such as LinkedIn to gain passwords and assume identities through ephemeral information gathering to attack other systems.
\nConsidering these things, the troubling realization is this is not the failure of the NSA, the GCHQ, the US presidential administration, or the lack of public outrage to demand change. The failure is in the infrastructure of the Internet itself. If anything, these violations of trust simply showcase technical flaws we have chosen not to acknowledge to this point in the larger system’s architecture. Endpoint encryption technologies like SSL became supplanted by forward versions of TLS because of underlying flaws not only in cipher strength, but in protocol assumptions that did not acknowledge all the ways in which the trust of a system or the interconnects between systems could be violated. This is similarly true for BGP, which has seen a number of attacks that allow routers on the Internet to be reprogrammed to shunt traffic to malicious entities that can intercept it: a protocol that trusts anything is vulnerable because nothing can be trusted forever.
\nWhen I state nothing can be trusted, I mean absolutely nothing. Your phone company definitely can’t be trusted - they’ve already been shown to have collapsed to government pressure to give up the keys to their part of the kingdom. The very wires leading into your house can’t be trusted, they could already or someday will be tapped. Your air-gapped laptop can’t be trusted, it’s being hacked with radio waves.
\nBut, individual, private citizens are facing a challenge Hollywood has for years - how do we protect our content? The entertainment industry has been chided for years on its sometimes Draconian attempts to limit use and restrict access to data by implementing encryption and hardware standards that run counter to the kind of free access analog storage mediums, like the VHS and cassette tapes of days of old, provided. Perhaps there are lessons to be learned from their attempts to address the problem of “everything, everybody, and every device is malicious, but we want to talk to everything, everybody, on every device”. One place to draw inspiration is HDCP, a protocol most people except hardcore AV enthusiasts are unaware of that establishes device authentication and encryption across each connection of an HD entertainment system. Who would have thought when your six year old watches Monsters, Inc., those colorful characters are protected by such an advanced scheme on the cord that just runs from your Blu-ray player to your TV?
\nWhile you may not believe in DRM for your DVD’s from a philosophical or fair-use rights perspective, consider the striking difference with this approach: in the OSI model, encryption occurs at Layer 6, on top of many other layers in the system. This is an implicit trust of all layers below it, and this is the assumption violated in the headlines from the Guardian and NY Times that have captured our attention the most lately: on the Internet, he who controls the media layers also controls the host layers. In the HDCP model, the encryption happens more akin to Layer 2, as the protocol expects someone’s going to splice a wire to try to bootleg HBO from their neighbor or illicitly pirate high-quality DVD’s. Today if I gained access to a server closet in a corporate office, there is nothing technologically preventing me from splicing myself into a network connection and copying every packet on the connection. The data that is encrypted on Layer 6 will be very difficult for me to make sense of, but there will be plenty of data that is not encrypted that I can use for nefarious purposes: ARP broadcasts, SIP metadata, DNS replies, and all that insecure HTTP or poorly-secured HTTPS traffic. Even worse, it’s a jumping off point for setting up a MITM attack, such as an SSL Inspection Proxy. Similarly, without media-layer security, savvy attackers with physical access to a server closet or the ability to coerce or hack into the next hop in the network path can go undetected if they redirect your traffic into rogue servers or into malicious networks, and because there is no chained endpoint authentication mechanism on the media-layer, there’s no way for you to know.
\nThese concerns aren’t just theoretical either, and they’re not to protect teenagers' rights to anonymously author provocative and mildly threatening anarchist manifestos. They’re to protect your identity, your money, your family, and your security. Only more will be accessible and controllable on the Internet going forward, and without appropriate protections in place, it won’t just be governments soon who can utilize the assumptions of trust in the Internet’s architecture and implementation for ill, but idealist hacker cabals, organized crime rings, and eventually, anyone with the right script kiddie program to exploit the vulnerabilities once better known and unaddressed.
\nWhy aren’t we protecting financial information or credit card numbers with media-layer security so they’are at least as safe as Mickey Mouse on your HDTV?
\n", "date_published": "2014-02-01T18:04:00+00:00", "url": "https://blog.seanmcelroy.com/2014/02/01/the-wires-cannot-be-trusted.html", "tags": ["Ethical Concerns","Privacy","Social Responsibility","Security"] }, { "id": "http://seanmcelroy.micro.blog/2013/12/04/scaling-enterprise-databasebound-applications-io.html", "title": "Scaling Enterprise Database-Bound Applications: I/O", "content_html": "Most software developers are building business (retail B2C, B2B API’s, or LOB’s), not scientific applications – and that means most are developing I/O-bound, not CPU-bound applications. Specifically, most business applications are creative user or application programming interfaces around relatively mundane CRUD operations on a data store. Even more complex applications that perform data synchronization or novel calculations of co-variance or multivariate regression consume maybe 5% of their time crunching data, and the other 95% of the time retrieving and sending it on.
\nSo, when you design an enterprise application and get past the ideation phase and start scaling out your next-generation game-changing application from a cute demo to a serious and robust application serving millions of requests, why would you bother with refactoring your string concatenation in loops into string builders, aiming for zero-copy, or optimizing for CPU performance? You should not and you should: You should not be optimizing for CPU performance, unless you have optimized all your slow accesses away – and you should be optimizing for CPU performance because hopefully you’ve already squeezed all the blood out of the I/O turnip you can.
\nBut you haven’t. I know you haven’t. You know you haven’t if you are being honest. Have you ever looked at your database queries per second for specific-entity queries? For instance, let’s say a user logs into your enterprise application, and a service on your application tier needs to retrieve the record of a user. That service might call another service to make a record of the user’s login. Then the user navigates to another page in your application 60 seconds later. How many times did any component of your system retrieve the user by their unique identifier? If the answer is, “I don’t know”, you haven’t scratched the surface of scaling an enterprise application, much less my most important axiom of doing so: “Don’t Repeat Requests”.
\nThis is a lot harder than you might think, because enterprise web application development lends itself to repeating requests, and it is not an easy problem to solve, because you are essentially creating state on an application tier for a web tier that hosts a stateless HTTP application protocol. When functionality is segregated into multiple services with distinct responsibilities, there is some duplication of I/O access that happens to fulfill a request that is unavoidable. Unless you and everyone on your team completely understands this disjoint and works collectively to design solutions that do not repeat requests, you will repeat requests as part of the natural design of any system.
\n(I say ‘usually’, because this depends on how you’re using your ORM. If you use your ORM as an expensive way to execute stored procedures, your ORM will be at best a pass-through for database methods and will not give you the benefit of entity caching that could be reused for multiple queries that include that entity as a result. As with all caching, YMMV depending on how you have designed your layers.)
\nOnce you enable caching, measure. Measure how many times you ask for that user record when a user logs in and performs some actions over time. You’ll be amazed that when you view this from a database request level, you will still be asking for the same user over and over again as long as not every component is using the cache for database entities with a consistent cache key. It’s very hard to get right, both from an application configuration and a caching server configuration perspective – do not assume, but do measure.
\nRemember, the most important thing to remember is not to get really fast answers to your repeated questions, but stop asking the same questions over and over again! Caching at the ORM is your tourniquet to stop the bleeding of your performance into database I/O buffers and wait times, but caching at the inter-component request level is critical. Let’s say you have an enterprise web application that retrieves a forecast for a city for a given period of time. The web client makes the request for the locale and date range to your application tier, which translates that into queries of whatever entities comprise your data model. With ORM second-level caching in effect, the next request for the same locale and date range will not ask the question of the database this time, but the answer will come instead from the second-level cache… but stop right there. The question was asked again at a higher level, you’re just answering it in a more intelligent way the second time around.
\nEnterprise web applications need to cache the responses of service requests using a cache key that accounts for the parameters of the request. Hopefully your web application faithfully implements a repository pattern, and if so, you implement a cache into this layer to eliminate repeated requests to the service layer to start with. This is not easy. This is hard because your ORM’s database caching is likely a black box implementation of complex cache expiration logic that performs all sorts of clever tricks to know when an entity has become ‘dirty’ and needs to be retrieved again from the underlying database rather than use the cached copy. If you’re developing business applications, you’re probably not accustomed to being clever at this level, and you will need to spend the time to implement this manually throughout your repository pattern (unless you thought ahead and can add caching as an aspect) and to bust your caches.
\nThis kind of task must be reserved for the architects and most senior engineers who know your system design and inter-dependencies inside and out to avoid data consistency errors. A key point is as long as all data validation logic is performed at the lowest layer under any custom caching work you perform, data consistency errors will at worst create a poor user experience. If you don’t - if you have critical client-side validation that is not mirrored under caching on the service-side of your architecture, you have bigger security risks and other problems than caching, but this will definitely impede your ability to deploy service request caching and scale your application.
\nDo not be afraid to accumulate. Do not be afraid to pass object graphs through method parameters to save I/O. You could think about the call stack as your cache here, and while you shouldn’t load it up as an unnecessarily heavy omnibus object to pass around to any method, and while you definitely should not front-load all your I/O before calling a logical method chain before conditional logic or exception management could make some of the calls unnecessary, intelligently design methods so they don’t take the smallest parameter set possible, but create the best scalability when working in concert.
\nYou should be using LocalStorage to cache and bust non-error responses to AJAX requests to your web services and REST endpoints. Just because you’ve thinned out the pipes from services to the database and from the web tier to the services tier, why stop there? Why continue to allow browsers to repeat requests to your web tier as a user moves back and forth between areas or pages? If you rest on your laurels on a job-well-done, but still repeat unnecessary I/O queries at a level higher up in the chain, then you’ve made your application more performant but not truly scalable – you’ve just shifted the blame.
\nBut, for what you do see, justify each and don’t make excuses for yourself:
\nOh yeah, and optimize query plans. This is important work, but it’s not the outer-most layer of the onion. It’s important to remember the difference between scalability and performance:
\nIt’s work you should do, but you shouldn’t do it first for scalability reasons.
\nThat being said, software developers, do not succumb to the pressure to deliver scalability improvements by reporting true but irrelevant statistics to management.
\nWhile most devs don’t do scientific computing, scaling applications is an empirical task that demands meaningful measurement in a realistic testing context. There is spec document or product owner guidance on improving scalability: you must treat it as a scientific experiment. Observe, hypothesize, have a control (the pre-change measurement), experiment, report data. If you fail to discretely value each change with before and after metrics, you’re just shooting in the dark. Cowboy coding gets teams into scalability messes, not out of them.
\nEspecially, though, don’t give updates on enhancements that you cannot verify improve scalability with before and after numbers. If you fix a problem that doesn’t improve the overall system scalability, which happens often in scalability improvement iterations, highlighting your accomplishments when there is no observable improvement suggest you are either ineffective or not working on the right items. Worse, in crunch times, providing such updates gives a false sense of accomplishment to management. Improving scalability, or performance for that matter, has no done-state. But providing meaningless accomplishment notes to management will accelerate the sense of “we’re done enough”, when in fact, you may not have even identified the most significant issue to your scalability for your particular scenario.
\nAnd if you haven’t, let me do it for you: You’re repeating your requests. Trust me on that one. :-)
\n", "date_published": "2013-12-04T23:49:32+00:00", "url": "https://blog.seanmcelroy.com/2013/12/04/scaling-enterprise-databasebound-applications-io.html", "tags": ["Programming"] }, { "id": "http://seanmcelroy.micro.blog/2013/08/22/a-brief-introduction-to-partofspeech.html", "title": "A Brief Introduction to Part-of-Speech Tagging", "content_html": "A field of computer science that has captured my attention lately is computational linguistics – the inexact science of how to get a computer to understand what you mean. This could be something as futuristic as Matthew Broderick’s battle with the WOPR, or with something more practical, like Siri. Whether it be text entered by a human into a keyboard or something more akin to understanding the very unstructured format of human speech, understanding the meaning behind parsed words is incredibly complex – and to someone like me – fascinating!
\nMy particular interest as of late is parsing – which from a linguistic perspective, means the breaking down of a string of characters into words, their meanings, and stringing them together in a parse tree, where the meanings of individual words as well as the relationships between words is composed into a logical construct that allows higher order functions, such as a personal assistant. Having taken several foreign language classes before, then sitting on the other side of the table as an ESL teacher, I can appreciate the enormous ambiguity and complexity of any language, and much more so English among Germanic languages, as to creating an automated process to parse input into meaningful logical representations. Just being able to discern the meaning of individual words given the multitude of meanings that can be ascribed to any one sequence of characters is quite a challenge.
\nIn this sentence, what is the function of the word beat? Beat functions as either a noun or a verb, but in this context, it is a noun. There are two general schools of thought around assigning a tag as to what part of speech (POS) each word in a sentence functions as – iterative rules-based methods and stochastic methods. In rules-based methods, like Eric Brill’s POS tagger, a priority-based set of rules that set forth language-specific axioms, such as “when a word appears to be a preposition, it is actually a noun if the preceding word is while”. A complex set of these meticulously constructed conditions is used to refine a more course dictionary-style assignment of POS tags.
\n![\"\"](\"https://seanmcelroy.micro.blog/uploads/2026/381cd991de.jpg\")
\nThis continues to be a good candidate for doctorial theses in computer science disciplines.. papers that have caused me to lose too much sleep as of late.
\n![\"\"](\"http://www.linguistik-online.de/50_11/eckerPict/eckerPict29.png\")
“Annabelle is a mean-spirited person. She shot my dog out of spite.”
\nA program could infer “my dog” is a dog belonging to the person providing the text. This has obvious applications in the real world if you can do this, and it has been done before. But, imagine the leap in context that is exponentially harder to overcome when resolving “She”. This requires not only an intra-sentence relationship of noun phrases, possessive pronouns, direct objects, and adverbial clauses, but it also requires the ability to carry context forward from one sentence to the next, building a going “mental map” of people, places, things – and building a profile of them as more information or context is provided. And, if you think that’s not hard enough to define .. imagine the two additional words appended on to this sentence:
\n, she said.
\nThat would to a human indicate dialog, which requires a wholly separate frame of Inception-style reference between contextual frames. The parser is reading text about things which is actually being conveyed by other things – both sets of frames have their own unique, but not necessarily separate, domains and attributes. I’m a very long-way off from ever getting this diversion in my “free time” anywhere close to functioning as advertised… but, then again, that’s what exercises on a weekend are for – not doing, but learning. :)
\n", "date_published": "2013-08-22T05:02:09+00:00", "url": "https://blog.seanmcelroy.com/2013/08/22/a-brief-introduction-to-partofspeech.html", "tags": ["Programming"] }, { "id": "http://seanmcelroy.micro.blog/2013/08/07/robustness-in-programming.html", "title": "Robustness in Programming", "content_html": "(For my regular readers, I know I promised this post would detail ‘a method by which anyone could send me a message securely, without knowing anything else about me other than my e-mail address, in a way I could read online or my mobile device, in a way that no one can subpoena or snoop on in between.’ A tall order, for sure, but still something I am working to complete in an RFC format. In the meantime…)
\nI have the benefit of supporting an engineering group that is seeing tremendous change and growth well past ideation and proof of concept, but at the validation and scaling phases of a product timeline. One observation I’ve made about the many lessons taught and learned as part of this company and product growth spurt have been the misapplication of the Jon Postel’s Robustness Principle. Many technical folks are at least familiar with, but often can quote the adage: “Be conservative in what you do, be liberal in what you accept from others”. Unfortunately, like many good pieces of advice, this is taken out of context when it relates to software development.
\nFirst off, robustness, while it sounds positive, it not a trait you always want. This can be confusing for the uninitiated, considering antonyms of the word include “unfitness” and “weakness”. On a macro-scale, you want a system to be robust; you a product to be robust. However, if you decompose an enterprise software solution into its components, and those pieces into their individual parts, the concerns do not always need to, and in some cases should not, be robust.
\nFor instance, should a security audit log be robust? Imagine a highly secure software application that must carefully log each access attempt to the system. This system is probably designed so that many different components of the system can write data to this log, and imagine the logging system is simple and writes its output to a file. If this particular part of the system were robust, as many developers define it, it must, as well as possible, attempt to accept and log any messages posted to it. However, implemented this way, it is subject to CRLF attacks, whereby a component that can connect to it and insert a delimiter that would allow it to add false entries to the security log. Of course, you developers say, you need to do input checking and not allow such a condition to pass through to the log. I would go much further and state you must be as meticulous as possible about parsing and throwing exceptions or raising errors for as many conditions as possible. Each exception that is not thrown is an implicit assumption, and assumptions are the root cause of 9 out the OWASP Top 10 vulnerabilities in web applications.
\nRobustness can, and is often, an excuse predicated by laziness. Thinking about edge cases and about the assumptions software developers make with each method they write is tedious. It is time consuming. It does not advance a user story along its path in an iteration. It adds no movement towards delivering functionality to your end users. Recognizing and mitigating your incorrect assumptions, however, is an undocumented but critical requirement for the development of every piece of a system that does store, or may ever come in contact with, protected information. Those that rely on the Robustness Principle must not interpret “liberal” to mean “passive” or “permissive”, but rather “extensible”.
\nIn the previous example I posited about a example logging system, consider how such a system could remove assumptions but still be extensible. The number and format of each argument that comprises a log entry should be carefully inspected - if auditing text must be descriptive, then shouldn’t such a system reject a zero or two-character event description? While information systems should be localizable and multilingual, shouldn’t all logs be written in one language and any characters that are not of that language omitted and unique system identifiers within the log languages' character set used instead? If various elements are co-related, such as an account number and a username, shouldn’t they be checked for an association instead of blindly accepting them as stated by the caller? If the log should be chronological, shouldn’t an event specified in the future or too far in the past be rejected? Each of these leading questions exposes a vulnerability a careful assessment of input checking can address, but which is wholly against most developers' interpretations of the Robustness Principle.
\nHowever, robustness is not about taking whatever is given to you, it is about very carefully checking what you get, and if and only if it passes a litany of qualifying checks, accepting it as an answer to an open-ended question, rather than relying on a defined set of responses, when possible. A junior developer might enumerate all the error states he or she can imagine in a set list or “enum”, and only accept that value as valid input to a method. While that’s a form of input checking, it is wholly inextensible, as the next error state any other contributor wishes to add will require a recompile/redeploy of the logging piece, and potentially every other consumer of that component. Robustness need not require all data be free-form, it must simply be written with foresight.
\nPostel, wrote his “law” with reference to TCP implementations, but he never suggested that TCP stack implementers liberally accept TCP segments with such boundless blitheness that they infer the syntax of whatever bits they received, but rather, they should not impose an understanding of the data elements that were not pertinent to the task at hand, nor enforce one specific interpretation of a specification upon upstream callers. And therein lies my second point – robustness is not about disregarding syntax, but about imposing a convention. Robust systems must fail as early and as quickly as possible when syntax, especially, has been violated or cannot be accurately and unambiguously interpreted, or if the context or state of a system is deemed to be invalid for the operation. For instance, if a receives a syntactically valid message but can determine the context is wrong, such as a request for information from a user who lacks an authorization to that data, every conceivable permutation of invalid context should be checked, not fail to consider each in a blasé fashion to leave room for a future feature that may, someday, require an assumption made in the present, if it is ever to be developed. This crosses another threshold beyond extensibility to culpable disregard.
\nIn conclusion, building a robust system requires discretion in interpretation of programming “laws” and “axioms”, and an expert realization that no one-liner assertions were meant by their authors as principles so general to apply to every level of technical scale of the architecture and design of a system. To those who would disagree with me, I would say, then to be “robust” yourself, you have to accept my argument. ;)
\n", "date_published": "2013-08-07T23:43:45+00:00", "url": "https://blog.seanmcelroy.com/2013/08/07/robustness-in-programming.html", "tags": ["Programming"] }, { "id": "http://seanmcelroy.micro.blog/2013/06/20/when-all-you-see-are.html", "title": "When All You See Are Clouds... A Storm Is Brewing", "content_html": "![\"\"](\"http://msnbcmedia.msn.com/j/MSNBC/Components/Photo/_new/130606-NSA-headquarters-tight-730a.380;380;7;70;0.jpg\")
The recent disclosures that the United States Government has violated the 4th amendment of the U. S. Constitution and potentially other international law by building a clandestine program that provides G-Men at the NSA direct taps into every aspect of our digital life - our e-mail, our photos, our phone calls, our entire relationships with other people and even with our spouses, is quite concerning from a technology policy perspective. The fact that the US Government (USG) can by legal authority usurp any part of our recorded life - which is about every moment of our day - highlights several important points to consider:
But before proposing some solutions, we must consider:
\n![\"\"](\"http://www.blogcdn.com/www.engadget.com/media/2009/05/090527-300baud-01.jpg\")
\nI posit an answer devoid of politics or blame, but on an evaluation of the present state of Internet connectivity and e-commerce. Arguably, the Internet has matured into a stable, reliable set of services. The more exciting phase of its development saw a flourishing of ideas much like a digital Cambrian explosion. In its awkward adolescence, connecting to the Internet was akin to performing a complicated rain dance that involved WinSock, dial-up modems, and PPP, sprinkled with roadblocks like busy signals, routine server downtime, and blue screens of death. The rate of change in equipment, protocols, and software was meteoric, and while the World Wide Web existed (what most laypeople consider wholly as “the Internet” today), it was only a small fraction of the myriad of services and channels for information to flow. Connecting to and using the Internet required highly specialized knowledge, which both increased the level of expertise of those developing for and consuming the Internet, while limiting its adoption and appeal - a fact some consider the net’s Golden Age.
\nBut as with all complex technologies, eventually they mature. The rate of innovation slows down as standardization becomes the driving technological force, pushed by market forces. As less popular protocols and methods of exchanging information give way to young but profitable enterprises that push preferred technologies, the Internet became a much more homogeneous experience both in how we connect to and interact with it. This shapes not only the fate of now-obsolete tech, such as UUCP, FINGER, ARCHIE, GOPHER, and a slew of other relics of our digital past, but also influenced the very design of what remains – a great example being identification and encryption.
\nFor the Internet to become a commercializable venue, securing access to money, from online banking to investment portfolio management, to payments, was an essential hurdle to overcome. The solution for the general problem of identity and encryption, centralized SSL certificate authorities providing assurances of trust in a top-down manner, solves the problem specifically for central server webmasters, but not for end-users wishing to enjoy the same access to identity management and encryption technology. So while the beneficiaries like Amazon, eBay, PayPal, and company now had a solution that provided assurance to their users that you could trust their websites belonged to them and that data you exchanged with them was secure, end-users were still left with no ability to control secure communications or identify themselves with each other.
\n![\"\"](\"http://www.naturalnews.com/gallery/articles/NSA-tech-companies-spy-scandal.jpg\")
A final contributing factor I want to point out is that other protocols drifted into oblivion, more functionality was demanded over a more uniform channel – the de facto winner becoming HTTP and the web. Originally a stateless protocol designed for minimal browsing features, the web became a solution for virtually everything, from e-mail (“webmail”), to searching, to file storage (who has even fired up an FTP client in the last year?). This was a big win for service providers, as they, like Yahoo! and later Google, could build entire product suites on just one delivery platform, HTTP, but it was also a big win for consumers, who could throw away all their odd little programs that performed specific tasks, and could just use their web browser for everything – now even Grandma can get involved. A more rich offering of single-shot tech companies were bought up or died out in favor of the oligarchs we know today - Microsoft, Facebook, Google, Twitter, and the like.
Subtly, this also represented a huge shift on where data is stored. Remember Eudora or your Outlook inbox file tied to your computer (in the days of POP3 before IMAP was around)? As our web browser became our interface to the online world, and as we demanded anywhere-accessibility to those services and they data they create or consume, those bits moved off our hard drives and into the nebulous service provider cloud, where data security cannot be guarenteed.
\nThis is meaningful to consider in the context of today’s problem because:
\n![\"\"](\"http://actualdownload.com/pictures/icon/encryption-shield-57903.jpg\"){kind=icon}
Over the next few blog posts, I am going to delve into a number of proposals and thoughts around giving control and security assurances of data back to end-users. These will address points #2 and #4 above as solutions that layer over existing web technologies, not proposals to upend our fundamental usage of the Internet by introducing opaque configuration barriers or whole-new paradigms. End-users should have choice whether their service providers have access to their data in a way that does not require Freenet's darknets or Tor's game-of-telephone style of anonymous but slow onion-routing answer to web browsing. Rather, users should be able to positively identify themselves to the world and be able to access and receive data and access it in a cloud-based application without ever having to give up their data security, not have to trust of the service provider, be independent to access the data on any devices (access the same service securely anywhere), and not have to establish shared secrets (swap passwords or certificates).\nAs a good example, if you want to send a secure e-mail message today, you have three categorical options to do so:
\n\n", "date_published": "2013-06-20T22:06:05+00:00", "url": "https://blog.seanmcelroy.com/2013/06/20/when-all-you-see-are.html", "tags": ["Ethical Concerns","Privacy","Security"] }, { "id": "http://seanmcelroy.micro.blog/2013/05/29/doing-your-due-diligence-on.html", "title": "Doing Your Due Diligence on Security Scanning and Penetration Testing Vendors", "content_html": "
All too often, development shops and IT professionals become complacent with depending on packaged scanning solutions or a utility belt of tools to provide security assurance testing of a hosted software solution. In the past five years, a number of new entrants to the security evaluation and penetration testing market have created some compelling cloud-based solutions to perimeter testing. These tools, while exceptionally useful for a sanity check of firewall rules, load balancer configurations, and even certain industry best practices in web application development, are starting to create a false sense of security in a number of ways. As these tools proliferate, infrastructure professionals are becoming increasingly dependent upon their handsomely-crafted reporting about PCI, GLBA, SOX, HIPPA, and all the other regulatory buzzwords that apply to certain industries. If you’re using these tools, have you considered:
\nSadly, I have found in practical experience this not to be the case. Too many times have I have enrolled in a free trial of a tool or actually shelled out for some hot new cloud-based scanning solution to find at best only existing known vulnerabilities are duplicatively reported by this new ‘solution’, with only false positives appearing as the ‘net new’ items to bring to my attention. Here in lies the rub – when new entrants to this market create competing products, there is an iterative reverse engineering that goes on – they run existing scanning products on the market against websites, check to see those results, and make sure they develop a solution that at least identifies the same issues.
\nThat’s not good at all. In any given security scan, you may see, perhaps, 20% of the total vulnerabilities a product is capable of finding show up as a problem in a scan target. Even if you were to scan multiple targets, you may only be seeing mostly the same kinds of issues in each subsequent scan. Those using this as a methodology to build quick-to-market security scanning solutions are delivering sub-par offerings that may only identify 70% of the vulnerabilities other scanning solutions do. eEye has put together similar findings in an intriguing report I highly recommend reading. Investigating the research and development activities of a security scanning provider is an important due diligence step to make sure when you get an “all clear” clean report from a scanning tool, that report actually means something.
\nHow do you judge your security vendor in this regard? Ask for a listing of all specific vulnerabilities they scan for. Excellent players in this market will not flinch at giving you this kind of data for two reasons: (1) a list of what they check for isn’t as important as how well and how thoroughly they actually assess each item, and (2) worthwhile vendors are constantly adding new items to the list, so it doesn’t represent any static master blueprint for their product.
\nMoreover, SaaS providers should consider DoS/DDoS weaknesses as security problems, not just customer relationship or business continuity problems. These types of attacks can cripple a provider and draw their technical talent to the problem at hand, mitigating the denial of service attack. During those periods, this can and has recently been used in high-profile fake-outs to either generate so much trash traffic that other attacks and penetrations are difficult to perceive or react to, or to create opportunities for social engineering attacks to succeed with less sophisticated personnel while the big-guns are trying to tackle the bigger attacks. Until weaknesses that can allow for high-load to easily take down a SaaS application are included as part of vulnerability scanning, this will remain a serious hole in the testing methodology of a security scanning vendor.
\nSo, seeing CVE identifiers and OWASP classifications for reported items is nice from a reporting perspective, and it gives a certain credence to mitigation reports to auditors, but don’t let those lull you into a false sense of security coverage. Ask your vendor what other types of weaknesses and application vulnerabilities they test for outside of the prescribed standard vulnerability classifications. Otherwise, you will potentially shield yourself from “script kiddies”, but leave yourself open to targeted attacks and advanced persistent threats that have created embarrassing situations for a number of large institutions in the past year.
\nIf you’re a serious player in the SaaS market, you have or will soon have a native mobile application or hybrid-native deliverable. If you’re like most other software development shops, mobile isn’t your forte, but you’ve probably hired specific talent with a mobile skill set to realize whatever your native strategy is. Are your architects and in-house security professionals giving the same critical eye to native architecture, development, and code review as they are to your web offering? If you’re honest, the answer is: probably not.
\nThe reason your answer is ‘probably not’ is because it is a whole different technology stack, set of development languages, and testing methodology where the tools you invested in to secure your web application do not apply to your native application development. This doesn’t mean your native applications are not vulnerable, it means they’re vulnerable in different ways that you don’t even know or are testing for yet. This should be a wake-up call for enterprise software shops: because a vulnerability exists only on a native platform does not mitigate its seriousness. It is trivial to spin up a mobile emulator to host a native application and use the power of a desktop or server to exploit that vulnerability on a scale that could cripple a business through disclosure or denial of service.
\nYour native mobile security scanning strategy should minimally cover two important surface areas:
\nVulnerabilities in the way the application stores data on the device in memory and on any removable media
\nVulnerabilities in the underlying API serving the native application
\nIf you’re not considering these, then you probably have not selected a native application security scanning tool checking for these either.
\nA disturbing trend in corporate IT departments everywhere is the introduction of SSL inspection proxies. This blog post explores some of the ethical concerns about such proxies and proposes a provider-side technology solution to allow clients to detect their presence and alert end-users. If you're well-versed in concepts about HTTPS, SSL/TLS, and PKI, please skip down to the section entitled 'Proposal'.
For starters, e-commerce and many other uses of the public Internet are only possible because the capability for encryption of messages to exist. The encryption of information across the World Wide Web is possible through a suite of cryptography technologies and practices known as Public Key Infrastructure (PKI). Using PKI, servers can offer a \"secure\" variant of the HTTP protocol, abbreviated as HTTPS. This variant itself encapsulates other application level protocols, like HTTP, using a transport-layer protocol called Secure Socket Layer (SSL), which as since been superseded by a similar, more secure version, Transport Layer Security (TLS). Most users of the Internet are familiar with the symbolism common with such secure connections: when a user browses a webpage over HTTPS, usually some visual iconography (usually a padlock) as well as a stark change in the presentation of the page's location (usually a green indicator) show the end-user that the page was transmitted over HTTPS.
SSL/TLS connections are protected in part by a server certificate stored on the web server. Website operators purchase these server certificates from a small number of competing companies, called Certificate Authorities (CA's), that can generate them. The web browsers we all use are preconfigured to trust certificates that are \"signed\" by a CA. The way certificates work in PKI allows certain certificates to sign, or vouch for, other certificates. For example, when you visit Facebook.com, you see your connection is secure, and if you inspect the message, you can see the server certificate Facebook presents is trusted because it is signed by VeriSign, and VeriSign is a CA that your browser trusts to sign certificates.
![\"\"](\"http://blueprintsforsuccess.wordpress.com/wp-content/uploads/2012/11/facebooksigneduntitled.png\" "\\"Secure")
So... what is an SSL Inspection Proxy? Well, there is a long history of employers and other entities using technology to do surveillance of the networks they own. Most workplace Internet Acceptable Use Policies state clearly that the use of the Internet using company-owned machine and company-paid bandwidth is permitted only for business use, and that the company reserves the right to enforce this policy by monitoring this use. While employers can easily review and log all unencrypted that flows over their networks, that is any request for a webpage and the returned rendered output, the increasing prevalence of HTTPS as a default has frustrated employers in recent years. Instead of being able to easily monitor the traffic that traverses their networks, they have had to resort to less-specific ways to infer usage of secure sites, such as DNS recording.
(For those unaware and curious, the domain-name system (DNS) allows client computers to resolve a URL's name, such as Yahoo.com, to its IP address, 72.30.38.140. DNS traffic is not encrypted, so a network operator can review the requests of any computers to translate these names to IP addresses to infer where they are going. This is a poor way to survey user activity, however, because many applications and web browsers do something called \"DNS pre-caching\", where they will look up name-to-number translations in advance to quickly service user requests, even if the user hasn't visited the site before. For instance, if I visited a page that had a link to Playboy.com, even if I never click the link, Google Chrome may look up that IP address translation just in case I ever do in order to look up the page faster.)
So, employers and other network operators are turning to technologies that are ethically questionable, such as Deep Packet Inspection (DPI), which looks into all the application traffic you send to determine what you might be doing, to down right unethical practices of using SSL Inspection Proxies. Now, I concede I have an opinion here, that SSL Inspection Proxies are evil. I justify that assertion because an SSL Inspection Proxy causes your web browser to lie to it's end-user, giving them a false assertion of security.
What exactly are SSL Inspection Proxies? SSL Inspection Proxies are servers setup to execute a Man-In-The-Middle (MITM) attack on a secure connection, on behalf of your ISP or corporate IT department snoops. When such a proxy exists on your network, when you make a secure request for [www.google.com](https://www.google.com), the network redirects your request to the proxy. The proxy then makes a request to [www.google.com](https://www.google.com) for you, returns the results, and then does something very dirty -- it creates a lie in the form of a bogus server certificate. The proxy will create a false certificate for www.google.come, sign it with a different CA it has in its software, and hand the response back. This \"lie\" happens in two manners:
This interchange would look like this:
![\"\"](\"http://blueprintsforsuccess.wordpress.com/wp-content/uploads/2012/11/interceptionproxy.png\" "\\"SSL")
It sounds strange to phrase the activities of your own network as an \"attack\", but this type of interaction is precisely that, and it is widely known in the network security industry as a MITM attack. As you can see, a different certificate is handed back to the end-user's browser than what www.example.com in the above image. Why? Well, each server certificate that is presented with a response is used to encrypt that data. Server certificates have what is called a \"public key\" that everyone knows which unique identifies the certificate, and they also have a \"private key\", known only by the web server in this example. A public key can be used to encrypt information, but only a private key can decrypt it. Without an SSL Inspection Proxy, that is, what normally happens, when you make a request to www.example.com, example.com first sends back the public key of the server certificate for its server to your browser. Your browser uses that public key to encrypt the request for a specific webpage as well as a 'password' of sorts, and sends that back to www.example.com. Then, the server would use its private key to decrypt the request, process it, then use that 'password' (called a session key) to send back an encrypted response. That doesn't work so well for an inspection proxy, because this SSL/TLS interchange is designed to thwart any interloper from being able to intercept or see the data transmitted back and forth.
The reason an SSL Inspection Proxy sends a different certificate back is so it can see the request the end-user's browser is making so it knows what to pass on to the actual server as it injects itself as a proxy to this interchange. Otherwise, once the request came to the proxy, the proxy could not read it, because the proxy wouldn't have www.example.com's private key. So, instead, it generates a public/private key and makes it appear like it is www.example.com's server certificate so it can act on its behalf, and then uses the actual public key of the real server certificate to broker the request on.
Proposal
The reason an SSL Inspection Proxy can even work is because it signs a fake certificate it creates on-the-fly using a CA certificate trusted by the end user's browser. This, sadly, could be a legitimate certificate (called a SubCA certificate), which would allow anyone who purchases a SubCA certificate to create any server certificate they wanted to, and it would appear valid to the end-user's browser. Why? A SubCA certificate is like a regular server certificate, except it can also be used to sign OTHER certificates. Any system that trusts the CA that created and signed the SubCA certificate would also trust any certificate the SubCA signs. Because the SubCA certificate is signed by, let's say, the Diginotar CA, and your web browser is preconfigured to trust that CA, your browser would accept a forged certificate for www.example.com signed by the SubCA. Thankfully, SubCA's are frowned upon and increasingly difficult for any organization to obtain because they do present a real and present danger to the entire certificate-based security ecosystem.
However, as long as the MITM attacker (or, your corporate IT department, in the case of an SSL Inspection Proxy scenario) can coerce your browser to trust the CA used by the proxy, then the proxy can create all the false certificates it wants, sign it with the CA certificate they coerced your computer to trust, and most users would never notice the difference. All the same visual elements of a secure connection -- the green coloration, the padlock icon, and any other indicators made by the browser, would be present. My proposal to thwart this:
Website operators should publish a hash of the public key of their server certificates (the certificate thumbprint) as a DNS record. For DNS top-level domains (TLD's) that are protected with DNSSEC, as long as this DNS record that contains the has for www.example.com is cryptographically signed, the corporate IT department of local clients nor a network operator could forge a certificate without creating a verifiable breach that clients could check for and then warn to end users. Of course, browsers would need to be updated to do this kind of verification in the form of a DNS lookup in conjunction with the TLS handshake, but provided their resolvers checked for an additional certificate thumbprint DNS record anyway, this would be a relatively trivial enhancement to make.
EDIT: (April 15, 2013): There is in fact an IETF working group now addressing this proposal, very close to my original proposal! Check out the work of the DNS-based Authentication of Named Entities (DANE) group here: http://datatracker.ietf.org/wg/dane/ -- on February 25, they published a working draft of this proposed resolution as the new \"TLSA\" record. Great minds think alike. :)
\n", "date_published": "2012-09-15T14:11:39+00:00", "url": "https://blog.seanmcelroy.com/2012/09/15/thwarting-ssl-inspection-proxies.html", "tags": ["Open Standards","Ethical Concerns","Privacy","Security"] }, { "id": "http://seanmcelroy.micro.blog/2012/07/16/cnn-lies-to-every-one.html", "title": "CNN Lies to Every One of Its Web Viewers", "content_html": "When is it okay to flat out lie to your users? I would argue: Never. But the website of one of the world’s most watched sources of news, CNN, does just that.
\nNear the bottom of every article is a section called “We recommend” and “From around the web”. These sections list about six links to other articles either on CNN itself, other Turner properties, or simply as a paid referral service for selected partners. So what’s my beef with this? It’s not the targeted marketing, it’s the outright lie I noticed they make when you hover over any of those links with your mouse.
\nFor some background, I’m a huge dissident against outbound link tracking. It’s fundamentally the same as gluing a GPS tracking device to your forehead and giving a a tracking device to the website you’re visiting. I have a problem with it because I think there is a fundamental freedom that is eroded by this technology - the freedom to consume information without being tracked for doing so. Do I have the right to pick up a magazine and browse through it without giving someone my telephone number? I would say yes – I think it is a natural right to be able to consume information without having your consumption observed.
\nBut my belief here isn’t realistic – tracking basic visitor behavior and consumer preferences is the basic monetization and sustainability model for most of the Web as we know it. So, this world doesn’t mesh with my perfect world, but at least I should know if someone is observing my behavior, right? Observing CNN’s privacy policy one can clearly see the word “link” is referenced twice, once in relation to third-party sites that may cookie you, and once for integration to social media or other partner sites that may have differing privacy policies.
\nOkay, fair enough, therefore I should expect that if I am surfing just CNN’s website, if I disable cookies, and if I turn on my do not track header, I should expect not to be tracked, right? No, and the reason is I cannot find out when I’m still on the CNN site to only stay within it. The reason is CNN has specifically coded it’s site to lie to me about when I’m staying within it or navigating away. For an example, if I were to hover over one example link in these two sections, I see the following in my browser status bar:
\nwww.cnn.com/2012/07/15/sport/jason-kidd-arrested/index.html\nI right-clicked the link in Chrome and copied the URL. Then curiously I noticed the link read differently in the browser status bar when hovering over it, this time reading:\n
[traffic.outbrain.com/network/r...](http://traffic.outbrain.com/network/redir?key=ad68e2a0a57f3eb04e4553bf2e80b6b2&rdid=349349184&type=MVLVS_d/t1_ch&in-site=false&req_id=968ab83e0a0f44e584d8744520d2aea0&agent=blog_JS_rec&recMode=4&reqType=1&wid=100&imgType=0&refPub=0&prs=true&scp=false&version=59070&idx=3)\nYouch, what's that, and why did it change? On closer inspection, by viewing the source of the page, I can see the target href of the link is exactly as reproduced above, going to traffic.outbrain.com. I peeked at some other URL's in the same section that I had not yet left-clicked or right-clicked and noticed this:\n
<a target=\"_self\" href=\"http://www.cnn.com/2012/07/15/sport/jason-kidd-arrested/index.html\" onmousedown=\"this.href='http://traffic.outbrain.com/network/redir?key=10b8398e7c07227c8a8786b1682f1707&rdid=349349184&type=WMV_d/t1_ch&in-site=false&req_id=968ab83e0a0f44e584d8744520d2aea0&agent=blog_JS_rec&recMode=4&reqType=1&wid=100&imgType=0&refPub=0&prs=true&scp=false&version=59070&idx=4';return true;\" onclick=\"javascript:return(true)\">Knicks' Jason Kidd arrested on suspicion of DWI</a>\nAnd herein is the deception -- this piece of inline JavaScript code changes the target of the link at the moment it is clicked to go to the traffic.outbrain.com address. Because target href originally reads to the final destination of the article, hovering over it gives the false impression that my click will directly take me to it. Instead, at the moment I click it, the target href is changed to the potentially unscrupulous third-party, and I have been given no browser notification this would happen prior to my click, and upon traffic.outbrain.com responding, it redirects me back to the original CNN article I initially wanted to view. On a broadband connection, you probably wouldn't even notice the superfluous page load and redirect back to CNN's site. Deceptive!\n
So, sure, why should anyone care? Isn’t this just plumbing, technology, and toolbox of tricks inherit of the Web? Maybe, but the problem here is the lie. You do not lie to your users. Ever. Outbound web tracking is not a web beacon. Web beacons are a different kind of “evil” - usually some JavaScript that opens an IFRAME to a third-party site that issues a cookie to track you; however, web beacons are covered by CNN’s privacy policy, so if they were equivalent, it’s all fair. Web beacons can be simply disabled by turning off third-party cookies in today’s browsers. This is precisely why outbound link tracking is becoming popular - it circumvents the privacy management tools most users have available and have knowledge of. Outbound link tracking is no more insidious than web beacons are, but the implementation of them often lies to the end user about what their action will do (a click in this case). An honest implementation would be to either clearly state in the privacy policy that any links you click may be link tracked or simply not to deceive the user by rewriting the target href the moment they click it to actually go to the link tracking site so the browser status bar is truthful on hover (Twitter’s t.co strategy).
\nWell, at least it’s just CNN at fault here. At least no one else would stoop to such shady tactics. Surely not Google (/url) or Facebook (l.php).. no, definitely not…
\n", "date_published": "2012-07-16T21:41:29+00:00", "url": "https://blog.seanmcelroy.com/2012/07/16/cnn-lies-to-every-one.html", "tags": ["Ethical Concerns","Facebook","Privacy"] }, { "id": "http://seanmcelroy.micro.blog/2012/05/02/the-cost-of-speed.html", "title": "The Cost of Speed", "content_html": "First off, I’m quite dissatisfied with my work.
\nBut then again, isn’t every architect? No matter how fantastically we break down and lay out complex enterprise systems, there’s always something to be dissatisfied with even the best logical designs, physical hardware, business logic, and user experiences. We know well enough that enterprise software development is never complete. Sure, user stories and discrete tasks can be marked “complete” in an issue tracking system, but large enterprise systems are virtual organisms that can be endlessly extended, refined, and improved upon. There is no finish line, but rather a multidimensional cube of gradients where each metric of success is defined and measured by different stakeholders. So, when I state I’m dissatisfied with my work, that’s not a state of being, it’s an acknolwedgement that architecting and developing these systems is a continuum of satisficing stakeholders, not a process that is ever truley complete. We should be dissatisfied, because if we are not, we are complacent.
\nMeasurements of Success
\nHowever, just because the composition of large and complex systems has no discrete end, it doesn’t mean success cannot be measured. There are a ton of metrics that can be derived to have some meaning to various parties in an ISV and the client ecosystem, some of which have meaning, and some of which can be predictors of success. When I look at a system, I intrinsically think about the technical metrics first - the layers of indirection, query costs, how chatty an interface is, cyclomatic complexity, interface definitions, the segregation of responsibility, patterns that are reusable and durable from one set of developers to the next, et cetera. But architects must understand that while these metrics do play a role in the ultimate success, re-usability, and appeal of a solution, they are not the same metrics a business user – usually those who define success at a more meaningful level for going concern of sustainable business – will consider. Instead, these technical metrics contribute to other metrics that are the ultimate way in which a product’s success will be measured and judged. Specifically, there are only three things that executive offices, sales, and prospects care about:
\nThat last item, “How fast does the system do it?”, seems out of place, doesn’t it? Now any whiney sales guy (I used to work with a lot of them, thankfully we have an awesome team where I’m at now) can tell you how a sluggish-feeling web page can tank a demo or blame a two second render time for a bacon he didn’t bring home last quarter, and cloistered developers are used to brushing off those comments. They really shouldn’t. Speed directly determines the success of a product in three ways:
\nUsers who have a slow experience are less likely to start to use the product
\nKISSmetrics put together a fantastic infographic on this subject that shows how page abandonment is affected by web page load times.
\n![\"\"](\"http://blueprintsforsuccess.wordpress.com/wp-content/uploads/2012/05/page-loading-time-by-kiss-metrics11.jpg\" "\\"Page-Loading-Time-by-Kiss-Metrics[1]\\"")
And let’s not fool ourselves – just because your product is served on an intranet, not for the fickle consumption of the B2C public Internet, your users are no yes fickle or demanding. Nor are you immune to this phenomenon because you utilize native clients or rich internet applications (RIA’s) to provide your product or service. Users will abandon your way to access their data if it’s too slow, even if you might think they are a captive audience. For instance, in a world where data liberation is a real and powerful force – where users demand to export their data from your system to use the interface of their choice, or even worse, where users demand you provide API’s to your data so they can use your competitor’s user interface – no audience is captive. Even worse for those of you providing a B2C public Internet service, page load times play into search engine optimization (SEO) ranking algorithms, meaning a slow slight is less likely to even enter the consciousness of prospects who depend on a search engine to scope their perception of available services.
\nUsers who have a slow experience are less likely to continue using a product
\nLet’s say you’ve enticed users with all your wonderful functionality and a slick Web 2.0 (I hate that term, for the record) user interface to visit your site, perhaps even sign-up and take it for a spin. Most developers fail to realize that a clunky web browsing experience in an application doesn’t just temporarily frustrate users, it affects their psychological perceptions about the credibility of your product (Fogg et al. 2001) as well as the quality of the service (Bouch, Kuchinsky, and Bhatti 2000). In one case which analyzed a large data site of an e-commerce site, a one second delay in page loads reduced customer conversion rates by 7%.
\n![\"\"](\"http://blueprintsforsuccess.wordpress.com/wp-content/uploads/2012/05/behaviour_model1.png\" "\\"behaviour_model[1]\\"")
The above graphic is a visualization of a behavior model by BJ Fogg of Stanford University about how users motivation and ability create a threshold to take action, and what triggers a product can use to entice users to cross that threshold depending on their position along this action boundary. Truly fascinating stuff, but to distill it down into the context of this blog post – the marketing of your product and the value proposition of your service should be creating a high motivation for your end users. What a shame then, if users never take action to use your product because you failed to reduce barriers to usage, reducing the ability and increasing complexity because your site was sluggish. Crossing that boundary is one hurdle to cross, but ISV’s have the ability to move the boundary in the way the market, design, and implement the product.
\nThe Cost-Speed Curvature
\nOkay, okay, you got it, right? The product needs to be fast. But how fast is fast enough? You can find studies from the late 1990’s that say 8-10 seconds is the gold standard. But back in reality, our expectations are closer to the 2-3 second threshold. The wiggle room is admittedly extraordinarily small in this minuscule window: it doesn’t accept any excuses due to the slow rendering speeds of ancient computers or low-powered mobile devices that might be using your site, the client’s low bandwidth, or buffer bloat in each piece of equipment between your server’s network card and your end user’s. Not to mention, most sites aren’t simply delivering static, cache-able content. They’re hitting web farms of web servers behind load balancers, often using a separate caching instance, subject to the disk and network I/O of a database server and any number of components in between to execute potentially long-running processes – all of which need to happen in a manner that still provides the perception of a speedy user experience.
\nNow, exactly how to get your product or service faster isn’t my concern, and it’s highly dependent on exactly what you do and exactly how you do it – your technology stack and specific infrastructure decisions. What I can tell you though is you need an answer to your executive suite, board, or ultimate impatient user who, no matter how performant (or not) your system is, asks, “How can we make this faster?” This answer shouldn’t be quantitative, as in, “We can shave 4 seconds off if we do Enhancement X, which will take two weeks”, unless you want to hear your words parroted back to you when you can’t deliver such an unrealistic expectation. Even if you have an amazing amount of profiled data points about each component of your system, quantifying improvements is a mental exercise with little predictable result in enterprise solutions.
\nWhy?
\nWell, in any serious enterprise software solution, there is obviously code you didn’t write and pieces you didn’t architect. Even if you were Employee #1, and not inheriting a mess by a predecessor team or architect, inevitably you’re using multiple black boxes in your interconnected system in the form of code libraries. Even if you’re a big FOSS proponent and can technically look at any of the source code for those libraries, face it, in a real business you never will have the time to do so, if the nerdy interest. While you can sample the inputs and outputs into each of those closed systems, you can predict but you cannot quantify how changing an input will affect the performance of a closed system creating an output. Don’t try it, you will fail.
\nInstead, remember my opening paragraph – performance optimization, much like “feature completeness”, is not a goal, it is a process that is continual over the life of the product. Obviously, developers start this process Googling StackOverflow et al. for “slow IOC startup” or “IIS memory issues in WCF services” or whatever the issue is with your particular technology stack, and will review the “me too” comments to see if they too did a “me too” misconfiguration or misdesign. Maybe it’s “whoops, forgot to turn on web server GZIP compression” or “whoops, forgot to turn off debug symbols when I compile”. Typically, these are low-hanging fruit – low risk to affect change with a high potential impact. But eventually you run out of simple “whoops!” Eureka moments or answers to simple questions, and you end up having to ask harder questions that have fewer obvious answers, thus requiring time spent specifically on researching those answers and developing solutions in-house. When you think about it, there’s a real escalating cost for each unit of performance gain over the lifetime of the product for this very reason. Graphed as a curve, I’ll call it the Marginal Cost of Speed:
\n![\"\"](\"http://blueprintsforsuccess.wordpress.com/wp-content/uploads/2012/05/marginal.png\" "\\"marginal\\"")
And this is, in fact, a reality that must be thoroughly understood inside a development team all the way up through the executive suite. Not dissimilar to how Einstein postulated the only way to achieve infinite speed was to harness infinite energy, the only way to get an instance page-load or a zero-latency back-end process completion is by spending an infinite amount of resources achieving that goal. I say this has to be understood at the development team level mostly because you will never, no matter how pragmatic and persuasive you are, convince the executive suite or the customer that you in fact cannot repeat the last thing you did that doubled performance, because the further you go down the performance optimization road, the narrower and longer it gets between mile markers. The development team needs to fully understand what constitutes low-hanging fruit and must have their efforts focused on those simple changes that affect the greatest change first, and not tackle such problems with an instinctive impulse to refactor.
\nLikewise, the executive and marketing teams need to understand the development of a lightning fast product is a last-mile problem, that reaching that nirvana will require an increasing amount of time (cost) and resources (cost) to achieve it. The effort is an exercise in satisficing the parameters to find an acceptable middle-ground. Usually, though, the realities of product development aren’t treated the same as the realities of other externally-governed factors, simply because they are perceived not to be governed by any absolutes since they are not external. Put another way, customers of Amazon.com might abandon the site because shipping times for purchases are too long, but the company can’t just start comp’ing overnight service for everyone. Well, they could do so, but the cost to acquire that customer just skyrocketed to a level that makes their business model unsustainable. Similarly, the time spent on performance optimization has a real and measurable cost, and it can actually be quantified as a cost to acquire and retain a customer when you think about how a performant site directly impacts customer acquisition and retention. Now, the business folks can definitely understand it in those terms. But, they’ll still want it faster anyway.
\nWhere To Sit
\nSo, where do you then sit on that curve? The real answer is, it doesn’t really matter how much you do or don’t want to make performance optimizations, particularly if they’re approaching the infinite cost asymptote of that graph. The answer is, you will have to sit wherever your competitors sit. Most of us out there building the next great thing aren’t making markets, we’re creating displacement products. For those of us doing so, we’ve got to chase after wherever your most successful competitor sits on the marginal cost of per speed graph. Now, to be fair, those guys have probably been working for a few years on their ascent up that cost-performance climb, and they probably have deeper pockets / more slack time to do so than you do if you’re breaking into a market, but there is a trade-off the suits can make. The accumulated cost to 90% of the graph is less than the whole last 10%, so put another way, if you can be at least performant to make 90% of those prospects who are 100% happy with your competitor’s product, that may well be enough to get displace enough business to let you keep tackling that last mile another day.
\nObviously, this question can’t be completely answered that way, because it’s highly dependent on your specific markets. Are you entering a market with a democratic offering of grass-roots, home-grown alternatives or are you tackling an oligarchy industry? Are you targeting disparate customers, or are your customers banded together in trade associations – which translates to – how much does your reputation change for each success or each failure? How are your customers allowed to back out of a contract if they find performance or other factors don’t match the vision sold to them? These answers may make the “how fast does it need to be” answer necessitate a disproportionately higher amount of resources and time to get it where it needs to be to have a good, marketable value proposition.
\nIn summary, you never really should sit anywhere on that curve, you should be climbing it. It will cost you more the further you climb, but you should never feel like you’re done optimizing performance, and you should never stop continuously reviewing it. Remember how I mentioned most of us are in the displacement business? Even if you’re not, someday, someone else will be, looking to displace you. That guy might be me someday, and rest assured, I won’t rest assured anywhere. :)
\n", "date_published": "2012-05-02T05:11:09+00:00", "url": "https://blog.seanmcelroy.com/2012/05/02/the-cost-of-speed.html", "tags": ["User Experience"] }, { "id": "http://seanmcelroy.micro.blog/2011/08/21/its-about-the-developers-stupid.html", "title": "It's About the Developers, Stupid!", "content_html": "Last week’s continued equity market shakeups were made even more volatile by a few headscratchers: Google purchasing Motorola Mobility for USD$12.5 billion (nearly $735 thousand per issued patent held by the company), and HP musing about spinning off its PC manufacturing business and potentially buying Autonomy to become a software and consulting house, an apparent IBM redux. Endless articles and commentaries are focusing on Google’s purchase of MMI, but the more interesting story to me is HP, and how their shift in business model is less about focusing on higher margin lines of business, but rather admitting failure in their purchase of Palm, and more generally, in building sustainable developer ecosystems.
\nWhen big companies spend big money on massive acquisitions, they take on huge amounts of explicit, intrinsic, and opportunity risk that only a carefully designed strategy will vindicate. When the stakeholders discuss only the balance sheet terms of deals they agree to, without really understanding the cultures of the external environments they depend upon, there’s a lot of unmitigated risk, and ultimately, a lot of avoidable waste. Arguably, Palm faltered and became an acquisition target for HP not because they had a inferior product or platform, but because they failed to nurture a strong developer ecosystem after Jeff Hawkins and Donna Dubinsky left to form Handspring. When iterations of the Palm OS failed to deliver critical platform feature requests to keep the offering competitive, Palm addressed the problem by releasing webOS, years later, and with a cavalier attitude that they could build a new developer community around the offering without needing to mend their fences with their long-time supporters.
\nWe know what happened there - Palm stumbled, and HP picked up a compelling technology offering in webOS. But HP made the same competitive mistake as Palm - it failed to foster a developer community to propel WebOS forward as the mobile operating system oligarchy was taking shape. It, like Nokia with Symbian, did not appreciate the role of a thriving developer ecosystem in building a mobile brand, nor did they continue to continuously invest into it. Great technologies attract bright developers, who in turn make direct contributions to the ecosystem in the forms of apps, frameworks, and cloud services, and indirect contributions by recommending technologies to ‘the suits’ who invest resources in leveraging them for their own ends. This generates a current of innovation that can become self-sustaining, and this fills out direct to consumer ‘app stores’ with features that intrigue consumers who make the ultimate platform selection through their purchases. Let’s face it, when you walk into a brick and mortar mobile phone store, you’re not confronted by displays that put “smart phones” on one wall, “camera phones” on another, with old-style candy bar phones somewhere in the back - that was so four years ago! Consumers today are targeted with marketing to compel them to choose an ecosystem – Android vs. iOS vs. Windows Mobile 7. The hardware is become less relevant as a purchasing decision, because there’s few physical differentiators other than form factor (which Apple continues to win, hands-down).
\nMicrosoft has understood this concept extremely well for decades, and they embrace their strategy by focusing on delivering excellent tool chains for developing applications that function on platforms (operating systems) they sell. Despite Steve Ballmer’s fanatical espoused enthusiasm on the matter, the company actually does make good on their word on investing in developers who invest in their technology. They virtually give away expensive integrated development environments to secondary and post-secondary schools and create extensive supportive curriculum, documentation, and living communities that attract bright people and encourage other young minds seeking to connect with the brightest of their peers working on their technology.
\nMicrosoft’s not alone in this strategy, but they’re notable for how well they execute it. Apple is one of the only notable exceptions to this process: attracting developers by rapidly building amazing market share. Apple is a force to be reckoned with, for sure, but at the end of the day, “suits” decide to support iOS because of it’s market share, not because their technologists and in-house developers extol the “amazing development experience” of iOS. Nokia tried this and failed. RIM is failing despite having a great market share position, at one time, for a mixture of technology capability and community support reasons.
\nThe lesson here, though, isn’t restricted to the multinational, large-cap platform developers – even small, agile start-ups must quickly understand the importance and formulate strategies for building synergies to succeed. Whether they’re implemented through open source software, direct-to-the-community adoption initiatives, or strategic partnerships between peer companies, small businesses depend upon the rich technological feedback for continous improvement they cannot generate internally due to constrained early-stage resources.
\nHP, though, doesn’t understand or doesn’t appreciate the “how” of building a real, working platform ecosystem is critical not only for innovative start-ups, but also for large-cap software firms. And though HP may be throwing in the towel for mobile devices, this is a lesson critically important for any software company no matter what their distribution channel is: mobile, tablet, desktop, or enterprise servers. The fact HP doesn’t get it or is too encumbered to act on it, is the biggest threat to HP spinning off their low-margin, but reliable revenue generating manufacturing segment and plugging ahead.
\nBallmer should do his good deed for 2011 and ring them up with a tip: It’s all about the developers, stupid!
\n", "date_published": "2011-08-21T17:54:53+00:00", "url": "https://blog.seanmcelroy.com/2011/08/21/its-about-the-developers-stupid.html", "tags": ["Technology Policies"] }, { "id": "http://seanmcelroy.micro.blog/2011/08/12/will-state-treasuries-get-wise.html", "title": "Will State Treasuries Get Wise to Geolocation?", "content_html": "Slowly, mobile users are becoming increasingly complacent with giving up the last remaining visages of privacy when it comes to using a mobile web browser or using mobile native apps to do the most rudimentary tasks. Just five years ago, imagine the adoption rate an application would have that required your exact geographic location and the rights to read the names and phone numbers of your entire digital Rolodex to let you read the front page headlines of news. It would fester in digital obsolescence through right-out rejection! Today, it’s a different ballgame.
\nThere’s some interesting changes I can foresee that will come out of these shifting norms that have nothing to do with the overblogged concepts of targeted advertising or the erosion of our privacy. There’s an awesome company called Square has a nifty credit card reader that plugs directly into the audio port of a mobile device to create instant point of sale devices with a lot of flexibility and little capital investment. Even this can’t be called new by today’s blogosphere standards, but something that caught my attention in beta testing this service was its requirement to continuously track your fine GPS location as an anti-fraud measure. Pretty sensical, but also, pretty telling of things to come.
\nAnyone’s whose been following the tech world recalls the recent tiffs between Amazon and various states, most recently of those being California, that have tried to get a slice of the revenue generated by sales addressed to their state. Large corporations can keep playing evasive maneuvers with state legislatures, and small business brick-and-mortar retailers as well as state coffers continue to feel the squeeze as shoppers become continuously comfortable and familiar with making large ticket purchases online, both to comparison shop, but also, quite obviously, to avoid paying state and local sales taxes. A looming federal debt crisis that is decades away from a meaningful resolution means less distributions to states, leaving each to pick up a larger share of the tab for basic services, infrastructure improvements, and some types of entitlements. States have reacted two-fold: to try to squeeze the large online retailers with legislation, and secondly, to require state taxpayers to volunteer their “fair share” by paying use tax.
\nWho accurately reports their online sales for the last tax year for the purposes of paying use tax? Anyone that knows me is well aware of my almost maniacal love for and usage of budgeting tools that allow me to easily pull up a report of every online purchase I’ve made in a given time period in a matter if seconds. But many people who owe hundreds in state use taxes file their returns the same as my parents, who purchase nothing online, and report zero in this box.
\nIt would be relatively trivial from a technology perspective, but predictably forthcoming from a policy perspective, that this free ride is about to end. One-third of smartphone owners have made a mobile online purchase from their phone, and a full 20% use their device as a fully-fledged mobile wallet. 47% of smartphone owners and 56% of tablet owners plan to purchase more products on their respective devices in the future. With the skyrocketing adoption of mobile as a valid, trusted payments platform, it won’t be long before a majority of physical goods transactions are made with these devices. In the name of “safer, more secure transactions”, consumers will likely be prompted to, and likely won’t think twice about, revealing their location from which they make that purchase.
\nNo matter how much we might muse to the contrary, legislators, nor their more technically savvy aides, aren’t oblivious to the coming opportunity this shift will provide: Imagine a requirement that any purchase made would log the location of the purchaser at the time the transaction was made, and charge online sales tax based on that location. Since most mobile users spend their lives in their home location, this would keep a high percentage of taxes collected in this manner in the municipalities that provide services to the end consumer, reclaiming unreported taxable sales in a manner consistent with the collections prior to this massive behavioral shift. It also levels the playing field for small retailers, who have to collect the same rates on their purchases.
\nIt’s an intriguing scenario, and one not far from reality. It may be this, and only this, that creates a consumer backlash against the complacent acceptance of leaking geolocation for anything other than maps or yellow page-type applications. It may create scenarios where people travel to an adjoining town which creates a digital “tax haven” by instituting free municipal WiFi and low tax rates to drive a new form of digital tax haven tourism.
\nIn any case, it’s definitely something to think about.
\n", "date_published": "2011-08-12T22:08:00+00:00", "url": "https://blog.seanmcelroy.com/2011/08/12/will-state-treasuries-get-wise.html", "tags": ["Privacy","Social Responsibility","Technology Policies"] }, { "id": "http://seanmcelroy.micro.blog/2011/02/20/sonys-poor-behavior-what-does.html", "title": "Sony's Poor Behavior: What does this say about learning in America?", "content_html": "Ask any technical recruiter, or any quickly-growing technology business, what the number one challenge in the external environment is to growth, and the answer might surprise you. In a resurgence reminiscent of the late 90’s in Silicon Valley in social media and associated technologies that connect people, ideas, and cash, there’s no lack of innovation, imagination, or good business ideas out there. With investment tax credits and freely-flowing capital fueled by low interest rates and desperate federal, state, and local attempts to ignite the engines of industry and the economy, lack of funding or tightness of credit isn’t the challenge it was two years ago. Rather, the lack of sufficiently knowledgeable and adequately trained professionals in highly technical fields is the biggest roadblock to the economic expansion of the services industry.
\nThe cost of labor of highly skilled software engineers is increasingly well above the rate of inflation, having increased over 25% in the past 8 years. (Just check out the term “computer systems software engineers median annual salary” on WolframAlpha.) Simply supply and demand sets the price points for wages in local markets, and this trend broadly realized over the entire world has to make one wonder: Where is the supply of new talent, and why is it not keeping pace with the growth demands of various technology-dependent industry sectors? I postulate there is a widening knowledge gap analogous to the wealth gap in America, driven by the policy, legal, education and cultural environments.
\nSpecifically, legislation built to protect corporate innovations, including software algorithm patents, anti-copyright mechanisms, and the Digital Millennium Copyright Act are two-edged swords that stifle learning by today’s technically-inclined youth by positioning technologies in untouchable black boxes. Consider for a moment a future electrical engineer in the 1950’s and what his potential contributions to his field would be if he couldn’t dismantle a radio and learn how its components work. What if programming languages were restricted from college classes to only corporations who could afford extortionate fees to access and learn technologies; would the networking revolution of the 1980’s and 1990’s have ever occurred? If young men couldn’t open the hoods of their cars without going to jail, would have have any more automotive innovation, even mechanics? While corporations must be able to earn protected profits to cover their costs of research and development, those same innovations must be allowed to be embraced and extended not only in the broader macro-economy, but also understood, adopted, and applied by the upcoming generation in higher education.
\nThe higher education system itself, however, has been unable to keep pace with the imparting of technical knowledge specifically in business applications, leading to B-schools churning out freshly minted grads that understand some of the ideas behind requirements analysis and abstract system design, but who lack technical depth that cannot be dismissed by specialization difference, but is required in today’s world where technology permeates every level of business, industry, and life. These b-school graduates then go out into the world, often with a deficient understanding of the application of technology required to manage technical resources or properly apply them to real-world processes. I believe this falls squarely in the fault of the lack of cross-disciplinary study plans that integrate related topics within a college, but fails to address the widening rift between engineers who are able to understand the inner workings of the technology, and the business majors who receive only a brush of experience with key concepts.
\nAs one university dean explained to me when I inquired why MIS majors were only required to take a single, general-purpose programming class without any exposure to reporting or datawarehousing concepts, upon which degreed candidates will be expended to understand in their first professional job, the answer was startling. That PhD replied, “We teach people to build businesses and manage technical talent. They don’t need to understand how the technical work is done.” Wrong. Dead wrong. Long past are the days when engineers can be enlisted for one-off projects and dismissed when their work is done. In today’s world, businesses that don’t integrate automation, networking, communication, and social media technologies are being quickly replaced by more savvy, and often foreign entities, that understand the importance of every corporate level, from the board room to the mail room, embracing a cross-functional understanding of technology application.
\nRestricting knowledge transfer is a sure-fire way to ensure you’ll never be able to procure enough of it. A great case in point of such ignorance and short-sightedness can be found in the Sony vs. George Hotz drama currently unfolding in technical circles. A young man, Hotz, dared to open his PS3 and learn how it works. Pages and pages of TOS’s, AUP’s, and EULA’s explicitly forbid him from doing so, and now in retribution for sharing what he learned about what’s inside the $600 black box he purchased, one of the largest companies in the world is actively suing him, and those he spoke to, to keep what they learned to themselves by applying the DMCA against them.
\nThe mass media has long abused and contorted the term “hacking” to apply to virtually any illegal, unethical, or criminal element that remotely involves technology. First and foremost, hacking in its true sense, is learning what’s not obvious. If we have effectively criminalized this learning process both legally and culturally, we can sit back and watch our economic output dwindle as other cultures and nations which either through their abandonment of intellectual property protections or permissive discovery and learning culture prepare a more capable generation of tinkerers, whom individually and in greater numbers will show us up. Sony’s behavior in attempting to sue young men attempting to learn how they do what they do is driven by the assumption that knowledge can be owned, controlled, and metered. While Sony may be able to apply punitive measures against a handful of the curious, the attempt to do so is not only futile (anyone remember what Napster did to the music recording industry?), but it creates a climate of fear and draconian policies that trickle down to further squelch off those who want to learn from being able to do so, both systematically by instilling a fear to do so will incur corporate wrath, or by discouraging institutions capable of imparting that knowledge from doing so as they attempt to shape ethical norms.
\nA society that fundamentally believes that some knowledge should not be learned nor shared is doomed to pay its dues to societies that value knowledge creation, knowledge transfer, and raising future generations with the desire and ability to become as competent as their forbearers and extend the reaches of their contributions.
\n", "date_published": "2011-02-20T20:23:39+00:00", "url": "https://blog.seanmcelroy.com/2011/02/20/sonys-poor-behavior-what-does.html", "tags": ["Social Responsibility","Technology Policies"] }, { "id": "http://seanmcelroy.micro.blog/2010/12/03/pp-dns-not-solving-the.html", "title": "P2P DNS: Not solving the real problem of centralized control", "content_html": "The more tech-savvy probably noted with passing interest the news blip this last week by Peter Sunde, co-founder of The Pirate Bay, a notorious website for finding BitTorrent .torrent files for everything from public domain books to copyrighted music, video, and warez of a new peer-to-peer Domain Name System in response to recent US authoritarian action in seizing domain names. The specific instance that is causing so much cyberangst is the Department of Homeland Security and Immigration and Customs Enforcement bowing to the pressures of media giants have shut down RapGodfathers.com. By “shut down”, these enforcement agencies didn’t just confiscate server equipment, they actually seized DNS hostnames assigned by their registrar, through ICANN. Long has the rest of the world complained that IANA and ICANN, bodies that assign all sorts of global numbering and addressing schemes, are puppets of the U.S. Government, and even a number of the American tech crowd that the actions of these bodies over time are counter to the perceived free and open nature of the Internet.
\nWhile DNS isn’t that important from a purely technological networking perspective, that is, it is simply a redirection service, almost no denizens of the web could find Google, Facebook, or Bing without it. DNS is a protocol that allows a simple name, such as example.com to be translated into an IP address, serving the role of a phone book of sorts. I’ll have to admit, just as I’d probably lose all my friends if I lost my EVO, since I depend on my address books over memorized phone numbers these days – I only know some of Google’s servers, my work, and my home IP address by heart, but for everything else, I’m dependent on DNS to tell me (and my browser) where to find things. In response to ICE’s attack on the perception that domain names should not be commandeered by governments, Sunde has started a project to offer up an alternative DNS service over peer-to-peer networks, to remove the ability for corporations or governments to seize domains. Unlike failed ‘alternate root’ schemes in the past, this shift in technology would, as the thought goes, allow the domain name resolution service to be operated by consensus. In such a world, ICE couldn’t have seized RapGodfathers.com domain, nor could any corporation with a similar name as a private individual file a copyright claim to take a domain name away from them. Do we have a fundamental right to allow the public to sign off on who gets to hold what URL properties?
\nThe rhetoric on the issue has been amusing at best and eye-rolling at worst, when people like Keir Thomas make outlandish claims that an alternate DNS scheme will be ‘heartily embraced by terrorists and pedophiles’. Sadly, such claims showcase the true lack of technical understanding about how the networking protocols of the Internet actually work. Coming back to my phone book analogy, a P2P DNS scheme would be akin to GOOG-411 providing phone numbers instead of my local phonebook (which sits unused, now 5 years old, mind you): Anyone can one a phone number or IP address, but the way you resolve a name to a number doesn’t really, on a true technical level, change anything about who controls access and availability to resources. If I could configure my computer to point cocacola.com to illegal content, that doesn’t change the fact the content was out there to point to in the first place, nor does it make it any easier to find for those not seeking how to access it.
\nThe real threat is when governments start mandating control over a protocol that hasn’t yet become a household name – BGP. Around in some form since 1982, BGP doesn’t translate human-recognizable names into network numbers, it actually describes where to route those numbers. When the Great (Fire)wall of China censors where its citizens can go, it does so by dictating that the numbers it doesn’t want you to dial call non-existent places, or more realistically in the network world, that the paths to route your request to are wrong or dead-end. Back to the analogy, controlling BGP is the end-game on the Internet– instead of taking over the phone book’s printing presses, you take over the phone company’s switching stations themselves. For those wishing to make the Internet more autonomous and decentralized, the future to securing the existing global communications network from superpowers' total control lies in alternatives to BGP, not DNS.
\nHowever, P2P BGP isn’t going to happen, because as DNS instructs your computers where to go to find information, an attribute you can control yourself, BGP instructs your ISP’s routers where to get their information, and you won’t ever control their hardware. And really, the fundamental issue is there’s no clear way to keep the current networking stack of protocols we collectively call the Internet free and open, as we like to believe it should be. Instead, for those wanting to leverage the crowd to free the Internet from tyrannous regimes or powerful special interests, your best bet for the future is Freenet or Tor, layers that sit on top of the Internet’s infrastructure and provide their own. They route requests and traffic through a “tunnel-atop-the-tunnels” approach that cannot be easily discerned nor controlled. If the history of Internet governance has taught us anything, it’s that if something can be controlled, the wrong entities end up controlling it. The approach that Freenet and similar onion routing networks take is to remove control and technologically favor independent voices. Instead of writing new technologies like P2P DNS to address yesterday’s problems, I heartily recommend those with the interest and aptitude look into key-routing networks like Freenet, which by their very design prevent eavesdropping and circumvent traditional control mechanisms. Just in their awkward teenage years, these will be the technology tools of digital patriots in the future, not P2P DNS on a network protocol stack that is increasingly being pulled out of the grasps of its grandfathers and architects.
\nI will have to commend Sunde’s efforts though, on the principal that if you do some Google keyword searching, ICE’s seizure of RapGodfathers.com was only a spec on the web’s map until Sunde’s project was announced. Raising awareness of who holds the keys to the words we write, read, and share is paramount in a world where most of the people who write, read, and share their thoughts over the Internet are generally otherwise without a clue to how their ideas are allowed or blocked by the powers above.
\n", "date_published": "2010-12-03T00:44:12+00:00", "url": "https://blog.seanmcelroy.com/2010/12/03/pp-dns-not-solving-the.html", "tags": ["Ethical Concerns","Privacy"] }, { "id": "http://seanmcelroy.micro.blog/2010/08/05/be-assimilated-or-be-ignored.html", "title": "Be Assimilated, Or Be Ignored", "content_html": "An interesting exercise is to visualize tidbits of data as material widgets, units of value that can be bought and sold in a marketing economy controlled by the forces of supply and demand. While completely relevant in an information and services-based economy, often we don’t stop to put data, news, or information in the same contextual terms as goods we find in supermarkets or services we can find in the phone book. (What is a phone book anyhow?) All the same rules apply, however.
\nFor example, if I can find white socks of equal quality cheaper at a store next door, for me, this is a viable substitute, and I will vote on retailers with my purchasing power. I am not, however, interested in purchasing raw cotton to spin my own socks. I’m just not equipped with the time or skill to take raw inputs of that nature to create the outputs I desire; the cost to do so is far greater than the opportunity cost I would incur doing things that make a lot more sense for my skill set. Similarly, if I can find my news on Twitter, Facebook, or other blogs, where others have distilled facts and raw data into commentary and analysis, and if I can determine the quality is sufficiently the same for my needs, then I won’t need to buy a newspaper, pay for online periodical archive access, or spend an opportunity cost in watching ads before each 30 second video on my local news channel’s website.
\nThat’s nothing new. What is new is, my economic substitutes, or other sources for information consumption, have a key feature my previous choices did not: aggregation. Now, I don’t even have to look at this information in the layout provided for me. I don’t need to view CNN’s promotions, Google’s AdWords, or Twitter’s obnoxious color schemes when all my news feeds come into my Microsoft Outlook or Google Reader tool. I pick and choose not only what I want to consume, but the manner in which it’s displayed. Were someone to make information unavailable for syndication or add inline advertisements to the syndicated content itself, I perceive there are many equally valuable substitutes, so I can nix any offending feed and replace it with another that meets my consumption demands.
\nAny viable business needs to consider not only the importance of providing their content for aggregation to vie for a user’s attention among other feeds, but also to build aggregation into their own offerings. As aggregators begin to control not only where a user looks, but provide more advanced options to filter what feeds are recommended for users and further, what portions of feeds are selected for inclusion into a dashboard view, they will become the most important gatekeeper of the next decade. They will control not only the screen real estate used to provide banner ads and inline contextual linkages to other promotional content, but they will also gain the power to shape what we think about and to what ideas we are exposed.
\nFor the rest of us, the bloggers and content providers, don’t worry so much on your layout and formatting. The way in which you deliver information loses relevancy compared to the actual value of the content you provide, and no matter how valuable you feel your analysis or commentary is, in a plugged-in world that encourages further social media interaction and feedback from smart people who may not be editorial experts, your offering is just a commodity. You will become increasingly disconnected from your consumers, who will use the product of your data and information over channels you do not control and of which you may not even be aware. You will lose your ability to monetize the delivery of your content, or at the very least, someone else will be giving you a faction of the channel they now own, a pay-for-play access fee to their aggregation or social network users. It may not be what we want, but it’s better than not being included in the new digital world order, as it were. Or, in other words, prepare to be assimilated, or prepare to be ignored!
\n", "date_published": "2010-08-05T16:48:54+00:00", "url": "https://blog.seanmcelroy.com/2010/08/05/be-assimilated-or-be-ignored.html", "tags": ["Social Media"] } ] }