The Black Liszt

Category: Big Business

  • Summary: Software in Government, Big Business and Big Tech

    This is a summary with links to my posts on the many ways that large organizations including government, big business, big tech and the rest diligently apply modern software procedures as taught in academia and required by professional management; they consistently produce disastrous results in software quality, cost, security and everything else that matters.

    There are of course issues that are common to all these large organizations, for example in cybersecurity.

    https://blackliszt.com/2015/06/systemic-issues-behind-the-cyber-security-disasters-at-opm-citi-anthem-etc.html

    Government

    Government software disasters are government-as-usual, so much so that disasters that wreck lives barely make the news. For example, over 10 million people world-wide enter a government-run lottery for immigration slots that can lead to US citizenship. How hard can picking a bunch of random numbers be? Apparently too hard for the government software people, with the result of horrible consequences for the declared lottery winners whose immigration slots were invalidated.

    https://blackliszt.com/2011/07/software-quality-horror-tales-electronic-diversity-visas.html

    Consider the sets "Excellence" and "Government IT." There is a great deal of evidence that these are non-overlapping sets. I learned there are organizations promoting and celebrating digital government. They hold awards ceremonies. I tried to find out what the winner had done to deserve winning. Surprise, surprise, the link at the organization’s website explaining it all was broken. Pathetic.

    https://blackliszt.com/2015/05/excellence-in-government-it.html

    Even simple things like making Social Security statements available on-line appears to be beyond them — including of course lying about it.

    https://blackliszt.com/2024/03/excellenece-in-government-it-the-social-security-administration-.html

    The NSA (National  Security Agency) has a budget of over $50 Billion and is touted as being the world’s best at cybersecurity. It turns out the only reason we know their super-top-secret budget is because their security was blatantly breached with massive internal data made public.

    https://blackliszt.com/2014/05/bureaucracy-regulation-and-computer-security.html

    Given that this army of highly-paid cyber geniuses can’t protect itself, it’s not surprising that its analysis of a high-visibility security breach may have sounded good to the public, but was in fact entirely fraudulent.

    https://blackliszt.com/2017/01/russia-hacks-dnc-podesta-email-fake-news.html

    What do you do with such a huge budget when you’re unable to do what you’re supposed to do even with your own secrets? You set up a massive program to teach students your excellent methods and hope to train over a million certified experts. I tracked the program from a local community college to the NSA’s own description of its program – which was both broken and insecure!

    https://blackliszt.com/2017/06/government-cyber-security-tops-the-oxymoron-list.html

    Unfortunately, this isn’t just about keeping information safe. Government ineptitude kills people. Instead of taking a quick, simple approach to preventing train crashes:

    https://blackliszt.com/2015/05/an-app-to-prevent-train-crashes-like-amtrak-philadelphia.html

    The government presses on with its super-expensive solution using obsolete technology, which leads to yet more preventable crashes and deaths.

    https://blackliszt.com/2016/10/scandal-hoboken-train-crash.html

    It’s not just big governments. The little government of several islands in the Caribbean managed to create a multi-front disaster using best practices to foist a digital currency system on its innocent citizens.

    https://blackliszt.com/2022/03/dcash-government-cryptocurrency-shows-why-fedcoin-would-be-a-disaster.html

    https://blackliszt.com/2022/03/what-is-behind-the-dcash-central-bank-digital-currency-disaster.html

    The US government continues to pursue a national digital currency of the kind that has already proved to be a disaster in the Caribbean. They do so ignoring the fact that the US Dollar is already largely digital, with extensive software support structures that are in place and working well..

    https://blackliszt.com/2020/12/we-dont-need-fedcoin-we-already-have-a-national-digital-currency.html

    Important things like voting systems are some combination of broken and insecure. I took the trouble to define a simple combination of tech and non-tech to build a modern, secure voting system that was auditable, with operations visible to every voter while keeping what they voted for secret. Will any government institution pay attention, much less implement it? We all know the answer.

    https://blackliszt.com/2025/03/voter-id-and-paper-ballots-dont-prevent-cheating.html

     

    Big Business

    Executives in big business want to succeed and advance, but this can only happen by avoiding risk. The best way to avoid risk is to do what “everyone else” is doing, what the experts say is best. That’s where industry advisory groups come in.

    https://blackliszt.com/2017/05/the-value-of-computer-industry-advisory-groups.html

    Giant advisory firms counsel their customers on how to make the best decisions. Getting your customers to like you is high on the list. Carefully crafted words are of supreme importance to such large organizations. Actions that match? Not so much.

    https://blackliszt.com/2016/07/gartner-group-big-company-customer-service.html

    A giant health insurance company “lost” the personal information of "tens of millions" of its members sometime in 2014; they're not sure how many, whose records were "lost," or when it happened. The details are an astounding illustration of big-corporate IT incompetence.

    https://blackliszt.com/2015/02/the-anthem-of-cyber-insecurity.html

    I soon found out that my information had indeed been stolen. The company’s response to the theft was right in line with their letting it happen.

    https://blackliszt.com/2015/02/my-anthem-account-was-hacked.html

    What company doesn't want to be part of the digital revolution and have an app? If you're a major health insurance company, why wouldn't you replace old-fashioned insurance cards with something always up-to-date that comes on an app? Here’s what ensued when one of the industry giants tried.

    https://blackliszt.com/2021/02/why-cant-big-companies-build-or-even-buy-sofware-that-works.html

    I've covered many big organization face-plants. The awfulness encompasses a broad range of consumer-dissing inconvenience, Here’s a case of some software that "works" but puts customer inconvenience front and center.

    https://blackliszt.com/2021/03/why-cant-big-companies-build-software-that-works.html

    Here’s a case of a giant company software issue that is low on the “it matters” scale, and high on the “a smart high school student could have done it better” scale. It’s the kind of issue that leads one to wonder whether we’d all be better off if they refused to hire any more people with college degrees for any job, and in particular, management.

    https://blackliszt.com/2021/05/anthem-needs-my-feedback-reveals-deep-problems.html

    Big Tech

    Whether the software is a cool social app, an academic website or a real business, there is a common theme: the software is poorly designed and, even worse, it just breaks. You might think the cool internet apps like Facebook and Twitter are an exception, but they’re not.

    https://blackliszt.com/2012/01/internet-software-quality-horror-shows.html

    How can you innovate? Did the leaders of the current big tech companies benefit from training in innovation? Once they became large, have the big guys like Google demonstrated excellence in innovation? Uhh, sorry, the facts indicate otherwise.

    https://blackliszt.com/2016/05/organizing-for-successful-innovation-recent-history.html

    The widely-accepted logic is: Facebook is wildly successful; FB is built on software; therefore, FB software must be excellent. I should hire people from FB to help me build excellent software! The history and facts support neither the logic nor the conclusion.

    https://blackliszt.com/2014/12/fb.html

    I looked at FB’s mobile app when it had over 700 million people using it. Over 20 million people had written reviews, more than 6 million of which were 3 stars or less. A random sample of those reviews yielded juicy results.

    https://blackliszt.com/2014/11/facebooks-software-quality.html

    The difference between image and reality at FB is astounding. Here is an interview and a recent book that should lead any ambitious young company to avoid hiring people from there.

    https://blackliszt.com/2017/03/software-giants-image-and-reality-facebook.html

    Large organizations have trouble building software. This has been true since the dawn of software history, and shows no signs of changing. The decades-long, rolling disaster of Microsoft Windows is a great example of this.

    https://blackliszt.com/2015/08/large-organization-software-fails-the-case-of-microsoft-windows.html

    Microsoft illustrated multiple issues relating to digital ownership in a case I dug into. Among other things they attempted to require use of their own pathetic browser.

    https://blackliszt.com/2014/05/giant-software-bureaucracies.html

    There are big problems with software quality. The social apps in particular have decided it's embarrassing. But instead of actually, you know, fixing the problems, they seem to have decided to mask the problems! Twitter is a great example of this disease.

    https://blackliszt.com/2013/05/twitter-software-quality-stinks.html

    I did detailed studies on Twitter and found that they do indeed produce provably bad search results.

    https://blackliszt.com/2013/05/twitter-software-quality-an-oxymoron.html

    People write and talk about what's "trending on Twitter" as though the trend meant something. It doesn't. It's based on deeply flawed Twitter search software that gives random, widely varying results.

    https://blackliszt.com/2013/05/the-bogus-basis-of-trending-on-twitter.html

    Twitter fired boatloads of software engineers in 2022 leading some to predict that software disaster will ensue. But then, most people don’t know much about software and don’t realize what a disaster Twitter software has been for years.

    https://blackliszt.com/2022/11/twitter-can-improve-software-quality-by-losing-most-of-its-engineers.html

    Then there is Apple, the high-prestige computer company making expensive devices. In 2016, terrorists killed a bunch of people in California. Law enforcement and the FBI worked hard to find out what happened and who else might have been involved. This required looking in the government-issued iPhones used by the killers. What happened? Apple did its best to protect the criminals. Here are the highlights.

    https://blackliszt.com/2016/03/the-apple-fbi-fiasco.html

    And here are the details:

    https://blackliszt.com/2016/03/apple-can-help-fight-crime-while-maintaining-privacy.html

    https://blackliszt.com/2016/02/apples-cancer-prevention-strategy.html

    https://blackliszt.com/2016/02/apples-approach-to-privacy-terrorists-and-criminals.html

    I reviewed a book about government security on Amazon. The author was impressive and had loads of experience. Many of the reviews were positive, with a few pointing to obvious bias. I wrote a review that pointed to the positive aspects, but also mentioned some of the bias. The review disappeared. I interacted with Amazon, and was told that suppressing the review was a mistake. It appeared again. Then it disappeared. I tried to write a review and was told I've been banned!

    https://blackliszt.com/2023/03/early-evidence-of-criticism-suppression-by-intelligence-agencies-.html

    Yelp isn’t as big as the industry giants, but it’s pretty big. A random plunge into their system demonstrates the same kind of slick surface with rotten underpinnings as their larger brethren.

    https://blackliszt.com/2021/05/yelp-big-tech-incompetent-corrupt.html

    Conclusion

    There is a better way! The winning methods aren’t even new – they’re proven in practice by small groups that need to win. See:

    https://blackliszt.com/2023/07/summary-software-innovation.html

    https://blackliszt.com/2023/07/summary-wartime-software-to-win-the-war.html

     

  • Anthem Needs my Feedback and Reveals Deep Problems

    I’m glad I have health insurance; I’ve had some health problems, now under control, that were expensive to fix. My insurance company, Anthem, has paid the claims.

    Still, I can’t help being impressed by how many actions of this health insurance giant are stupid, wasteful, incompetent and worse. To be clear: I don't claim they're worse or better than the others; they just happen to be the one that I have.

    I’ve just had a new stupidity inflicted on me by them that is low on the “it matters” scale, and high on the “a smart high school student could have done better” scale. What's interesting is that while the stupidity itself is minor in the overall scheme of things, it's the result of a serious dysfunction causing widespread trouble and hassle for patients and providers, while adding expense to Anthem. The extra painful thing is that, for anyone with a moderate amount of software knowledge, it could easily be fixed!

    When I think about all the big, important, highly paid executives involved in and in charge of this, it does make me wonder whether we’d all be better off if they refused to hire any more people with college degrees for any job, and in particular, management.

    The only reason this story is worth telling is that it's not about Anthem; it illustrates how  the techniques taught in business and computer science schools and embodied in endless standards and regulations keep large organizations locked into acting in ways that are both expensive and ineffective.

    Summary

    The root of the issue is my company negotiating a new plan for its employees. Anthem then sent an email (how modern!) welcoming me and inviting me to go digital with card images and an app. I suspect there were self-congratulatory executive meetings about the wonderful modernity of all this. See this for the inexcusably bumbling reality.

    A few weeks ago I went to the dentist, got an exam and cleaning and then scheduled for a crown. My dentist submitted claims as usual and got denied, and I received denial EOB's in the paper mail. I sent my new digital card to the office, and that led to another denial and another paper EOB. Finally the office person was able to get through to a human being at Anthem and somehow resolved the issue.

    Then I received a survey from one of Anthem's contracted firms to see how impressed I was by Anthem's new plan signup experience. The survey design and implementation resembled a 1995 paper-based process cluelessly "updated" to use, uhhhh, computers.

    The whole mess could have been avoided by having a simple system that took incoming claims and checked them against a database of patients and plans, including plans that had been updated and/or replaced. When a claim arrived for a person with a no-longer-in-effect plan, a simple lookup would enable translating it to the new plan information and everything would take place seamlessly, with ZERO action, trouble or inconvenience to patient or provider. You wouldn't even need to send out a survey to ask how the transition went, because it would have been seamless.

    An Email asking for Feedback

    Some Anthem exec wanted to get some customer feedback about how their new plan roll-out was going. I got an email. Here’s the lead paragraph:

    1

    As you’ll guess from my past blog posts regarding Anthem (a summary with links is below), I clicked, hopeful that they’ve gotten moderately competent, but suspecting that some new sophomoric amateur-hour performance will ensue. (BTW, I’ve always been curious about the use of “sophomoric,” since it is the second year of a four year education program. Shouldn’t a performance that’s worse than “sophomoric” be called “freshmanic” or something?)

    Sure enough, I clicked, my browser came up and showed me … a blank screen. Yup, nothing. Nada. I refreshed, tried again, same results. Then I thought, “remember this is Anthem we’re dealing with here; what would someone who dropped out of Computer Engineering 1.01 do? Sure, they’d test their stuff on their favorite browser and declare it working!” I strongly suspected that the kid’s (or highly paid seasoned professional with the skill of a kid who dropped out) browser was today’s most widely used one, Chrome. The next one is Safari, Apple’s browser on the Mac. Behind Safari’s share but still with substantial use are Firefox and Microsoft’s Edge. I copied the URL, brought up Edge and plugged it in. It worked! But it failed totally with my Firefox browser. I visit a large number of websites, and this is the first time in years that I’ve gone to a site that managed a complete face plant on Firefox. It takes a certain kind of perverse skill to pull off a feat like that!

    With such a great start, who knows what joys would follow? Could this be a candidate for “freshmanic” status??

    The Survey

    The very first question puts this survey in the running for un-great-ness. Would you take a look at the never-before-seen way for formatting answers to a multiple choice question? With the major choices ending with “that”?

    2

    The second question continues the fun.

    3

    After asking me whether I remembered receiving this or that communication and answering “no,” the next question was always an huge graphic reproducing what they sent with the question (in effect) now do you remember? Not once — several times.

    Finally came a request to allow Anthem to follow up with me to help improve their experience. To see what would happen, I said yes.

    I then got a form in which I was supposed to enter my name, telephone number, email, and best time to reach me. Right. You already know all this. Anthem gave you my email with an ID. All you need to do is give them my answers with the associated ID. That would make it easy for me. Instead, in typical brain-dead, big-corporate manner, you make me enter all the information you already know AGAIN. So NO, I’m not going to fill it in. So I hit Next. What do I get? You can guess, I bet. Yup!

    4

    Lots of red with the same questions in red backgrounds. Of course if this stupidity was supposedly all about protecting my privacy, they could have said something about that, apologizing for the inconvenience of entering information they already have. But no. Just ERROR. And I've already agreed to have my answers and identity shared with Anthem!

    I could have just closed the browser, but they provided a button labelled “log off.” Which leads to a screen titled “Logged Off,” informing me of my “success” in logging off. But I haven’t “exited” the survey! They close with “Please close your browser to exit the survey.” OMG! If I don’t close the browser I haven’t “exited” the survey even though I’ve “logged off of this survey.” All these years in the business and I’m only just now hearing a new level of sophistication, about how “exiting” and “logging off” are different. Could I possibly screw things up here?

    The Anthem Market Strategy and Insights team

    I went back to the original email, and clicked on the link “To find out more about Anthem’s research surveys.”

    I found out lots of interesting things like how “we’ll never ask for any personal data in our surveys, like social security or ID numbers.” I guess my name, phone, email and (on another question) age aren’t personal. Who knew?

    I found out that the geniuses who put together this survey are just one of a crowd:

    5

    Always friendly, always available to answer your questions:

    6

    I could go on, but what’s the point. Anthem is incapable of doing the simplest kind of market research on their own, so they turned to a bevy of outsiders who can’t do it either, but Anthem can’t tell the difference between good and bad, so who cares? Except it takes a certain kind of genius to consistently pick the worst on multiple dimensions. I guess that’s why they have a “team” working on it – no mere “department” could do it on their own.

    How Anthem could leap decades ahead and get to 2010

    Anthem doesn't need to invent a thing. All they have to do is copy methods that are decades ahead of the ones they use — and are quicker, cheaper and more effective to boot. Here are things they could do:

    • Make the transition of employees to a new plan seamless.
      • The best thing would be to keep the plan number the same and put in a switch to adjudicate and give plan information according to the date of the switch-over. No new cards or numbers, digital or otherwise!
      • The next best thing would be to issue new numbers but keep a database of employees/patients enrolled in the prior plan, and automatically update things like newly arriving claims with the old numbers to the new ones as needed.
      • If you do one of these you won't need a survey!
    • Dump the whole survey thing.
      • Track customer actions in detail like modern web companies do and detect when things don't go as they should. If someone isn't getting what they want, detect it and fix it, and give them an opportunity to tell you what's wrong at the time they experience it.
    • Dump the whole paper thing.
      • I've opted out of paper every way I can at Anthem. I still get loads of paper mail. Why is it so hard?
    • Fire your design department and start over.
      • I get lots of to-the-point communications from Amazon. Copy them!
      • Keep it simple. Test everything in small scale with real people before inflicting it on your customers.

    Those are just highlights. There's lots more Anthem could do if their august team of highly experienced professionals would stoop to it. See the next section for some samples.

    Past Anthem issues

    Just to make it easy for anyone researching giant health insurance company outstanding achievements, here is the list of Anthem issues that I’ve looked into.

    Again, I make no claim that Anthem is better or worse than any other insurance company. It’s just the one I experience, so theirs are the stories I tell.

    Sometime in 2014 Anthem “lost” tens of millions of patient records. And completely botched telling their customers about it.

    I discovered on my own that my data was in the stolen data. Anthem then offered a worse-than-useless plan to “help” me.

    Somebody at Anthem decided that I would be cheaper to insure if I acted better and made an expensive botch job of offering a pre-paid card as an incentive.

    At one point they decided to send me an email to get me to use a primary care physician they had selected for me. Everything about the experience was a nightmare. A case study in stupidity.

    Then I discovered on an offer on Anthem’s website to enable me to pay my patient co-pays. Wonderful idea! Except it doesn’t work. Not just a big screw-up, a huge face-plant.

    While I was discovering the joys of Anthem’s inability to help with co-pays, I got and responded to a customer survey. It was a model of how to p*ss customers off, not to mention being decades behind standard practice.

    Most recently I discovered that my insurance ID changed and that they botched the change and the wonderful new app that was supposed to replace my card.

    Conclusion

    Big companies can't build software that works. They can't even do surveys. Hiring people from Big Tech companies doesn't help, because they can't do it either. The good news is, that gives LOTS of room for small, innovative groups that get stuff done that people need, building effective software that works quickly along the way. Hooray!

  • Why Can’t Big Companies Build Software that Works?

    Years ago when I encountered big-company software flubs that were screamingly bad I would wonder about them: all those people, all that money, all those MBA-led processes and controls — how could they screw up basic software things? What's wrong with them?

    Now I know more. I know the flubs I saw weren't exceptions. They were and are the rule. I know that big corporate and government organizations are incapable of building reliable, cost-effective software. I have identified many of the contributing causes to this disturbing phenomenon, but there's much that isn't known. What's worse, all of the important people, leaders of academia, government and industry, ignore and/or deny this fact. 

    Yet Another Inexcusable, Simple Stumble

    I've covered many big organization face-plants in the past. In order to make it clear that the awfulness isn't just crippling software fails but encompasses a broad range of consumer-dissing inconvenience, I'll show some software that "works" but puts customer inconvenience front and center.

    I got a bill from my gas utility provider. Utility company billing has been around for awhile, right? Treating your customers decently and getting paid should be high on the list of any company's priorities, right? How long have auto-pay and e-billing been around — just flew out this year, didn't they? Bad joke. We all know that auto-pay of some kind has been around for decades, and so has some form of e-billing. So how long do you think it should take for a giant, lumbering company to modify their billing software so that auto-pay is handled reasonably well? Common sense says it should only take a couple months, but rich, giant company maybe a year? Two years? Let's say that by any reasonable measure it should have been completely nailed by 2010, over a decade ago.

    Let's look at a bill I just got from my local gas utility. It was an e-bill — hooray!

    Sj 1

    I read the above. I immediately think — hey, I've got this account on auto-pay; what's this with Click here to make a payment?? I'm on auto-pay!

    I keep reading and next see this:

    Sj 2

    Great. If — If — If I'm enrolled in Auto Pay? What??? You guys don't know if I'm enrolled in auto-pay?? Your software went to my account master record in your database. You extracted my name, billing address, e-mail address and current amount owed. It's hard to imagine that my auto-pay status isn't also there — after all, you knew enough to send me this e-mail.

    Couldn't you possibly, while you were there with your ancient gnarly claws on my account data, also have extracted my auto-pay status and congratulated me for being enrolled? Couldn't you have assured me that this e-mail was for my interest only and was being paid as usual?

    Of course you could have. A trivial addition. Instead you make me wonder whether you have somehow dis-enrolled me as a sneaky way of tacking on a late charge? How can I find out? Then you give this:

    Sj 3

    Yeah, dial that number and wait around. Send an email. Sure. Or go to your generic website and try to pass the obstacle course to getting the information I need. That's what I did. I got through the obstacle course and verified that I'm in auto-pay and in fact owe nothing. Thanks for nothing, South Jersey Gas.

    This is such an obvious thing. Maybe they're too small? It turns out that South Jersey Gas is a public company with $1.5 Billion in annual revenue and 1,100 employees. They serve 400,000 customers.

    Maybe they're desperately searching for programmers? I looked at their job openings. No programmers. But there is a technical kind-of job that they're seriously looking to fill:

    Sj 4

    Yup, a complete b.s. job, whose only relevance is to produce reports and do things to make the bureaucrats in government agencies who wouldn't know a line of code if it bit them on the ear stay calm.

    Here is the first sentence of the job description, which is followed by paragraphs of similar verbiage:

    The role will be responsible for establishing appropriate Critical Access and SOD rule sets for different applications (Workday Financials and HCM, Oracle Customer Care & Billing (CC&B), Maximo and Hyperion), managing IT organizational policies and standards in support of legal and regulatory compliance needs, designing and testing general IT and organizational information security controls and interfacing with Internal Audit and business leaders to ensure that controls are designed and operating effectively. 

    Got it? I looked through the rest of the details, and nowhere was "common sense" or "assure at least moderate levels of customer respect" mentioned or even hinted at.

    Conclusion

    The South Jersey Gas e-mail bill notice is a hardly-worth-mentioning issue given the massive fails that these organizations regularly commit. But when you consider that auto-pay is a good thing for customers and for the gas company and that about 5 minutes of real work is all it would take to fix the problem, and you think back on the important, well-paid job they're advertising for, you realize that the software problem is wired into the system and that everyone whose opinion matters thinks things are fine.

    Maybe they should buy software instead of building it? They do buy software. They can't handle that either. See this for a great example with links to more.

     

  • Why Can’t Big Companies Build or even Buy Software that Works?

    Many large companies depend on software. They often have staffs of thousands of people using the best methods and project management techniques, supported by professional HR departments and run by serious, experienced software management professionals. They can afford to pay up so that they get the best people. Why is it, then, that after all these years, they still can't build software that works?

    Some of these giants recognize that they can't build software. So they buy it instead! Surely with careful judging and the ability to pay for the best, they can at least slap their logo on top-grade software, right? Sadly, the facts lead us to respond … not right.

    What company doesn't want to be part of the digital revolution and have an app? If you're a major health insurance company, why wouldn't you replace old-fashioned insurance cards with something always up-to-date that comes on an app?

    As an Anthem customer, I can see that they've gotten with the program. I got this email from them:

    Capture

    An app, huh? Why is it called Sydney? First, let's keep it simple. They say I can now download a digital version of my ID card, so let's try that first.

    I clicked on the link, which brought me to the main Anthem login. I logged in. What I expected was normal website behavior, a deep link to the right page, but  having to login before getting there. This "exotic" technique, standard practice for over a decade with websites that care about their users, was beyond the professionals at Anthem. After logging in, I got to my profile. Where's my digital card?? I guess it's one of their intelligence and mental status tests, where they count the clicks and the time it takes for you to get where you're going.

    Hoping to succeed, I scrolled down in the Profile section and hit gold. I saw this:

    Capture

    That wasn't too hard! Mobile ID cards! Let's see.

    Capture

    Nothing about seeing it, printing it or emailing it. Just an option to turn off getting a physical card in the mail, and a casual mention (with no link, of course) to "our Engage mobile app." What happened to Sydney??

    I thought I had gotten through the usual Anthem obstacle course in record time. Nope. Dead End. There are a lot of people these days screaming about how bad disinformation is and how it needs to be stopped. Hey, guys, over here….!

    Back to the home page. Look at all the menus. Check all the drop-down lists. Under "My Plans" there's something called "ID Cards." Bingo! An image of our cards, front and back, with options to print, email, etc. as promised!

    Nothing about an app, Engage, Sydney or anything else.

    Alright, Anthem, I've had enough of your website. Let me go to the Play Store and check out Sydney. Here's what they say it is:

    Capture

    Sounds pretty good, right? What can it do? Let's see:

    Capture

    Seems like it can do HUGE amounts of stuff!  Let's keep going.

    Capture

    OK, I've got it. Maybe "Engage" is something Anthem's own army of programmers built. Maybe it was crap and management decided to buy some best-of-breed software. Makes sense. Perhaps some of the hundreds of programmers no longer working on Engage can be assigned to update the website and make it kinda sorta accurate and usable, you think?

    No doubt Anthem management exercised great care to assure that CareMarket did a great job and was giving them a proven app that customers loved so that when it went out named Sydney, Anthem's reputation would go up. Let's see the reviews:

    Capture

    Over 2,600 reviews. That line by the "1" rating is pretty darn long. Looks longer than 2 to 5 added up. I guess Anthem had trouble threatening enough of their employees into giving 5 star reviews to get the job done, right?

    Let's sample a couple of reviews. Here's the top one when I looked:

    Capture

    "This is the worst app I've ever encountered." Error messages. Failed searches. There's a response from the vendor:

    Capture

    Hey guys, she already gave you "a brief description." Do you test your software? Give it to normal people to try before inflicting it on your innocent, unsuspecting customers? Skimming down, I see that pretty much the same response is given to every each tale of woe. Pathetic.

    Here's a sample of other reviews:

    Capture

    Capture

    Capture

    Capture

    Capture

    Get the general drift…?

    This app has been downloaded 500,000 times!! The pain and frustration Anthem is causing is hard to fathom. Why is anyone at Anthem involved with Sydney still employed there? Silly question. Did anyone lose their job after the giant hack at Anthem and the catastrophically bad response to it that I've described?

    Maybe they should hire people from the big tech companies to do stuff like this. Those people really know how to build great software! Uhhhh, not so much. Here is specifically about Facebook's app. For more see this and this and this.

    This big-company software effort is bad beyond belief. I can't comprehend how it is that they pay people big bucks and come out with stuff like this. From what I can tell, though, governments are in close competition for the "prize" of doing the worst job of building and managing software. It's like there's a competition. See this and this.

    The whole world is up in arms about the pandemic. Big powerful people and organizations are taking it seriously and making changes with the intention of fixing the problem. When it comes to the software pandemic, however, everyone just whistles and waltzes along like there's no problem. Everyone just expects and accepts awfulness, acting like it's just how life is.

    It doesn't have to be this way.

  • The IRS Anti-fraud Contract with Equifax is Good

    First there was the furor that Equifax was hacked, putting millions of confidential consumer records in criminal hands. Next there was the furor about Equifax's response. Now, our in-bred elites are outraged that the IRS would award a sole-source contract to Equifax for, of all things, anti-fraud! Outrageous! Equifax can't protect itself, and now our genius IRS awards them millions of dollars?!

    Sadly, this is yet another example of pathetically ignorant people expressing outrage about a perfectly normal and sensible action by the IRS that has nothing to do with Equifax's inexcusable malfeasance in protecting consumer data.

    Here's the story in a nutshell.

    Equifax

    Equifax is one of a handful of companies that gathers and sells information about consumers, much of it confidential. It is a public company that provides an essential service to its customers, which are predominantly credit-granting businesses. The core of their business is receiving detailed transaction data from banks, aggregating it and selling it.

    The Equifax breach and follow-ons

    As usual with breaches, it happened long before the company became aware of it. Also typically, the company waited a long time before making an announcement. Equifax executives added an extra unsavory twist to the events by selling stock before the breach was announced. The response of Equifax to the event, which included a bogus offer of consumer protection against identity theft, was awful. Extremely little hard-core information about the breach has been released.

    With this breach, Equifax joins the ranks of large institutions, private and government, that demonstrate their inability to keep their data assets safe. This is an ongoing scandal for which there are solutions, but none that major institutions care to use. I have written extensively about this.

    The IRS contract

    The IRS awarded a sole-source contract to Equifax for access to confidential consumer credit data — exactly the same kind of service that Equifax provides to most of its customers. Public figures were outraged!

    Capture

    If the IRS contracted with Equifax to help apply its expertise to keeping IRS data secure, the outrage might have been justified. But Equifax does not sell those services. What they sell is data, data that the buyer can use for many purposes — often for credit-worthiness, but sometimes to help verify consumer identity. The data was valuable for this purpose before the breach, and remains valuable today.

    The data that was stolen was, of course, a snapshot of what Equifax had at the time of the theft. Since then, data has continued to pour into Equifax, updating and augmenting the data it already had. By using this additional data in special ways, the IRS could improve its ability to prevent identity thieves stealing taxpayer refunds, for example. I have no idea if the IRS will be smart enough to do this (I suspect not), but in any case they need the data! Without it, the IRS will be even more vulnerable to theft and fraud than it already is.

    For the Senators to castigate the IRS for buying data from Equifax shows that they don't have a clue about computers and software — they don't care to know the difference between services and data, for example. But we already knew that.

    Clueless about Technology

    What this is really about is that most people, including business, government and media elites, are clueless about technology. Which doesn't stop them from pronouncing about it with great confidence. As it turns out, I wrote about before, using the IRS and e-mail to illustrate the hapless opining of public figures about Bitcoin and Blockchain.

    When things go wrong, "experts" are called in, and more money is spent doing the same useless things that let the problem happen in the first place. With the side effect that everything is even slower, more error-prone and vulnerable than it is today. The current round of posturing by public figures helps nothing. Sad.

     

  • The Value of Computer Industry Advisory Groups

    The value of the famous computer industry advisory companies is much less than most people think. Take the example of Gartner Group, whose exemplary customer service I discussed elsewhere. Gartner employs some highly knowledgeable, helpful people. Gartner wants you to think is that it's the place to go to find experts. And you can find them there, as I explain here. But as a company, Gartner and its kin are mostly formalized gossip services for big-company IT folks.

    The origins of advisory services

    Imagine, in pre-Gartner days, groups of IT execs getting together by geography or at industry conferences. They all naturally want to learn what others have done in terms of purchasing gear, because it’s expensive and things change a lot. As they exchange information, it’s clear that some are kind of behind the times, others have made similar choices to everyone else, a few are out there – with new stuff from the usual vendors, or with something new from a new vendor.

    Everyone wants to avoid career risk. There’s a strong tendency to reversion to the mean, i.e., doing what most of the others are doing. The tendency is strongest concerning vendors, and after that products within a vendor; e.g. “I always buy GM cars, when I get more money I’ll upgrade to a Caddy.”

    Gartner comes along with a deal: you tell me what your choices are and a bunch about your business, and I’ll put everyone’s choices together and feed back to you a better, broader-based version of the results of the networking and gossiping you used to do, so you don’t have to spend the time. Even better, I’ll put it in an authoritative package so that if you’re ever questioned about your choices by non-IT people in your organization, you have our endorsement to fall back on; e.g., “yes we had a disaster, but it happened to everyone who made the best available choice at the time.” What a huge win, and cheap for the value.

    For example, here's how they explain their most famous graphic, the "magic quadrant."

    Gartner quadrant

    What is it? Yes, there's some dressing, but a vendor only gets to the best place, the upper right, if lots and lots of people buy their products. It's little but a graphical illustration of products by popularity.

    This is the value they add. All their categories and analysis is just a prettied-up version of what everyone tells them.

    The value of advisory services

    So who actually listens to Gartner and follows their advice? Exactly the kind of buyers who avoid risk at all cost. They go to Gartner, who tells them what people like them are buying. So they can buy the same thing. And be safe!

    If you're running a big-organization IT operation, commercial or government the way most people do, that's exactly what you want. Your operation is, almost by definition, bloated, inefficient, over-the-top expensive and riddled with problems. Saving a few bucks or doing things a little better isn't going to get you promoted. When a disaster strikes, the fact that your decisions were "mainstream" tends to bullet-proof you against recriminations.

    In this context, Gartner is indispensable. Your decisions weren't just mainstream; you can point to Gartner — Gartner says they're excellent decisions. So there!

    Advisory services and innovation

    What if you run an organization and for some reason are really motivated to innovate? What if you're a hot young tech group building next-generation products and want to find buyers? What is Gartner's role?

    It's simple: if you really want to innovate, avoid experts at all cost. Period. Gartner and anyone like them included. Here is lots of detail and a juicy example of why.

    The people and organizations who value Gartner are least likely to buy from a 1% market share vendor with a product that’s ahead of the market. Who is most like to buy such a product from such a vendor? Someone who is in big trouble, desperate, or one of those thinks-for-themselves buyers at the front end of the Geoff Moore adoption curve.

    How about the small to mid-market? Similar rules. It's true that the buyers mostly don’t know or care about Gartner. Who cares about people who buy giant, expensive systems from HP, IBM, EMC and the rest? They focus on their business, and don’t give a hoot about products and vendors – though someone they’ve heard of would be nice. They’re like homeowners buying a heating/cooling system – mostly they’re buying from the local dealer, who they depend on to sell them something good and then support it. The dealer matters as much as the product. Gideon Gartner can just Giddy-up out of town, he doesn’t matter in this world. But at the same time, the buyers are still mostly failure-avoiding. They don't want innovation. They want works and cheap.

    Product Innovation

    There aren’t a lot of ways to break in with a new technology product. The ways I’ve seen are pretty much summarized in this blog post, which includes links to further material, including my book on the subject.

    The most important concept is simple. 95+% of the market will never buy from you. Ignore them and their gossip-aggregators. The vast majority of the big, Gartner-esque buyers won’t give you the time of day. You need to find some narrow market niche to focus your energy on, and dominate that tiny sub-market. Then you can grow from there.

  • The Ransomware Hack Attack: Lessons from the Experts

    The Wannacrypt ransomware attack is in the news because it's causing havoc world-wide in major corporations and government institutions. It's a textbook lesson in a number of subjects including (but not limited to): the hopeless incompetence of major institution management in general, and IT management in particular; the worthlessness of most people said to be experts; how dead simple most cyber-security is; the rank illiteracy of otherwise highly educated journalists about computing; the incompetence of our super-spook institutions.

    The authoritative New York Times

    Of course, we turn to the venerable NYT to get the facts about this important story. Here's the head:

    A1

    It's clear from the headline that the substance of the story is beyond the grasp of the generally super-bright Times authors (look at the bottom of the story, the author had lots of help), so we're going to have a treat: lots of experts!

    First some facts

    Let's start with a couple simple facts.

    The software in question is "ransomware" that users are tricked into running on their computers. The software is normally an attachment to an email message that an unwitting user (being kind here) clicks on. Once it runs, the software encrypts all the files on the computer, making them unusable. It then displays a helpful announcement of what it's done and how to get your files back. Here's a sample, taken from a nice summary of the situation:

    A2

    At this point, most people panic. Loads of hospitals in the UK were infected, for example, and mostly shut down.

    There's more! Once installed, the software probes all the computers connected to the same network, and tries to infect them with the ransomware using an error in some deep-in-the-guts thing normal users would never encounter called SMB. This means that once a single user in an organization has fallen for the bait and gotten the software, it quickly spreads. This part of the evil software is the "worm."

    Here's how the New York Times describes it:

    A3

    The underlying reality — the important facts

    Here are the most important things to know about this "audacious global cyberattack."

    • The ransomware spread by the usual means: emails to gullible users. To their credit, the Microsoft Windows Defender group quickly identified the problem and released an update that detects and removes it.
    • Only obsolete and improperly maintained Microsoft Windows computers were affected by the worm. Loads of systems were hit in hospitals running Windows XP, which Microsoft stopped supporting years ago. Supported versions of Windows that had installed all recent patches were not hurt. The relevant patch was released months ago. It is the worm part of the malware that infects servers, which is particularly harmful.
    • The bad guys are only charging $300 in Bitcoin to unlock your computer. That's a small price to pay to learn the lesson of keeping your system up to date!
    • If you really don't want to pay, all you have to do is wipe your machine and restore it from a backup. And then maintain it properly. I gather from all the furor that on top of using obsolete software, the affected sites fail to follow standard backup procedures.
    • The bad software itself has been publicly available for months, ever since being walked out of the NSA and published. It was only a matter of time.
    • It's not exactly genius software. A clever guy managed to do a simple thing that disabled the worm aspect of it worldwide! Details here from the guy himself.

    The Experts weigh in

    Since my regard for experts could hardly get lower, the NY TImes article changed nothing. But perhaps some examples might be amusing.

    I love this one:

    A4

    The price goes up to $600 if you delay. Let's assume everyone delays but pays. That means no less than $1B/$600 = 1,666,667 sites would have paid, if the experts are right. I checked the relevant Bitcoin accounts a few minutes ago, and the total had yet to exceed $30,000. Way to go, experts!

    I also love the choice given: "pay the digital ransom or lose data." Right. First of all, you're stupid because you're running obsolete software. Then, you can't restore from a backup? You deserve to lose all your data, and then your job — remember, we're not talking about naive consumers here, we're talking about richly paid computer professionals!

    Our next expert dares to be named:

    A5

    Here's the part I like: "Despite people's best efforts, this vulnerability still exists…" Of course it does! Updating Windows makes the problem disappear. You can't make people update their software — even though it's their job to maintain it!!

    "…experts said that computer users in the United States had so far been less affected than others because a British cybersecurity expert inadvertently stopped the ransomware from spreading."

    First, the guy who stopped the worm part was brilliant. He did what he did very much on purpose — he just referred to what he did as something "accidental," being sleep-deprived and modest. Second, what he stopped wasn't the initial infection into a site, but the spread of the worm once it was in. There were loads of US sites infected — the numbers are random, as you would expect from whatever email list the bad guys used, and the odds of professional users clicking on the attachment.

    The Times itself attempts to explain how the clever guy managed to halt the worm aspect of the malware. Completely screwed it up. Sorry guys, maybe you should stick to quoting experts who get it wrong instead of being obviously wrong yourselves.

    Then we have security experts weighing in:

    "Yet security experts said the [Microsoft] software upgrade, while laudable, came too late for many of the tens of thousands of machines that were locked and whose data could be erased."

    The Microsoft software upgrade was made months ago. It was not too late. It's the people responsible for the machines in question who are too late. If they let their data be erased it's on them — either pay up, wipe and restore from backup, or slink away in shame.

    As to the NSA that created and released the software in question: shame on you. You probably have yet to implement the measures that would prevent more of the same in the future.

    Summary

    When you read stories like this, it's natural to form a set of impressions, including:

    • There are mysterious hackers out there who are really smart and really bad.
    • The evil hackers can cause havoc.
    • All we can do is bring in experts and try to clean things up quickly.
    • Let's hope it's not worse next time.

    All these are reasonable thoughts for a layperson to have, reading the published material.

    The truth of the matter is closer to the following:

    • The richly funded NSA develops evil software and can't keep it secure, in spite of having a budget larger than most countries.
    • Opportunistic hackers comb through stuff and sometimes put together something that could make some money.
    • A shocking fraction of the big government agencies and corporations fail to follow the most basic computer maintenance procedures (keeping software up to date and making backups), in spite of spending megabucks on IT, and so are vulnerable.
    • The experts quoted in news stories are ignorant and/or wrong, along with the stories themselves.
    • The guy who stopped the worm part of the software from working was at the opposite end of the competence spectrum from all the highly-paid executives who weren't doing their jobs.
    • Most organizations will change nothing, so something very similar will happen again.

    Sigh.

  • Software Giants: Image and Reality at Facebook

    I am perpetually amazed at the flood of reverent articles about the wonderful big software companies that are inflicted on us. How great are their leaders! How wonderful it is to work there! Everyone should emulate their business practices! Their products are awesome!

    The reason why the sycophantic flood continues is based in simple economics, but why most people appear to buy the b.s. is beyond me. You don't have to be a hardened cynic to see past the image.

    This subject is worth a book or two. I've contributed just a couple blog posts. This post is another one on the wonderful Facebook, which (supposedly) does so much to demonstrate software excellence and contribute to our betterment.

    Why Facebook is Wonderful

    An article has just appeared about the wonderfulness of Facebook. The article is an interview by John Battelle with Lori Goler, who is "VP of People" at Facebook, leading the company's growth from 500 employees to 15,000. Here she is:

    Lori Goler

    She sounds like a really nice person. I've worked with the interviewer, John Battelle, at one of his prior ventures, and he's a great guy.

    The whole article is worth scanning. But the subhead gives you the idea: Facebook is "the world's most admired employer." Here are a couple quotes from Ms. Goler:

    We are really looking for builders…What goes along with that is a learning mindset.

    Being a strengths-based organization is a place where you are really looking to put people in roles where they are doing work they enjoy that plays to their strengths. … It’s where you get the best teamwork. It’s where the people are able to do the best work of their lives.

    For us, the mission is, “To make the world more open and connected,” so it makes sense that our culture is open and connected. Then internally, we reflect that culture.

    What we find is that what it really means is that people have all the context they need to be able to work with great autonomy in the organization, which of course leads to greater innovation and greater impact. It’s been a virtuous cycle for us.

    According to the article, Facebook is a great place with a socially uplifting mission, populated by great people who are always learning and in roles where their strengths are tapped and their work has impact is fulfilling, in a completely open and supportive environment. Wow. Who wouldn't want to work there?

    Another view of Facebook

    For a contrasting view of Facebook, I recommend reading this book:

    Chaos monkeys

    Warning: I had to force myself to get through the book; the author's self-described behavior was distasteful, to put it mildly. But it made the rest of his descriptions the more credible, and he said nothing that contradicted my inside knowledge.

    The book has gotten lots of attention. It's been reviewed by major media, for example the New York Times:

    NY Times

    And by tech journals, for example Tech Crunch, which declared it was the "year's best non-business book about business"

    Tech crunch

    The book is #1 in several categories at Amazon. The top-rated review is telling:

    Chaos review 1

    ….

    Chaos review 2

    Perhaps you can see that there is a contrast, shall we say, between the wonderfulness of Facebook as presented by its leaders and the reality. But this makes sense. What was the job of the VP People before getting that job? Marketing! In other words, telling stories to get you to buy stuff. She is continuing to do her job well, i.e., selling Facebook as a great place to work.

    Facebook's Product

    Well, maybe it isn't such a great place to work in spite of all the propaganda, but at least those 15,000 people turn out a great, high-quality product, right?

    Here's a post about software quality issues with a section on Facebook, and here are details about the inability of those 15,000 engineers to turn out a product that has reasonable quality, even after many attempts — as judged by their own users.

    It's not just Facebook. It's Google and the rest. Think about this: with such wonderful employees and huge cash reserves, why can't they make their own products work, much less innovate? If they're so innovative, why does so much of what they "innovate" come from acquisitions? See this for details.

    You might ask, if their software quality is so awful, how did they become so big and valuable? Good question. Zuckerberg made some world-class smart business strategy moves to get it going. See here for details.

    Why this matters

    These observations about image and reality at the big famous software companies have huge practical implications for small companies, managers and programmers.

    I have often observed that when board members want to hire a top executive, or when managers want to fill an important software position, they often value highly a candidate's having done a stint at one of these famous giants. They'll think something like "Facebook has a product that nearly everyone uses; I want to build a product that nearly everyone uses; therefore, I'll hire people from Facebook, and I'll get a product that everyone uses."

    Of course rarely will someone come out and say something like that, but the Facebook (or Google or whatever) aura is so strong, people often act as though they believe it. On the other hand, if you really get the perspective about the inept software giants described here and confirmed widely, you will tend to avoid hiring people from Facebook (or wherever) because you know you're likely to drag down your company to its abysmal level!

    Lots more detail on this and related subjects is in my book on Software People. Or for an illustration from a whole different direction, consider the incompetent doctor and nurse in the PBS civil war hospital drama Mercy Street.

    Hastings
    Nurse Hastings frequently brings up the fact that she served with Florence Nightingale as proof that her opinions are the best.

    Conclusion

    Nothing is going to change. Major corporations of all kinds, even more so the big tech ones, will beat the drums of self-promotion, selling themselves to customers and potential employees. It's in the interest of groups that hunger for money and attention from the big companies to make nice, and trumpet the self-congratulations. The big companies will continue to be unable to innovate, and will instead buy innovative companies. Sometimes the contrast between the image that is widely promoted and the reality gets to me. At minimum, my hope is that you're working at a place that's far better than Facebook, and that you avoid error of attribution I have described here.

  • Innovation: the Barriers

    It's hard to be an innovator. You have to come up with cool new stuff, make it work, and get people to use it. Not easy! Depending on your situation, there can be barriers, active and passive, to being a successful innovator. Lots of people in business and government love to talk about how they're innovative, and how they foster innovation. Hah! In all too many cases, what they actually do is build and sustain barriers so strong and so high that innovation is nearly impossible.

    If you look at my earlier posts on innovation, you may think that I'm a cynic. The reality is that I'm an enthusiastic, life-long believer in innovation. My sarcasm is targeted exclusively at the hollow, creativity-killing rhetoric that too often passes for support for innovation.

    Active barriers to innovation

    What about big companies who innovate? That's mostly rumor and self-promotion, rarely a reality.

    What if you're a small company trying to innovate? The barriers are mostly put up by the large businesses that dominate the field in which you want to innovate.

    Will the big business itself innovate? In spite of all the talk, probably not. It's likely they want to be seen as modern, with it and innovative. It's highly unlikely that they actually want change. This post goes into some detail about the reality behind giant companies that supposedly are great innovators. Why can't big companies innovate? Who knows, but I think the attitude of the pointy-haired boss is a hint:

    Dilbert

    There is lots of information and a few stories about how to out-fox the giants that want to keep you down in my book on building a growing business from a startup. But it's tough. The big guys hold most of the cards.

    Passive barriers to innovation

    Governments are the main source of "passive" barriers to innovation. The barriers are usually in the form of regulations — regulations that can quickly morph into active barriers once you get caught in the cross hairs of one of these innovation-killing agencies.

    You think those regulations are no big deal? The current code of federal regulations is massive, and getting bigger every day. Here's a quick glance at its size:

    CFR

    Of course, no government agency will ever admit that what they are doing is preventing innovation. They are protecting consumers! Enforcing fairness! Doing good stuff, the peoples' business! That's what they say. Sometimes it's even true. But in most cases, what they are really doing is protecting existing businesses and professionals from competition. They do this by putting increasingly burdensome and expensive barriers to new products and services entering the market, and competing with the establishment.

    Regulatory barriers to innovation are everywhere, in nearly every industry. Why isn't there a huge outcry? Simple:

    • The companies and people that are on the "inside," benefiting from the barriers, vociferously support "protecting consumers" or whatever the b.s. cover story is.
    • The people who would benefit from the innovation don't see the innovations, because they don't exist yet, and so can't really lobby against the barriers.
    • It's just the way things are. Who has the energy to "fight City Hall," particularly when the innovative benefits don't exist yet because of the barriers?!

    The barriers are everywhere, preventing innovation or worsening convenience and price. The barriers are in old, tangible things like a store being able to sell liquor or a car company being able to sell its cars. More importantly, they're in newer, life-issue things like nearly every aspect of healthcare.

    Barriers to innovation in healthcare are massive, and getting worse. The barriers aren't called that, of course. The government agencies are protecting our health and privacy! But when you lift the covers, it is easy to see that what is really going on is a rapidly metastasizing federal bureaucracy that prevents life-enhancing products and drugs from being invented, and massively increasing the cost and slowing down the relatively few innovations that squeeze through the gauntlet.

    Conclusion

    We're clearly in the middle of an innovation bubble. Everyone says they want it. Companies and government agencies claim to be fostering and promoting it. I'm someone who has worked in the innovation trenches for decades. I try to innovate myself, and help others to do it. It's not easy. That's why I get so cynical about all these innovation-smothering institutions who are so loudly in favor of innovation. Their words say one thing and their actions say another. All their innovation amounts to is a pile of marketing rhetoric, an attempt to make themselves appear to be modern.

  • Gartner Group: Showcase of Big Company Customer Service

    Giant, powerful organizations nearly always do two things really well:

    1. Wax eloquent about how concerned they are with respect, privacy and customer service.
    2. Treat their actual customers like disposable pieces of crap.

    I've seen lots of examples of this over the years. I've written about it, for example illustrating how HP disrespects its customers with simple things like hard-to-get-out-of email subscriptions you never subscribed to. I've just encountered an even grosser example inflicted on me by the world's leading IT consultant firm, Gartner Group.

    Gartner Group

    I've known a number of Gartner employees over the years, and most have been hard-working, respectful, knowledgeable people. But Gartner is a big place. They purport to teach the world's companies how to do IT. So how does Gartner itself do IT?

    Here's the basic story with Gartner:

    2 Gartner

    In addition to thousands of employees, they're worth billions of dollars:

    1 stock

    Their range of activities is amazing. It's clear that they teach IT best practices to important companies all over the world:

    3 Gartner

    It's hard to believe that Gartner's own IT practices wouldn't themselves be world-class. Wouldn't you expect a music teacher to be a master musician?

    Gartner email

    Somehow I ended up getting spammed by Gartner. I'm not sure how. I got this email:

    1 gartner

    I didn't ask for it, and I don't want it.

    So I went to the bottom, and was assured that Gartner is committed respect, privacy and all the usual big-company boiler plate. And even better, I can unsubscribe!

    2 Grtner

    So what happened? Did I get that satisfying one-click experience that responsible spammers provide? You know, the one that immediately says, "you're out! But if you'd be so kind, please tell us why you're going?" You know, like this:

    11

     

    No. Apparently, Gartner emails are much too important to be simply unsubscribed from. When the page popped up, my eye first went to this, which by itself sets a new record for customer disrespect:

    44

    They know my information — they're emailing me! But filling out the form for me? I guess this standard practice is beyond the geniuses at Gartner. Or beneath them. Or they kindly want to make sure I'm qualified to live without their wisdom. Or something.

    Then I studied the top part of the page, which provided the detailed instructions that must be meticulously followed in order to unsubscribe. If you're not good at reading and following instructions, the penalty is eternal pounding by unwanted junk mail from Gartner:

    Gartner

    I have nothing more to say. Gartner, the billion-dollar advisory firm, leading the way, demonstrating the customer respect that big company customer service is all about. Also demonstrating how carefully crafted words are of supreme importance to such large organizations. Actions that match? Not so much.

  • Systemic Issues Behind the Cyber-Security Disasters at OPM, Citi, Anthem, etc.

    Our personal data is stored in the computers at large corporations and government organizations. We now have abundant proof that these large organizations are incapable of protecting our data. This is not a string of bad luck that will soon pass. These large organizations never had good security — they just weren't being attacked. Unfortunately, the security flaws are a direct outcome of the dysfunctional technical and management practices that lead to large-organization IT failures across the spectrum.

    Recent Security Disasters

    The security disaster at the government Office of Personnel Management (OPM) has been in the news recently. Here is a summary, and here is a timeline. OPM knew all about security, and tried its darndest to be secure, spending over $4.5 Billion dollars on a system to prevent breaches, including a recent $218 million upgrade on the security system known as Einstein. All for naught. 

    In the private sector, there was the breach at Anthem, preceded by a string of security disasters at major banks and retailers involving tens of millions of consumer records.

    The Response to the Attacks

    We're seeing the usual responses to the problems.

    First and foremost, try to avoid letting anyone know there's a problem.

    Second, try to draw attention to all the attacks that were thwarted. The OPM is actually bragging about all the attacks they defend against! That's like, when the bank has been totally cleaned out, bragging about how many attempts had been thwarted.

    Finally, talk about how much you care, offer completely counter-productive services to consumers, and spend even more money on the stuff that didn't, doesn't and won't work. Ignore the fact that the incentives are all wrong, that in fact no one cares.

    No one is losing their job. No significant changes are being made. No one is running around like their hair's on fire. Ho-hum, it's business as usual.

    Systemic Issues are behind the Disasters

    Security in large organizations is broken. But that's just a side effect of the fact that IT in large organizations is broken. Not in detail — in principle. When the foundation of a building is made out of jello instead of concrete, you don't fix it by adding more jello, trying a new flavor of jello, or getting everyone to walk slowly and carefully. You replace it with reinforced concrete — pronto! When the foundations are the wrong kind of stuff, making new foundations out of jello will never help. Even if it's jello that costs billions of dollars.

    The Systemic Issues

    This is a subject that is long and deep. All the problems come down to two simple core thoughts: (1) computers are just like all the other things to which management techniques are applied, so standard-issue "good management" will solve any problems; and (2) computer security is just like all the other computer issues, and can be managed using the same standard techniques.

    Wrong and wrong.

    Computers and software in general are radically different than anything else we encounter in our normal lives, and evolve more quickly by orders of magnitude than anything else in human experience. Managing a software building project as though it were a home building project leads to results that are, at best, 10X worse than optimal methods, and at worst, complete disaster.

    Computer security in particular is not just another issue to be managed using standard techniques, which in any case yield horrible results. In computer security, we're dealing with smart and motivated attackers who are at war with us, and naturally use the latest "weapons" in a rapidly evolving arsenal. While our attackers are at war with us, we plod along at a peace-time pace, scheduling security issues like just the other items in prioritized lists. When the armed gang breaks through the back door of the warehouse, we eventually discover the break-in and schedule a response for sometime in the next couple of months. By the time we've installed new alarms, the gangs are already on their third generation of tools for defeating them.

    Computers are different than the other things we manage

    Computers evolve at a pace that is completely unprecedented in human experience.

    Most of the things that managers do to manage computers is modeled on what they do for everything else, and make things worse.

    Computers are incredibly complex! But somehow, we imagine that people with no actual experience with computers can manage them, when we would never let someone who never saw a baseball game manage a team, or someone who never wrote an article manage writers.

    The vendors of hardware, software and services have evolved to provide incredibly expensive, ineffective products and services that are packaged to make top managers feel great.

    Computer security requires war-time actions, not peace-time ones

    Translating from physical security, managers insist that security is about walls, guards and kevlar vests. The bad guys are out there, our job is to keep them out. Wrong. The vast majority of security breaches result from either conscious or unknowing cooperation of insiders. Including OPM.

    The bad guys are at war with us. By the time we've figured out that we've been robbed, the bad guys are long gone. By the time we're just wrapping up the requirements documents for our response, the bad guys have cleaned us out again.

    Once we finally deploy our best defense, the art of war has advanced and our defenses are useless, just like the Maginot Line in World War I.

    Conclusion

    We all know that the definition of insanity is repeating the same actions and expecting different results. In that sense, the approach that large organizations, private and public, take to computer security is insane. All the people in charge propose is doing what they've always done, only somehow harder and better. The alternative approach, while radically different from the current one, is simple, clear and actionable. The people in charge actively resist it today. They've got to embrace it if there is to be any chance at all of improvement in cyber-security.

  • My Anthem Account was Hacked

    I get my health insurance through Anthem. Corporate Anthem was hacked, and the company has made a mess of their customer relations after the hacking, as I've described from receiving their "help." I now see evidence that my personal information was accessed, and Anthem has never told me.

    Anthem and HIPAA

    Anthem is really committed to HIPAA. Here's how they explain it on their website.

    Anthem hipaa

    It's clear from this that Anthem is very committed to privacy and security. Both! Here's some of what they say about privacy.

    Anthem privacy

    And here's some of what they say about security.

    Anthem security

    Anthem clearly had all the bases covered. Except they didn't. What's mind-blowing to me is that, in spite of all the security-privacy-lah-de-dah, someone walked off with the personal information of tens of millions of customers — and no alarm even went off! The breach was actually discovered by an alert grunt in the trenches.

    Anthem sys admin
    Hacking David Black

    Anthem has communicated to its members that they would let them know when they discovered whether any particular member was among those who had been hacked. I haven't heard a thing from them. But I now know that it's likely that my information was stolen.

    I went into the standard Anthem consumer portal a little while ago.

    Anthem header

    I poked around a little, and discovered this little bombshell:

    Anthem last visit

    In other words, "I" had logged in at quarter after one in the morning on Saturday, Jan 31, 2015. However, I personally wasn't logged into Anthem at that time. I was asleep.

    The Good News

    There's good news here! I already knew that Anthem either didn't know whether I'd been hacked or had decided to not tell me, so no change there. My opinion of Anthem was already subzero, so it didn't get noticeably lower. Furthermore, in spite of all this, Anthem executive management will continue to rake in millions, and they're pretty sure that profits won't be harmed:

    Anthem won't hurt earn
    What a relief!

    Conclusion

    Nothing new here. Big corporations comply with all the burdensome regulations, and tens of millions of private records somehow get stolen. The result: lots of face-saving talk that does no one any good, and increased competition-stifling regulation that does nothing to solve the problem. Nothing to see here, people … move along…

     

  • The Anthem of Cyber-Insecurity

    I'm hoping that people will start writing songs about cyber-insecurity, and that a good one will emerge that will be acclaimed as the "Anthem of Cyber-Insecurity." It will be sung quietly by groups of computer users who hold hands as they hear the details of yet another massive computer breach. While singing, some of the much-abused users will be silently praying that their "protectors" get bombed by Facebook friend requests by identity-thieved replicas of themselves, while others will pray for the end of "help" that isn't.

    The Anthem Attack

    I'm one of those praying users, because I'm a member of Anthem, the company that "lost" the personal information of "tens of millions" of its members sometime in 2014; they're not sure how many, whose records were "lost," or when it happened. Here's a personalized communication I received from Anthem:

    Anthem When

    Anthem has made a priority of communicating with its customers about the attack. When you're in the glare of publicity like this, I'm sure great care has gone into each statement on the case. That's probably why I have received more than one missive with the same date that spins things in different ways. For example, the Feb 13 note above refers simply to "cyberattackers" who "tried to get" private information, raising the possibility that their efforts were foiled by the valiant workers at Anthem.

    Check out the identically-dated but substantially different Feb 13 note below.

    Anthem 1
    In this second attempt, Anthem tells us about "cyber attackers" (now two words instead of one) who executed a "sophisticated attack," and "obtained personal information" "relating to" their customers. I guess it was successful? But maybe not, because the behavior of these guys isn't a felony, it's merely "suspicious activity" that "may have occurred." Furthermore, they carefully state that the personal information wasn't the customer's actual personal information, but merely "related to" said personal information. Hmmm….

    What "May Have Been" Lost

    So what information may have been lost during this incident that may have occurred at some unknown time? A fair amount.

    Anthem 2

    Again, what's clear is that Anthem isn't clear. The information "accessed" (wasn't it stolen?) "may have included names, …" But maybe not, we are led to believe. If the information that may have been accessed may have included my Social Security number, why isn't it possible that all sorts of other information was also accessed? We are supposed to be reassured that "there is no evidence at this time" that this actually took place — a nearly ideal way of phrasing something that is supposed to sound like reassurance, but provides full CYA.

    Anthem Provides Protection

    Anthem has a whole website set up to let its members know what's going on, and to let customers know how they can get protection against the possible unauthorized access of their personal information.

    Anthem header

    Here's what Anthem will do: they'll pay a third party to help you out.

    Anthem protections

    If you get in trouble, you can call the service, and they'll help you out. Meanwhile, your personal information may be in the hands of people who were unauthorized to access it. If they are the kind of people who will do "unauthorized" things, who knows what perfidy they'll stoop to?

    Anthem's Additional Protection

    The basic service you get isn't protection at all, as they make clear. Nonetheless, "For additional protection…" — on top of the non-protection they already provide — you can sign up for more. What exactly is this more? Quite a bit! Here's some of it:

    Allclear features

    Wow, and all for free! Let's sign up!

    So you enter your e-mail, and get a code, go to the website, enter the code, and finally get to register for protection.

    What happens next? Here's the page:

    Allclear register

    Wow, this is amazing!

    I have a chance to enter into a website a good fraction of the private, personal information entrusted to a giant insurance company which, while under their stewardship, "may have been accessed" by "unauthorized" entities.

    The security geniuses who kept my information secure want me to give it again to a company that they endorse as being wonderful security experts. Anthem was just terrific at keeping my information secure — it goes without saying that their endorsement of the security of this partner they've just picked is rock-solid.

    These guys are bureaucrats. Read this about bureaucratic security cred. And for more, this.

    Summary

    Anthem's revenues are greater than $60 Billion. They can afford to keep customer data secure.

    Anthem's executives are paid enough to do their jobs well. Last year, the CEO made over $16 million and the CFO over $7 million.

    And yet…

    It took a guy at the bottom rung of the ladder to pay attention and notice something was wrong; had he not cared, the outflow of personal data would still be going on, as it had been for an indeterminate amount of time before the alert employee's observation.

    No system or procedure established by the rich, giant entity had anything to do with noticing the breach, much less preventing it.

    Everything about what they've done since exhibits the same lack of attention to detail and I-don't-care attitude that made the breach possible. What they mostly seem to want is to dash off letters riddled with errors and assurances, focused above all on their public image.

    Their offer of "protection" is a cruel joke, exposing the gullible who accept the offer to further dissemination of their private information.

    Conclusion

    I'm waiting for that anthem as I sit, holding hands in a circle with my fellow users, thinking dark thoughts. And I'm as likely to enter my personal data into the Anthem authorized "protection" service as I am to publish it on this blog.

  • Bureaucracy, Regulation and Computer Security

    There always seems to be a bureaucracy ready to tell you how to keep your computer systems secure; or, worse, to tell you what you must do to be in compliance with the regulations promulgated by the bureaucracy. "It's for your own good," they say.

    If you are forced to comply with some regulation or other, you'd better comply. But you're a fool if you confuse compliance with keeping the assets of your business actually, you know, secure.

    Bureaucrats can't keep simple physical things secure

    Computers are complicated. Construction sites? Not so much. Fences, cameras, sensors, guards and an alert, well-managed staff should do the trick. But when bureaucrats are in charge? Forget it.

    David Velazquez was in charge of security at the World Trade Center construction site. Mr. Velazquez is a Columbia University graduate and had a 31 year career at the FBI, ending as head of the Newark field office. You might think well of the FBI, I don't know, but what I do know is that it's a giant government bureaucracy, and Mr. Valazquez appears to have applied the lessons he learned there on his new job.

    Here is one of the crack guards "on duty" at the work site:

    Sleeping guard
     

    That may explain why a group of guys was able to get to the top and jump off, recording video all the way down:

      Base jumper

    Then a kid slipped through a fence and made it all the way to the roof, unheeded by sleeping guards:

    Security kid

    The biggest, baddest bureaucrats of all can't keep their own computers secure

    Alright, maybe the FBI are amateurs. Let's go to the best of the best, the scariest cybersecurity experts of all, the NSA.

    NSA

    These guys are in charge of keeping us secure from the worst of the worst. A cover story in Wired Magazine told us all about it.

    Wired cover

    Loads of people using piles and piles of super-secret cyber magic are on the case:

    Wired story 1

    If anyone can achieve cyber-security, surely these guys are it:

    Wired story 3

    But we all know how that turned out. It just took one moderately clever person with bad intentions and all the vaunted cyber-wonderfulness was for naught. Among Mr. Snowden's myriad revelations was the previously secret budget of the cyber-bureaucrats of the NSA, an astounding $52 billion. Do you think if they doubled the budget they could have done a better job? Hmmmm.

    Bureaucrats and Security

    Why should you listen to someone who can't do it themselves? If you want to stop smoking, do you eagerly take the advice of someone who smokes? If you want to get rich, do you take advice from poor people? Bureaucrats are sure they're right — because they have no competition, and there's no one who has the power to tell them otherwise.

    Why this matters

    The laughable ineffectiveness of bureaucratic security in general, and cybersecurity in particular, can matter a great deal to you. Here's why:

    • If you do what the bureaucrats tell you to do, you'll spend a lot of money.
    • Following the regulations makes everything slower and less efficient. You'll hurt your business.
    • If you get conned into thinking that following the regulations means that you're secure, you're in big trouble. You will be more vulnerable to business-damaging breach than ever before.

    What you should do is simple: establish effective and efficient security by the best means available, which will typically be unrelated to what the authorities solemnly declare. Then, do as much regulation-following as you need to do, whether it's PCI or any of the rest of the alphabet soup, to avoid punishment.

    Is this cynical? Of course! But it's also real life.

  • Giant Software Company Bureaucracies

    It is the nature of giant bureaucracies to coerce and control the populations they "serve." Giant bureaucracies also tend to resist change, protect themselves at all cost, operate with laughable inefficiency, and become increasingly disconnected from their supposed mission. This is true whether the bureaucracy is a government agency (illustrated on a small, local scale by the wonderful movie Still Mine)

    Still mine

    or a software company. When the bureaucracies are giant software companies, the coercion is often masked in a sickly-sweet cover story about trying to help you, or assuring that things happen with high quality, which just rubs it in.

    I recently ran into an example of this with Microsoft. I was trying to play WMA (Windows Media Audio) files that I had created for my own use from CD's I had purchased. In other words, I was trying to do something I should have been able to do.

    Why CD's? I had bought them a long time ago, why should I purchase them again digitally when it's legal to create a personal digital copy. Why WMA? At the time, it was technically slightly better than the MP3 easily available to me.

    The Random House example (apologies to Random House)

    Imagine I had bought a paper book years ago. Now I was trying to open it to re-read a section. When I tried to open it, it won't open! The book was stuck, and there was a knock on my apartment door. There's a loud voice coming from outside: "Open up! Open up! This is Random House!" OMG! What's this about? I can't open my old book, and suddenly some publisher is pounding at my door??

    I go to the door, open it, and there's a couple scary-looking guys. They say, "We understand you're trying to open a Random House book. Before you open it, we need to verify that you have the right to do so."

    I say, "What do you mean? IT'S MY BOOK! I BOUGHT IT! I'VE OWNED IT FOR YEARS! WHAT RIGHT DO YOU HAVE TO POUND ON MY DOOR AND QUESTION ME?"

    They reply, "We're Random House. We're the publishers. You may think you own this book, but we're the publishers. How do we know you own the book legally? We've got to make sure you have the proper rights for this book. Until we receive that assurance, you will not be able to open the book you claim to own."

    "OK," I say guardedly. "What do I have to do to convince you I own the book I own?"

    "It's simple. Just replace all your phones and your phone service with Random House's. Then our book will be able to call our office and make sure you have the rights you say you have."

    "I've heard about the Random House telephone service. It's really crappy. It's full of static. That's why fewer people use it every month, even though it's free. Even worse, crooks have figured out how to use it to see when I'm not home, so they can break in and steal my stuff. If you insanely want to somehow have the book you published be able to 'phone home,' why not just use the phones I've already got, which work great?"

    "They're not Random House phones. We can't guarantee their quality or appropriateness. Random House books only work with Random House phones. You can say what you want — but we say that we put our name on it and we stand behind them — and they're the only phones we'll use."

    I get the message. I kick myself for being so deluded that I thought buying a book from Random House was a good idea. There's no way I'm trading my secure phones for ones that practically fly a flag to alert all the criminals in the area when the house is vulnerable. I hand the book that I bought and paid for, but which I cannot use, to the agents from Random House, and dis-invite them from my house.

    Microsoft and WMA

    This is what Microsoft did, acting just like the imagined Random House of my example.

    I tried to play my WMA file. It wouldn't play. Instead, just like the agents from Random house pounding on my door, I get this:

    Microsoft fail

    Note the copyright, literally ten years ago! Tens of thousands of supposedly super-bright programmers, and they can't manage to keep things up to date?

    They "don't support" my web browser, which (on this machine) is Firefox. They insist on using IE, which is of course their own browser. Whose utilization has plummetted from over two-thirds in 2009 to about the same as Firefox last year.

    Usage_share_of_web_browsers_(Source_StatCounter).svg

    Why do I care? First of all, they shouldn't care. It's outrageous that they do. Second, here's one reason among many why I care:

    IE vulnerability

    I might as well fly a flag from my house saying "hey, all crooks in the area, c'mon over, the pickin's are good!" And this isn't the first time — IE is famous for being about the most inept, dangerous-to-use browser in existence. Imagine, a free product with a plummeting market share!

    Conclusion

    This experience didn't teach me anything I didn't already know. Microsoft isn't unique. It's like every other giant, bumbling bureaucracy: it's an elephant, we're mice, and you'd better look smart and be careful or you'll get crushed. But somehow, when your nose gets rubbed in it, and they effectively steal something from you from your own house (computer), and there's nothing you can do about, I at least get aggravated in spite of myself.

     

Links

Recent Posts

Categories