The Black Liszt

Category: Summaries

Summary: Software in Government, Big Business and Big Tech

This is a summary with links to my posts on the many ways that large organizations including government, big business, big tech and the rest diligently apply modern software procedures as taught in academia and required by professional management; they consistently produce disastrous results in software quality, cost, security and everything else that matters.

There are of course issues that are common to all these large organizations, for example in cybersecurity.

https://blackliszt.com/2015/06/systemic-issues-behind-the-cyber-security-disasters-at-opm-citi-anthem-etc.html

Government

Government software disasters are government-as-usual, so much so that disasters that wreck lives barely make the news. For example, over 10 million people world-wide enter a government-run lottery for immigration slots that can lead to US citizenship. How hard can picking a bunch of random numbers be? Apparently too hard for the government software people, with the result of horrible consequences for the declared lottery winners whose immigration slots were invalidated.

https://blackliszt.com/2011/07/software-quality-horror-tales-electronic-diversity-visas.html

Consider the sets "Excellence" and "Government IT." There is a great deal of evidence that these are non-overlapping sets. I learned there are organizations promoting and celebrating digital government. They hold awards ceremonies. I tried to find out what the winner had done to deserve winning. Surprise, surprise, the link at the organization’s website explaining it all was broken. Pathetic.

https://blackliszt.com/2015/05/excellence-in-government-it.html

Even simple things like making Social Security statements available on-line appears to be beyond them — including of course lying about it.

https://blackliszt.com/2024/03/excellenece-in-government-it-the-social-security-administration-.html

The NSA (National Security Agency) has a budget of over $50 Billion and is touted as being the world’s best at cybersecurity. It turns out the only reason we know their super-top-secret budget is because their security was blatantly breached with massive internal data made public.

https://blackliszt.com/2014/05/bureaucracy-regulation-and-computer-security.html

Given that this army of highly-paid cyber geniuses can’t protect itself, it’s not surprising that its analysis of a high-visibility security breach may have sounded good to the public, but was in fact entirely fraudulent.

https://blackliszt.com/2017/01/russia-hacks-dnc-podesta-email-fake-news.html

What do you do with such a huge budget when you’re unable to do what you’re supposed to do even with your own secrets? You set up a massive program to teach students your excellent methods and hope to train over a million certified experts. I tracked the program from a local community college to the NSA’s own description of its program – which was both broken and insecure!

https://blackliszt.com/2017/06/government-cyber-security-tops-the-oxymoron-list.html

Unfortunately, this isn’t just about keeping information safe. Government ineptitude kills people. Instead of taking a quick, simple approach to preventing train crashes:

https://blackliszt.com/2015/05/an-app-to-prevent-train-crashes-like-amtrak-philadelphia.html

The government presses on with its super-expensive solution using obsolete technology, which leads to yet more preventable crashes and deaths.

https://blackliszt.com/2016/10/scandal-hoboken-train-crash.html

It’s not just big governments. The little government of several islands in the Caribbean managed to create a multi-front disaster using best practices to foist a digital currency system on its innocent citizens.

https://blackliszt.com/2022/03/dcash-government-cryptocurrency-shows-why-fedcoin-would-be-a-disaster.html

https://blackliszt.com/2022/03/what-is-behind-the-dcash-central-bank-digital-currency-disaster.html

The US government continues to pursue a national digital currency of the kind that has already proved to be a disaster in the Caribbean. They do so ignoring the fact that the US Dollar is already largely digital, with extensive software support structures that are in place and working well..

https://blackliszt.com/2020/12/we-dont-need-fedcoin-we-already-have-a-national-digital-currency.html

Important things like voting systems are some combination of broken and insecure. I took the trouble to define a simple combination of tech and non-tech to build a modern, secure voting system that was auditable, with operations visible to every voter while keeping what they voted for secret. Will any government institution pay attention, much less implement it? We all know the answer.

https://blackliszt.com/2025/03/voter-id-and-paper-ballots-dont-prevent-cheating.html

Big Business

Executives in big business want to succeed and advance, but this can only happen by avoiding risk. The best way to avoid risk is to do what “everyone else” is doing, what the experts say is best. That’s where industry advisory groups come in.

https://blackliszt.com/2017/05/the-value-of-computer-industry-advisory-groups.html

Giant advisory firms counsel their customers on how to make the best decisions. Getting your customers to like you is high on the list. Carefully crafted words are of supreme importance to such large organizations. Actions that match? Not so much.

https://blackliszt.com/2016/07/gartner-group-big-company-customer-service.html

A giant health insurance company “lost” the personal information of "tens of millions" of its members sometime in 2014; they're not sure how many, whose records were "lost," or when it happened. The details are an astounding illustration of big-corporate IT incompetence.

https://blackliszt.com/2015/02/the-anthem-of-cyber-insecurity.html

I soon found out that my information had indeed been stolen. The company’s response to the theft was right in line with their letting it happen.

https://blackliszt.com/2015/02/my-anthem-account-was-hacked.html

What company doesn't want to be part of the digital revolution and have an app? If you're a major health insurance company, why wouldn't you replace old-fashioned insurance cards with something always up-to-date that comes on an app? Here’s what ensued when one of the industry giants tried.

https://blackliszt.com/2021/02/why-cant-big-companies-build-or-even-buy-sofware-that-works.html

I've covered many big organization face-plants. The awfulness encompasses a broad range of consumer-dissing inconvenience, Here’s a case of some software that "works" but puts customer inconvenience front and center.

https://blackliszt.com/2021/03/why-cant-big-companies-build-software-that-works.html

Here’s a case of a giant company software issue that is low on the “it matters” scale, and high on the “a smart high school student could have done it better” scale. It’s the kind of issue that leads one to wonder whether we’d all be better off if they refused to hire any more people with college degrees for any job, and in particular, management.

https://blackliszt.com/2021/05/anthem-needs-my-feedback-reveals-deep-problems.html

Big Tech

Whether the software is a cool social app, an academic website or a real business, there is a common theme: the software is poorly designed and, even worse, it just breaks. You might think the cool internet apps like Facebook and Twitter are an exception, but they’re not.

https://blackliszt.com/2012/01/internet-software-quality-horror-shows.html

How can you innovate? Did the leaders of the current big tech companies benefit from training in innovation? Once they became large, have the big guys like Google demonstrated excellence in innovation? Uhh, sorry, the facts indicate otherwise.

https://blackliszt.com/2016/05/organizing-for-successful-innovation-recent-history.html

The widely-accepted logic is: Facebook is wildly successful; FB is built on software; therefore, FB software must be excellent. I should hire people from FB to help me build excellent software! The history and facts support neither the logic nor the conclusion.

https://blackliszt.com/2014/12/fb.html

I looked at FB’s mobile app when it had over 700 million people using it. Over 20 million people had written reviews, more than 6 million of which were 3 stars or less. A random sample of those reviews yielded juicy results.

https://blackliszt.com/2014/11/facebooks-software-quality.html

The difference between image and reality at FB is astounding. Here is an interview and a recent book that should lead any ambitious young company to avoid hiring people from there.

https://blackliszt.com/2017/03/software-giants-image-and-reality-facebook.html

Large organizations have trouble building software. This has been true since the dawn of software history, and shows no signs of changing. The decades-long, rolling disaster of Microsoft Windows is a great example of this.

https://blackliszt.com/2015/08/large-organization-software-fails-the-case-of-microsoft-windows.html

Microsoft illustrated multiple issues relating to digital ownership in a case I dug into. Among other things they attempted to require use of their own pathetic browser.

https://blackliszt.com/2014/05/giant-software-bureaucracies.html

There are big problems with software quality. The social apps in particular have decided it's embarrassing. But instead of actually, you know, fixing the problems, they seem to have decided to mask the problems! Twitter is a great example of this disease.

https://blackliszt.com/2013/05/twitter-software-quality-stinks.html

I did detailed studies on Twitter and found that they do indeed produce provably bad search results.

https://blackliszt.com/2013/05/twitter-software-quality-an-oxymoron.html

People write and talk about what's "trending on Twitter" as though the trend meant something. It doesn't. It's based on deeply flawed Twitter search software that gives random, widely varying results.

https://blackliszt.com/2013/05/the-bogus-basis-of-trending-on-twitter.html

Twitter fired boatloads of software engineers in 2022 leading some to predict that software disaster will ensue. But then, most people don’t know much about software and don’t realize what a disaster Twitter software has been for years.

https://blackliszt.com/2022/11/twitter-can-improve-software-quality-by-losing-most-of-its-engineers.html

Then there is Apple, the high-prestige computer company making expensive devices. In 2016, terrorists killed a bunch of people in California. Law enforcement and the FBI worked hard to find out what happened and who else might have been involved. This required looking in the government-issued iPhones used by the killers. What happened? Apple did its best to protect the criminals. Here are the highlights.

https://blackliszt.com/2016/03/the-apple-fbi-fiasco.html

And here are the details:

https://blackliszt.com/2016/03/apple-can-help-fight-crime-while-maintaining-privacy.html

https://blackliszt.com/2016/02/apples-cancer-prevention-strategy.html

https://blackliszt.com/2016/02/apples-approach-to-privacy-terrorists-and-criminals.html

I reviewed a book about government security on Amazon. The author was impressive and had loads of experience. Many of the reviews were positive, with a few pointing to obvious bias. I wrote a review that pointed to the positive aspects, but also mentioned some of the bias. The review disappeared. I interacted with Amazon, and was told that suppressing the review was a mistake. It appeared again. Then it disappeared. I tried to write a review and was told I've been banned!

https://blackliszt.com/2023/03/early-evidence-of-criticism-suppression-by-intelligence-agencies-.html

Yelp isn’t as big as the industry giants, but it’s pretty big. A random plunge into their system demonstrates the same kind of slick surface with rotten underpinnings as their larger brethren.

https://blackliszt.com/2021/05/yelp-big-tech-incompetent-corrupt.html

Conclusion

There is a better way! The winning methods aren’t even new – they’re proven in practice by small groups that need to win. See:

https://blackliszt.com/2023/07/summary-software-innovation.html

https://blackliszt.com/2023/07/summary-wartime-software-to-win-the-war.html

November 17, 2023
Summary: Software Experts

This is a summary with links to my posts on software experts. For medical experts see this:

https://blackliszt.com/2023/08/summary-the-medical-industrial-complex.html

The hype about experts is so extreme, it’s important to take a bit of time mocking it. After all, experts confirm that experts are super-smart and never make mistakes!

https://blackliszt.com/2021/04/experts-are-super-smart.html

Medical doctors considered blood-letting to be a standard part of medical practice until well into the 1800's. They continued to weaken and kill patients with this destructive "therapy," even as the evidence against it piled high.

The vast majority of software experts strongly resemble medical doctors from those earlier times. The evidence is overwhelming that the "cures" they promote make things worse, but since all the software doctors give nearly the same horrible advice, things continue.

https://blackliszt.com/2019/02/what-software-experts-think-about-blood-letting.html

The Wannacrypt ransomware attack caused havoc world-wide in major corporations and government institutions. It's a textbook lesson in a number of subjects including the worthlessness of most Experts and the rank illiteracy of otherwise highly educated journalists about computing.

https://blackliszt.com/2017/05/the-ransomware-hack-attack-lessons-from-the-experts.html

Governments are filled with Experts, particularly on important subjects such as security. In 2017 the US government declared that the Russian government hacked important US entities. The official report was filled with impressive-sounding evidence that demonstrated the incompetence and/or duplicity of the agencies that issued it. The majority of the US press simply echoed the nonsense.

https://blackliszt.com/2017/01/russia-hacks-dnc-podesta-email-fake-news.html

Big Data is one of those subjects that is widely touted but opaque to most people. It’s a classic forum for Experts to have their statements taken as infallible. A famous expert in Big Data and elections made an error in a recent election that clearly illustrates typical Expert behavior of seeking an outcome instead of the truth.

https://blackliszt.com/2017/04/big-datas-big-face-plant.html

Experts and anointed authorities of various kinds, both academic and commercial, have been the front lines of resistance to innovation for centuries. They are the firewall keeping rogue ideas outside the environments they oversee, protecting them from bad influences that their naïve but innocent charges might inadvertently adopt.

https://blackliszt.com/2020/05/experts-vs-innovation-new-book.html

Groups like Gartner sell expertise as a service. Go to them if you want to know the best thing to do! The real value of such groups is to help executives avoid risk and avoid blame when things go wrong. Innovation? Industry-acknowledged experts are not the way to go.

https://blackliszt.com/2017/05/the-value-of-computer-industry-advisory-groups.html

If you're doing something new and want to do it right, it's natural to seek the help of someone who's been there and done that. If you want to do the thing in an innovative way, that's all the more reason to seek expert help. The trouble is, however useless experts are in general, they’re worse when it comes to innovation. The history of manned flight is a good example.

https://blackliszt.com/2016/07/innovation-and-experts.html

November 8, 2023
Summary: Regulations

This is a summary with links to my posts on regulations.

Regulations are supposed to make things better. Most of the time regulations make things worse by preventing innovation, increasing costs and failing to achieve the goals for which they were created. The negative impact of regulations, whether government or corporate, is greatly magnified in software and technology.

The reason why our ever-growing number of regulations fail to protect us is simple. In the vast majority of cases, they spell out, often in great detail, how to accomplish the goal, instead of plainly and simply defining the goal and leaving it up to the person or company regulated to figure out how to get it done.

https://blackliszt.com/2011/12/regulations-goals-or-directions.html

Regulations that define a goal enable innovators to find new ways to accomplish the goal, whether it’s a better medical device, computer security, whatever.

Another way to think about effective regulations is the criminal law, which regulates behavior. There aren’t ever-growing mountains of regulations telling you how to avoid murdering someone, just a simple statement that murder is something you must not do.

https://blackliszt.com/2012/03/lets-criminalize-our-regulations.html

Regulations are front-and-center in the bureaucracy-driven battle to prevent innovation. Regulations are ever-growing mountains of words written by lawyers and bureaucrats. The current federal regulations have more than 100 times the number of words than the collected works of Shakespeare.

https://blackliszt.com/2016/09/innovation-the-barriers.html

Regulations and standards can be good; without standard steering wheels and brake pedals, no one would be able to drive a rental car. Software is different. The misguided effort to impose standards and regulations on software development has played a key role in the nonstop cybersecurity disasters and software failures that most organizations try to minimize and ignore.

https://blackliszt.com/2021/11/the-destructive-impact-of-software-standards-and-regulations.html

Medical device regulations increase costs and prevent innovation. The FDA device regulations provide an excellent example, declaring in massive detail how exactly to achieve quality in classic how-type method. The what-type (criminal version) would simply declare that the device must perform its declared function, accurately and well.

https://blackliszt.com/2016/12/regulations-that-enable-innovation.html

A similar story plays out in the field of medical imaging. The essential devices could be vastly improved if the regulators got out of the way.

https://blackliszt.com/2023/01/how-to-reduce-the-cost-of-medical-imaging-and-pacs.html

The bureaucrats who write regulations for software are ignorant of software. It’s literally invisible to them! Their understanding, such as it is, tends to be based on false metaphors and is wildly inappropriate. They end up requiring expensive, obsolete methods for building software that no sensible company would use.

The current mountains of regulations should be replaced by something like “We don’t care how you build your software, but it’s your responsibility to assure that the software performs its stated job without fail. If the software has errors that cause medical harm, you are responsible for the damage it causes, and you may be barred from supplying software to the medical market in the future.”

https://blackliszt.com/2020/05/heres-how-the-fda-can-reduce-medical-device-costs-while-improving-healthcare.html

Regulation is also about important things like making trains safe so that, even when something goes wrong, you don’t have a crash that kills people. That’s one the government has been all over. They devised a system called PTC (positive train control) to prevent crashes. The cost to implement it was tens of billions of dollars and took many years. Years after it was mandated there was a crash that killed people and injured hundreds.

https://blackliszt.com/2015/05/an-app-to-prevent-train-crashes-like-amtrak-philadelphia.html

The crash in Philadelphia wasn't a one-off. The problem is that PTC is built on computing technology that belongs in a museum, not supposedly protecting our lives.

https://blackliszt.com/2016/10/scandal-hoboken-train-crash.html

The technology exists to enable a more effective, inexpensive system to be built using modern technology. But of course the regulators ignore it.

Whenever the government wants to step in to improve a company’s software, beware. When has such a move ever had a positive impact on anything, much less the stated goal of the regulation?

https://blackliszt.com/2015/04/the-government-wants-to-help-ubers-software-quality.html

There has been a recurring furor about the unfairness of the internet. According to the critics, regulators at the FCC should step in and impose “net neutrality” to make service fairer for everyone. In fact, what few problems there are have been caused by regulation. And where regulation has been imposed elsewhere, government-mandated severe censorship has quickly followed.

https://blackliszt.com/2014/11/net-neutrality-it-aint-broke-dont-fix-it.html

It’s not just the government. Big bureaucracies to “control and improve” software emerge in giant companies of all kinds. Even in a software company, the in-house regulators can impose insanity. Here’s a case involving Microsoft and digital goods.

https://blackliszt.com/2014/05/giant-software-bureaucracies.html

Regulations have an outsized role in causing the on-going disaster of computer security. See these for examples and explanations:

https://blackliszt.com/2017/05/security-regulations-vs-security.html

https://blackliszt.com/2014/05/bureaucracy-regulation-and-computer-security.html

https://blackliszt.com/2013/07/cyber-security-standards-are-ineffective-against-insiders-like-edward-snowden.html

For more about security and how regulations make things worse, here is the summary post.

https://blackliszt.com/2023/05/summary-the-ongoing-problem-of-computer-security.html

November 7, 2023
Summary: Computer Data Centers and Networking

This is a summary with links to my posts on computer data centers and networks. The two subjects are intimately related because the whole point of networks is to connect computers with each other.

Like everything else in computing, fashions have a strong impact. A bit over a decade ago, the world of computing started yammering about “the cloud” and “virtualization.” These things were the hot subjects all the cool kids talked about. If you weren’t driving towards moving to the cloud, you were obsolete. The reality of course was much different.

https://blackliszt.com/2011/12/the-name-game-of-moving-to-the-cloud.html

The Cloud is just another virtue-signaling fashion word like Big Data.

https://blackliszt.com/2012/04/im-tired-of-hearing-about-big-data.html

The fact that the underlying reality hardly changes at all is way beyond the technical knowledge of the vast majority of the people who talk about it.

https://blackliszt.com/2012/05/the-cloud-and-virtualization.html

When something like “the cloud” heats up, all the vendors related to it rush to promote their products as ideally suited to the new thing.

https://blackliszt.com/2013/04/storage-vendors-in-the-cloud.html

The managers of large in-house data centers rarely know much about what they’re buying and what the alternatives could be. As a result, they often spend way more than they need to on equipment, something which competing cloud vendors are less likely to do.

https://blackliszt.com/2014/08/data-center-managers-spend-too-much-on-equipment.html

Like anything where people are involved, there are politics and fights over who should perform a given function. One of the classics is making replications of the data in DBMS’s for safety purposes. The data center people want to use their mechanism and the DBMS people want to use theirs. There is a clear right answer, but it only wins sometimes.

https://blackliszt.com/2014/03/replication-good-idea-storage-replication-nah.html

One of the key things people who write applications want is for them to “scale,” i.e., be able to handle any load without slowing down. The way some systems applications are built, based on decades-old designs, can get in the way. But there are solutions.

https://blackliszt.com/2014/02/obstacles-to-scaling-centralization.html

People also want their applications to continue to be available in the event of data center failure. For people working in data centers, there are perverse incentives at work.

https://blackliszt.com/2010/04/is-your-site-working-do-you-really-care.html

To be fair to data center managers, applications originally written decades ago expect to be able to run in modern data centers. It’s tough!

https://blackliszt.com/2010/01/paleolithic-mainframes-discovered-alive-in-data-center.html

Way back in 2015, it was clear that hardware had evolved to make dramatic improvements possible in software.

https://blackliszt.com/2015/08/the-data-center-of-the-future.html

With software people ignorantly focused on network-connected services as the way to build scalable applications, they assure that the incredible hardware power will be minimally utilized.

https://blackliszt.com/microservices/

Network neutrality was a hot topic and still comes up. The idea is that everyone should be charged the same for internet access and services. When you dig into the subject with actual knowledge of the technology, you see that the whole furor by virtue-signaling ignorance.

https://blackliszt.com/2014/11/net-neutrality-it-aint-broke-dont-fix-it.html

One of the arguments made about net neutrality and legal privacy provisions was unusually divorced from reality.

https://blackliszt.com/2017/04/on-the-internet-youre-naked-and-for-sale.html

Years later, a weak form of net neutrality was repealed. There were demonstrations with US Senators making predictions about the disaster that would ensure. Here’s an analysis of the non-disaster a year later.

https://blackliszt.com/2019/06/the-aftermath-of-the-net-neutrality-disaster.html

August 31, 2023
Summary: The Medical-Industrial Complex

Modern medicine can do wonderful things. I benefited from remission of an extremely rare form of cancer that was made possible by advances in the last couple of decades. At the same time, a great deal of what is done in medicine is controlled by the Medical-Industrial Complex, which causes untold waste and harm. These posts document the tip of that iceberg as my health journey has led me.

I've long had a concern that what doctors do often doesn't follow the clear evidence. For example, here's a case of blatantly ignoring standard practice with something simple.

https://blackliszt.com/2016/12/what-can-cats-teach-us-about-healthcare.html

No big deal. But then I encountered something far more serious. I tell the start of my story here, about heart symptoms caused by blood pressure pills.

https://blackliszt.com/2022/07/the-destructive-treatment-of-hypertension.html

The next part of my story is when I discovered there’s a large study demonstrating that taking blood pressure pills more than doubles your chances of getting AMD, which makes you blind.

https://blackliszt.com/2022/07/blood-pressure-pills-can-make-you-blind.html

It’s a study that none of the relevant doctors ever tells you about.

I backed up and studied hypertension. Expert opinion on the subject is united.

https://blackliszt.com/2022/05/the-experts-are-clear-control-your-blood-pressure.html

When you dig past the pronouncements of authorities, you discover that hypertension isn’t even a disease.

https://blackliszt.com/2022/06/the-facts-are-clear-hypertension-is-not-a-disease.html

So what happens when you stop taking the pills? At least in one anecdotal case, things change in good ways, and in any case, no harm.

https://blackliszt.com/2022/11/how-to-cure-amd-macular-degeneration.html

More digging led me to the bogus, incredibly destructive diet-heart hypothesis, proven false by the evidence but promoted on food packages and everywhere else.

https://blackliszt.com/2021/12/trusting-science-the-whole-milk-disaster.html

The idea is that eating red meat and full-fat dairy leads to heart disease.

https://blackliszt.com/2022/02/the-experts-are-clear-dont-eat-much-saturated-fat.html

When you dig past the propaganda, you learn that saturated fat is an essential and healthy part of your diet.

https://blackliszt.com/2022/03/the-facts-are-clear-eat-lots-of-saturated-fat.html

Eating saturated fat is supposed to increase your blood cholesterol, which leads to heart trouble, they say.

https://blackliszt.com/2022/02/the-experts-are-clear-keep-your-cholesterol-low.html

The widely-prescribed drugs that lower cholesterol don’t help with heart disease and cause problems of their own.

https://blackliszt.com/2022/04/the-facts-are-clear-dont-take-cholesterol-drugs.html

There is strong evidence that bad diet recommendations based on the bogus diet-heart hypothesis is a leading cause of the obesity epidemic that continues to worsen.

https://blackliszt.com/2022/04/the-forbidden-question-what-caused-the-obesity-epidemic.html

The current recommendations for diet and medical treatment of obesity continue the madness.

https://blackliszt.com/2022/09/the-medical-treatment-of-obesity.html

Exactly what ingredients are in the food you eat is crucial. Places that advertise that they're healthy can be tricky and require careful study of ingredients:

https://blackliszt.com/2021/06/ingredients-whole-foods-sneaks-in-sugar.html

The ingredients of things that aren't food should be read carefully:

https://blackliszt.com/2021/04/ingredients-and-truth.html

Many people receive reminders to get things like blood pressure and cholesterol checked so that drugs can be prescribed “if necessary.”

https://blackliszt.com/2023/02/be-healthy-and-dont-schedule-your-heart-health-visit.html

Disease Prevention and Testing

Disease prevention sounds like a great idea. So does early detection of bad things. When you dig into the evidence and the numbers, a different picture emerges.

Screening for colon cancer is a multi-billion dollar industry. The only large-scale study that’s ever been done shows that it doesn’t result in longer life.

https://blackliszt.com/2023/01/value-of-colon-cancer-screening.html

Everyone is supposed to get an annual flu shot. The CDC’s own numbers and massive studies show that you’re better off without it.

https://blackliszt.com/2022/12/flu-shots-propaganda-reality.html

Vaccine efficacy is often mentioned. Its technical meaning can be found, but the authorities rarely mention it. A large efficacy can still mean that you only have 1 chance in a hundred of being helped by the drug.

https://blackliszt.com/2022/09/does-vaccine-efficacy-of-95-mean-i-wont-get-sick.html

Your chances of being helped (NNT, Number Needed to Treat) must be considered along with the chances of being harmed, a thing that is too-often ignored.

https://blackliszt.com/2022/09/nnt-for-benefits-and-for-harms.html

The studies that are supposed to show treatment effectiveness are too often biased, and (shockingly) backed by data that is kept secret by law. If the treatment is wonderful, why keep the data secret?

https://blackliszt.com/2022/11/revolutionize-health-by-making-medical-data-and-studies-open-source.html

There is a proven path to make drug discovery and testing a quantum leap better. All the authorities and experts ignore it.

https://blackliszt.com/2017/01/using-software-methods-to-speed-drug-discovery.html

Given the results, the common-sense idea of wellness visits stops making sense in most cases.

https://blackliszt.com/2023/02/be-healthy-and-schedule-your-annual-wellness-visit.html

Of course there is a great deal more to be said on this subject. There are true experts, far more qualified than I am, some of whom are referenced in the above posts. My intention is these posts was to detail the journey that a normal patient took from trusting and naive to the opposite. Again, there are wonderful benefits for patients from doctors and hospitals; but not everything that is recommended is wise to take/do.

August 31, 2023
Summary: The Business of Healthcare, Medicine and Insurance

After spending years learning about the software used in medical offices, hospitals and insurers, I came to the conclusion that healthcare technology is the trailer park of the computer world.

I have since realized that I was wrong in the comparison; trailer parks actually work to house their inhabitants, effectively and inexpensively, unlike most medically-related computer systems. Trailer parks of the world, I humbly apologize.

https://blackliszt.com/2015/12/healthcare-problems-and-opportunities.html

Here are some of most revolutionary areas for change.

https://blackliszt.com/2025/01/how-to-reduce-medical-costs-and-improve-health.html

Here is a way of understanding what needs changing the most:

https://blackliszt.com/2025/01/medical-services-before-during-and-after-an-event.html

Healthcare Computers and Software

Hospitals have trouble keeping their computers working. When they fail, they often try to hide the failure, which puts patients at risk.

https://blackliszt.com/2015/05/healthcare-it-disfunction-the-secret-computer-outage-at-mount-sinai-hospital.html

In contrast, other institutions let their customers know about computers failures and even apologize for the inconvenience.

https://blackliszt.com/2015/05/computer-troubles-at-the-hospital-and-at-the-symphony.html

The contrast between putting meals on the table and assuring that data is available for use on providers’ screens couldn’t be more stark.

https://blackliszt.com/2015/05/meals-at-downton-abbey-and-it-in-healthcare.html

Even when their computers are working, the people in charge brag about the fancy new things they’re doing instead of assuring that basic processes like patient scheduling work as intended.

https://blackliszt.com/2014/07/innovation-made-simple.html

There’s no doubt that hospital computer systems are a horror show that hurt patients and that managers try to hide. But there’s a far worse largely hidden problem leading to patient deaths.

https://blackliszt.com/2017/05/hospital-computer-disasters-and-iatrogenic-disease.html

Is making systems work reliably and well a wide-spread problem? Yes. Are there proven solutions? Yes. Hospitals do an outstanding job of ignoring them.

https://blackliszt.com/2023/04/summary-software-quality-assurance.html

Electronic Medical Records (EMR)

The data about your health and health history are essential for guiding medical interactions of any kind. The quality of the data is essential. All too often, the data is wrong or missing.

https://blackliszt.com/2016/06/healthcare-innovation-emrs-and-data-quality.html

Most medical organizations have EMR’s and also loads of paper records, which both patient and provider fill out.

https://blackliszt.com/2016/06/healthcare-innovation-emrs-and-paper.html

Have you ever tried to get your own medical records from the institution that has them – your records?? Nightmare.

https://blackliszt.com/2016/06/healthcare-innovation-getting-our-data-out-of-emr-prison.html

They all say they have easy-to-use systems to enable patients to get their own records. Hah. Just try it.

https://blackliszt.com/2018/01/who-owns-your-health-data.html

Here are details of the horrible obstacles a major systems places between the patient and his “easy-to-access” data.

https://blackliszt.com/2018/10/medical-testing-getting-the-results.html

https://blackliszt.com/2018/10/medical-testing-getting-the-results-2.html

Most people have been to more than one institution, each with its own records about you. EMR interchange is supposed to make it easy for the place you’re visiting to get your records from other places.

https://blackliszt.com/2016/10/healthcare-emr-why-portability-matters.html

https://blackliszt.com/2016/07/healthcare-innovation-how-to-achieve-emr-interchange.html

EMR interchange works? Patients sometimes have to go to ridiculous lengths to get essential data from where it is to where it’s needed.

https://blackliszt.com/2016/09/healthcare-emr-interchange-breakthrough.html

The bottom line is that medical institutions should prioritize giving patients their data. There should be an EMR app for your data just like there are money apps like Venmo and Zelle for your money.

https://blackliszt.com/2016/01/i-want-an-emr-app.html

One of the many reasons for EMR nightmares is that the software is a huge problem and “just” buying and installing it is a job of “epic” proportions.

https://blackliszt.com/2016/05/healthcare-innovation-emr-procurement-is-broken.html

Healthcare Business: Scheduling, Testing, Billing and Payment

Medicine is all about helping people get and stay healthy, but it’s also a business.

https://blackliszt.com/2018/07/medicine-as-a-business-overview.html

Scheduling, billing and payment are different for medical care than any other service we require. Here are highlights of the differences.

https://blackliszt.com/2018/07/medicine-as-a-business-billing-overview.html

You need to get a simple test like an X-Ray? No problem. Bad data at the provider, the insurer and the EMR will ensure that lots of time is wasted.

https://blackliszt.com/2017/01/my-cat-taught-me-about-the-state-of-healthcare-provider-data.html

Scheduling sounds simple, right? Maybe for a cat at a vet, but not for a person at a hospital.

https://blackliszt.com/2019/04/whats-wrong-with-medical-scheduling-and-why-it-matters.html

The amount of time and effort on everyone’s part to schedule a medically-required MRI is amazing. Without patient persistence, it won’t happen.

https://blackliszt.com/2018/09/medical-testing-scheduling-the-test.html

Covid testing illustrated the widespread problems with scheduling – it was about the systems and software, not Covid.

https://blackliszt.com/2021/12/the-nightmare-of-covid-test-scheduling.html

Once you’re scheduled, you have to go to the right place, which can be its own adventure. Fortunately, you can usually rest afterwards by filling out reams of paperwork.

https://blackliszt.com/2016/10/hospital-wellness-innovation-breakthrough.html

For costlier tests like MRI’s, the extent of useless, costly processes is mind-boggling.

https://blackliszt.com/2018/09/medical-testing-doing-the-test.html

MRI’s are an amazing technology. The reports from the specialists are something else again. It turns out that in spite of endless years of training, there are no standards for reading and reporting – even “simple” things like the size of a tumor.

https://blackliszt.com/2018/11/medicine-as-a-business-medical-testing-5-the-results.html

What I’ve described aren’t one-off’s. The problems from scheduling to getting the results are business-as-usual in the world of medicine.

https://blackliszt.com/2022/02/medicine-as-a-business-medical-testing-6-another-test.html

And then there’s billing. I went for an MRI, and got bills from different places with bad information and a broken online payment system.

http://blackliszt.com/2018/07/medicine-as-a-business-billing-1.html

The second bill was also a mess of bad data, with an offer to pay online but the bad data prevented me from paying. So I wrote a check.

https://blackliszt.com/2018/07/medicine-as-a-business-billing-2.html

When you dive into the insurance payment part, the nightmare morass of bad data and bad software is overwhelming. Business as usual.

https://blackliszt.com/2018/07/medicine-as-a-business-billing-3-insurance.html

My health insurer announced a major new service to help patients pay provider bills. Strike out.

https://blackliszt.com/2019/02/giant-health-insurer-reveals-patient-billing-break-through-or-is-it-a-break-down.html

Here is a summary of what’s wrong with medical billing and insurance payments.

https://blackliszt.com/2018/08/medicine-as-a-business-billing-4-whats-wrong.html

Medical Costs

Medical costs are outrageous. They don’t have to be.

Hospitals could stop spending money on useless fancy innovations and concentrate on simplifying and automating the everyday nuts and bolts of the business.

https://blackliszt.com/2014/07/innovation-made-simple.html

Removing the regulatory straight-jacket on medical equipment would be huge.

https://blackliszt.com/2016/12/regulations-that-enable-innovation.html

https://blackliszt.com/2020/05/heres-how-the-fda-can-reduce-medical-device-costs-while-improving-healthcare.html

https://blackliszt.com/2023/01/how-to-reduce-the-cost-of-medical-imaging-and-pacs.html

Some of the same principles apply to drug discovery and costs.

https://blackliszt.com/2017/01/using-software-methods-to-speed-drug-discovery.html

https://blackliszt.com/2022/11/revolutionize-health-by-making-medical-data-and-studies-open-source.html

Health insurance

Health insurance companies maintain a huge trove of your personal data. They are incapable of keeping it secure, and bungle the response when there’s a major breach they can’t cover up.

https://blackliszt.com/2015/02/my-anthem-account-was-hacked.html

https://blackliszt.com/2015/02/the-anthem-of-cyber-insecurity.html

Health insurers think providing patients incentives to keep healthy is a win all around. Sadly, their incentives tend to be embarrassing failures. Here’s an example with giving a Prepaid debit card.

https://blackliszt.com/2018/11/patient-incentives-in-healthcare.html

The marketing departments of health insurers want to communicate with patients to make good things happen. Most of them appear never to have heard of the concept of personalization.

https://blackliszt.com/2019/02/patient-outreach-by-health-insurers-case-study.html

https://blackliszt.com/2022/04/health-insurance-company-tries-to-keep-me-healthy.html

The same marketing gurus reach out to patients for feedback and reveal that they flunked Marketing 1.01

https://blackliszt.com/2021/05/anthem-needs-my-feedback-reveals-deep-problems.html

Here’s an example of why they get terrible response rates when asking for customer feedback.

https://blackliszt.com/2019/02/adventures-with-health-insurance-software-customer-feedback.html

Every serious company has to have an app, right? It makes sense that the insurer would spend piles of money to build one that customers hate.

https://blackliszt.com/2021/02/why-cant-big-companies-build-or-even-buy-sofware-that-works.html

AI and fashionable algorithms in Medicine

Doctors too often get the wrong answer. This is the kind of thing that makes some people hope that automation could do a better job:

https://blackliszt.com/2016/12/what-can-cats-teach-us-about-healthcare.html

There are many examples of AI in healthcare in this summary of AI in general.

https://blackliszt.com/2023/07/summary-ai-machine-learning-big-data-math-optimization.html

Here are some of the AI posts relevant to healthcare. A major theme of AI in healthcare is that well-paid managers are desperate to keep up with the AI fashion, and spend lots of money and generate lots of publicity about how wonderful they are. The trouble is that there are simpler, common-sense things that are fully understood that could be done right away that would yield major improvements in patient health; these things are too often ignored or neglected.

https://blackliszt.com/2015/07/cognitive-computing-and-healthcare.html

https://blackliszt.com/2015/08/human-implemented-cognitive-computing-healthcare.html

https://blackliszt.com/2016/05/healthcare-innovation-can-big-data-and-cognitive-computing-deliver-it.html

https://blackliszt.com/2016/09/healthcare-innovation-from-washing-hands-to-ai.html

https://blackliszt.com/2017/06/how-to-avoid-cutting-off-breasts-by-mistake.html

https://blackliszt.com/2018/08/getting-results-from-ml-and-ai-4-healthcare-examples.html

https://blackliszt.com/2025/02/can-ai-improve-medical-diagnosis.html

August 31, 2023
Summary: Computer Software History and Evolution

This is a summary with links to my posts of the history and evolution of computer hardware and software.

In computers and software, history is ignored and its lessons spurned. What little software history we are taught is often wrong. Everyone who writes or uses software pays for this, and pays big. This is true at a large scale for computers.

https://blackliszt.com/2012/03/computer-history.html

Software history can be spectacularly bad about important things.

https://blackliszt.com/2019/08/the-comes-before-is-not-causation-fallacy-in-software-evolution.html

Bad software history is also rife on a smaller scale, for example cryptography.

https://blackliszt.com/2020/10/elizebeth-smith-friedman-the-cancelled-heroine-of-cryptography.html

Software history is largely ignored.

https://blackliszt.com/2015/06/dont-know-much-about-history.html

When you learn math, you are learning its history. When you get to calculus, you’re essentially up to the 1600’s. You build from the ground up. Not in computing, to its detriment.

https://blackliszt.com/2015/05/math-and-computer-science-vs-software-development.html

Studying history would show modern software people who think they’re solving big problems that the problem is decades old and that the box of failed solutions is overflowing.

https://blackliszt.com/2021/08/the-crisis-in-software-is-over-50-years-old.html

Software development is a giant mess. Studying its history and the attempts to clean up the mess is an essential part of constructive change.

https://blackliszt.com/2022/04/how-to-fix-software-development-and-security-a-brief-history.html

The history of other fields can have lessons that are highly relevant for software.

https://blackliszt.com/2012/07/what-can-software-learn-from-steamboats-and-antiseptic-surgery.html

https://blackliszt.com/2014/02/lessons-for-software-from-the-history-of-scurvy.html

https://blackliszt.com/2019/02/what-software-experts-think-about-blood-letting.html

Some aspects of computer history are so important and relevant that, when understood and applied, they make dramatic improvements to software development.

https://blackliszt.com/2013/12/fundamental-concepts-of-computing-speed-of-evolution.html

Software Evolution

Software evolves. But it evolves differently than other things studied by science. Virtually no one studies software history in any way, much less the patterns of evolution that become apparent when you study that history.

https://blackliszt.com/2019/06/the-evolution-of-software.html

Why should anybody bother to study software evolution? Knowing software evolution can help you predict the future! How? Software does not evolve the way most people think it does

https://blackliszt.com/2019/09/understanding-software-evolution-helps-you-predict-the-future-of-software.html

When studying software evolution it’s important to understand the underlying principles of automation – the things that computers and software are capable of doing. The fundamentals will tell you WHAT will happen, but not WHO or WHEN.

https://blackliszt.com/2020/01/the-fundamentals-of-computer-automation.html

The way you learn software evolution is to study both its history in general and how it has been used in different industries and domains.

https://blackliszt.com/2014/04/continents-and-islands-in-the-world-of-computers.html

One clear pattern of evolution is that a kind of software will be established in one domain and then slowly, sometimes over decades, appear in other domains, one by one.

https://blackliszt.com/2019/08/the-slow-spread-of-linear-programming-illustrates-how-in-old-vation-in-software-evolution-works.html

A similar pattern often occurs with simple kinds of software.

https://blackliszt.com/2019/09/simple-data-entry-technology-illustrates-how-in-old-vation-in-software-evolution-works.html

A peculiar aspect of software evolution is software fashions. They appear, grow strongly and often fade away as their awfulness becomes hard to avoid. Then they come back, usually re-named.

https://blackliszt.com/2019/05/recurring-software-fashion-nightmares.html

A typical result of a fashion-driven software trend is to increase costs without delivering benefits, which can happen even when it involves an intrinsically good thing like workflow automation.

https://blackliszt.com/2019/09/laser-disks-and-workflow-illustrate-the-insane-fashion-driven-nature-of-software-evolution.html

One of the forces of evolution is resistance to automation.

https://blackliszt.com/2020/01/luddites.html

Software Programming Language Evolution

Software people love to talk about progress and innovation. Most of what they claim as progress is little but milling around in a small box.

https://blackliszt.com/2020/09/software-programming-languages-50-years-of-progress.html

There is an odd evolution in programming languages, in which the relationship between data definitions and programs cycles from inside to outside the program and back.

https://blackliszt.com/2015/06/innovations-that-arent-data-definitions-inside-or-outside-the-program.html

However, there have been a couple major advances in the evolution of programming languages.

https://blackliszt.com/2020/09/the-giant-advances-in-software-programming-languages.html

While some truly powerful advances have become standard, others have been thoughtlessly discarded.

https://blackliszt.com/2021/09/software-programming-language-evolution-structures-blocks-and-macros.html

Other powerful advances, widely used for a time in the market, ended up being ignored or abandoned by academia and industry in favor of a productivity-killing combination of tools and technologies.

https://blackliszt.com/2020/11/software-programming-language-evolution-beyond-3gls.html

The emergence from academia of “structured programming” and the associated effort to find all GOTO statements and burn them at the stake was a particularly shameful instance of the evolution of programming languages.

https://blackliszt.com/2021/09/software-programming-language-evolution-the-structured-programming-goto-witch-hunt.html

While loads of people focus on language, programming tools that make huge contributions to the productivity of programmers have evolved: libraries and frameworks.

https://blackliszt.com/2021/10/software-programming-language-evolution-libraries-and-frameworks.html

Some programming languages that were supposed to be so much better than older ones led to major failures. But failures are expected in the normal world of software development, so the bad new languages kept marching along.

https://blackliszt.com/2021/01/software-programming-language-evolution-credit-card-software-examples-1.html

The small world of functional software languages has an interesting history.

https://blackliszt.com/2021/05/software-programming-language-evolution-functional-languages.html

Software Applications and Systems Evolution

One of the recurring evolution patterns is that functionality emerges on a new platform in roughly the same order as it emerged on earlier platforms. The timescale of the emergence may be compressed; the important aspect of the pattern isn’t the timing but the order. Here is an explanation of the general concept and how it worked for operating systems.

https://blackliszt.com/2020/10/software-evolution-functionality-on-a-new-platform.html

Here is how the same pattern of functionality on a new platform worked out in security services.

https://blackliszt.com/2020/11/software-evolution-functionality-on-a-new-platform-security-services.html

Here’s the pattern of existing functionality on a new platform as seen for remote access software.

https://blackliszt.com/2020/11/software-evolution-functionality-on-a-new-platform-remote-access.html

Just because you’re building a version of existing functionality for a new platform doesn’t mean you’ll succeed. You can screw it up in various ways, including being too early – if the market doesn’t think it has a problem, it won’t buy a solution.

https://blackliszt.com/2020/11/software-evolution-functionality-on-a-new-platform-market-research.html

Transaction monitors are a classic example of the pattern of functionality emerging on a technology platform, and then emerging in pretty much the same way in the same order on other platforms.

https://blackliszt.com/2021/02/software-evolution-functionality-on-a-new-platform-transaction-monitors.html

Once an application appears on a software platform, there is a consistent way the category of applications evolves on that platform. It goes from a custom application to a basic product, then parameters are added and finally a workbench.

https://blackliszt.com/2020/01/the-progression-of-abstraction-in-software-applications.html

Here are examples of the progression from prototype through increasing levels of abstraction.

https://blackliszt.com/2021/03/the-progression-towards-abstraction-on-a-software-platform-examples.html

Software automates human effort to varying degrees. One dimension of automation is depth, in which software evolves from recording what people do through helping them and eventually to replacing them.

https://blackliszt.com/2021/04/the-dimension-of-software-automation-depth.html

There is surprising pain and trouble of going from one stage of automation depth to the next. Unlike the progression to increasing levels of abstraction, customers tend to resist moving to the next stage of automation for various reasons including the fear of loss of control and power.

https://blackliszt.com/2021/05/the-dimension-of-software-automation-depth-examples.html

Here’s how the automation depth pattern plays out in “information access,” the set of facilities that enable people to find and use computer-based information for decision making.

https://blackliszt.com/2021/12/the-dimension-of-automation-depth-in-information-access.html

The patterns of automation play out in multiple dimensions. In addition to automation depth, there is automation breadth.

https://blackliszt.com/2021/11/the-dimension-of-software-automation-breadth.html

Here are a couple of clear examples of the evolution of software applications to increasing automation breadth.

https://blackliszt.com/2022/07/the-dimension-of-software-automation-breadth-examples.html

The evolution of spreadsheets is a good example of how these patterns work out in history, and clearly demonstrate the predictive power they have for anyone who cares to look.

https://blackliszt.com/2023/09/software-evolution-spreadsheet-example.html

As software evolves in these ways, it becomes increasingly nuts for a potential customer to build it for themselves. However, there are a surprising number of cases where a smart user can build a private version of commercially available software and win with it.

https://blackliszt.com/2023/09/software-evolution-build-buy-then-build.html

You would think that user interfaces would be thoroughly understood. When you look at a variety of UI’s you see that the underlying principles are clear but usually ignored.

https://blackliszt.com/2020/12/software-evolution-user-interface.html

https://blackliszt.com/2021/01/software-evolution-user-interface-concepts-whose-perspective.html

https://blackliszt.com/2020/01/how-to-design-software-user-interfaces-that-take-less-time.html

Does Software Always Evolve?

There is a large body of core software that evolves extremely slowly.

https://blackliszt.com/2023/07/does-software-evolve-rapidly.html

Once a kind of software gets built, it tends to live long past the problem for which it was the solution. Data was so big you needed storage. Special SW invented to handle it. Now it will all fit in memory (for most applications). But the SW and code practices live on!

https://blackliszt.com/2010/09/databases-and-applications.html

Yes, they’ve been obsoleted by modern hardware – but needed by software.

https://blackliszt.com/2010/01/paleolithic-mainframes-discovered-alive-in-data-center.html

While there are things that change in software and management, a surprising number of common things just get new names. The “cloud” is a typical example.

https://blackliszt.com/2011/12/the-name-game-of-moving-to-the-cloud.html

Another example is pure software, shown with SOA and micro-services.

https://blackliszt.com/2021/03/micro-services-the-forgotten-history-of-failures.html

One of the explosive growth areas of the internet boom was the invention and spread of social media. Most of the people who look at it ignore its deep roots in non-digital media.

https://blackliszt.com/2018/10/social-media-has-a-long-history.html

If and when serious study of computer software history and evolution finally starts to take place, perhaps Computer Science will start on the path to being, you know, scientific. And normal software development will stop being dominated by fashions.

https://blackliszt.com/2023/04/summary-computer-science.html

https://blackliszt.com/2023/07/summary-software-fashions.html

August 31, 2023
Summary: Occamality and Software Architecture

This is a summary of my posts on the single most important principle of software: Occamality (non-redundancy). This principle applies to everything from the simple concept of what makes a piece of software "good" to developing and evolving software quickly, efficiently and with high quality. It drives good software architecture and all other aspects of development, from requirements through QA.

This summary also includes my posts on software architecture, mostly explaining why widely accepted architectures like microservices are terrible.

To start, it's worth pointing out that software people don't know what makes a piece of software "good."

https://blackliszt.com/2023/09/how-do-you-know-if-a-given-piece-of-software-is-good.html

Software people often have strong thoughts about software languages and architecture. However, it is extremely rare for those opinions to be grounded in or related to the goals of software architecture. What are the goals? Here’s my proposal.

https://blackliszt.com/2022/05/the-goals-of-software-architecture.html

Here are the key specific things you do to accomplish the goals.

https://blackliszt.com/2020/02/how-to-build-applications-that-can-be-changed-quickly.html

Here is a layman's, common-sense explanation of the same idea:

https://blackliszt.com/2022/10/how-to-improve-software-productivity-and-quality-common-sense-approach.html

Here is a specific explanation explaining why Occamal programs are better than non-Occamal ones and why you should care.

https://blackliszt.com/2023/09/why-should-you-build-occamal-programs.html

How do you apply Occamality in practice? Here is a short, simple list of the practical things you do.

https://blackliszt.com/2023/09/understanding-occam-optimality-practically.html

Occamality isn't another entrant in the myriad of design principles competing for attention in the chaotic world of software — it's an overriding principle, one that stands above and ranks all the contenders.

https://blackliszt.com/2023/09/occamality-and-other-design-principles.html

Occamality isn't confined to writing software. It applies to all stages of the development lifecycle, from requirements through QA and support.

https://blackliszt.com/2023/09/occam-optimality-applies-to-all-stages-of-the-software-life-cycle.html

The value of reducing redundancy isn't confined to software; it's a general principle.

https://blackliszt.com/2020/05/lessons-for-better-software-from-washing-machine-design.html

Saying that you should reduce redundancy in a program sounds simple, but once you get past trivial examples, it's not. Here's an analysis of the increasingly sophisticated kinds of redundancy in programs that should be addressed.

https://blackliszt.com/2023/10/moving-towards-occamal-software.html

Reducing redundancy is accomplished by taking a declarative approach to programming instead of a purely imperative one. There are many examples of this.

https://blackliszt.com/2021/07/software-programming-languages-the-declarative-core-of-functional-languages.html

Databases are an excellent, proven example of applying the principle of Occamality.

https://blackliszt.com/2023/09/occamality-in-databases.html

It's not just databases — Occamality is a thread that weaves through much of the history of software.

https://blackliszt.com/2023/09/occamality-in-software-history.html

You might think that reducing redundancy is an obviously valuable thing to do. The trouble is, modern software orthodoxy endorses the notion of collections of code that are separated by high walls (components, services, layers, objects, etc.), which typically leads to huge amounts of redundancy.

https://blackliszt.com/2023/09/occamality-the-problem-with-layers-components-and-objects.html

There is a simple idea that shows the basic approach to eliminating redundancy in programs: instead of stating how a thing should be accomplished, you concentrate on defining what is to be accomplished.

https://blackliszt.com/2023/09/achieving-occamality-what-not-how.html

The optimal way to reduce redundancy includes recognizing that in addition to instructions and data, programs include varying amounts of metadata. Metadata is an easy concept for those who use it, but many programmers don't get past the idea of parameters. Here's a way to understand metadata:

https://blackliszt.com/2023/09/achieving-occamality-through-definitions.html

In broader context, here's how metadata fits into a whole program, as the third dimension of software architecture.

https://blackliszt.com/2020/02/the-three-dimensions-of-software-architecture-goodness.html

For all dimensions, lack of redundancy is the main virtue. As a group, the more functionality is expressed in metadata and the less in code, the better.

A focus on metadata is similar to having a generic direction-generating program that refers to an easily-changed map.

https://blackliszt.com/2020/06/the-map-for-building-optimal-software.html

Here’s more theoretical depth on the role of metadata in a software system, with a comparison to theories of the solar system.

https://blackliszt.com/2022/10/how-to-improve-software-productivity-and-quality-code-and-metadata.html

Why put as much application knowledge into metadata as possible? It's the easiest thing to change, and above all, it's the best place to eliminate redundancy, which is the enemy of fast, error-free change.

https://blackliszt.com/2020/03/william-occam-inventor-method-for-building-optimal-software.html

Here is a more extensive explanation of the history and context of Occam's Razor and its relevance to software.

https://blackliszt.com/2023/09/occams-razor-the-key-to-optimal-software-development.html

How do you achieve this ideal architecture for a body of code? Not all at once! You avoid the usual nightmare of useless, ever-changing requirements and do something that makes a customer happier than they were. Then fix it. Here’s the process, to which I’ve given a fancy name.

https://blackliszt.com/2022/09/better-software-and-happier-customers-with-post-hoc-design.html

Here’s another statement of the basic idea:

https://blackliszt.com/2011/06/software-how-to-move-quickly-while-not-breaking-anything.html

https://blackliszt.com/2020/03/how-to-pay-down-technical-debt.html

Here is more detail and explanation of how to use increasing amounts of metadata to help build applications quickly, which of course should be a major goal of software architecture.

https://blackliszt.com/2022/05/how-to-improve-software-productivity-and-quality-schema-enhancements.html

Here's a short case study from early in my career that demonstrated to me the incredible value of taking an Occamal approach to building an end-user business application.

https://blackliszt.com/2023/09/achieving-occamality-through-definitions-case-study.html

Here is a more recent case study of a system based on extensive use of metadata and what happened when technology-fashion-driven executives took over the company.

https://blackliszt.com/2023/09/case-study-replacing-metadata-with-fashionable-software.html

One of the most basic aspects of software architecture is the data and where it is stored. The default choice for most architecture is to use a standard DBMS. Given the steady advance of Moore's Law, this is often no longer the best choice.

https://blackliszt.com/2010/09/databases-and-applications.html

Given the huge advantage of taking a metadata approach to software, why isn't it widely used? It's because all of software has been obsessed with procedural language as the core focus of programming. While necessary for the first decades of computing, it's now the core reason, never discussed, for the near-universal dysfunction in software development.

https://blackliszt.com/2024/08/why-is-writing-computer-software-dysfunctional.html

Bad Software Architectures

Software is infected with architectural religions, none of them with a sound basis in logic or real-world experience. It’s not that you can’t build software that sort of eventually kinda works with them – but it’s like building a car with a steam engine.

Sadly, some programming languages and programming concepts encourage redundancy.

https://blackliszt.com/2014/03/how-to-evaluate-programming-languages.html

Starting a couple decades ago the idea of “distributed computing” as an architecture become the thing all the cool kids gravitated to.

https://blackliszt.com/2015/04/the-distributed-computing-zombie-bubble.html

A modern incarnation (with a new name and rhetoric of course) is micro-services, which is supposed to boost programmer productivity.

https://blackliszt.com/2021/03/how-micro-services-boost-programmer-productivity.html

Not only does micro-services boost programmer productivity, it supposedly is a “scalable” architecture – in sharp contrast to the evil “monolithic” architecture … a word which is usually pronounced with a sneer.

https://blackliszt.com/2020/06/why-is-a-monolithic-software-architecture-evil.html

The trouble is, microservices make about as much sense as blood-letting did in medicine. It's widely accepted as useful, but entirely without evidence.

https://blackliszt.com/2019/02/what-software-experts-think-about-blood-letting.html

Programmers seem to like to layer their software, often without thinking about it.

https://blackliszt.com/2012/06/layers-in-software-fuss-and-trouble-without-benefit.html

Similarly when they link together pieces, a key decision is whether the coupling is loose or tight.

https://blackliszt.com/2012/08/coupling-in-software-loose-or-tight.html

Components and layers have been promoted for a long time.

https://blackliszt.com/2021/03/micro-services-the-forgotten-history-of-failures.html

https://blackliszt.com/2021/09/software-components-and-layers-problems-with-data.html

https://blackliszt.com/2021/08/the-dangerous-drive-towards-the-goal-of-software-components.html

For the best results, it’s good to focus on the goals of software architecture described above, and assure that everything that you do contributes to those goals. Part of how you do this is to avoid the always-present temptation of following software fashions.

https://blackliszt.com/2023/07/summary-software-fashions.html

August 31, 2023
Summary: AI Machine Learning Big Data Math Optimization

This is a summary with links to my posts about Big Data and the fancy algorithms and math that are variously called Artificial Intelligence, Cognitive Computing, Machine Learning, Big Data Analytics, math optimization, etc. I have worked as an insider in these fields for over fifty years.

Huge amounts of time and money are spent on these things, but getting practical, real-world results is not so easy. You can make things worse.

https://blackliszt.com/2023/03/making-things-worse-with-ai.html

How do you make things better? It starts with data.

https://blackliszt.com/2018/03/getting-results-from-ml-and-ai-1.html

Everyone knows that data is important. There’s even a field called Big Data.

What people call Big Data often isn’t nearly as big as people think.

https://blackliszt.com/2013/02/big-data-some-little-observations.html

Do the arithmetic!

https://blackliszt.com/2012/04/im-tired-of-hearing-about-big-data.html

For all the talk about Big Data, it’s clear that making sure that the data is accurate and complete is too much to ask.

https://blackliszt.com/2019/03/nobody-cares-about-data.html

Among other things, the status hierarchy in Data Science makes it clear that anyone with ambition needs to get as far away from the actual data as possible.

https://blackliszt.com/2019/03/the-hierarchy-of-software-skills-data-science.html

Without data integrity, your analytics is screwed. It’s worse when the data is about your health. The integrity of data in healthcare electronic medical records is a major issue.

https://blackliszt.com/2016/06/healthcare-innovation-emrs-and-data-quality.html

Even simple insurance provider network data is too often wrong, putting yet more obstacles between patients and the services they need.

https://blackliszt.com/2017/01/my-cat-taught-me-about-the-state-of-healthcare-provider-data.html

Even if all the data were perfect, no one seems to ask why exactly is big data better?

https://blackliszt.com/2015/07/fatal-flaws-of-big-data.html

Is Big Data delivering results? Here’s what happened with Big data and Hadoop at Yahoo.

https://blackliszt.com/2019/01/using-advanced-software-techniques-in-business.html

With all the money going to Big Data, people are ignoring the massive benefits of leveraging little data.

https://blackliszt.com/2016/08/little-data-vs-big-data.html

The few people who study computer history have noticed that Big Data is remarkably similar to what used to be called EDW, Enterprise Data Warehouse.

https://blackliszt.com/2015/10/big-data-and-data-warehouses.html

Same thing with a new name. Hmm. Could it possibly be that Big Data is just a fashion trend?

https://blackliszt.com/2013/02/the-big-data-technology-fashion.html

https://blackliszt.com/2015/03/big-data-the-driving-force-in-computing.html

At least for insiders, there's lots of humor to be found in data. There's even a book of the fun highlights.

https://blackliszt.com/2022/01/data-humor-book-by-rupa-mahanti.html

After data, you move to the amazing variety of techniques available, each of which has specific areas of applicability.

https://blackliszt.com/2018/04/getting-results-from-ml-and-ai-2.html

Even a narrow-sounding field like Machine Learning has incredible variety and lots of things you need to do until the algorithm can be effective.

https://blackliszt.com/2017/02/learning-machine-and-human.html

The simple concept of closed loop is essential to getting and keeping good results, but isn’t applied as often as it should be.

https://blackliszt.com/2018/04/getting-results-from-ml-and-ai-3-closed-loop.html

The application of the principles to healthcare has proven to be challenging, with high-profile failures. But there are successes.

https://blackliszt.com/2018/08/getting-results-from-ml-and-ai-4-healthcare-examples.html

I've discussed the application of the principles to fintech, with a focus on anti-fraud. The current leader in the field displaced the incumbent who was using neural network technology. Here is the background on HNC, which crowed about how it used neural networks to catch credit card fraud. Which it did. Sorta.

https://blackliszt.com/2016/06/how-blockchain-will-deliver-value.html

Here's the story of how the absolute winner of credit card fraud detection was displaced by a superior algorithm and concentrated on more relevant data.

https://blackliszt.com/2019/12/getting-results-from-ml-and-ai-5-fintech-fraud.html

Natural language AI has proved notoriously hard to make practical. Here’s an example of how it’s been done in fintech customer support.

https://blackliszt.com/2019/12/getting-results-from-ml-and-ai-6-fintech-chatbot.html

All too often, AI research is conducted in isolation. There is a great deal to do to assure that any results that are achieved can be integrated into production systems.

https://blackliszt.com/2022/06/how-to-integrate-ai-and-ml-with-production-software.html

Given all the issues in making AI work in practical reality, how likely is it that generative AI, the hot fashion in 2024, is likely to decimate the job market?

https://blackliszt.com/2024/07/how-many-jobs-will-ai-eliminate.html

As is true in most fields, in healthcare the money and attention tends to go to expensive, fancy methods requiring PhD’s instead of simple things that actually work.

https://blackliszt.com/2015/07/cognitive-computing-and-healthcare.html

https://blackliszt.com/2016/09/healthcare-innovation-from-washing-hands-to-ai.html

https://blackliszt.com/2016/05/healthcare-innovation-can-big-data-and-cognitive-computing-deliver-it.html

You don’t need AI or cognitive computing to discover or promulgate the new discoveries that humans make.

https://blackliszt.com/2015/08/human-implemented-cognitive-computing-healthcare.html

There are no technical obstacles to having computers do what doctors do. There is a clear path to getting it done. But the people in charge can't or won't make it happen.

https://blackliszt.com/2025/01/ai-can-automate-what-doctors-do.html

There are barriers to medical innovation, but there are terrible risks.

https://blackliszt.com/2025/02/can-ai-improve-medical-diagnosis.html

And then there's generative AI, which spews out expert-sounding language like a doctor. Is it better?

https://blackliszt.com/2025/07/will-ai-give-better-healthcare-advice.html

https://blackliszt.com/2025/07/chatgpt-covid-vaccine.html

There are many ways to spend lots of time and not get practical results.

https://blackliszt.com/2019/03/the-hierarchy-of-software-skills-data-science.html

Even when an optimization technique is perfected and proven in practice at scale, it can take decades for it to be used in other relevant fields.

https://blackliszt.com/2019/08/the-slow-spread-of-linear-programming-illustrates-how-in-old-vation-in-software-evolution-works.html

Impressive analytics performed by recognized experts are subject to getting the results the Expert wants.

https://blackliszt.com/2017/04/big-datas-big-face-plant.html

Corruption of data or process can lead to bad results anywhere.

https://blackliszt.com/2019/10/surprising-bugs-at-amazon-shows-how-ai-can-lead-to-disaster.html

For cynical definitions in the tradition of Ambrose Bierce of Big Data, Machine Learning, Deep Learning, Cognitive Computing and AI, see these.

https://blackliszt.com/2017/01/devils-dictionary-for-21st-century-computing.html

https://blackliszt.com/2017/01/devils-dictionary-for-21st-century-computing-2.html

https://blackliszt.com/2017/03/devils-dictionary-for-21st-century-computing-3.html

August 29, 2023
Summary: Wartime Software to Win the War

The vast majority of software development is organized to deliver software that meets a set of requirements on time and on budget. In other words, it’s organized to meet expectations.

What if you don’t have the time or money to do this? What if you’re focused on a need unmet by existing software and you’re driven to create software to meet that need? This is wartime software – get it done or lose the war. It’s optimized for speed, with customer satisfaction and unfailing quality as a given.

In this summary, we’ll cover relevant blog posts and books.

No matter how you do it, there are things about the software development process that are common.

https://blackliszt.com/2013/03/software-development-process-simple-terms.html

How do you measure success in software development? Agreeing on the measure of success is crucial to achieving it.

https://blackliszt.com/2013/03/software-postulate-the-measure-of-success.html

Why is so much software so bad? there are multiple reasons.

https://blackliszt.com/2011/06/why-computer-software-is-so-bad.html

https://blackliszt.com/2022/04/how-to-fix-software-development-and-security-a-brief-history.html

The near-universal focus on procedural language is the most important underlying cause of bad software development. It's not something a few tweaks can fix.

https://blackliszt.com/2024/08/why-is-writing-computer-software-dysfunctional.html

Peacetime Software Development

Peacetime software optimizes for predictability, i.e., delivering the requirements on time and budget. Wartime software optimizes for speed.

https://blackliszt.com/2010/10/software-project-management-dates-are-evil.html

Being on time and budget means making an estimate, which experienced people know is nearly impossible.

https://blackliszt.com/2012/06/the-nightmare-of-software-estimation.html

Estimates are also hard because software productivity is hard to measure.

https://blackliszt.com/2012/09/what-is-software-productivity.html

Whatever the productivity is, doubling the amount of work to be done using popular peace-time development techniques is sure to make it worse.

https://blackliszt.com/2012/09/software-productivity-divisors.html

Peacetime development involves loads of activities that aren’t programming.

https://blackliszt.com/2012/09/software-productivity-the-abp-factor.html

Project Management is supposed to assure that software development is “under control.” It’s actually a method to assure that programmers spend as little time as possible programming.

https://blackliszt.com/2012/10/software-project-management-book.html

https://blackliszt.com/2012/10/the-disease-of-software-project-management.html

The vast majority of software development bears a striking resemblance to the meticulous planning and building of Frankenstein’s monster.

https://blackliszt.com/2012/04/the-dirty-secret-of-peace-time-software-development.html

You get to near the end of a software project that has been “going well” and you suddenly discover the disaster.

https://blackliszt.com/2014/06/building-software-the-bad-old-way-and-the-good-new-way.html

With the usual measures, standard methods deliver terrible results.

https://blackliszt.com/2011/07/software-quality-theory-and-reality.html

The normal methods make common sense, like planning and making a Thanksgiving meal – and usually yield the classic mess you have after the meal.

https://blackliszt.com/2011/11/thanksgiving-meals-and-software-turkeys.html

Most peacetime software managers know they should act like they’re modern, criticizing old-fashioned waterfall and embracing Agile. Little change in results.

https://blackliszt.com/2013/03/software-comparing-waterfall-and-agile.html

Classic software development efforts focus on process instead of the substance of the software itself.

https://blackliszt.com/2013/03/process-and-substance-in-software-development.html

Most important decisions about software aren’t made by programmers, which is a big factor in things going wrong.

https://blackliszt.com/2014/01/who-makes-the-software-decisions.html

https://blackliszt.com/2014/10/joe-torre-and-software-development.html

Standard development embraces the metaphor of “building” software like a house, imagining that there’s a time when it’s done.

https://blackliszt.com/2013/10/when-is-software-development-done.html

Another factor in building software is “just” having competent people who do their jobs.

https://blackliszt.com/2014/04/delivering-software-is-a-nightmare.html

Why do all the experts cling to the expectation-based approach to software? The history of scurvy is a close parallel to software development diseases.

https://blackliszt.com/2014/02/lessons-for-software-from-the-history-of-scurvy.html

A major issue with expectation-based software is incentives. Among other things, when you’re judged on expectations, you’re incented to make huge estimates of the amount of work.

https://blackliszt.com/2015/04/software-problems-the-role-of-incentives.html

You might think that because software is related to math that it’s precise. There are huge factors like the exponential growth of computer power that are ignored in Computer Science and peacetime development.

https://blackliszt.com/2015/05/math-and-computer-science-vs-software-development.html

There are many paths people take to feel better about software development; they may sell well at the beginning but make things worse.

https://blackliszt.com/2015/06/software-problems.html

Software development is bad, and what makes it even worse it that people try to hide the failures, re-brand failures as success, and above all avoid tracking the outcomes.

https://blackliszt.com/2024/08/what-is-your-software-project-managers-batting-average.html

Wartime Software Development

The essence of wartime software development is that everything is optimized for speed of development.

https://blackliszt.com/2013/05/wartime-software-optimizing-for-speed.html

Great software development resembles growing a baby.

https://blackliszt.com/2011/06/software-how-to-move-quickly-while-not-breaking-anything.html

Just like with growing a baby, wartime software can’t be allowed to die.

https://blackliszt.com/2012/04/field-tested-software.html

A good way to see the contrast between the mainstream way of software development and the method used by software ninjas is to compare building bridges in peacetime and war time.

https://blackliszt.com/2012/03/bridges-and-software-in-peace-and-war.html

When you focus on speed, there is a simple concept that explains how it’s better.

https://blackliszt.com/2015/12/the-fundamentals-of-speed-optimized-software-development.html

https://blackliszt.com/2015/10/building-better-software-more-quickly.html

Is wartime software development like regular development only faster with fewer distractions? No. There are also essential differences in method and code standards.

https://blackliszt.com/2020/02/how-to-build-applications-that-can-be-changed-quickly.html

If software is always being changed, you want to make change easy. It’s what the average lay person wants from software.

https://blackliszt.com/2022/10/how-to-improve-software-productivity-and-quality-common-sense-approach.html

Here’s the idea of code without redundancies.

https://blackliszt.com/2020/06/the-map-for-building-optimal-software.html

While the software world may not have caught on to the value of no redundancy, the ordinary world of machine design is all over it.

https://blackliszt.com/2020/05/lessons-for-better-software-from-washing-machine-design.html

Here’s a broader exploration of focusing on redundancy.

https://blackliszt.com/2014/03/how-to-evaluate-programming-languages.html

Technical debt should mostly mean too much redundancy.

https://blackliszt.com/2020/03/how-to-pay-down-technical-debt.html

Technically, the way to eliminate redundancy is to move concepts from code to metadata.

https://blackliszt.com/2022/10/how-to-improve-software-productivity-and-quality-code-and-metadata.html

More about how to make quickly-changeable code.

https://blackliszt.com/2022/05/how-to-improve-software-productivity-and-quality-schema-enhancements.html

Code without redundancy is simple. It’s an ancient idea.

https://blackliszt.com/2020/03/william-occam-inventor-method-for-building-optimal-software.html

Here’s a summary of the method and techniques.

https://blackliszt.com/2022/09/better-software-and-happier-customers-with-post-hoc-design.html

Here's an example of how one group evolved from trying to improve quality and availability to wartime software:

https://blackliszt.com/2024/06/the-amazing-path-to-high-quality-fast-to-change-software.html

Of course, following good business and product strategies is an essential aspect of wartime software — among other things, carefully selecting customer focus is key, Like conquering Europe in World War 2 by landing on a small number of beaches in Normandy instead of invading the whole coastline.

https://blackliszt.com/2023/07/summary-software-innovation.html

Wartime Software Book

Here is a brief introduction with links to my book on Wartime Software.

https://blackliszt.com/2013/05/wartime-software-book-available.html

July 23, 2023
Summary: Software Innovation

This is a summary with links to my posts on software innovation. It includes posts on barriers to innovation and how to grow a winner.

Software is taking over the world. The pace and scope of the transformation of human activity has no precedent. People often assume that this is the result of fierce innovation in software. While brand-new software is built every day, the actual innovation is the result of computer innovation – which does indeed proceed at an unprecedented pace.

https://blackliszt.com/2013/12/fundamental-concepts-of-computing-speed-of-evolution.html

https://blackliszt.com/2014/03/innovation-with-computers-and-slow-things-1.html

The fact is that innovation in software is incredibly simple. It rarely involves fancy stuff like AI, but mostly figuring out the best way to accomplish things and getting the computer to do as much of the work as possible.

https://blackliszt.com/2014/07/innovation-made-simple.html

Here are some of the most important patterns of the small groups that innovate their way to success in software.

https://blackliszt.com/2017/01/the-science-of-innovation-success.html

The general impression is that software is all about innovation and rapid evolution. Programmers are on the front lines, constantly making new things. Sadly, this general impression, which is shared by most programmers, doesn't hold up under examination.

https://blackliszt.com/2023/07/does-software-evolve-rapidly.html

People love to brag about the software innovations they’ve invented. The fact is, most of the fundamental innovations in software are proven and in place; they’re ignored by practically everyone, including the experts.

https://blackliszt.com/2015/06/fundamental-innovations-in-software.html

This holds true even when you look at a narrow field of application such as financial technology.

https://blackliszt.com/2016/03/fintech-innovation-the-drivers.html

An amazing fraction of what appears to be innovations are little but taking advances that are proven in a narrow domain and applying them to a new one. Here is the story of an algorithm that was standard practice in oil refinery operation over 50 years ago that, decade by decade, is still crawling its way into new domains.

https://blackliszt.com/2019/08/the-slow-spread-of-linear-programming-illustrates-how-in-old-vation-in-software-evolution-works.html

Here is an example of a truly beneficial innovation proven over 50 years ago that is still not used in medical imaging.

https://blackliszt.com/2019/07/barriers-to-software-innovation-radiology-1.html

https://blackliszt.com/2019/08/barriers-to-software-innovation-radiology-2.html

It’s not just fancy algorithms that are proven and waiting for application in new domains. It’s simple things like production human data entry, where widely proven “heads down” methods are at least 5 times more efficient than what is normally done.

https://blackliszt.com/2019/09/simple-data-entry-technology-illustrates-how-in-old-vation-in-software-evolution-works.html

Here are details of the advantages of “heads down” data entry and how it was ignored at a huge project at Sallie Mae.

https://blackliszt.com/2019/10/software-professionals-would-rather-be-fashionable-than-achieve-10x-productivity-gains.html

One of the common ways to ignore a major innovation such as “heads down” data entry is to concentrate on a method that is highly fashionable, even though it doesn’t do much good.

https://blackliszt.com/2019/09/laser-disks-and-workflow-illustrate-the-insane-fashion-driven-nature-of-software-evolution.html

Given this, it’s all the more amazing that companies have Chief Innovation Officers whose job is usually to “foster innovation.” Heh.

https://blackliszt.com/2016/04/the-innovation-bubble.html

Barriers and Resistance to Innovation

Software innovation faces huge barriers from people and established practice, like innovation in medicine, where a cure for scurvy was proven in practice for decades before the authorities grudgingly accepted it.

https://blackliszt.com/2014/02/lessons-for-software-from-the-history-of-scurvy.html

Innovation has been strongly resisted for a long time.

https://blackliszt.com/2020/01/luddites.html

https://blackliszt.com/2016/05/innovation-some-history.html

Have you heard the story of Samuel Pierpont Langley, the renowned expert on manned flight? He’s a case study in how experts prevent innovation.

https://blackliszt.com/2016/07/innovation-and-experts.html

Major advisory institutions reflect common thinking and prevent their customers from "making mistakes" with innovative companies.

https://blackliszt.com/2017/05/the-value-of-computer-industry-advisory-groups.html

Here is the story of how the British military resisted a huge innovation for combating submarines in World War 2.

https://blackliszt.com/2021/10/deep-seated-resistance-to-software-innovation.html

Even the experts in entrepreneurship resist innovation. Amazing. Experts!

https://blackliszt.com/2020/05/experts-vs-innovation-new-book.html

Perhaps you think that the big tech companies are great at innovation? When you look at how they actually innovate, they don’t look so great.

https://blackliszt.com/2016/05/organizing-for-successful-innovation-recent-history.html

Often important innovations don’t require software at all, which doesn’t seem to stop people spending loads on money on exotic software.

https://blackliszt.com/2016/09/healthcare-innovation-from-washing-hands-to-ai.html

https://blackliszt.com/2016/05/healthcare-innovation-can-big-data-and-cognitive-computing-deliver-it.html

Healthcare is a rich source of examples of how to screw up and fail to take advantage of obvious, long-overdue “innovations” like electronic medical records.

https://blackliszt.com/2016/06/healthcare-innovation-emrs-and-paper.html

https://blackliszt.com/2016/05/healthcare-innovation-emr-procurement-is-broken.html

https://blackliszt.com/2016/06/healthcare-innovation-emrs-and-data-quality.html

While experts and big companies put up active resistance to innovation, regulations are an important source of passive resistance.

https://blackliszt.com/2016/09/innovation-the-barriers.html

The huge cost of medical imaging systems is a clear example of how regulations prevent innovation and keep costs high.

https://blackliszt.com/2023/01/how-to-reduce-the-cost-of-medical-imaging-and-pacs.html

When you look into sample sets of regulations, you see how onerous they are and how they hamstring innovation.

https://blackliszt.com/2016/12/regulations-that-enable-innovation.html

There’s a clear path to eliminating regulatory resistance while making the enforcement power of the regulations even stronger, by shifting from massive how-type regulations to simple but effective what-type ones.

https://blackliszt.com/2011/12/regulations-goals-or-directions.html

https://blackliszt.com/2012/03/lets-criminalize-our-regulations.html

When you look at how innovation works in practice, it's hard to avoid thinking that there's a conspiracy to prevent innovation. It's mostly not a conscious effort on the part of the conspirators, but it has the same net effect.

https://blackliszt.com/2024/06/the-conspiracy-to-prevent-innovation.html

Growing the Innovation to Success

Once you’ve got your software company up and running, there are strategic moves that will keep you on the track to success.

https://blackliszt.com/2010/05/from-start-up-to-real-success.html

https://blackliszt.com/2010/12/from-startup-to-success-costs.html

Would you like to follow Facebook’s growth path? Their success wasn’t about great software development. It was due to a classic product/company growth strategy.

https://blackliszt.com/2014/12/fb.html

Your strategy may be good, but unless you build applications that can be changed quickly, you’ll lose the race.

https://blackliszt.com/2020/02/how-to-build-applications-that-can-be-changed-quickly.html

When you’re moving quickly, classically-trained programmers may start whining about the growing amount of “technical debt” and how important it is to pay it down. Here’s how to do it.

https://blackliszt.com/2020/03/how-to-pay-down-technical-debt.html

If you follow classic software development methods instead of fast ones, the chances are you could be hit with a big surprise at the end of the project.

https://blackliszt.com/2014/06/building-software-the-bad-old-way-and-the-good-new-way.html

There is a way to avoid disasters of this kind. It's practicing wartime software development, optimizing your methods for speed instead of meeting expectations.

https://blackliszt.com/2023/07/summary-wartime-software-to-win-the-war.html

When you’ve built a successful narrow-focus software company, how can you grow it further?

https://blackliszt.com/2022/07/the-dimension-of-software-automation-breadth-examples.html

The Book

Here are highlights of winning growth strategies and the book with more stories and details.

https://blackliszt.com/2016/04/software-business-and-product-strategy-book.html

https://blackliszt.com/2016/05/innovation-from-startup-to-success.html

Here are highlights of the companies used as examples in the book.

https://blackliszt.com/2016/07/innovation-stories.html

July 11, 2023
Summary: Software Fashions

Software is a precise discipline. A single bit out of place can crash a program. It’s like math. Fashion is the exact opposite – skirts are short or long, you wear a tie or you don’t, whatever. It’s all opinions and taste, nothing objective about it. How could there be fashions in bit-driven, nerdy software? This is a summary of my writing on software fashions.

https://blackliszt.com/2018/11/what-are-software-fashions.html

Here’s a survey of some current software fashions and a couple that have become standard practice.

https://blackliszt.com/2019/05/recurring-software-fashion-nightmares.html

Software people are, after all, people. People want to have status. One of the main ways to get status is to get identified with a fast-rising fashion trend and ride it.

https://blackliszt.com/2018/10/the-hierarchy-of-software-status.html

The status-driven attraction of fashion is so powerful that software people will often go with the fashionable approach and ignore software methods that would achieve dramatically better results.

https://blackliszt.com/2019/10/software-professionals-would-rather-be-fashionable-than-achieve-10x-productivity-gains.html

https://blackliszt.com/2019/09/laser-disks-and-workflow-illustrate-the-insane-fashion-driven-nature-of-software-evolution.html

Other things that become fashions are OK ideas taken way beyond where they make sense. These fashions often appear repeatedly, usually with new names.

https://blackliszt.com/2021/03/micro-services-the-forgotten-history-of-failures.html

Another pattern is wild swings from one end of a spectrum to the other — and back again. You'd better be on the right end of the spectrum or risk being "out of it!" In computer programs whether data is defined inside the program logic or outside of it is a decades-long example.

https://blackliszt.com/2015/06/innovations-that-arent-data-definitions-inside-or-outside-the-program.html

Computers and software are all about innovation — so how could innovation itself be a fad? Well, for a few years, innovation was a fashion-forward as you could get. Until it faded back to its usual place.

https://blackliszt.com/2016/04/the-innovation-bubble.html

Sometimes technical fashions are picked up by nontechnical people, who wildly promote the fashion without having clue about what the underlying technology is about, and therefore get it wrong. DLT (blockchain, the technology underlying cryptocurrencies) is an example of something that is promoted as a solution for problems for which it's not relevant.

https://blackliszt.com/2015/11/the-magic-of-block-chain.html

And for problems which have existing solutions that are often thousands of times better.

https://blackliszt.com/2017/11/blockchain-a-sailboat-without-sails.html

It wasn’t so long ago that everyone was talking about Big Data:

https://blackliszt.com/2015/03/big-data-the-driving-force-in-computing.html

https://blackliszt.com/2013/02/the-big-data-technology-fashion.html

https://blackliszt.com/2013/02/big-data-some-little-observations.html

https://blackliszt.com/2012/04/im-tired-of-hearing-about-big-data.html

Here’s an example of how the focus on fashion with Hadoop (a fashion-driven approach to Big Data) led to bad results.

https://blackliszt.com/2019/01/using-advanced-software-techniques-in-business.html

Large bunches of data, shared computers (the cloud) and virtualization are things that have been around since the early days of computing, are still around and always will be. But for a period of time they were what fashion-forward people talked about.

https://blackliszt.com/2012/05/the-cloud-and-virtualization.html

https://blackliszt.com/2011/12/the-name-game-of-moving-to-the-cloud.html

How is it possible that fashion can play such a large role in software decision-making? It’s because evidence is rarely part of software decision-making.

https://blackliszt.com/2017/02/evidence-based-software.html

https://blackliszt.com/2014/02/lessons-for-software-from-the-history-of-scurvy.html

In a broader sense, fashion rules in software because, in spite of the name, Computer Science is not a science.

https://blackliszt.com/2023/04/summary-computer-science.html

July 3, 2023
Summary: Software Fundamentals

You might think that computer fundamentals would be the first things taught and often returned to in Computer Science. To the contrary, some of the most important things about software may be known to programmers if you ask them, but understood and practiced? Not so much.

https://blackliszt.com/2012/02/fundamental-concepts-of-computing.html

https://blackliszt.com/2012/04/where-does-software-come-from-what-is-software-.html

What do you do with software — aside from wasting huge amount of time? You automate!

https://blackliszt.com/2020/01/the-fundamentals-of-computer-automation.html

In any field you can think of, there are clear, agreed-upon measures of success. In software? Not so much.

https://blackliszt.com/2013/03/software-postulate-the-measure-of-success.html

If you have trouble measuring success, how can you measure productivity?

https://blackliszt.com/2012/09/what-is-software-productivity.html

If you have trouble measuring productivity, practices can become widely used that, if you step back from them, you can see obviously reduce productivity.

https://blackliszt.com/2012/09/software-productivity-divisors.html

Programming is taught as though all programming is similar. Nothing could be further from the truth. Is arguing a criminal case in a courtroom the same as acting in a Shakespeare play? They both use English, right? Programming has distinctions that are just as sharp.

https://blackliszt.com/2023/09/classifications-of-software-programs.html

One of the most obvious but least examined fundamental is that software is data. The fact that there is a memory that contains a mix of instructions and data … and that the instructions are data … is fundamental, with many implications.

https://blackliszt.com/2014/04/fundamental-concepts-of-computing-software-is-data.html

Diving into and understanding the possible relationships between data and instructions is bedrock core to being a real programmer.

https://blackliszt.com/2021/08/the-relationship-between-data-and-instructions-in-software.html

Everyone knows about numbers and counting. But how many times do people stand back from their work and apply some appropriate numbers to them? Not often enough.

https://blackliszt.com/2012/10/fundamental-concepts-computing-counting.html

Closely related to counting is speed. The speed of computing has increased unlike anything else in our existence. If cars got 10X faster, we would change lots of things, including whether to drive or fly. But as computing has gotten 100X or more faster, we too often fail to adapt the way we program.

https://blackliszt.com/2013/12/fundamental-concepts-of-computing-speed-of-evolution.html0

When we walk or drive, we always look ahead and change details of our walking or driving to adapt to ever-changing conditions. Adapting what you do to what you see and learn is fundamental common sense. All too often, it is not part of programming.

https://blackliszt.com/2013/02/fundamental-concepts-of-computing-closed-loop.html

Some fundamental concepts of software aren’t taught and most people, including programmers, have the wrong ideas in their head about them. One of those concepts is that a program is a passive thing that needs to be built, instead of a set of actions that needs to be choreographed.

https://blackliszt.com/2021/08/fundamental-concept-software-is-not-a-passive-thing-it-is-actions.html

Flowcharts and workflow express the actions of a program. They have been around since the start of programming. They're still around, but largely ignored as a fundamental.

https://blackliszt.com/2022/06/flowcharts-and-workflow-in-software.html

You think of building a house, after which it’s mostly unchanged. Software is different. Being built is like being born – software spends most of its “life” being changed.

https://blackliszt.com/2013/10/when-is-software-development-done.html

https://blackliszt.com/2021/08/fundamental-concept-software-is-rarely-built-mostly-it-is-changed.html

What we call “building” software works much better if we treat as “growing a baby.”

https://blackliszt.com/2011/06/software-how-to-move-quickly-while-not-breaking-anything.html

It’s easy to think that most of software has been figured out, and that innovation is possible only at the nether reaches of the field. Kind of like how innovation in math isn’t needed or performed from arithmetic through algebra and calculus and way beyond. Not true in software! Some of the most fundamental things, like quality, are ripe for fundamental innovation.

https://blackliszt.com/2015/06/fundamental-innovations-in-software.html

Speaking of math, computing started as a way to do math and grew as part of math departments. The misconception that math and Computer Science are closely related is widespread. This is deeply harmful.

https://blackliszt.com/2015/05/math-and-computer-science-vs-software-development.html

https://blackliszt.com/2015/04/math-and-computer-science-in-academia.html

The usual practice in computing is to dive right into the details. Perspective and putting it in a broader context is tremendously helpful if you aspire to be truly excellent in computer software. Here are some sample books that provide that context.

https://blackliszt.com/2023/05/inspiring-books-for-writing-software.html

Computing and software would greatly benefit by establishing the fundamental concepts of the field. Individual programmers can become vastly more productive by understanding the fundamentals and practicing them.

June 23, 2023
Summary: Software Programming Languages

This is a summary with links to all my posts on software programming languages.

Computers need to be “told” what to do. Each computer has a native language it understands, a “machine language.” Each machine language has a human-readable version called “assembler language.” Starting in the 1950’s “high level” languages for telling the computer what to do (programming it) have been developed.

Here are the basics of that amazing progression.

https://blackliszt.com/2020/09/the-giant-advances-in-software-programming-languages.html

Software programming language core concepts

How do you learn a programming language so you can be a programmer? No one wants to tell you that the best way to learn is on your own!

https://blackliszt.com/2025/01/how-to-learn-to-be-a-computer-programmer.html

One of the core features of all programming languages is the subroutine. Everyone who programs uses subroutines, but their power can be underappreciated.

https://blackliszt.com/2021/03/fundamental-concepts-of-computing-subroutine-calls.html

Among the earliest features added to languages were structures, blocks and macros. Each of these contributed to more concise and understandable programs.

https://blackliszt.com/2021/09/software-programming-language-evolution-structures-blocks-and-macros.html

Once you get past the basics of language, you find that widely used languages have rich collections of subroutine libraries and sometimes what are called frameworks, which contribute a large fraction of any program’s solution.

https://blackliszt.com/2021/10/software-programming-language-evolution-libraries-and-frameworks.html

The way programs flow is a key aspect.

https://blackliszt.com/2022/06/flowcharts-and-workflow-in-software.html

Programs consist of a combination of instructions and definitions of the data on which the instructions operate. Exactly how those relate to each other has been the subject of major disputes – which continue to this day.

https://blackliszt.com/2015/06/innovations-that-arent-data-definitions-inside-or-outside-the-program.html

Some programming "advances" basically try to restrict the possible relationships betwen code and data. Bad idea.

https://blackliszt.com/2021/08/the-relationship-between-data-and-instructions-in-software.html

Software programming language evolution

When talking about software languages, the term high level language was introduced early. Here’s what it means.

https://blackliszt.com/2020/10/what-is-high-about-a-high-level-language-in-software.html

Decades ago a couple of languages were invented and enjoyed an explosion of use that were major steps ahead of the standard languages of the time in terms of productivity.

https://blackliszt.com/2020/11/software-programming-language-evolution-beyond-3gls.html

Not long after these amazing, productivity-enhancing languages had their rapid growth, the modern DBMS came onto the scene, with its own language for data definition, query and update. The impact of the DBMS revolution was huge.

https://blackliszt.com/2021/01/software-programming-language-evolution-impact-of-the-dbms-revolution.html

One of the major reactions to the DBMS revolution was the introduction and rise to prominence of what were called 4-GL’s, all which of incorporated the DBMS as an integral part.

https://blackliszt.com/2021/02/software-programming-language-evolution-4gls-and-more.html

There has been an explosion of programming language invention.

https://blackliszt.com/2014/04/continents-and-islands-in-the-world-of-computers.html

It just goes on and on and on and on, with no end in sight.

https://blackliszt.com/2022/04/software-programming-language-cancer-must-be-stopped.html

Software programming language innovations

Experts tout the major advances in programming languages made since the early days, but they don’t amount to much.

https://blackliszt.com/2020/09/software-programming-languages-50-years-of-progress.html

A comparison to the virtues of various human languages makes this clear.

https://blackliszt.com/2020/12/the-bronte-sisters-and-software.html

Attempts have been made by experts to fix problems that have been widely noted for many decades.

https://blackliszt.com/2021/09/software-programming-language-evolution-the-structured-programming-goto-witch-hunt.html

One of the serious detours into unproductive swamps taken by programming languages has been what’s called object-orientation. It had a promising start and would have been fine had it stopped there.

https://blackliszt.com/2021/11/the-promising-origins-of-object-oriented-programming.html

Lots of people have things to say about O-O languages. There are the academics and other authorities and then there are the people with real experience who aren’t groupies.

https://blackliszt.com/2022/01/object-oriented-software-languages-the-experts-speak.html

Here are some facts and logic to explain why O-O should be referred to exclusively in the past tense.

https://blackliszt.com/2022/03/why-object-orientation-in-software-is-bad.html

O-O is so bad that people who actually know stuff take pleasure from mocking it:

https://blackliszt.com/2022/05/making-fun-of-object-orientation-in-software-languages.html

Another category of hyped but harmful programming languages have been “functional” ones. These arose early in programming evolution and keep popping up.

https://blackliszt.com/2021/05/software-programming-language-evolution-functional-languages.html

The attraction of functional language has a strong basis. It’s true that increasing the declarative aspect of a total solution brings strong benefits. But embedding it in the language neuters it.

https://blackliszt.com/2021/07/software-programming-languages-the-declarative-core-of-functional-languages.html

Here are some detailed examples of how supposed advances in programming languages have worked out in practice, including failures of Java and 4-GL’s and a huge success with a brand-new system written in that ancient bit of history called COBOL.

https://blackliszt.com/2021/01/software-programming-language-evolution-credit-card-software-examples-1.html

https://blackliszt.com/2021/02/software-programming-language-evolution-credit-card-software-examples-2.html

https://blackliszt.com/2022/08/software-programming-language-evolution-credit-card-software-examples-3.html

After all those failed advances, there's the amazing case of HTML, a wildly successful language without all the bells and whistles that takes the winning principle of "just get the job done."

https://blackliszt.com/2010/05/html-assembler-language-easy-to-use-tools.html

Software programming language evaluation

Promoters of new languages tout how they prevent programmers from making errors (ha!) and result in cleaner programs that somehow promote productivity – which is never measured or tested.

The fact is, there is a standard by which programming languages should be judged – along with the support systems and metadata that are an integral part of the solution. It’s simple: how many places do you have to go to make a change? The right answer is one. Here’s the idea.

https://blackliszt.com/2014/03/how-to-evaluate-programming-languages.html

June 13, 2023
Summary: Blackliszt and David B Black Writing

I have written over 540 blog posts on Blackliszt.com; they have gotten over 170,000 views. I’ve published published five books on computer software. This is an introduction with links to summary posts on the topics I’ve written about.

Throughout my career I followed the low-level strict rules, i.e., my programs compiled, ran and did what they were supposed to do. As the years passed, I learned that most of the widely accepted high-level rules lead to huge time and effort and produce mostly bad software. I first learned this as a programmer and CTO and then as a VC investor, seeing that the winners often follow a different set of rules. I’ve tried to explain the status quo and the better alternatives in my blog and books.The contrarian ideas and methods I describe are ignored and/or forbidden in software teaching and management. They represent a revolution in quality, efficiency and effectiveness, leading to superior results in a fraction of the elapsed time and often less than a tenth of the labor. Along the way, I enjoy making fun of empty fads like blockchain as strong examples of the fashion-driven, floating-in-the-clouds nature of much software “thinking.”

I have recently applied my software approach to the issue of building systems for voting in public elections. blackliszt.com/category/voting-systems

I have had extensive experience with the medical system as an investor and patient. To say that massive improvements are possible is both obviously true and false. My posts detail some of the problems and solutions, which supports the “obviously true” response. However, so long as the current crew of players (insurance, financing, pharma, regulation and training, incentives for treatment), the only sane response is to recognize that while trivial improvements are possible, massive ones are not.

For details of my experience see this. https://www.linkedin.com/in/david-black-49b2b4a2

The Summaries

Computer Science doesn’t come close to being a science. It takes baseless beliefs and promotes them as proven truths.

https://blackliszt.com/2023/04/summary-computer-science.html

For something that claims to be a science, software is amazingly fashion-driven.

https://blackliszt.com/2023/07/summary-software-fashions.html

Because software is invisible to most people, it’s described in metaphors, which are mostly inappropriate and lead to trouble. Even the programmers use the metaphors, which often support methods that are consistently bad.

https://blackliszt.com/2023/06/summary-bad-software-metaphors-lead-to-bad-results.html

If you want to understand what’s going on in software, it’s natural to turn to the experts, insiders who share their knowledge so that most of us can understand the mysteries of software. There’s just one little problem: software experts are as valuable as doctors were when blood-letting was standard practice.

https://blackliszt.com/2023/11/summary-software-experts.html

Everyone wants to know what’s “new” in software; the new thing is most often just a fashion. Innovation? It’s supposed to be what fuels software and changes the world. In reality, there is almost no innovation in software itself, and software-driven innovation is most often what I call in-old-vation, i.e., an advance that’s proven in one domain that often takes decades to crawl into new domains. This summary also covers business strategies for taking software innovations to success.

https://blackliszt.com/2023/07/summary-software-innovation.html

How do you win with software? Nearly everyone starts with requirements, makes estimates and is judged whether they deliver on time and on budget. This optimizes for expectations and is a proven path to failure. The winning path optimizes for speed, customer satisfaction and continuous quality. It’s what the people who need to win do.

https://blackliszt.com/2023/07/summary-wartime-software-to-win-the-war.html

Software people don’t learn the fundamentals of computing; they may be aware of any one of them if asked, but don’t apply them. In any other field, knowing and applying the fundamentals — like blocking and tackling — is demanded. Not in computing.

https://blackliszt.com/2023/06/summary-software-fundamentals.html

A key aspect of software fundamentals is software architecture. A surprising amount of the received wisdom about architecture is counter-productive, the goals are rarely discussed and the key to effective software architecture, Occamality, is largely ignored. Occamality is a foundational tool of wartime software: get the job done quickly and flawlessly, meet the real need and beat the competition.

https://blackliszt.com/2023/08/summary-occamality-and-software-architecture.html

If you want to architect and build good software, a major helpful factor that is usually ignored is to learn the history and evolution of software. Patterns repeat and knowing where you are in an evolution can help you predict what is likely to happen. Not to mention avoiding going down paths that usually fail.

https://blackliszt.com/2023/08/summary-computer-software-history-and-evolution.html

What do software people do? They write programs. What language do they use? What are programming languages, anyway? Are there good ones and bad ones? How have things changed?

https://blackliszt.com/2023/06/summary-software-programming-languages.html

Software people are indeed people, but a good fraction of them are different from non-programming people, with differences that are important to understand. Similarly, there are important differences in how to manage them most effectively.

https://blackliszt.com/2023/05/summary-software-people.html

Regulations are a problem in many areas, but are particularly pernicious in software and technology. By commanding that developers use largely obsolete methods for building technology, a huge increase in cost and time results, with no improvement in quality or outcome.This is a direct result of ignorant bureaucrats telling technical people in great detail HOW to get things instead of simply defining WHAT should be accomplished, like the criminal law does when it states simply that murder should not be committed.

https://blackliszt.com/2023/11/summary-regulations.html/

Computer security is assured by mountains of regulations, tools, standard practices and audits. But computer security continues to be breached without pause. There are major improvements possible that are largely ignored.

https://blackliszt.com/2023/05/summary-the-ongoing-problem-of-computer-security.html

Software quality assurance is an important specialty within the non-science of computing, but in spite of all the time and money spent, quality continues to be a major issue. There are solutions that have been proven in practice that are ignored by the experts and authorities.

https://blackliszt.com/2023/04/summary-software-quality-assurance.html

Everyone knows that software project management is essential to producing software that works, on time and on budget. In spite of decades of “innovation,” it doesn’t get better. The winners follow a different set of rules.

https://blackliszt.com/2023/04/summary-software-project-management.html

Huge amounts of attention are paid to AI, ML, Big Data, and the rest. It’s a fashion that too often results in vast amounts of money being spent with little to show for it. Results can be amazing when achieved, but it takes a combination of deep knowledge and common sense to do it.

https://blackliszt.com/2023/08/summary-ai-machine-learning-big-data-math-optimization.html

Data centers and networks are the underpinning of computing. The transition to the Cloud has been a big factor, along with the rapid evolution of hardware and the glacial evolution of software. Net Neutrality has been an annoying furor, utterly divorced from reality.

https://blackliszt.com/2023/08/summary-computer-data-centers-and-networking.html

The vast majority of major organizations studiously follow the established methods of software design, construction, operation and security. They learn from academia, industry advisory groups and leading experts. The results are consistent. Disastrously consistent. This is true not only of governments but cool big tech companies including Google, Facebook, Twitter and the rest.

https://blackliszt.com/2023/11/summary-software-in-government-big-business-and-big-tech.html

Cryptocurrency and blockchain are highly visible for many reasons, including the fact that crypto fuels the explosion of ransomware. Leading thinkers are convinced that blockchain is a technology that will revolutionize computing.

https://blackliszt.com/2023/01/cryptocurrency-and-blockchain-summary.html

Learning about the mechanics and technology of the healthcare system provides an education on the huge opportunities to reduce costs, increase efficiency and make things better for patients and providers.

https://blackliszt.com/2023/08/summary-the-business-of-healthcare-medicine-and-insurance.html

I’ve had some personal experiences with the healthcare system that have resulted in a sobering education about what really goes on. Healthcare is incredibly valuable with many dedicated professionals involved, but there are some serious flaws.

https://blackliszt.com/2023/08/summary-the-medical-industrial-complex.html

I have published five books on these topics. The books are Software Business and Product Strategy, Software Quality, The Disease of Software Project Management, Wartime Software, and Software People. Here are introductory blog posts to them with links to the books.

https://blackliszt.com/2023/05/summary-david-b-black-books.html

June 10, 2023
Summary: Bad Software Metaphors lead to bad results

Most of the people who talk about software can’t create it. They can’t even read it with understanding. So how is anyone who not only can’t write software but can’t even see it supposed to understand it, much less manage it?

https://blackliszt.com/2016/12/managing-what-you-cant-see.html

https://blackliszt.com/2017/04/managing-software-thats-invisible-to-you.html

A standard way about talking about software has emerged among the people to whom it’s invisible: physical-world metaphors. The metaphors are so ubiquitous that they are also used by programmers.

Because the metaphors are nearly always seriously inappropriate and people act as though they are great, horrible things result.

https://blackliszt.com/2016/11/software-myths-and-myth-conceptions.html

https://blackliszt.com/2021/09/the-catastrophic-effect-of-bad-metaphors-in-software.html

Wouldn’t it be nice if we could churn out software as reliably and predictably as a factory?

https://blackliszt.com/2012/04/how-effective-are-software-factories.html

The building metaphor is widely used. Buildings have architects, and so does software.

https://blackliszt.com/2018/12/software-planning-is-impossible.html

https://blackliszt.com/2017/06/software-myth-conceptions-building-materials.html

You architect and build a house. It’s a passive physical object. Software is NOT a physical object; it’s mostly is a set of actions to be taken by a computer.

https://blackliszt.com/2021/07/fundamental-concept-software-is-not-a-passive-thing-it-is-actions.html

The notion that the core organizing principle of software is that it's a collection of “objects” has had catastrophic effects.

https://blackliszt.com/2021/11/the-promising-origins-of-object-oriented-programming.html

Once you’ve built a house, you hope that modifying it is rare. Software, by contrast, undergoes nearly continuous modification.

https://blackliszt.com/2013/10/when-is-software-development-done.html

https://blackliszt.com/2021/07/fundamental-concept-software-is-rarely-built-mostly-it-is-changed.html

We maintain our cars, homes and devices. Heating and cooling systems need regular maintenance. So do our bodies! If we don’t care for our bodies properly, they break down! Software, by sharp contrast, never needs to be maintained. NEVER!

https://blackliszt.com/2022/01/software-never-needs-to-be-maintained.html

Waterfall and Agile are metaphors widely used in software project management. They hurt without helping.

https://blackliszt.com/2013/03/software-comparing-waterfall-and-agile.html

Software is amazing. We interact with it when using our computers and phones and when playing video games. Understanding what goes on behind the scenes is a whole different thing. It's understandable that metaphors are how people think about it. Sadly, the inappropriate metaphors result in bad software management.

June 5, 2023
Summary: Software People and Management

Guess what – software people aren’t like regular people! There are differences in the ways they think and act, judging results can be difficult, and hiring and managing is challenging. This is a summary of my writing on the subject.

Skills, Training and Status

The education establishment has taken over the world of skilled jobs. They've convinced the world that getting a degree in Computer Science is the only way to learn how to be a computer programmer. They are wrong.

https://blackliszt.com/2025/01/how-to-learn-to-be-a-computer-programmer.html

When you get a degree in CS or learn a programming language, you’re all set, right? Actually, those things are pretty much the same as knowing how to hammer a nail when building a house. It’s a nice start, but there are levels and levels of skill beyond the basics.

https://blackliszt.com/2018/09/the-hierarchy-of-software-skills.html

There is perennial conflict between young and old in software, perhaps more than other fields.

https://blackliszt.com/2010/12/experience-and-youth.html

You might think that getting a degree in Computer Science prepares you to be an effective programmer, much like getting an MD prepares you to be an effective doctor. A comparison of the two makes it clear that CS education is sadly deficient.

https://blackliszt.com/2020/07/job-requirements-for-software-engineers-should-stop-requiring-cs.html

Does status in software correlate with your skills, kind of like how hitting more home runs gives you status in baseball? Sadly, no. One aspect of software status is that the more layers there are between you and real customers, the more status you have.

https://blackliszt.com/2011/10/status-in-software-silliness-and-stupidity.html

There are other dimensions, for example the more management layers between you and fingers-on-keyboard programming, the greater the pay and status. And there are others!

https://blackliszt.com/2018/10/the-hierarchy-of-software-status.html

There is a similar skills and status hierarchy in data science

https://blackliszt.com/2019/03/the-hierarchy-of-software-skills-data-science.html

One of the sad things about software is that writing great code and getting credit for it are largely unrelated to each other.

https://blackliszt.com/2020/10/elizebeth-smith-friedman-the-cancelled-heroine-of-cryptography.html

Programming skill and power in the programmer is far more important than the differences between programming environments.

https://blackliszt.com/2010/05/what-is-the-best-programming-environment.html

At least in some environments, high-performing nerds are highly valued.

https://blackliszt.com/2011/07/the-new-top-gun-is-top-nerd.html

Understanding Programmers

When you think about hard-core programmers, the word "nerd" comes to mind. In the poplar imagination, "nerd" often seems to mean guys who play lots of games. And have attitude. Of course there are such people. But the very best programmers, in my experience, don't match that description.

https://blackliszt.com/2012/02/most-nerds-are-introverts.html

Some programmer-nerds go beyond introversion.

https://blackliszt.com/2012/03/nerds-autism-deficiency-advantage.html

Many of the best programmers rub "normal" people the wrong way. They're not "team players," They refuse to follow industry-standard norms, etc.On the other hand, what's so great about being "normal?"

https://blackliszt.com/2025/07/whats-so-great-about-being-normal.html

Temple Grandin's views are always valuable.

https://blackliszt.com/2010/02/nerds-and-the-yakkityyaks.html

In the end, the best nerds like to get stuff done — and have fun.

https://blackliszt.com/2011/07/top-nerd-activities-work-hard-save-the-day-and-have-fun.html

https://blackliszt.com/2010/03/nerds-in-norway.html

https://blackliszt.com/2011/07/top-nerd-nerd-values-and-attitudes.html

And of course, normal people can react to a passionate desire to get things done in the best way possible as arrogance.

https://blackliszt.com/2012/02/when-you-call-the-programmer-arrogant-are-you-committing-libel.html

Hiring

Hiring software people is a challenge. There is widespread bad advice that interviewing for a software job should involve lots of chatting and relationship building.

https://blackliszt.com/2017/02/how-to-get-a-software-job.html

There are definitely better ways to select software people than the normal interview process.

https://blackliszt.com/2011/12/interviewing-software-people.html

The CTO of an organization is a special case. The requirements for the job are frequently impossible to meet.

https://blackliszt.com/2014/10/hiring-a-cto-the-impossible-dream.html

While CTO’s should be tops in technology, the best of them understand finances.

https://blackliszt.com/2012/10/cto-cfo-cfbco.html

Managing

Managing software people isn’t easy. Most managers tend to focus on process instead of substance.

https://blackliszt.com/2013/03/process-and-substance-in-software-development.html

https://blackliszt.com/2013/11/people-and-substance-in-software-management.html

Many companies have what's called a software architect. What should that person do?

https://blackliszt.com/2010/04/the-chief-architects-role-in-a-tech-company.html

If instead of just paying attention to abstract process managers paid attention to what the programmers are actually doing, things would improve.

https://blackliszt.com/2010/04/what-are-all-the-programmers-doing.html

One of the reasons software managers tend to focus on process instead of substance is that they have zero experience building software. Worse, it's literally invisible to them.

https://blackliszt.com/2016/12/managing-what-you-cant-see.html

As you can see from Dilbert and coal mining, you do a whole lot better managing something you've done or at least can see.

https://blackliszt.com/2017/04/managing-software-thats-invisible-to-you.html

Non-technical managers naturally want to find a way to feel better about what they're doing. Here are some of the favorite methods.

https://blackliszt.com/2015/06/software-problems.html

All too often, nontechnical managers make important decisions instead of software people.

https://blackliszt.com/2014/01/who-makes-the-software-decisions.html

When non-technical people do all the managing, the programmers who know stuff either leave or learn to keep their mouths shut in the face of stupidity and lie.

https://blackliszt.com/2024/06/well-trained-managers-create-software-failure.html

In other fields, it’s mostly people who DO the highly skilled thing who become managers of it. It should be the same in software.

https://blackliszt.com/2014/10/joe-torre-and-software-development.html

Non-technical managers think that MBA skills are the main thing that's required to be a good manager. Wrong.

https://blackliszt.com/2016/01/software-business-and-business-school.html

Here's how I learned from a course at Harvard Business School that the MBA teaches confident ignorance in terms of software.

https://blackliszt.com/2023/01/software-management-and-business-school.html

So with all this, should a skilled software aspire to be a manager? Dilbert makes an excellent argument.

https://blackliszt.com/2023/02/should-a-software-engineer-aspire-to-become-a-manager.html

It’s easy to focus on relationships instead of technical substance when managing.

https://blackliszt.com/2017/02/software-management-and-relationships.html

There is also the classic case of ego. It's been a big problem in other technology-based fields, and it's a big problem in software.

https://blackliszt.com/2012/07/a-lesson-from-joseph-lister-ego-the-killer-of-software-projects.html

https://blackliszt.com/2012/07/what-can-software-learn-from-steamboats-and-antiseptic-surgery.html

The way group management changes as the group grows can be destructive.

https://blackliszt.com/2011/05/how-great-software-teams-can-go-wrong.html

It’s clear from the pay scales that most companies value management much more highly than the people who write code. Unlike baseball, for example, where the people who play on the field get the highest pay.

https://blackliszt.com/2016/04/software-people-you-get-what-you-pay-for.html

Does having a CS degree lead to the best pay? Not as much as you’d think.

https://blackliszt.com/2015/05/how-much-is-a-computer-science-degree-worth.html

There are many ways otherwise good programmers can go wrong. One big one is the result of perverse incentives.

https://blackliszt.com/2015/04/software-problems-the-role-of-incentives.html

Programmers who are considered to be “stars” often have issues unique to software that hurt group efforts and are rarely recognized.

https://blackliszt.com/2015/04/high-iq-programmers-the-problems.html

On the other hand, the whole process of training and promoting the most talented programmers is broken. It can learn a lot from ballet.

https://blackliszt.com/2013/04/what-can-software-learn-from-ballet.html

Book

The above are highlights of what there is to learn about software people. For more, see my book.

https://blackliszt.com/2015/03/software-people-book-just-published.html

May 17, 2023
Summary: David B Black Books on Software

This is a brief description of the Building Better Software Better series of books. I wrote the books to explain the non-standard methods of producing great software that I first discovered over a couple decades of writing code in various environments, and then observed in small groups of programmers over a couple decades of VC investing. It's the only way to explain how tiny teams can beat industry giants at their own game. It's not about everyone being a so-called 10X programmer — it's about working smarter.

Here is the story of the background of writing the books.

https://blackliszt.com/2015/10/secrets-of-software-super-developers.html

Software Business and Product Strategy

Here is a summary with links to my posts on software innovation and growth strategies.

https://blackliszt.com/2023/07/summary-software-innovation.html

Here are some of the business and product strategies described in the book, including links to the book:

https://blackliszt.com/2016/07/innovation-stories.html

https://blackliszt.com/2016/04/software-business-and-product-strategy-book.html

https://blackliszt.com/2016/05/innovation-from-startup-to-success.html

Wartime Software

Building bridges for civilian use takes years and thousands of people. Building bridges in wartime takes hours – it’s done under enemy fire, and the load and quality requirements are higher. Most software is built the way bridges are in peacetime. Some groups — those who are “under fire” or just don’t have time or money to do it the “right way” — build software using wartime rules – software that is better than peacetime software. They do this not by cutting corners, but by following a different set of rules and methods.

Best to start: https://blackliszt.com/2012/03/bridges-and-software-in-peace-and-war.html

Summary of posts with links: https://blackliszt.com/2023/07/summary-wartime-software-to-win-the-war.html

Book: https://blackliszt.com/2013/05/wartime-software-book-available.html

Project management

The title is the Disease of Software Project Management, because project management has in fact been a disaster for software. The idea that software development should be managed by generic project management techniques is accepted nearly universally — the only disputes are about which of the many minor variations to choose. This book is a deconstruction of project management, its origins and its application to software development. If phrases like "that will probably take us three sprints to get done" spring from your lips without sarcasm, you need to read this book. The core idea is that all forms of project management optimize for expectations, while in wartime you optimize for speed.

Best to start: https://blackliszt.com/2010/10/software-project-management-dates-are-evil.html

https://blackliszt.com/2013/03/software-comparing-waterfall-and-agile.html

Summary of Posts: https://blackliszt.com/2023/04/summary-software-project-management.html

Book: https://blackliszt.com/2012/10/software-project-management-book.html

Software QA

Things are done faster in wartime, but that's only possible if they're done differently. QA is one of the key areas of difference. There are core ideas to understand, and detailed methods that are different, involving different skill sets than are normally involved in QA.

Best to start: https://blackliszt.com/2012/04/a-simple-framework-for-software-quality-assurance.html

Summary of Posts: https://blackliszt.com/2023/04/summary-software-quality-assurance.html

Book: https://blackliszt.com/2012/10/software-quality-assurance-book.html

People

Managers tend to apply the same templates to managing software people that they apply to everyone else. They foolishly think they can manage people who are doing something they don't understand and can't even see. This book covers some of the things that are unique to software people.

Best to start: https://blackliszt.com/2014/10/joe-torre-and-software-development.html

https://blackliszt.com/2014/01/who-makes-the-software-decisions.html

https://blackliszt.com/2013/11/people-and-substance-in-software-management.html

Summary of posts: https://blackliszt.com/2023/05/summary-software-people.html

Book: https://blackliszt.com/2015/03/software-people-book-just-published.html

May 14, 2023
Summary: The Ongoing Problem of Computer Security

This is a summary with links to my posts on computer security.

Computer security is important for the same reason as home security – you don’t want bad guys breaking into your house, wrecking it and taking your stuff! Here are brief summaries of the unending computer security problems and how to solve them.

https://blackliszt.com/2023/04/how-to-prevent-the-leak-of-confidential-documents.html

https://blackliszt.com/2017/05/computer-security-problems-solutions.html

In the invisible world of computers, security is elaborate, expensive and an ongoing nightmare of failure and ineptitude. But computer attacks are largely ignored, while physical security problems often make the news.

https://blackliszt.com/2014/09/cyber-security-and-cyber-insecurity.html

Managers treat managing computer security no differently than anything else. Huge mistake.

https://blackliszt.com/2015/06/systemic-issues-behind-the-cyber-security-disasters-at-opm-citi-anthem-etc.html

https://blackliszt.com/2015/06/what-baseball-can-teach-us-about-opm-anthem-and-other-cyber-thefts.html

Everyone in the physical world realizes that security falls into two entirely distinct categories. First is for places when they’re closed: security is stopping anyone unauthorized from entering. Second is for places when they’re open: keep visitors and employees from stealing things. Computer security focuses almost exclusively on keeping the bad guys out, largely ignoring the cases of visitors and (above all) employees from stealing things.

Here’s the basic idea of applying retail store methods to computer security.

https://blackliszt.com/2015/03/methods-for-effective-cybersecurity.html

Libraries too.

https://blackliszt.com/2017/03/libraries-are-more-secure-than-computers.html

Here are famous examples of the “insider threat” in the physical and software worlds.

https://blackliszt.com/2013/07/edward-snowden-daniel-ellsberg-ineffective-security-then-and-now.html

https://blackliszt.com/2013/07/cyber-security-standards-are-ineffective-against-insiders-like-edward-snowden.html

Experts like to insist on particular solutions to the security problem. Requiring MFA (2FA) is a popular one to demand. There's just one problem: it doesn't improve security.

https://blackliszt.com/2024/12/mfa-wont-fix-computer-security-problems.html

Although largely ignored, there are practical ways to implement insider-threat computer security.

https://blackliszt.com/2017/03/lets-fix-cia-cybersecurity-using-machine-learning.html

The government is terrible at cybersecurity.

https://blackliszt.com/2017/06/government-cyber-security-tops-the-oxymoron-list.html

https://blackliszt.com/2021/07/the-irs-could-have-prevented-the-tax-data-leak.html

And there is general ignorance about basic aspects of it.

https://blackliszt.com/2017/10/the-irs-anti-fraud-contract-with-equifax-is-good.html

Government security experts demonstrate deep ignorance in high visibility cases, confusing outside hacking with typical ignorant-user phishing.

https://blackliszt.com/2017/01/russia-hacks-dnc-podesta-email-fake-news.html

In spite of its own incompetence, the government produces mountains of regulations that companies are required to follow. The regulations largely ignore insider threats and don’t work for outside attackers. Even the famous NSA fails at security.

https://blackliszt.com/2014/05/bureaucracy-regulation-and-computer-security.html

https://blackliszt.com/2017/05/security-regulations-vs-security.html

One of the many reasons the regulatory approach to security fails is that the “experts” are always fighting the last war.

https://blackliszt.com/2014/09/cyber-insecurity-and-the-maginot-line.html

Institutions that are hacked and lose mountains of customer information make a big show of concern, but don’t in fact help their customers.

https://blackliszt.com/2015/02/the-anthem-of-cyber-insecurity.html

https://blackliszt.com/2015/02/my-anthem-account-was-hacked.html

Some of those giants wait way too long and then lie like crazy to their customers.

https://blackliszt.com/2011/04/chase-data-theft-exemplary-handling.html

Smaller tech-oriented companies do the same thing.

https://blackliszt.com/2019/01/computer-security-breach-response-excellence.html

Part of why the data theft parade continues is that the people in charge have no real motivation to make security work.

https://blackliszt.com/2015/03/how-to-achieve-cybersecurity-motivation.html

Outside hacks succeed in part because unwitting employees open the door and let the bad guys in.

https://blackliszt.com/2019/11/cybersecurity-almost-impossible-to-achieve-heres-why.html

https://blackliszt.com/2015/04/internet-drivers-licenses-needed-for-users.html

Ransomware is a new way for hackers to profit from security breaches. Not many of the attacks make the news, but there’s an epidemic of it.

https://blackliszt.com/2021/06/the-colonial-pipeline-cyber-security-disaster-in-context.html

https://blackliszt.com/2022/12/new-york-citys-metropolitan-opera-crippled-cyber-attack.html

Articles about famous ransomware attacks shows the profound ignorance of “experts” on the subject.

https://blackliszt.com/2017/05/the-ransomware-hack-attack-lessons-from-the-experts.html

There are proven ways to protect against and recover from ransomware, which are sadly not widely used.

https://blackliszt.com/2023/05/how-to-protect-and-recover-from-ransomware.html

Computer security is rife with specialized terminology and abstruse concepts. It can be hard to understand. But the core concepts are easily understood when you compare hack attacks to physical things like car dealerships or gated communities.

https://blackliszt.com/2020/02/why-the-equifax-hacking-disaster-wouldnt-happen-at-a-car-dealership.html

https://blackliszt.com/2021/06/gated-communities-help-us-understand-the-ongoing-cyber-security-disaster.html

There is a special security case of law enforcement agencies with a legitimate need to look inside a consumer device. Apple does its best to protect the criminals. Here are the highlights.

https://blackliszt.com/2016/03/the-apple-fbi-fiasco.html

And here are the details:

https://blackliszt.com/2016/03/apple-can-help-fight-crime-while-maintaining-privacy.html

https://blackliszt.com/2016/02/apples-cancer-prevention-strategy.html

https://blackliszt.com/2016/02/apples-approach-to-privacy-terrorists-and-criminals.html

Until the uniform wisdom and practice of the ruling experts changes, computer security disasters will continue unabated in spite of massive spending to conform to the regulations intended to achieve security.

May 9, 2023
Summary: Software Quality Assurance

This is a summary with links to my posts on the optimal method of achieving software quality, and then a sample of the endless quality problems of not using those methods, including in the supposedly smart big tech companies,

Software quality results from the whole process of specifying and building software, not just the formal quality and testing disciplines, as noted here:

https://blackliszt.com/2011/07/software-quality-theory-and-reality.html

There are groups of software people who use proven, decades-old methods to build high quality software quickly and well, methods that violate the regulations and professional methods that dominate the field. Here is an introduction to the winning method of software QA.

https://blackliszt.com/2021/07/the-revolutionary-championchallenger-method-of-software-qa.html

The concept of logical redundancy in addition to physical redundancy is a key aspect of the champion/challenger approach to quality.

https://blackliszt.com/2011/04/single-point-of-failure-logical-vs-physical.html

The typical methods of QA lead to endless work, added to already-long development.

https://blackliszt.com/2015/12/speed-optimized-software-qa-or-cancer.html

Good QA recognizes that one method should be used for getting software to do what you want and a different method for keeping it right, a.k.a. “regression testing.”

https://blackliszt.com/2012/04/a-simple-framework-for-software-quality-assurance.html

A remarkable thing about traditional QA is that it’s all built and run in the lab, while good QA is focused on the only environment users care about, essentially field-testing.

https://blackliszt.com/2012/04/field-tested-software.html

Software Quality Problems

Software has a quality problem. It's big and widespread. It affects nearly all software development efforts.It's not getting better. It's not just the QA process. Here are some highlights of the horrors and why it's so bad.

https://blackliszt.com/2011/06/why-computer-software-is-so-bad.html

Is anybody worried about this? It doesn't appear that way, even though software quality is right up there with "government efficiency" and "northern hospitality."

https://blackliszt.com/2011/06/software-quality-horror-failure-tragedy-and-ineptitude.html

Even simple-sounding things like making a web site work reasonably well for the people who use it is apparently too much to ask of many software departments.

https://blackliszt.com/2011/01/what-do-consumers-want-web-site.html

Given this, you would think that expensive and time-consuming methods that don’t work would have long-since been discarded. But the bad methods are taught in academic Computer Science and broadly supported throughout the industry. The people in charge of quality say they’re doing the best they can – and they are, given the terrible methods they use.

Big companies with all their resources can’t build – or even buy – quality software.

https://blackliszt.com/2015/09/software-quality-at-big-companies-united-hp-and-google.html

https://blackliszt.com/2021/02/why-cant-big-companies-build-or-even-buy-sofware-that-works.html

https://blackliszt.com/2021/03/why-cant-big-companies-build-software-that-works.html

It's not just "core" software; quality problems infect software wherever it's used, including customer service and surveys.

https://blackliszt.com/2016/07/gartner-group-big-company-customer-service.html

https://blackliszt.com/2021/05/anthem-needs-my-feedback-reveals-deep-problems.htm

Big software companies aren't any better.

https://blackliszt.com/2015/08/large-organization-software-fails-the-case-of-microsoft-windows.html

You might think that those cool internet companies would get it right.

https://blackliszt.com/2012/01/internet-software-quality-horror-shows.html

Twitter is a good example of cool-company software failure.

https://blackliszt.com/twitter/

Like Google and FB, they mask errors in their search and sequencing algorithms giving you whatever answers their error-prone software comes up with as facts. All sorts of major organizations have endorsed Twitter results as factual when those results are riddled with error.

https://blackliszt.com/2013/05/the-bogus-basis-of-trending-on-twitter.html

Facebook is as bad as the rest. Here are the details on FB’s mobile app, including what its users have to say.

https://blackliszt.com/2014/11/facebooks-software-quality.html

And generally about quality at FB:

https://blackliszt.com/2017/03/software-giants-image-and-reality-facebook.html

When you look at a financially successful company based on software, it’s natural to think that they must know how to build great software. Facebook grew to its strong position in the market in spite of its poor software quality. Here’s an explanation of FB’s path to success, which shows all FB had to do was not be drastically worse than their competitors in terms of quality.

https://blackliszt.com/2014/12/fb.html

Here’s an example of a government software QA disaster with serious human consequences.

https://blackliszt.com/2011/07/software-quality-horror-tales-electronic-diversity-visas.html

Naturally, since government is so great at software, sometimes it intervenes to "make things better."

https://blackliszt.com/2015/04/the-government-wants-to-help-ubers-software-quality.html

Bad software QA goes way beyond screwing things up; it can kill people.

https://blackliszt.com/2012/03/why-software-quality-stinks.html

There is absolutely no excuse for this, when proven methods are available.

Here’s an overview of the book on the subject.

https://blackliszt.com/2012/10/software-quality-assurance-book.html

April 28, 2023