The Black Liszt

Category: People

  • Summary: Software People and Management

    Guess what – software people aren’t like regular people! There are differences in the ways they think and act, judging results can be difficult, and hiring and managing is challenging. This is a summary of my writing on the subject.

    Skills, Training and Status

    The education establishment has taken over the world of skilled jobs. They've convinced the world that getting a degree in Computer Science is the only way to learn how to be a computer programmer. They are wrong.

    https://blackliszt.com/2025/01/how-to-learn-to-be-a-computer-programmer.html

    When you get a degree in CS or learn a programming language, you’re all set, right? Actually, those things are pretty much the same as knowing how to hammer a nail when building a house. It’s a nice start, but there are levels and levels of skill beyond the basics.

    https://blackliszt.com/2018/09/the-hierarchy-of-software-skills.html

    There is perennial conflict between young and old in software, perhaps more than other fields.

    https://blackliszt.com/2010/12/experience-and-youth.html

    You might think that getting a degree in Computer Science prepares you to be an effective programmer, much like getting an MD prepares you to be an effective doctor. A comparison of the two makes it clear that CS education is sadly deficient.

    https://blackliszt.com/2020/07/job-requirements-for-software-engineers-should-stop-requiring-cs.html

    Does status in software correlate with your skills, kind of like how hitting more home runs gives you status in baseball? Sadly, no. One aspect of software status is that the more layers there are between you and real customers, the more status you have.

    https://blackliszt.com/2011/10/status-in-software-silliness-and-stupidity.html

    There are other dimensions, for example the more management layers between you and fingers-on-keyboard programming, the greater the pay and status. And there are others!

    https://blackliszt.com/2018/10/the-hierarchy-of-software-status.html

    There is a similar skills and status hierarchy in data science

    https://blackliszt.com/2019/03/the-hierarchy-of-software-skills-data-science.html

    One of the sad things about software is that writing great code and getting credit for it are largely unrelated to each other.

    https://blackliszt.com/2020/10/elizebeth-smith-friedman-the-cancelled-heroine-of-cryptography.html

    Programming skill and power in the programmer is far more important than the differences between programming environments.

    https://blackliszt.com/2010/05/what-is-the-best-programming-environment.html

    At least in some environments, high-performing nerds are highly valued.

    https://blackliszt.com/2011/07/the-new-top-gun-is-top-nerd.html

    Understanding Programmers

    When you think about hard-core programmers, the word "nerd" comes to mind. In the poplar imagination, "nerd" often seems to mean guys who play lots of games. And have attitude. Of course there are such people. But the very best programmers, in my experience, don't match that description.

    https://blackliszt.com/2012/02/most-nerds-are-introverts.html

    Some programmer-nerds go beyond introversion.

    https://blackliszt.com/2012/03/nerds-autism-deficiency-advantage.html

    Many of the best programmers rub "normal" people the wrong way. They're not "team players," They refuse to follow industry-standard norms, etc.On the other hand, what's so great about being "normal?"

    https://blackliszt.com/2025/07/whats-so-great-about-being-normal.html

    Temple Grandin's views are always valuable.

    https://blackliszt.com/2010/02/nerds-and-the-yakkityyaks.html

    In the end, the best nerds like to get stuff done — and have fun.

    https://blackliszt.com/2011/07/top-nerd-activities-work-hard-save-the-day-and-have-fun.html

    https://blackliszt.com/2010/03/nerds-in-norway.html

    https://blackliszt.com/2011/07/top-nerd-nerd-values-and-attitudes.html

    And of course, normal people can react to a passionate desire to get things done in the best way possible as arrogance.

    https://blackliszt.com/2012/02/when-you-call-the-programmer-arrogant-are-you-committing-libel.html

    Hiring

    Hiring software people is a challenge. There is widespread bad advice that interviewing for a software job should involve lots of chatting and relationship building.

    https://blackliszt.com/2017/02/how-to-get-a-software-job.html

    There are definitely better ways to select software people than the normal interview process.

    https://blackliszt.com/2011/12/interviewing-software-people.html

    The CTO of an organization is a special case. The requirements for the job are frequently impossible to meet.

    https://blackliszt.com/2014/10/hiring-a-cto-the-impossible-dream.html

    While CTO’s should be tops in technology, the best of them understand finances.

    https://blackliszt.com/2012/10/cto-cfo-cfbco.html

    Managing

    Managing software people isn’t easy. Most managers tend to focus on process instead of substance.

    https://blackliszt.com/2013/03/process-and-substance-in-software-development.html

    https://blackliszt.com/2013/11/people-and-substance-in-software-management.html

    Many companies have what's called a software architect. What should that person do?

    https://blackliszt.com/2010/04/the-chief-architects-role-in-a-tech-company.html

    If instead of just paying attention to abstract process managers paid attention to what the programmers are actually doing, things would improve.

    https://blackliszt.com/2010/04/what-are-all-the-programmers-doing.html

    One of the reasons software managers tend to focus on process instead of substance is that they have zero experience building software. Worse, it's literally invisible to them.

    https://blackliszt.com/2016/12/managing-what-you-cant-see.html

    As you can see from Dilbert and coal mining, you do a whole lot better managing something you've done or at least can see.

    https://blackliszt.com/2017/04/managing-software-thats-invisible-to-you.html

    Non-technical managers naturally want to find a way to feel better about what they're doing. Here are some of the favorite methods.

    https://blackliszt.com/2015/06/software-problems.html

    All too often, nontechnical managers make important decisions instead of software people.

    https://blackliszt.com/2014/01/who-makes-the-software-decisions.html

    When non-technical people do all the managing, the programmers who know stuff either leave or learn to keep their mouths shut in the face of stupidity and lie.

    https://blackliszt.com/2024/06/well-trained-managers-create-software-failure.html

    In other fields, it’s mostly people who DO the highly skilled thing who become managers of it. It should be the same in software.

    https://blackliszt.com/2014/10/joe-torre-and-software-development.html

    Non-technical managers think that MBA skills are the main thing that's required to be a good manager. Wrong.

    https://blackliszt.com/2016/01/software-business-and-business-school.html

    Here's how I learned from a course at Harvard Business School that the MBA teaches confident ignorance in terms of software.

    https://blackliszt.com/2023/01/software-management-and-business-school.html

    So with all this, should a skilled software aspire to be a manager? Dilbert makes an excellent argument.

    https://blackliszt.com/2023/02/should-a-software-engineer-aspire-to-become-a-manager.html

    It’s easy to focus on relationships instead of technical substance when managing.

    https://blackliszt.com/2017/02/software-management-and-relationships.html

    There is also the classic case of ego. It's been a big problem in other technology-based fields, and it's a big problem in software.

    https://blackliszt.com/2012/07/a-lesson-from-joseph-lister-ego-the-killer-of-software-projects.html

    https://blackliszt.com/2012/07/what-can-software-learn-from-steamboats-and-antiseptic-surgery.html

    The way group management changes as the group grows can be destructive.

    https://blackliszt.com/2011/05/how-great-software-teams-can-go-wrong.html

    It’s clear from the pay scales that most companies value management much more highly than the people who write code. Unlike baseball, for example, where the people who play on the field get the highest pay.

    https://blackliszt.com/2016/04/software-people-you-get-what-you-pay-for.html

    Does having a CS degree lead to the best pay? Not as much as you’d think.

    https://blackliszt.com/2015/05/how-much-is-a-computer-science-degree-worth.html

    There are many ways otherwise good programmers can go wrong. One big one is the result of perverse incentives.

    https://blackliszt.com/2015/04/software-problems-the-role-of-incentives.html

    Programmers who are considered to be “stars” often have issues unique to software that hurt group efforts and are rarely recognized.

    https://blackliszt.com/2015/04/high-iq-programmers-the-problems.html

    On the other hand, the whole process of training and promoting the most talented programmers is broken. It can learn a lot from ballet.

    https://blackliszt.com/2013/04/what-can-software-learn-from-ballet.html

    Book

    The above are highlights of what there is to learn about software people. For more, see my book.

    https://blackliszt.com/2015/03/software-people-book-just-published.html

     

  • What’s So Great about being Normal?

    What's so good about people who are "normal?"

    I'm a software guy. As in hard-core programming for decades. Many of the great programmers I've known aren’t "normal," Managers usually don’t like them. They don’t “fit in.” They’re not “team players.” And so forth. Society puts labels on such people, and thinks they need “treatment” to “fix” them. Would such programmers have been "better adjusted" had they been diagnosed early in life and given all sorts of therapy?

     https://blackliszt.com/2012/03/nerds-autism-deficiency-advantage.html

    Great programmers aren’t perfect, of course. But then who is? If you want great software, you should want great programmers. You should also want software development methods that fit their preferred way of working, instead of the usual MBA-style project management that is dictated, resulting in the near-universal disaster of software development.

    The way things work, all the criticism flows from the world of the “normal” to the small group of outstanding programmers. It’s long overdue to turn the tables. As a start on that project, here are some of the highlights of how many "normal" people act:

    Loads of "Normal" people:

    lie a great deal.

    rarely say what they really think;

    sow misinformation and confusion constantly;

    can't remember much or focus on anything for long;

    are hopelessly vague and consumed with generalities;

    are incapable of paying attention to details, or remembering them;

    constantly try to trick others into doing things they wouldn't otherwise do, or thinking things they wouldn't normally think;

    are emotionally cruel and bullying;

    form cliques and feel superior, while trying their best to make others feel inferior;

    become masters of incredibly complex, ever-changing games involving interpersonal relationships;

    use their ability to manipulate others to advance themselves;

    associate success at this game with personal superiority;

    pride themselves at shaping their opinions and behaviors to the situation, thinking of this as being adaptable and flexible, instead of untrustworthy and without moral fiber.

    This list might strike some people as being a description of people who are psychopaths or sociopaths. I wouldn't disagree. The elites of academia, business and society are heavily populated by the highest-performing “normal” people in terms of the list above.

    Isn’t “normal” wonderful? Normal people run the world. Do you think it’s possible that many of the things that are screwed up about the world have something to do with the domination of “normalcy?”

    Hey, "normal" people — if you actually want what you say you want, which is software that works and does what it's supposed to do, produced quickly, then maybe you should think about getting software people who are good at producing that and let them have at it in the way they think best. 

  • How to Learn to be a Computer Programmer

    Lots of people will tell you how to become a computer programmer. Go to college and major in Computer Science is the most wide-spread advice. In the last few years things called “coding academies" have emerged as quicker, more affordable paths to that goal. There is a faster, better way to acquire the skill of computer programming that is rarely discussed. While this post is about computer programming, the educational concepts discussed are broadly applicable.

    How Skills are Acquired

    Stepping back from programming, let’s think about how skills are acquired. Physical skills like playing various sports is a good place to start. How much classroom time is required? How about text books? The basic way you learn sports or other physical skills is watching someone play them. With any sport involving a ball, you pick one up and try to throw it. Then catch it when thrown to you. Someone may give advice, but basically you try a lot, gradually learn from your mistakes and get better. It’s important to note that you see the results of your effort. You see if the ball went where you intended it to go, for example. This can continue for years.

    How about a more intellectual skill? We all learn how to talk and listen with understanding. The way we learn is similar to learning a sport – by watching, listening and then emulating. The world is incredibly complex and varied, with words associated with a huge number of things and actions. You start with a few words, and spend years adding many thousands more to them. Yes, parents give lots of feedback – not unlike the feedback you get when you see whether the ball you threw went where you wanted it to, whether the receiver caught it. Did you catch what I was trying to say there? Yup, like that.

    The language of software

    By contrast to any human language, a computer language is amazingly simple – partly because the “world” in which it “lives” is incredibly narrow and abstract. The nouns are all data of a couple different types, basically numbers and letters. You give each place that can hold data a name. The verbs are just a handful of simple actions that just grab the data that’s in a named place on your “desk,” do something with it, and put the result back or into another named place.

    A great deal of the power of software comes from what’s called the subroutine library, which is basically an attentive gang of little robots waiting to do your bidding. If you want to write something on a screen, you “call” the relevant robot, give it your instructions, and send it off. There are robot librarians and file clerks that are excellent with card catalogs and endless file cabinets, fetching what you ask for and putting away what you’re done with. There are robot butlers at each “door” (keyboard, mouse, etc.) that politely inform you when something has arrived and give you the package. Another butler will also send data where you want it to go. While central to programming is learning how to “talk,” learning about the team of robots at your disposal and what each can do for you is also important.

    Yes, there are lots of different software languages. But they’re really like local dialects. They all conform to the basic description I just gave. The gang of robots available in a given language typically vary depending on what the language is mostly used for. There are business-oriented languages that pay special attention to things like financial data, account data collections like accounts, and others that have robots that are really good at fancy math. But the way of thinking about them and writing programs is remarkably similar.

    This should make you wonder exactly why you need years of courses “taught” by fancy professors, when programming is like doing things in an amazingly simple world of data using a small set of instructions that are easy to visualize and understand. The answer is that you can do it on your own, given access to a computer, some widely available tools and a bunch of organized exercises that start super-simple and get gradually more elaborate. That’s how I learned!

    How I learned to program in ancient times

    I went to high school at the newly-opened Morris Knolls HS in Rockaway, NJ. It took students from a couple of towns and had courses that were explicitly college and non-college. There were vocational tracks such as HVAC, auto repair and secretarial. The towns weren’t poor but definitely weren’t elite, with few professionals.

    My introduction to programming and computers was a course offered in my junior year of high school, the academic year 1966-7. One of the high school teachers wanted to move from teaching to software and had no training or other way to get there. He somehow persuaded the administration to let him teach a course on programming. His break-through was arranging for the class to have computer time at a nearby company, Reaction Motors.

    The teacher found a FORTRAN textbook to serve as the basis of the class. While he was the “teacher,” it was clear that his teaching amounted to trying to be a couple chapters ahead of the class in the book. But it didn’t matter. The important thing was having a book that laid things out, along with exercises and answers. And above all, the Saturday time slot when we had access to the computer.

    Programming involved writing the program on paper, and then keying the program onto cards using one of the punch card machines that were in a room adjacent to the main machine room. We got to operate the room-sized computer, something which in the normal corporate environment was an absolute no-no, as I later learned. It was terrific fun.

    I got lucky the summer after my senior year, and got a job at a local company, EMSI, Esso Mathematics and Systems Inc. The company applied computers and math to a wide variety of issues running the oil company. I started by fixing bugs and making changes to their math optimization FORTRAN program that helped them run their oil refineries better. Did I “know” how to do this? In each case I dove in and figured it out.

    Then they needed some test programs written a new language, PL/1. No one there knew the language, so I dove into the manual and wrote the programs. Then they needed some accounting programs to help run their facilities in Asia. They wanted them written in COBOL, which made sense for financial programs, so I learned the language as I went along. I quickly fell into the pattern of getting some data from my boss, writing code that did what he wanted with it, showing him the output, and cycling for more. Each cycle required less self-education in COBOL, so I got faster. I tell more of this story here.

    What I did wasn’t that unusual at the time. Most of the early programmers basically figured things out as they went along. They needed to get stuff done. They studied programs written by other people and learned how to write their own. Compared to learning to read and write the German language, I found FORTRAN to be amazingly simple by comparison; in fact, it was a relief compared to the endless complications and words of any human language, not to mention idioms.

    How you can learn in modern times

    You won’t have to deal with card and card punch machines. You won’t have to go to a place with a room-sized computer. You have the internet to give you access to all the information about any language you want, and tools you can access to enable you to write code in that language, try to run it and see how it goes. To start you can use formal exercises with programmed solutions you can study. You can learn and advance at your own pace.

    If you want, you can poke around for a local or remote job. It’s good to start with some narrow domain of problems and tools that you’re interested in and have gained skill in. The first job doesn't even have to be programming — it could be testing, for example. Just get in somewhere and start producing. Once you prove value, you’ll move up from there, because people who can produce results aren’t that easy to find, and a proven producer is a keeper. The important thing is to keep learning and move up the skills hierarchy.

    Finally, count your blessings that the ridiculous world of job certification hasn’t yet latched its evil claws into programming very much. Yes, there are ignorant HR people who insist applicants have a degree in Computer Science, but the best programmers with degrees are good in spite of their mis-education in college.

  • Elizebeth Smith Friedman: The Cancelled Heroine of Cryptography

    The unheralded Elizebeth Smith Friedman is a textbook example of the vast gulf that too often separates achievement in a field from getting credit for the achievement.

    Eliz pic

    She was a true pioneer of cryptography and code-breaking, leading multiple efforts against the international criminal mob and the Axis in World War II. Unlike most people called “leaders,” she was actually the best at what she did, personally cracking “uncrackable” codes and personally pioneering new methods. She was a leader in the true sense: the manager/boss of the long distance runners AND the runner far in front of everyone else who gets there first AND helps all her fellow runners speed up.

    Technical History Issues

    Making an advance in technology is hard. Not many people try to do it, and a tiny fraction of those who try seem to succeed. Many of those apparent successes burn out – they weren’t advances after all. Sometimes a true advance, for various reasons, is never adopted. When it is adopted, there is often a race to claim credit for the advance. The race isn’t so much a race as it is a no-holds-barred war. To win the war, you usually need the support of loads of people who have no idea what the advance is about. These ignorant people create history, along with its winners and ignored achievers.

    Is this cynical? Yes. Is it an accurate description of what happens? In all too many cases, sadly yes. Here are examples of from the war for credit for inventing the computer.

    In most cases, technical invention isn’t like a giant comet streaking to earth and creating a big boom. It’s more like a sequence of parallel, overlapping efforts to solve a problem or make something better. Often an advance is made by more than one person or group without involvement with the other. What in retrospect is described as the big advance is usually a step forward, one of many, building on earlier work. Sometimes the advance isn’t an advance so much as a commercialization. Matt Ridley describes this with many examples in his mostly excellent book on Innovation. Elizebeth stands out as being a true innovator on multiple dimensions.

    Elizebeth Smith Friedman

    Getting to the truth about inventors and technology innovation is a problem in general. In the case of Ms. Friedman, the problem was made worse by the credit-taking actions of government leaders. The truth has only emerged recently with the release of previously concealed documents and the ending of secrecy periods.

    Here are some highlights of her career:

    In the 1930s, Elizebeth Smith Friedman became America’s and indeed the world’s best-known codebreaker. She inflicted severe damage on the interests of organized crime and at times needed to be protected by bodyguards. The evidence she gave in criminal trials describing how she cracked encrypted messages passing between mobsters made her a newspaper sensation.

    Later, during World War 2, she broke coded messages sent on Germany’s Enigma machines. These messages revealed a plot by the Argentinian government to help Germany replace South American governments with Nazis, giving Germany bases from which to attack America. Her discoveries allowed the western allies to thwart the Argentinian and German plans.

    Elizebeth Smith Friedman’s wartime codebreaking work was so secret that she was forbidden to mention it in public. She died many years before government archives were brought to light showing what she had done. During and after World War 2, J. Edgar Hoover and the FBI took the credit for work Elizebeth and her U.S. Coast Guard team had carried out.

    Her whole story is fascinating. Among other things she is a wonderful example of the power of bureaucracies (education, government and corporate) to control and often suppress outstanding talent, and how sometimes, when the bureaucracy is desperate for results, it will break its own rules to achieve a goal – and then claim credit. It is particularly striking in Elizebeth’s case because she was NOT trained in math or STEM of any kind; her fascination was with literature and philosophy.

    There is a book about her life which is includes previously classified and/or ignored documents about her career:

    Woman smash cover

    If you’re at all interested in people like Alan Turing and Grace Hopper and/or computing history, it’s worth reading. She was completely “unqualified” to do what she did – and she became the best at it, for example cracking the German Enigma machine without the huge staff and machines at Bletchley Park that have become famous.

    Likewise, I had never heard of her husband William Friedman, who was also accomplished as a code-breaker both by himself and working with Elizebeth. Here are a couple links that give some highlights, though I still recommend the book.

  • Job requirements for software engineers should stop requiring CS

    I've read thousands of job requirements for computer programmers over the years, and written or edited quite a number. I’ve interacted with hundreds of software groups and seen the results of their work. I’ve spent a couple decades cranking out code myself in multiple domains. There are a couple near-universal problems with job requirements that, if changed, would improve the quality of software groups and their productivity.

    Of course, it’s not just the job requirements and what the hiring people do – it’s also the managers, from the CEO down. They also have to not just support but champion the changes I describe. If they do, everyone will enjoy better results from the software team.

    In this post, I'm going to concentrate on just one of the issues: academic degrees

    A near-universal job requirement is an academic CS degree. When analytics, ML or AI is involved, the requirement is often “upgraded” to a Master’s or PhD.

    There are many capable programmers who have degrees of this kind. Often it doesn’t seem to hurt or hold them back much. But all too often it does! The more specialized the training, for example in project management or quality, the more likely the “education” the person has received is an active impediment to getting good work done.

    Here’s the simple, raw, brutal fact: Getting a degree in Computer Science does NOT train you to become an effective programmer in the real world. All too often, the degree results in the person performing worse than someone with self-training and apprentice experience.

    That is the fact. Surprised?

    Let’s do a little compare and contrast with medicine. Yes, I know that an MD is a graduate degree, while CS is often undergrad. Medical training has evolved to what it is now after decades and centuries of finding out what it really takes to help make people healthy. By contrast, CS degrees just started being granted 50 years or so ago, and are far from even trying to figure out what kind of training helps create good programmers.

    First let’s look at the training and testing:

    • You don’t even get into med school without taking the MCAT, a challenging test that takes over 7 hours that few do well on.
    • Once you’re in med school you take a four year course to get your MD.
      • The first two years are academic, including hands-on labs. Then you take the USMLE-1. If you don’t pass this challenging test you’re out. End of med school.
      • The second two years are clinical! You’re in hospitals, clinics and offices seeing patients under supervision. And you’re graded. And then you take the USMLE-2, which is harder than part 1 and has lots of clinical stuff. If you fail, you’re not an MD.
    • To practice, even as a general practitioner, you have to apply and be accepted into a Residency. Depending on specialty, this can be 3-7 years of mostly hands-on practice, under close supervision.
      • During your first year, you have to take and pass the USMLE-3. Fail and you’re out.
      • During your last year you have to take and pass the test specific to your specialty. Fail and you’re out.

    Here’s the equivalent of the training and testing in CS:

    • There is NO equivalent in CS. No entry testing. No exit testing. Just grades on courses determined by professors who usually pass everyone.

    A little compare and contrast between medicine and CS:

    • Medicine is taught by doctors who practice medicine
      • CS is taught by professors, most of whom have never practiced programming in the real world.
    • A large part of medical training is working with real patients with real problems, under the supervision of practicing doctors.
      • CS is primarily classroom teaching with textbooks and homework exercises. You have to write programs as exercises, but it’s completely artificial. There is nothing apprentice-like or truly clinical about it.
    • Medical training is led by doctors who are incented to produce great doctors.
      • CS training is led by academic PhD’s with no real-world experience who are incented to publish papers read by people like them.
    • Medical journals publish essential information for practicing doctors, giving advances and new discoveries.
      • CS journals are read by the tiny group of academics who publish in them. Practicing programmers pay no attention for good reason.
    • Bad doctors are fired for incompetence and barred from practicing.
      • CS graduates are rarely fired for incompetence. If CS graduates can’t program well, they usually shift into using their non-skills in “management.”
    • In medicine, best practices are increasingly codified. You rapidly fall under scrutiny for deviating.
      • CS grads seek out and follow fashions that are the software equivalent of blood-letting, enthusiastically promoting them and getting them adopted with disastrous results.
    • Hospitals are compared with each other in terms of results. It’s not hard to find which are the best hospitals.
      • Groups of CS grads make it impossible to make comparisons between groups, with the result that huge groups produce major disasters at great expense, while tiny groups of effective programmers perform 10X or more better.

    All this doesn’t make things uniformly wonderful in medicine. But it goes a long way towards explaining why software is so bad. It’s awful. The awfulness is so widespread that it’s rarely talked about!. If bridges fell down at 1/100 the rate that software projects fail, there would be a revolt! Instead, everyone in the industry just sighs and says that’s the way things are.

    You think things are great in software? Check out a couple of these:

    https://blackliszt.com/2015/09/software-quality-at-big-companies-united-hp-and-google.html

    https://blackliszt.com/2014/12/fb.html

    https://blackliszt.com/2014/02/lessons-for-software-from-the-history-of-scurvy.html

    The fact is, CS training leads to horrible results because Computer “Science” is roughly at the same level as medicine was when bleeding patients was the rule. See this:

    https://blackliszt.com/2019/11/computer-science-is-propaganda-and-computer-engineering-is-a-distant-goal.html

    Conclusion

    There are lots of things you can do to improve the results of hiring software programmers and managers. Here's how the usual interview process goes; here is specific advice about interviewing. There is a whole pile of advice in my book on software people. If all you did was drop the CS degree requirement, you would have taken a big step forward in quality improvement.

  • The Hierarchy of Software Skills and Status in Data Science

    There is a striking hierarchy of skills in software, as I've explained here. When you dive into any particular aspect of software, you usually find that it's got a hierarchy all its own. Data science is a subject of intense interest these days, so in this post I'll explain some of the basics of the data science skills hierarchy.

    A skills hierarchy is very much an insider's game. What most people care about is status. I talk about the basics of software status here. Remember, the skills hierarchy is a whole world away from the status hierarchy that most people care about. Don't confuse the two!

    Data Science skills

    First and foremost, it's important to understand the incredibly broad range of subjects covered  by the term "data science." I attempt to explain the basics of the range in this blog post. You can be just amazing in one of those subjects, while being a neophyte in one that the outside world may consider to be "related," but which in practice is not just down the hall, but in a different building on a different campus.The general understanding of this range is SO pathetic that "data science" is typically managed as a completely independent, free-standing group. Which makes about as much sense as believing that a sous-chef belongs in something that isn't a kitchen. Or that everything that everyone does in a kitchen is basically the same thing.

    Here's one cut at the hierarchy in data science, starting from the base:

    1. Tool users.These are people who have learned how to use some software tool, or maybe a couple. Most "data scientists" fall into this category.
      • They don’t understand how the tools are built or any of the underlying software
      • They may know their tool, but aren't real clear on what the other tools are about, much less when you might consider using one.
    2. Some have broader knowledge of tools
    3. Very few have the sophistication to understand real data analysis, per my series of AI/ML; see this and the links in it: https://blackliszt.com/2018/04/getting-results-from-ml-and-ai-3-closed-loop.html
    4. Of those, even fewer can understand the underlying algorithms and follow the latest research literature
    5. Of those, even fewer can make real algorithmic advances and implement them as tools and deliver practical value.
    6. Of anyone who can do all of the above, it is rare to meet anyone who can address and solve deep tool-level problems in other domains needed to make their code practical in the real world.
    7. Finally, it is extremely rare to meet people who, in addition to their deep and broad prowess in data science and relevant skills needed to make it real-world practical, have similarly deep skills in an associated domain needed to make the data science fully effective for a business.

    The issues of this hierarchy are compounded by the usual over-selling by people who are good promoters but little else, and corporate/government big-wigs who don't want to be bothered with details, but are keen to be seen as "doing something" on such a high-visibility topic. Getting results that make a difference is pretty low on the typical priority list.

    And then of course there are the "data scientists" themselves, who most often are sincere people who are trying to do a good job as they've been taught to do it — mostly by professors and others who have no idea what real-world success looks like, much less how to bring it about.

    Finally, there is the usual "manage something that's invisible to you" phenomenon I have often discussed in this blog, which leads to so much dysfunction and so many wonderful Dilbert cartoons.

    Conclusion

    People talk as though "data science" were a thing, with the usual kind of hierarchy based on level of management and/or "experience." Those typical patterns of hierarchy just don't cut it for understanding what's going on in data science, just like they don't cut it for understanding software development. We will continue to see waste and dead-end efforts until we at least make a start at making our understanding of data science more sophisticated, and aligned with the facts on the ground.

  • The Hierarchy of Software Status

    You might think that the hierarchy of status in software closely tracks the hierarchy of software skills, which I explained here. Hah! It is true that there is a small but important subset of people whose internal sense of status tracks the skills hierarchy reasonably well. But not for most programmers. The reason is simple: non-programmers' assessment of software status is unrelated to skill! It's a whole different hierarchy that has nothing to do with the ability to conceive and write good code that works!

    Here is an earlier attempt to explain this issue.

    This is NOT just an "academic" subject, BTW. It is absolutely crucial to getting good software done. See this for another angle at addressing the same subject.

    Status

    Status is part of human existence. There has always been a status hierarchy. Not long ago, we had lords and peasants in which the status differences were overt:

    MedievalLordAndPeasant

    Status itself has barely changed since those days, though the clothing and other means of expressing it has changed a great deal.

    Status is expressed in clothing, language, money, power relations, human interactions and in much of what we do. It would be shocking if generic human status relations did not apply to software. What's interesting is the translations that are made, which track medieval lord-and-peasant relationships quite well.

    Status in Software

    The easiest way to explain how status works in software is to look at how "close" you are (along multiple dimensions) to real people using code today. The closer you are to actual code and/or people using code, the lower your status; the farther away, the higher your status.

    • One dimension is time. The highest status of all is enjoyed by people who think great thoughts about how some software might be built by some undefined group at some undefined future.
    • One dimension is management layer. In any group, the "lowest" person in terms of management levels has the lowest status, the manager of the worker's manager has twice-higher status.
    • One dimension is closeness to a real user. Anyone working in any way on code that isn't yet used by anyone has greater status than anyone working on code that is in actual use.
    • One dimension is tied to the flow of interactions with users. If you work in any way on the flow of code or changes to code that is on its way to users (i.e., to production), you have higher status than anyone working in any way on the flow of information and communications that flow from users to the supplier or deliverer of the software. In other words, building software is higher status than customer service.
    • One dimension is trendiness. It's really important to identify yourself with tech trends that are exploding in fashionable talk.
      • If you're too early, you lose status. When someone hears you babble on about something, if they check it out on conferences, publications and the web and it's obscure, you seriously lose status.
      • If you're too late, you lose status — you're pegged as someone who's part of the crowd. Better late than never, of course. You also lose status if you keep bringing up something that's starting to fade away.
      • Maximum status is early-peak attention.
      • You can't actually know anything or do anything in the fashionable area and have any chance of maintaining status. You've got to direct, hire, establish, prioritize or strategize. Those all keep your hands safely clean and enhance status.
    • A dimension that becomes crucially important during the hiring process, and which often persists long after the hiring date, is the visibility and perceived success of the person's prior company. The more visible and successful, the higher the status.
      • No one knows whether the person had positive impact on the company's success or the opposite. The person from Google has a halo, regardless of what they did.
      • This is a subject I treat in considerable detail in the Software People book, along with other important hiring considerations.

    Naturally, there are nuances.

    • For the people who deal with the flow of data back from users:
      • There are people who respond to customer complaints and problems. Super-low status.
        • Some of those customer issues are real bugs in the software! The very lowest status goes to the people whose job is to patiently educate the stupid users whose inability to do the simplest things with such a wonderful application is obvious to everyone except themselves. The people who get called in to see if there really might be a bug have higher status.
        • The people who fix such bugs have even higher status, but still remarkably low.
      • There are people who analyze conversion rates and details of customer use of the application.
        • All these have higher status than anyone involved with customer complaints or bugs.
        • If they deal with things like conversion rates, that's related to high-status strategy and marketing and therefore higher status than anyone whose job is to tweak the application to make it better for users.

    There's another important aspect of status. I suppose I could jam this into the "dimensions" list, but I think it's simpler to say this other important thing about status this way: the more code you write, the lower your status. Writing no code is, with some important exceptions and subject to the general dimensions above, higher status than writing any amount of code.

    Yes, writing code requires highly specialized knowledge. But so do lots of other low-status things, like repairing garage door openers or installing lawn sprinkler systems. The people who manage the repair people are, of course, higher status than the people who get dirty or use their muscles doing physical work. Similarly, the people who metaphorically dirty their minds while doing demanding, hard mental work are lower status than those who direct them in any way.

    Another way of thinking about this is asking, what's your main focus? To the extent that it's a body of code, your status is lower. To the extent that it's people, your status is higher, with some exceptions, see customer service above.

    Like anything else, there are exceptions. I have met highly productive coders who are older and well-regarded. But they're rare, and usually they're tucked away in a corner if they're tolerated at all.

    Conclusion

    The way status is perceived in software is perverse, and accounts for a great deal of the dysfunction we see in the software world. It is sad. Even sadder is that, with all the fashion trends that sweep through the tech world, none of them addresses this issue. People with high software skills often perceive this and shake their heads, knowing there is nothing they can do about it.

    There are, of course, tiny groups of people where status is conferred as it should be: to the people who can conceive and then build their way to victory, rapidly and effectively. It's from those warriors of the abstract world that I learned the principles I discuss in my various books related to wartime software. There is more information and context on this and related subjects in my book on Software People.

  • The Hierarchy of Software Skills

    Outsiders generally have no knowledge of the hierarchy that exists among programmers in terms of skills. While everyone can understand that a programmer has had a job in a particular industry or company, and everyone can understand what place they have in the management hierarchy, the place that a programmer has in the skills hierarchy is invisible to most outsiders; moreover, it's not something they're interested in.

    I've talked before about the difficulties of managing programmers, particularly when you're not a programmer yourself. I've talked about how strange this is, particularly in comparison to other fields like sports, music and journalism. My book on Software People addresses this subject. It turns out that understanding the skills hierarchy is important not just for understanding the differences among programmers, it has HUGE impact on innovation and the success of entrepreneurial companies.

    The usual view

    Most outsiders know there are software people, programmers. They know there are professors and that you can get degrees in Computer Science. They know that experience with software in banking is different than software in healthcare, but they're not sure how. They know there are levels of management. Many people even know some of the common buzzwords referring to languages and other things. But then the knowledge, such as it is, dribbles to nothing, other than things that apply to everyone: is s/he a "good" programmer? A team player? Etc.

    When new software fashions emerge, many people know the basic buzzwords. Everyone at this point knows about AI, and even Machine Learning. Terms like Big Data are fading in frequency now, while Data Science is on the rise. Self-respecting managers want to be on top of things, and want to assure their organizations are investing in certified AI experts, and leading-edge Data Scientists — otherwise, we'll lose out!

    Sadly, most of these decision-makers don't bother to learn the first thing about what they're spending money on. Or if they do, it's just details that sound impressive, but don't matter. This explains in part the billions of dollars wasted on software fashion, while true innovators working far from the fashion runways make real-life, practical advances. I've illustrated this phenomenon with AI in healthcare.

    The hierarchy

    The skills hierarchy in software is generally not fashion-driven, though programmers can get caught up in fancy trends like anyone else. So I don't say that programmers are immune to fashion, but that the skills hierarchy itself changes very slowly.

    A book could be written on this subject — it deserves a book, both about the hierarchy itself, its evolution and how it impacts everyone involved. Actually, a chapter in a book has already been written about it, in this book. This is a long way of saying … this blog post is just a brief introduction to an important subject that is broad and deep and consequential, but sadly largely ignored.

    The first thing there is to understand is the incredible depth and breadth of highly specialized knowledge that is required to get anything meaningful done in the world of software. The start of the chapter may give you a flavor that we're talking about WAY more here than "learning a language."

    A tremendous amount of specialized knowledge is required to work quickly and well in practical software environments. The knowledge begins with fluency in one of the many extensive, exacting and demanding computer languages. The language itself, however demanding it may be, is just the beginning. The associated “library” (depending on the times, it could be called the run-time library, the foundation class library, or various other things) has to be mastered as well, because it provides the functions that actually get a good deal of your work done. Just by comparing the bulk of documentation, you can easily deduce that the library is as large as the language itself. You can think of the library as being the set of idiomatic expressions, like “raining cats and dogs” that extend any natural language. Depending on the situation, you may need to know two or more complete languages and associated libraries in order to do anything. For example, if you are writing a database-oriented application, you may need to know the application language (e.g., Java), the language’s library (J2EE), the database’s query language (SQL), the database’s stored procedure language (e.g., PL/SQL), and the special library that connects the application language to the database (e.g., JDBC).

    Languages and libraries, however elaborate and challenging to master, are just the raw materials for a real program. Knowledge of these is like a carpenter knowing how to cut and nail lumber – he can work with the raw materials and perform the low-level operations on it. Using raw materials to build a wall of a house requires knowledge of the “design pattern” of walls, and the typical way of assembling them, with the special way of framing windows and doors. While the saws, hammers and nails enable you to build a wide variety of wall-like structures, there is just one way to build them that accommodates standard size windows, doors, sheet rock, etc. There’s a special design pattern for when two walls meet to form a corner, and another one when one wall meets another in a “T.” Once you know all that, you know how to build frame walls, and you get to start over learning how to frame a roof, which itself has many details and variations. Design patterns like this exist in software, unfortunately not as standardized as in home construction, but still important and real. You really don’t want someone building a transaction system who is thinking about the issue for the first time. Just as a person building a wall for the first time might forget about windows or not know that walls are typically built on the ground and then raised up, a person building a transaction system for the first time might not think about audit trails or know that paying attention to row-level locking is a typical concern.

    Unfortunately, once you know all that, you are still just at the beginning. If you know them well, you are as well-qualified as a college graduate with a major in English would be to lead a corporation’s response to a lawsuit – while it is true that all the legal documents are written in English and all legal proceedings are conducted in English, there are structures and procedures and patterns of speaking, writing and interaction intrinsic to legal English, and unless you conform to them, you and your case will be summarily dismissed. While the law-challenged English major may think in terms of making a date with the opponent and a judge and talking things through, there are formal complaints, responses, interrogatories, motions for discovery, rules of evidence and procedure and on and on and on.

    The chapter goes on to cover things like all the tools you have to know to write, build, test and run your program, not to mention find and fix the bugs.

    Once you've done all that, you've mastered a single set of myriad tools that are out there to build and test software, in a single application environment. When you move on to the second set of such tools, you begin to appreciate just how different those different environments are — at least as different as classical Chinese is Chinese
    from modern American English (this blog post), from medieval Celtic runes.

    Runes

    Having learned a couple of these amazingly different language systems and the wildly different worlds in which they "live," you're still pretty low on the skills hierarchy.

    One of the dimensions "up" from "simple" ability to work successfully within one of these language systems is the ability to find patterns in extensive bodies of code and invent ways to leverage the patterns to transform the code in valuable ways. One frequent transformation takes knowledge out of code and puts it into editable data of some kind, with the net result that there's less code and that changes can be made more quickly and safely by changing the data instead of the code.

    A closely related dimension to pattern-finding is working with the tools. If there's a problem with one of the tools you use, can you fix it? Can you build a better tool?

    This may not sound like a big deal, but there is a HUGE difference between a programmer who is able to write an application that uses a DBMS (database) and a programmer who is able to create a DBMS. Among programmers who work with the internals of a DBMS in any way there is an extensive hierarchy. At the bottom of that hierarchy are people who don't change source code but are nonetheless expert in the use of it, running through various steps up to those who can make strategic changes or even create a new DBMS. This is a dramatic a hierarchy as starting with people who can drive a car, onto car mechanics who can tune up an engine and going up to engineers/scientists who can create a whole new kind of engine.

    Within tools, there's even a broad range. For example, someone who can work with a text editor tool is normally worlds away from someone who can build a compiler or a code generator. As different as calculus is from algebra.

    This brings us to a whole new dimension. There are people who are so skilled that, not only can they master the available tools to build effective, working software quickly, they can conceive, create and build tool-building engines to automate the process of using the tools to build software. I won't try to describe this here, but it's a kind of level-shift, much like algebra is a level above arithmetic, fully encompassing it while going way beyond it — while still "boiling down" to numbers in the end.

    With tools, we begin to dip our toes into the deep waters of "systems software." Things like the operating system or networking software or storage software or machine/device control software, each of which constitutes a whole world in itself. The people who are proficient in any of these things are truly rare, and near the top of the skills hierarchy.

    Just to take one small example, in the world of the internet there are levels of software and skills that are a little exposed to people who are administrators, but like with most of these things, the bulk of the iceberg is hidden from the surface. Something that everyone uses for example is DNS, which is the software that translates the URL that appears in the top of your browser to the low-level IP address that identifies the server which will deliver content to you. DNS operates in a way that is mysterious to the vast, vast majority of software people, even those who are deeply skilled in other areas. You can be skilled in DNS and still know little about the devices and software that get the moving job done, the TCP/IP protocol engines (and extensive supporting software) that moves data through the network.

    Hey, you might say, isn't that esoteric stuff only relevant to the nerds who keep things running, like of like the maintenance people in the basement of our office building? Yup, that's typical thinking. There are maintenance people, sure. And there are people who understanding things so well that they can create a new heating and cooling system for your building that is silent, effective, and uses a fraction of the power of the old one.

    Conclusion

    There's a great more to be said about the multi-dimensional, multi-domain software skills hierarchy than I've said. I've only scratched the surface. But I hope you've gotten as a take-away that the depth of skills in software is much greater than most outsiders, and even most normally-skilled programmers, imagine. Understanding this skills hierarchy is essential to attracting and retaining the very best software people, and unleashing their considerable talents to transform software-using businesses in dramatic ways.

     

  • How to Get A Software Job

    There is lots of advice from authoritative places about how to get a job. Any job! Let's apply those principles to software jobs. If you're a great programmer and the techniques work — you don't want to work there! If you're a software manager and the techniques work on you — I hope you work for the government or a big corporation, you'll do well.

    How to Interview Well

    There are lots of places where you can find essentially the same advice, but a recent article in the Wall Street Journal did it well. Here's the key — and note, it's hot off the press!

    Rapport

    Rapport, huh? Sounds suspiciously like what a car salesperson does, your instant new best friend. Who are experts in this rapport thing? Somehow, I suspect it's not top programmers:

    Researchers

    Wow. And "researchers" have found this. So it must be true!

    Here's a lady in a professional job. Where'd she learn her key skills? Tending bar. So much for the value of computer science!

    Bar

    One thing that's recommended is using humor in the interview. Here are a couple models to follow:

    Humor

    The scary part is the study referenced in the WSJ article, in which ratings for competence were strongly influenced by the extent to which rapport was built in the first couple minutes of the interview — all other things being equal, you got higher marks for competence by the interviewer if you were an effective schmoozer. Sad.

    There are Alternatives. Winning Groups use them!

    I strongly suspect that the author's advice is excellent. Aside from my snark about being proven, it's true that most software interviewing is mostly about interpersonal relationships. Groups that famously try to do it differently need to try again. Here's my analysis.

    It doesn't have to be this way. Do it the Joe Torre way! It's all about the substance, people! Just get your head out of software and look at other substance-centric fields. All will be clear.

  • Software Management and Relationships

    The worldly and wise among us repeat without end: relationships and getting along are everything in the world of work. I grudgingly admit that they are sort of right. But I also insist that groups of highly technical people with highly technical jobs, like programmers, outshine the competition by 10X or more by following a different set of rules.

    Yes, relationships are important. But high-performing software people build real relationships based on the common substance on which they work — software — rather than interpersonal junk.

    Sick Sheep in England

    This isn't just about software. It's about knowledge and substance vs. interpersonal relationships. There's a section of Thomas Hardy's novel Far From the Madding Crowd which illustrates the point nicely.

    Bathsheba has inherited a farm, and she knows little about farming. She is totally focused on her changed circumstances and her status relationships with the people around her. Because of his own troubles, a neighbor named Gabriel Oak has been working for her. Gabriel has deep knowledge of sheep. Bathsheba disapproves of the way he relates to her, and dismisses him. Then her sheep break through a fence and start eating young clover, which makes them very sick. They start to die.

    Hardy sick sheep

    Her men tell her that only Gabriel knows how to pierce their stomachs and cure them, but she refuses to ask for his help. Then another sheep dies, and she asks him to come. He refuses until asked "properly," with respect. She asks, he comes, he operates on all the sheep. The day is saved.

    Hardy people

    Bathsheba has learned the important lesson that — substance and knowledge matter! — and so asks him back and he accepts, now that his technical skill is respected, and front-and-center.

    Sick Sheep in Software

    I have seen far too many situations in software groups run by Bathshebas who have yet to go through a dying-sheep episode. I have also seen far, far too many situations in software groups in which software sheep are dying by the day … and the Bathsheba in charge simply refuses to change her or his attitudes and reconcile with the equivalent of Gabriel Oak.

    Conclusion

    Do you want to win an award for most congenial software group? Or do you want to have a software group that bonds over the work it's doing, and hits it out of the park? This isn't a binary choice, but you definitely do have to decide your priorities. If someone is marginally productive and starts getting nasty, it's time for them to go. If someone is a great producer, you find ways to make the nerd happy and fulfilled as well. By doing great work. This is a tough subject and an important one. For more, see this. For lots more on software people, see this.

  • Software people: You get what you pay for

    What's more important: programmers who are really good, really productive and consistently turn out high-quality code that helps the business? Or software managers, whose technical skills are not at issue, and whose job is to hire and manage programmers? Or, if they're really important, they hire managers of managers. Based on where most put their money, it's the managers we value most.

    If you want things to get better in software, you may want to consider rewarding the people who actually, you know, do the work.

    Management bias

    Who's in charge of your organization? Let me guess … a manager. Am I right? So when it comes down to the important things, who do you think gets valued the most highly … other mangers, perhaps?

    Most organizations have a pay scale that rises with increasing "responsibility," i.e., how "high" you are in the management hierarchy. I use quotes here because I'm deeply cynical about those things. Since when do the top people truly take responsibility for things, instead of blaming some poor underling? And height in the hierarchy implies goodness and value. 

    Look at this fascinating list of the best-paying jobs:

    Jobs

    This list is clearly incomplete because it doesn't have things like CEO on it. But still, you'll notice that the people at the very top are individual contributors, doctors and lawyers. Right behind them are a couple kinds of technology managers. Then a "strategy manager." What about the next one, Software Architect? Sadly, in most organizations, the person with that title may not manage many people, but is considered way too "important" to actually write code, you know, like the peons do. Then we've got our first actual value-producing job category, an engineer! Not software, though. Of the last four jobs listed, three are explicitly management, and the fourth, "solutions architect," is a person who wouldn't be caught dead lowering himself to writing code.

    It doesn't have to be this way

    Many interesting organizations don't work this way. One example I've explored in detail is baseball, where the players are valued most highly, and where the vast majority of the managers were themselves excellent players, usually for many years.

    In more detail, here are some of most highly paid managers:

    MLB manager

    Here are some of the most highly paid players:

    MLB highest paid

    Notice a difference in pay scale? Baseball has clearly decided that while managers are important, it's the players who win games. What an idea!

    Conclusion

    If what you think your organization really needs is lots of managing, and you figure the software will somehow get itself written, then you should pay your people like everyone else seems to. But if you actually need high-quality software that works and meets your needs, you may want to consider a different strategy.

    p.s.: If you want to learn more about software people, see my book on the subject.

  • How much is a computer science degree worth?

    The median annual wage of a college grad with a computer, math or statistics degree is over $70,000. This is better than the vast majority of college majors, and compares really well with the median annual wage of high school grads, which is under $40,000. The conclusions are clear:

    • Go to college
    • Major in computers, math, statistics, architecture or engineering
    • Otherwise, you’re screwed.
    • Well, all right, majoring in education or psychology leads to crappy salaries, but at least it’s better than being just a high school grad.

    Here is the data: Wages of college grads

    This is a test!

    Trigger Warning! From here to the end of this post could trigger feelings of inadequacy among certain people. Others could feel anger towards the author, causing potentially dangerous heightening of the pulse rate. Others could feel that the author is hopelessly arrogant or elitist, resulting in generally uncomfortable feelings. So read on at your own risk.

    This post is a test of whether you’re qualified to be a top computer programmer, or an outstanding achiever in any technical/quantitative field. The thoughts in this post up to this point summarize what the article accompanying the chart intends you to conclude, and what most people will think on looking at the chart.

    The author of the article clearly failed the test.

    Did you?

    Understanding the data

    If you haven’t already, look at the chart again. Note the big, fat explanation at the top. The endpoints of the lines represent 25th and 75th percentiles. The 75th percentile for high school grads is about $50,000. This means that a quarter of high school grads have salaries above that. The 25th percentile for computer etc. grads is roughly $50,000, perhaps a little more. Which means that a quarter of the computer etc. grads make less than $50,000. In summary: a quarter of high school grads have salaries that are greater than a quarter of college grads with degrees in computers, math or statistics. Read that sentence again. Get it? Did you figure it out before reading this?

    Implications for Hiring Computer Programmers

    I hope you’ve just seen why, when I’ve hired people, I really haven’t given a %^* about their education or their degree – in fact, the higher the education and the fancier the degree, the more concerned I am to weed out the folks with bad attitudes, the ones who have been granted the knowledge and the certification to prove it, and want to spend their lives resting on and/or milking their degrees. Some of the best programmers I’ve met in decades of programming did not have college degrees. Most of the ones who are less than excellent and/or have “risen” in management are experts at glancing at things and reaching the wrong conclusions. Like most people do when looking at the salary chart above. FWIW, here are some good examples of drop-outs who did pretty well. Including the Wright Brothers — after all, how hard can inventing the airplane be?

    The people who are best in computing combine big-picture, visual/conceptual abilities with an utterly uncompromising attention to detail. Computer programs shouldn’t have even a single byte wrong, and the bytes should be selected and arranged according to a deep conceptual understanding of the problem at hand. Amateurs and pretenders don’t do well at either of these jobs, much less in combination.

    Conclusion

    If you care about attracting, selecting and retaining the very best software people, you would be well advised to alter your hiring practices as required to select the people who … get ready for it … can actually do the work! Really well! Having degrees or whatever is not nearly as correlated to that outcome as you might think.

  • High-IQ Programmers: the Problems

    You've got really smart programmers. Problem solved, right? NOT! They can have issues, too. Just different ones.

    Smart Programmers

    Everyone knows that programming is hard. Everyone knows that really smart programmers can be many times more productive than average programmers. Everyone knows that your project's chance of success go way up if you have smart programmers working on it. But not everyone knows that there is a collection of flaws to which really smart programmers are particularly susceptible.

    The reason is pretty simple. Programmers are people, and people have problems! But different kinds of people often have different kinds of problems, and truly exceptional people may have problems many of us are not familiar with. A person who's really tall can have problems with bonking his head on entries that most of us don't have. A person who's really famous can have problems having a quiet meal in a restaurant most of us don't have. And there can be a dark side of a really smart programmer's most admirable qualities.

    The "Problem" of being good at solving really hard problems

    My new book on Software People (here's a description, and here's the book on Amazon) has a whole section on the problems endemic to high-IQ programmers. Here is an excerpt:

    Are good at solving hard problems. The ability to solve hard problems distinguishes them from other people. They ignore simple problems. They disdain working on them. When a simple problem can’t be avoided, they go to great lengths to turn it into a hard problem. People who are good at solving hard problems like hard problems, and can find them in places where other people see no problems at all. Sometimes this is a good thing, like when you encounter a genuinely hard problem that can’t be avoided. Smart people get bored easily. Smooth, straight roads are boring. Some smart people will actively change directions and seek out a problem they suspect will be hard because it is hard. Often, smart people seeking hard problems overlook sophisticatedly simple solutions because they are simple, or spend loads of time solving a really hard problem that actually didn’t need to be solved.

    If you've got a hard problem, you darned well better have people who are good at solving problems that are hard. Otherwise, you're screwed. But it happens often enough that programmers who are good at solving hard problems are really proud of that fact (why shouldn't they be?). Their self-identity is tied up in that ability. 

    What you really want is a programmer who is capable of solving really hard problems, but feels no need to demonstrate that ability unless it's really needed. I've definitely met people like this, but boy are they rare! You're talking about a super-nerd who is amazingly humble.

    Climbing a Mountain

    Suppose there's a mountain your team has to ascend. There's only one good mountain-climber in your group, and he's an amazing one — famous for his ability to tackle near-impossible climbs. You and your team are standing at the foot of a mountain. Naturally, you turn to your expert.

    Your expert, being an expert, scopes out the mountain. He sees lots of things that the normal people in your group miss. He spots a hard-to-see, tricky path that avoids the tough parts and makes the ascent a piece of cake. He also sees a route that starts out looking smooth, but has a pulse-pounding section that no one could make without his expert knowledge, experience and guidance. And then there are the other routes.

    The expert has to choose between these two reactions at the end of the climb:

    1. Boy, what an easy climb! That mountain wasn't so tough after all!
    2. We got to the top, but we almost died on the way. If it hadn't been for X's amazing skills, we would be calling for helicopters to remove the injured and the dead at this point. Thanks, X!

    Choice number 1: X's amazing skills are nearly invisible, because it "wasn't so hard after all" — but only because X uniquely saw the hard-to-see route that avoided the difficulties.

    Choice number 2: X's amazing skills are on full display, demonstrated in vivid 3-D to the team members, as he accomplishes something no normal mortal could pull off.

    Hmmmm. Choice 1: make a hard thing simple, which only I could do, but in the end, everyone is left with the impression of how simple and easy it was. Choice 2: take a tough-but-possible route, in which my amazing powers are on full display. The smart person may not even be aware of how his guts and ego are pulling him to Choice #2. It's just human nature.

    Conclusion

    There's lots more in the Software People book, where this came from. On the one hand, outstanding software people are people. On the other hand, they have issues that are unique to their smartness. You want smart people on your team. Definitely. But you also want to help your smart people be even better than they already are by confronting and overcoming their unique problems.

  • Software Problems: the Role of Incentives

    When lots of human beings work at something for a long time, they tend to figure out how to do it. Building software appears to be a huge exception to that rule. With decades of experience under our belt, why is it that we still can't build good software?

    One of the reasons software projects so often fail and improved methods aren't used appears to be that the people involved have perverse incentives.

    Incentives

    Everyone knows about incentives. They work. Even when we know someone is using incentives to get us to do something, we're more likely to do the thing with incentives than without them.

    Perverse Incentives

    Whether an incentive is perverse or not is in the eye of the beholder. From the incented person's point of view, an incentive is an incentive, and as we know, incentives work. But we normally call incentives "perverse" when they incent people to do something that most other people would agree is a bad thing.

    Perverse Incentives: Mortgages

    The housing boom leading up to the financial crash of 2007 was clearly driven by perverse incentives on multiple fronts. Borrowers were tempted to take what seemed to be easy money. Mortgage companies could make piles of money in fees by packaging up risky mortgages and passing them on. Rating agencies could collect loads of fees by not looking too closely. And the bankers at the top of the food chain made themselves lots of money by creating and selling fancy instruments that ignored the underlying realities and the ultimate consequences of their actions. Then it all came crashing down. Many were hurt, the big guys who made the most money least of all.

    Perverse Incentives:The VA System

    It has recently come out that more than 120,000 veterans are experiencing long waits for care at VA hospitals, even while official reports showed minimal wait times, enabling managers to collect incentive payments. If there ever was a case of perverse incentives leading to bad behavior, this is it.

    VA incentives

    Perverse Incentives in Software

    Software is so rational, so organized, the people involved are so smart and well-educated — surely perverse incentives aren't driving behavior in software, are they?

    Sorry, sweetie, perverse incentives are a human issue. Humans respond to incentives, perverse or otherwise. And as it turns out, there is a rogue's gallery of perverse incentives operating in software — I will only scratch the surface here!

    Estimates

    Estimates are perverse all by themselves.

    They are also a GIANT BILLBOARD incenting EVERYONE involved in the process to make any estimate as long as they can possibly get away with; and since very few people (often including the programmer involved!) has any idea how long something *should* take, the estimates are typically accepted as is; but then, manager often double the estimates before passing them on. Why is this perverse?

    The organization probably would like to get something done in the shortest reasonable time. But the programmers and project people are measured on whether they beat or miss the estimate. The longer the estimate, the better the chances of avoiding failure. It's that simple. It just makes it all the more maddening that, even with inflated estimates, things still go wrong!

    Requirements

    The whole modern software development process starts from requirements. Gamesmanship around requirements is therefore front-and-central. Estimates are based on requirements, and therefore controlling and fixing the requirements is central to the effort of creating "success." The system may fail, the users may hate it, but if it meets the "requirements," the people running the project get to declare "success." What you'd like is for the project to succeed when the needs of the business are met. The perverse incentive is for the people delivering the system to define "meeting the requirements" and then control the requirements to assure that they're met, regardless of what disasters happen to the business.

    False reporting

    Just like at the VA, project managers are highly incented to avoid reporting problems — typically using big fancy reports that are chock full of meaningful-seeming stuff but are in fact just garbage. Just like in the mortgage-driven financial crisis, everyone involved is incented to declare success, take their rewards, and kick the can down the road for the next guy. Eventually, with shocking speed, it all comes crashing down, just like the financial system, and just like the mere 4 days between the laudatory article about how great Cover Oregon was going to be and the admission of total failure.

    False Assessments

    Here's where the rubber meets the road. Who is incented to blow the whistle on a failing software project? How, when and by whom is a software project judged to have failed? Most importantly: what are the consequences of having failed?

    We all know the answer. Who has even heard of a software engineer who was fired for failure to deliver? And the people in charge? Never. It wasn't their fault! And the project didn't fail anyway! The requirements changed every month, the target kept moving, and blah, blah, blah.

    Conclusion

    Your kid comes up to you and asks, "can I play my video game now?" You briefly think about how your question when you were that age was "Can I go out and play now," but the kid isn't interested, and is bouncing around waiting for your "sure." Being the aspiring adult you are, you act responsibly and ask "Have you done your homework?" There's a brief pause. The kid is doing a quick risk-reward ratio calculation. If he says "yes," he probably gets to do what he wants. But you might ask to check. Hmmm.

    This is the breeding ground of perverse incentives. We all learn to balance honesty, openness and getting what we want. Some of us go for honesty and openness, deciding that anything else just isn't worth it. But loads of people make an informed judgment on a case-by-case basis, much like the kid and his homework.

    Whatever the morality of the case, the facts are clear: software projects fail left and right, and perverse incentives are a significant factor in making them fail. Without changing the incentives, we're unlike to abandon the Bad Old Way of building software and achieve success.

  • Software People: Book just Published

    I've just published my book on Software People — an insider's look at what programmers are like. It's got the same tacky cover design as the three books already publicly available:

    BBSB cover People

    I attempt to cover material in the book that I haven't seen elsewhere. Here are some of the topics:

    • A description for outsiders of all the stuff you've got to know in order to be a programmer — learning a language is just a tiny bit of it!
    • A statement of the programmer's dilemma — how all-consuming mastering even a slice of software usually is, and the difficult trade-off's you're then faced with involving the other skills you need to succeed in an organization and in life.
    • A discussion of how there are levels and levels of software skill — it isn't like learning to drive a car. Similarly with productivity.
    • A extensive discussion of the cultural divisions and wars that blaze through the software community, with mutually incompatible "religions" living in separate colonies, looking with disdain and pity at those who follow false gods.
    • How people who are excellent at software, far from being honored, are often diminished and marginalized.
    • Lots of material about hiring. Who decides, on what basis, common mistakes.
    • A discussion of the deep-seated cynicism that infects a large number of programmers.
    • Technology organizations, managers and decision making.
    • Typical patterns I've seen in software people.
    • An extensive discussion, with examples, of the flaws that are characteristic of high-IQ programmers.
    • Finally, a discussion of the role of the CEO in a company where software plays a key role.

    I've been at work for a long time on my series of books on how to Build Better Software Better. The books in the series have circulated in draft form, and each has undergone multiple revisions over a period of years. I've already released my basic books on Software QA, Software Project Management and Wartime Software. The one on People underwent 9 major revisions. Software People is less technical and more readable by civilians than the others.

    I have a couple more that are no longer undergoing revisions and are about ready for general circulation. They are:

    • Software Business Strategy. There are some things that are unique to running a software business that apparently are not taught in business schools, and are common errors in the software businesses I see. I spell out the problems and solutions in this book.
    • Software Product Design. You'd think we'd have it down by this point. But I see software product design happening all the time, and mistakes made over and over. In this book I describe the best methods for creating successful software products and avoiding the common mistakes.
    • Software Evolution. When you see software built over decades and decades, patterns emerge — and it's far from just onwards and upwards! These software patterns are strong and they repeat, like the well-known Innovator's Dilemma, only much more software-specific. They have amazing predictive power.

    I will publish the rest of the books as time permits. Meanwhile, I'm pleased that I've finally released the Software People book for Kindle, more than 12 years after I circulated version 1.

  • Hiring a CTO: the Impossible Dream

    I've been a CTO several times. I've worked with many CTO's. Last time I checked, every one of them was a natural-born human being. No cyborgs, no alien intelligences controlling a homo sapiens body. So why is it when most organizations try to hire a CTO, their requirements clearly demand capabilities and accomplishments that no human being could ever achieve?

    The Impossible Dream

    I've seen a whole pile of CTO job specifications over the years. They are remarkably similar. If I were in the executive recruiting business, I'd be embarassed to deliver my "custom-built" CTO job requirements document, knowing that all I've really done is some light editing from previous efforts. The word "plagarism" somehow comes to mind.

    But that's chicken feed compared to the blue whale in the room — this is a way bigger issue than a mere elephant in a room. It's the fact that no actual, living human being born of human parents (to rule out the alien connection) could ever possibly satisfy the requirements! The "requirements" aren't requirements — they're an impossible dream.

    The Impossible Dreamer

    Here are the first couple of verses from the song, the central song in the 1965 musical Man of La Mancha

    Impossible dream

    The song and the musical are about Don Quixote, hero of one of the greatest and most influential novels of all time. Don Quixote and his faithful companion Sancho Panza go off on a quest to revive chivalry and do good works.

    Don Quixote

    One of the most famous episodes is Don Quixote going after the windmill, which he imagines to be a giant. Things don't go well.

    Windmill

    This is the origin of our phrase "tilting at windmills," in which you are taking on an impossible task.

    Tilting at windmills

    Impossible CTO Requirements

    Part of the trick in writing these requirements is to make it not seem to be an impossible task. I admire the clever writing needed to slide the impossibilities past the average reader, who, frankly, pays little attention to the wording.

    The requirements amount to something like the following list:

    • The successful candidate will be over 7 feet tall and be able to successfully pass under a limbo pole exactly 2 feet high without touching the ground.
    • The successful candidate should be able to crank out bug-free code at the rate of hundreds of lines per hour, while being an empathetic, nuturing leader of technical talent.
    • The successful candidate should have a proven track record of hiring technical talent that dresses in board-of-directors-friendly "business casual," works during normal working hours at the designated location, and is consistently cheerful and friendly.
    • The successful candidate will have demonstrated the ability to be a self-starter, while at the same time executing management directives enthusiastically and without deviation or failure.

    Have I exaggerated? No! I've only slightly edited actual requirements that I've seen in order to protect the guilty.

    Conclusion

    Trying to hire an in-your-dreams-only CTO is business-as-usual in my experience. I try not to get upset about it. And in-your-dreams is not even the biggest problem in CTO hiring; even worse is making requirements that actively filter out the most promising candidates! This madness is yet another side effect of people without a technical bone in their bodies thinking they can manage it, because after all, technology is just another thing to manage. This absurdity is so widely accepted in management circles that even solid examples of how ridiculous it would be to act this way in other fields just whittle away at the deep-set conviction.

    Postscript

    I know, I know, I'm being mean. The recruiters are, after all, doing the best they can with what they have to work with. They won't get very far with their customers (the companies doing the hiring) if they say "that's a stupid requirement, I won't put it in." And in the end, they're measured on the success of the person who eventually gets the job, which is way beyond a document — which people don't take terribly seriously anyway.

    But there's a serious point here anyway: the document could be meaningful and helpful if it specified something that could be satisfied by something less than a celestial being, and if people could get clear about avoiding "there are at least 15 top priority items here — yes, they're all equally important!"

     

  • Joe Torre and Software Development

    Joe Torre had an outstanding run as manager of the NY Yankees baseball team. While managing baseball seems pretty distant from managing software development, there are nonetheless a couple of important lessons to be learned. Put simply, baseball has it right and software has it wrong: if we chose software managers using the common-sense methods that are widely accepted in baseball, our software development track record would emerge from its current long, dismal, always-agonizing depression.

    Joe Torre

    Maybe not everyone knows who Joe Torre is. Now retired, he was a baseball player and manager.

    JoeTorre1982
    Joe Torre had an excellent career as a player, from 1960 to 1977. He was an all-star 9 times, was NL MVP once, and was the NL batting champion once. Unusually for a baseball player, he had extended playing time at multiple positions: catcher, first base and third base.

    Joe_Torre 2005
    He went on to have a stellar career as a manager, from 1977 to 2010. His Yankees won the World Series 4 times. He was AL manager of the year twice. His NY Yankees #6 was retired.

    Players and Managers

    Are most managers former players? Is Joe Torre the exception? Loads of baseball fans imagine they can do a better job than their home team manager. The owners have their own opinions on the subject. How hard can it be?

    I looked into this question. There is a list of every baseball team manager from the start of the game. The list gives lots of information, including the manager’s history as a player (or not).

    Here are the facts: as of today, there have been 686 managers of major league baseball teams, starting in 1871. Of those, 566 were former players, while 120 were never players. So the numbers show that the vast majority of managers have been former players. Just 17% of managers since 1871 were never players.

    Is it just Sports?

    My go-to example in music is, of course, Franz Liszt, who excelled as a performer, composer and conductor.

    Liszt
    But he was hardly alone. The NY Times says

    Times have not really changed. In Bach's day composers played their music at keyboards and conducted the instrumentalists about them. Beethoven conducted. So did Berlioz, Mendelssohn, Wagner, Mahler and Strauss. In our day composers are still conducting…

    Here’s Gerard Baker, managing editor of the Wall Street Journal. Gerard Baker WSJ
    Mr. Baker is an accomplished writer, an excellent reader and judge of other peoples’ writing.

    A pattern seems to be emerging here

    Yes, it’s a pattern. Can you imagine a CFO who can’t add? A managing editor who not only can’t write, but can’t even read? How about a museum director who not only isn’t an artist, but can’t see?

    Let’s apply the pattern to software!

    Oh! Ummmm…

    "Well, software is just another thing that can be managed by good management techniques!"

    "I don’t need to know the details – I manage for results!"

    Can we talk about something else now please?

    Conclusion

    The best qualifications for managing software in general and programmers in particular has never been a hot topic. In spite of all the evidence of massive failure, I doubt it will become a hot topic any time soon. But it should be! Just think about the basics here: however peculiar you may think writers are, do you really think editors don’t need to be able to read and write themselves? You may think of accountants as people with thick glasses hunched over desks with green-shaded lights, but do you really think the CFO doesn’t need to be able to add? Programmers may be weird, but doesn’t similar thinking apply?

    Postscript

    While 83% of baseball managers were players, 17% were not, among them some excellent managers. I'm not saying that only former programmers can manage programming efforts, and I know a couple truly excellent non-programmer managers. But in each case, they do interesting special things that are not widely understood that enable them to achieve excellent results.

  • Who Makes the Software Decisions?

    When the home team loses game after game, everyone starts wondering who's in charge, and shouldn't changes be made? Well, this is exactly what's happening in software. It's gotten so bad that it's making the front page of the tabloids.

    Post
    Who's in charge in software? Who makes the decisions? Could it possibly be that the wrong people are empowered to make crucial decisions in software, and that things will only get better if we make a change?

    Software decision makers

    Who makes important decisions in software? The answer is obvious: anyone but programmers (ABP)! Programmers are the ones who do the real work: create and modify source code. They work with various tool sets in one or more programming environments and create the code that leads to the results needed by the business. You might think that programmers, therefore, would be front-and-center in selecting the toolsets and methods to use to get the desired results most effectively. This is rarely the case. Such decisions are typically made by people who are not, and in many cases never were, programmers.

    What the industry thinks

    I receive e-mail solicitations every day for various software products. I'm often invited to attend seminars or webcasts. Here is a typical example I received recently: What

    I'm not drawing attention to it because it's exceptional in any way. The next part of the solicitation is also typical, who should attend: Who

    Who should attend? ABP, of course! Again, the company that sent the solicitation is not doing anything wrong. In fact, they're being smart. They are inviting the people who make important decisions about programming, i.e., ABP.

    How things work in other fields

    In just about any field you can think of, the more highly specialized and skilled the person doing the work, the more involved that person tends to be in all important decisions about the work. While kids starting out in baseball are told what gloves and bats to use, accomplished players have their own gloves and bats they have selected.

    Even when the front-line people in other fields don't make the ultimate decisions, the important managers who do tend to be former front-line people.

    Software is different

    Things are different in software, of course. In sports, anyone can watch the game. They see the players on the field or court. TV commentators can circle a player on the screen and tell you to watch the thing they did, whether stupid or smart. Whatever it was, it makes sense to the viewer.

    There is no equivalent in software! Software is invisible! (to everyone but programmers…) All those important decision-makers ever see is reports. Not only don't they open the hood and look underneath, they can't even see the car! The decision-makers largely rely on rumors and hearsay, but nonetheless develop strong opinions about how best to win on a playing field they can't see, where a game is played they can't play, following rules about which they are entirely ignorant. Hmmm, how is this going to turn out, I wonder???

    It doesn't have to be this way!

    There are places where business-as-usual in software decision-making is … blatantly violated! Someone who … wrote the code! … is actually in charge of things. Of course, since he's a guy entirely without management training — OMG, he doesn't even have an MBA, that's how bad it is! — the place must be a disaster, right?

    One such place is Athena Health. Athena powers doctor's offices. I first encountered them more than ten years years ago, when I had the pleasure of having a phone interview with the guy who wrote their code. I was hearing lots of skepticism, which is why I was having the call. This was still the time when internet bubble thinking ruled technology, and the rumor was that this guy was using "toy" technology and building something that "wasn't scalable." Heh.

    The real problem was that the guy who wrote the code's skill was … get ready for this … writing good code, and making excellent decisions about writing code along the way! He had no experience in or talent for telling ignorant investors what they expected to hear. Bless him!

    We invested, and the company has done, and continues to do, great. A couple years ago, I was pleased to have Ed Park, who is now EVP and COO of the company, attend my nerdfest, a gathering of top CTO's of companies I'm associated with. Here he is explaining something.

    2011 07 03 Nerdfest Sunday 006s
    Ed wrote Athena's original code. He still knows stuff, and continues to make decisions based on the substance, not just go-through-the-motions process.

    Conclusion

    While lots of spinning goes on to disguise the fact, software projects typically fail, and even ones that "succeed" have crappy software. The Post was right (see above) to feature the question, "who makes the software decisions?" The industry's answer is clearly and unambiguously ABP (anyone but programmers). If you want the software your organization produces to sort of, actually, you know, work, you might want to think about removing the "ABP" restriction from the job requirement for software decision-makers.

  • People and Substance in Software Management

    What's the right focus for managing a bunch of software engineers? Should you focus first on assuring that you have a congenial team with good working relationships, or should you focus first on assuring the substance of what they're creating is the best it can be? This is a similar question to the choice of process and substance that I covered in a previous post and it's just as important.

    Substance vs. Process and People

    Just as schools of education purport to teach people how to educate without reference to the subject that is being educated (math, history, etc.), management schools (you know, the kind that give MBA's) purport to teach people how to manage without reference to the kind of work that is being done or the kind of people who are being managed. The idea that "good management" and "management skills" are entirely distinct from the actual work being done pervades our work environment. The more esoteric the work skills, and thus the more difficult it is for a manager to have a clue about what his people are doing, the more this seems to be the case. Software development is right up there with particle physics in its ability to induce glassy eyes among the non-initiated.

    Substance vs. People

    It's really tough if you're a manager and you're trying to manage something when you have no clue what the people doing the work are doing. This is what leads to the strong tendency for anyone who claims to be a good manager to focus on process instead of substance, and to focus on people and human relationships rather than substance. Truth be told, if you don't have a clue about what people are doing or talking about, you don't have a lot of choice. A normal human being observing an exchange between two engineers can easily discern that engineer A is being nasty and disrespectful to engineer B, but how can that normal human being see that engineer B is proposing a completely stupid approach that will take a lot of time and then fail, while engineer A is being quite restrained (relatively speaking) in his reaction to the uneducated drivel drooling out of engineer B's mouth? The manager is likely to react to the only thing he actually sees, i.e., the disrespect being given by engineer A to engineer B, and therefore chastise engineer A, when actually it's engineer B who should be sent back to the farm teams, if not worse.

    I am not claiming that there is a binary choice between people and substance, that if you focus on one you can't pay attention to the other. Of course, both are important. But as I look at software shops, I frequently see managers treating one of these as the primary goal, while figuring that the other will mostly take care of itself if they get the primary one right. The less they know about the substance, the more likely this is to be the case, but even experienced engineers who are trying to make the transition to management get a clear message on this subject from their superiors: good management is independent of substance, and a "good manager" makes sure above all that the human relationships in his group are in good shape, and that if they are, good software will naturally be the result.

    Management focus and company success

    In the best of all possible worlds, a manager would create harmony and excellent relationships among the programmers, while at the same time achieving maximum productivity towards making the company successful. That's a nice fantasy. But that's all it is. While being nasty doesn't imply great programming any more than being misunderstood implies that you're a genius, the fact is that there's a strong correlation between a focus on substance and company success.

    The reason putting major emphasis on substance (i.e., "who's right" is more important than "who's more respectful of other opinions") is a winner is simple. The groups who all agree that writing the best possible software to meet the company's needs are more likely to … write the best possible software to meet the company's needs! Compare this to groups who all agree that having personal harmony among its members is more important than achieving the best possible software — doesn't it make sense that you'll make all sorts of compromises in software to optimize everyone's feelings? Oooo, better give everyone a medal, we don't want to risk diminishing anyone's self-esteem, do we?

    Organizational Size

    I've noticed that the larger the overall organization, the more that general, content-free management is the norm, and the more likely it is to be imposed on groups that develop software. This is completely understandable from a human psychology point of view. Everyone thinks that the skills and knowledge they have are the really important ones, and the others (which they don't have) pale in importance by comparison. The big bosses are likely to come from sales, finance or consulting and think that having an MBA is a good thing. Why should software be different from, say, running a store, or making deliveries? Or making tubes of toothpaste? If you're a … drum roll, please … manager, then that's what you do: manage. Your skill trumps all the others!

    This is the dominant view of people who run organizations, or are on the management ladder. In order to advance, you have to adopt the organizational management-centric creed. Otherwise, you'll be "stuck" at the bottom of the management ladder, along with all the unwashed, unrewarded hordes who have to sit quietly with their heads down as various management fashions sweep through the grasping folks with sharp elbows who occupy the rungs of management ambition. If you even try to say "software doesn't work that way," all you've done is further disqualify yourself from advancement in rank.

    If you've ever wondered why it is that small, under-funded organizations are nearly always the one who build ground-breaking software, and that giant, "well-managed" organizations with hundreds or thousands of programmers rarely do, this is your answer. "Good management practices" assure that good software rarely emerges from their large organization, and that outstanding programmers with superior ideas about how to do things are marginalized and otherwise made unproductive.

    Software and Baseball

    This may sound cynical and harsh. But think about, say, baseball. While it is tempting to think that all those beards


    131003153809-red-sox-beards2-single-image-cut
    had something to do with the Red Sox winning the 2013 World Series, it is more likely that their outstanding pitching, fielding and hitting enabled them to defeat an excellent Cardinals team.

    The 2012 season was awful for the Sox. It was their first losing season since 1997, and worst season since 1965. How did they respond? Did they update their project management methodology? Did they send in HR specialists to get everyone to be nicer to each other? Did they sideline people based on failing to high-five someone when they struck out with men on base? They did none of these things. Here's what they did:

    • They fired the manager, even though he had a year left on his contract.
    • They got rid of a bunch of players based on … their poor performance!
    • They acquired a bunch of players based on … their superior performance!
    • Finally they focused on their energies on, you know, … winning games! Doing whatever it takes!

    Building good software is surprisingly like building a winning baseball team. Everyone does their job and puts everything they have into delivering great results. If someone isn't performing, too bad — you're not good enough to be a Red Sox! (Oh, just think how badly his feelings must hurt. How cruel!) Even so, sometimes someone is so awesomely good that it seems like he's carrying the team on his shoulders — and with Series MVP David Ortiz, you can make a strong argument that's exactly what he did with the excellent Sox.

    Conclusion

    People should be nice to each other. There should be mutual respect. But software is about results. The difference between average and great can easily be 10X for an individual, and 100X for an organization. You don't get "great" by focusing on process issues or good general management practices. You get "great" by … wait for it … focusing on software! Just like a musician focuses on the music and a painter focuses on the painting, the best software people focus on the software. The substance.

  • What can Software Learn from Ballet?

    Most people think that software and ballet are distant topics, completely unrelated. While you can imagine a program that helps a correographer keep track of things, what could software possible learn from ballet? Answer: a great deal. It would take many posts to describe it all.

    First Some Ballet

    The annual Youth America Grand Prix competitions just concluded.


    Yagp0

    These are a big deal, and people in the field rave about it:


    Yagp9

    The movie First Position was recently released about the competition, and it is well worth seeing.

    First_Position_2011

    While the competitions are for young people, professionals are involved in everything. During the Gala that ends the season, the program consists of the youthful winners of the competition and some of the best professionals in the world. Here are a couple of pros performing at a recent Gala:

    Yagp1

    And here are a couple of the amazing kids:

    YAGP2

    I attended the 2013 Gala; it was totally amazing. What was most striking was that all the kids were in the audience, cheering and generally having the time of their lives. They knew what they were seeing, and were all at some stage of training to be able to do it. They totally got it, and appreciated every nuance.

    Software

    Watching the whole spectacle made me think about what it would be like to substitute "programming" for "ballet."

    There are strong similarities. Both are hard to do. Not many people can do it well. It takes years of hard work and dedication to get good at it. Huge discipline, focus and concentration are required. Small mistakes can wreck an otherwise perfect performance. While it's primarily an individual discipline, group performances are often required, and are even harder but can lead to even better results. The best performances seem effortless. Beauty and symmetry are important aspects of successful performances. And in both cases, you are orchestrating a flow through time.

    There are of course huge differences between the two. The most important differences have nothing to do with what you wear to work.

    Here is one of the young competitors doing something completely amazing:


    YAGP4

    There are young programmers who can do the programming equivalent of this, but how can they get identified, rewarded and encouraged? What opportunity do they have for watching and learning from people who are way more advanced then they are? Even at a more advanced level, when they take courses, they're sitting in class and being taught mostly irrelevant stuff by academics, most of whom aren't serious programming practitioners, and don't even respect it! They think their papers and conferences are much deeper and more important. Sad. It's always best to be taught by someone you want to emulate, rather than by people who look down their noses at actually, you know, writing code.

    They got all the kids on stage at the end. They did a lot of amazing things a picture can't capture, but here's one anyway, from last year:


    YAGP3
    Who would have thought that the field of ballet would be, in many ways, a role model for the transformation of software training and organization? But now I realize that it is, and I encourage others to pick up this ball and run with it.There is quite a bit we can learn from "artsy" fields, including architecture, music and sculpture. Not to mention steamboats and antiseptic surgery.

Links

Recent Posts

Categories