The Black Liszt

  • Demand Media’s Approach to New Media

    Demand Media's angle on new media is to provide content creators the vetting, ideas, channel and money they need to become professional content creators. It provides a direct path to success for talented, hard-working people. As my previous post points out, this is a new path to success. In the traditional, mainstream media, only the elite few really win, while there is (almost) no path to professionalism in user-generated content.

    Demand's approach is perfect for those with talent, skills and a desire to work … but who may not have a paying outlet for their efforts, or a system to structure and focus their work.

    One of the things that is particularly interesting about Demand is that it's not a boutique. It's an industrial-scale enterprise, capable of producing far more quality content than the vast majority of content-creating entities today. But unlike traditional publishing empires, it leverages the internet and works on a fully scalable, distributed basis. There is no concern about tapping out on a certain kind of talent within commute distance of "the office," not when the office is the internet. There is workflow, there are defined roles for people to play, there is automation (lots of it), so in that respect, it's very industrial.

    "Industrial" can be used as a critical word, but it's also a word that describes a system that is designed to produce a large amount of a desired thing at low cost and with predictable, consistent quality. Back in the 1980's, a whole industry got started to process structured content, the kind you have when you fill out a form, for example. This was when concepts like "automated document processing" and "workflow" began to be implemented, and pioneering companies like FileNet applied industrial templates to massive document handling problems. I personally worked on such an automation effort at Sallie Mae, where even then there were over 40 million documents to be processed per year, with a couple thousand people doing the work — definitely an industrial problem.

    What's fascinating here is that, a couple decades later, Demand Media is leading a similar new wave of document automation. The differences include:

    • DM  produces documents; before we handled documents received
    • DM is highly distributed; most earlier efforts were centralized
    • DM deals with unstructured documents; the earlier efforts were directed at structured documents

    But the commonalities are striking. In both cases:

    • Workflow software is the heart of the industrial engine
    • The work is highly structured, with quality measures taken at each step
    • The process is scalable to massive volumes

    And of course, there is a key new process. In first-generation document processing, we were consuming documents. When you are producing documents, you have to make sure you're producing stuff that people actually want to consume. How the heck do you do that at scale?!

    Well, that's part of the magic of Demand Media, and perhaps will be the subject of a future post.

    What does this mean to the person who would like to work in professional content creation? It means that Demand Media, by pioneering Professionally Generated Content (see prior post), has a job and value-creation system in which all the participants win. And because its industrial system has been built on a secure foundation (workflow, etc.) with appropriate updates and variations, we can be confident it will continue to scale and work as it does.

  • Media for a New Era, Demanded and Federated

    There is a new kind of media. We all know the old media produced by the elites, and we've all heard of user-generated content. There is a new kind of media shepherded by new kinds of companies. It combines the best of the old media, and is generating value for the people who create it and the people who consume it.

    Among other reasons, I'm paying attention to this because a couple of companies I know are in the heart of enabling this new kind of content. That’s partly why I like Demand Media so much. It’s also one of the reasons I’m a fan of
    Federated Media.

    The vast majority of the media we consume is generated by an elite, largely
    full-time, generally well-compensated, tiny group of people, usually working in
    large organizations. Let's call it EGC, Elite Generated Content. EGC is generally a pyramid, with a tiny number of highly
    visible people at the top, a large supporting cast, and often a very large
    number of under-compensated people in support, many of whom aspire to climb up
    the pyramid. Katie Couric, sitting at the top of the CBS News pyramid with her
    $15M annual salary    , is a great example. She can do her job only because of her
    supporting cast of hundreds (thousands?), the vast majority of whom will never
    have the camera focused on them except while saying “testing, testing.” They do the work and she takes the credit. While
    the example I’ve used is broadcast news, there is a similar structure in
    newspapers, magazines, books, music, movies, theater, etc.

    The internet has brought us a new kind of media: UGC, user
    generated content. UGC is generated by essentially anyone who wants to. There
    are a vast number of UGC-er’s (millions!), who do what they do mostly on their
    own, mostly for free. Examples include blogs, comments, everything on Facebook,
    product ratings, Wikipedia, etc.

    Some of the people who create UGC are every bit as talented,
    hard-working and effective as the elite producers of EGC, sometimes more so.
    Some UGC creators produce content for which they know there’s a market, but the
    existing rigid structure and cruel, anti-democratic hierarchy of EGC provides no mechanism for them to tap that
    market.

    Some members of the old media have recognized and
    capitalized on the fact that there is a huge pool of frustrated, under-valued
    and ambitious UGC-er’s out there, and have created mainstream media vehicles
    for tapping into this vast labor pool. Perhaps the best examples are TV shows like
    American Idol. For every Susan Boyle who leaps from complete obscurity to fame
    and fortune, there are tens of thousands of applicants who either waste their
    time, or worse, become the on-air fodder for the laughter and cruelty of the
    wealthy, self-appointed judges. In typical old-media fashion, it dangles the
    hope of success to millions and convinces tens of thousands to donate
    their time and effort to the success of the elite handful at the top, for
    example the well-named Simon Cowell, who is surely the Simon Legree of
    show business aspirants.

    Is there an alternative to old-media EGC and
    chaotic, unstructured and unpaid UGC? Simply put, yes. Demand Media and
    Federated Media each are pioneering new paths to enable UGC-er’s to advance to
    professional success. There is no accepted name for this new category (alert!
    naming opportunity!), so until a better name shows up, I’ll just call it PGC,
    professional-generated content.

    What is PGC? Simple: it combines many of the most desirable characteristics of EGC and UGC. Here are the highlights:

    • Like EGC, the content is vetted, it is good quality.
    • Like EGC, there is a market for the content, and a way to reach that market.
    • Like EGC, the participants in the process actually get money for their efforts.
    • Like UGC, the participants in the process actually get credit for their efforts.
    • Like UGC, there is no hierarchy barring the door to aspirants.
    • Like UGC, the process is highly distributed and democratic.

    Sometime soon, I'll show how DM and FM are each, in their own ways, leading the way to developing the market for PGC, professionally generated content, and thus providing opportunities for people and filling previously unmet needs.

  • Mainframes and Storage blades: Xiotech’s ISE

    In computing, the transition from mainframes to blades is
    well established. What about storage? There is exactly one choice: the Xiotech
    ISE.

    We throw words around like crazy in computing. Once a word
    starts to become associated with something good, every person in marketing in
    the universe latches on to that word, and finds a way to associate it with
    their product. Most of the analysts aren’t much help here.

    The opposite applies, too. Once a word is no longer in fashion,
    no vendor admits to selling it, and no buyer admits to owning one. A case in
    point is “mainframe.” Some time ago, you had to have a mainframe to be in the
    big leagues of computing; otherwise, you were in the minor leagues, dealing
    with unimportant problems. Now, even though mainframes are alive and well and
    broadly used, it’s hard to get anyone to admit it. “Everyone” knows that
    mainframes are old-fashioned.  

    In spite of their un-cool-i-tude, let’s talk mainframes for
    a quick minute. What’s a mainframe? In computing, it’s a big single thing that
    gets all of many big jobs done at the same time. It is your central computing
    resource. It’s valuable. It takes lots of time and attention to manage, but
    rewards the attention by being the work-horse of your data center. As your
    workload grows, your mainframe can start to get overloaded; no problem.
    Capacity measurement and planning is a key skill with mainframes, and even
    better, mainframes are built to be expanded. That’s why they’re called “main
    frame.” It isn’t the CPU – it’s the frame (the main one!) that structures your
    computing engines. Sometimes your mainframe needs more compute-power; no
    problem, you can add it in. Sometimes your mainframe needs more I/O channels
    or local memory; no problem.

    What’s cool today, if not mainframes? We all know the
    answer: it’s racks and racks of servers or blades. Each one is powerful but
    inexpensive. You increase capacity by buying more of them. That’s why
    virtualization is such a powerful trend in the data center: VM-Ware (and
    similar products) helps you use all those servers more efficiently. Everyone
    has lots and lots of servers; therefore, the need to use them as effectively as
    possible is ubiquitous; therefore, Hyper-V and its brethren are hot.

    What’s going on in the world of storage? Do we have storage
    mainframes? Of course not! Heaven forfend! “Mainframes” are the bad old,
    un-cool thing, so there’s no way my storage is a mainframe – it’s a SAN, a
    storage area network! It’s a network, see, it’s cool!

    Now let’s cut through the verbiage, and apply the criteria
    of mainframe to storage. A mainframe (see above) is:

    • A big single thing that gets all of many big storage jobs
      done at the same time; check.
    • Your central storage resource; check.
    • It’s valuable; check.
    • It takes lots of time and attention to manage; check.
    • Capacity measurement and planning is a critical function;
      check.
    • You expand capacity by augmenting it, adding things into it;
      check.

    There’s a simple rule here. Suppose you buy a “small, simple”
    mainframe. When you’ve added huge amounts of capacity to it, how many do you
    have? If the answer is “one,” you’ve got a mainframe. If you’ve got a small,
    simple server/blade collection, you’ve already got a bunch of them. When you’ve
    added loads of capacity, you’ve got loads more of them. You’ve never got just
    one. You start with some, you grow to lots, and expand to lots and lots.

    Applying our rule to storage, it is undeniable that while vendors
    will avoid the “m-word” like crazy, what they’ve all got is storage mainframes.
    They are single things, just like a railroad train is a single thing regardless
    of the number of engines at the front or freight cars at the back. Even if they
    choose to call it “cloud storage,” the brutal fact is that it is still a
    monolithic, single, unbroken entity – a mainframe, in short!

    There is exactly one vendor in the market who has created
    for storage what blades are for computing, and that is Xiotech, with its ISE
    product. You do not expand an ISE; you buy another one. Each ISE is a separate,
    free-standing, inexpensive but powerful resource, directly connected to a
    switch – just like a server.

    The Xiotech ISE is the anti-mainframe, and is the
    natural choice for server-farm data center storage.

  • Demand Media and the Media

    Members of the mainstream media seem to have trouble looking at Demand Media, understanding its success and the fact that it is pioneering a new and valuable form of content creation. This trouble is illustrated perfectly by an article by David Carr, the media specialist for the New York Times. The article is surprisingly fair, but the author's feelings are easy to read.

    At least from the article, it appears that the author (along with nearly all mainstream-media writers on this subject) either isn't familiar with the Innovator's Dilemma, or is unable or unwilling to apply the concepts to his or her own industry.

    It's really not that different from the transitions between disk drive form factors, an example used by Clayton Christensen in his classic book (The Innovator's Dilemma: When New Technologies Cause Great Firms to Fail) on the subject.

    In each case, the people who are invested in the older technology look at the newer one as inadequate, cheap, failing to meet crucial requirements, and so on. They fail to appreciate how the new technology opens up new markets, fills unmet needs, and satisfies new classes of customers.

    In this case, the old-media establishment is stuck on the notion that any form of media production must meet the following criteria:

    • a small number of
    • high-priced,
    • full-time workers in
    • central offices, producing
    • small amounts of content that are released in a
    • small number of
    • elite outlets that
    • charge readers for access

    By contrast, Demand Media is pioneering media that meets the following criteria:

    • a large number of
    • moderately paid,
    • part-time workers in
    • distributed locations, each producing
    • as much content as they choose, which is
    • released on the internet and is
    • free to readers,
    • supported by advertising

    Since Demand Media knows it's pioneering, it is experimenting and iterating its way to success, largely driven by direct feedback from its new class of customers. They are learning new things every day.

    Again, the Times article was a relatively fair one; but critical feelings about Demand Media are completely understandable — simply put, the people who make them often feel threatened by DM's success. But the day is long past when a small number of highly influential media figures can sway the thinking of the public. In the end, the internet is enabling consumers to vote with their "feet" (actually, their fingers and clicks), and from all the evidence, the number of votes in favor of Demand Media is growing at a rate that the management of old media can only dream about.

  • Kayak and Customer Intimacy

    Kayak is a case study in how to get engineers to interact with customers to move your product or service ahead rapidly.

    Paul English, their excellent CTO, put up a recent blog post on the subject that humorously features a red phone.

    Redphone
    I continue to be amazed at how frequently organizations try to protect those "precious, critical path resources," their engineers.

    The engineers are often complicit in this. They do all this project planning and get signed up for an ambitious string of deliverables. Then things happen — new requirements come along, some things are harder and take longer, they are late and feeling the pressure. How do think they react when some well-intentioned, idealist jerk comes along and talks about what a great idea it is to have engineers handling customer requests and complaints directly? R-rated, to say the least.

    It's the whole way organizations are set up that leads to everyone thinking that the "red phone" is a terrible idea. And that's a good test for your organization — if the "red phone" seems like a terrible idea, then maybe your organization needs to change.

  • Xiotech Nerdfest Attitudes

    Xiotech's Nerdfest was a wonderful thing. One of the best thing of course was the nerds in full display, wearing their "colors."

    Naturally, one expects a good nerd to have some attitude … err, I mean self-confidence. CTO Steve Sicola's t-shirt displayed an attitude that was widely shared at the fest:

    2010 01 19 Nerdfest Sicola

    Naturally, there were also comments on various technical-religious issues. Here is a particularly out-there one:

    2010 01 19 Nerdfest Gates

    (Pictures courtesy of one of the nerd attendees, Marshall Midden.)

  • Xiotech and the Strange World of Storage Administration

    Storage administration is a world of its own, controlled by storage administrators, running by their own set of rules. It's got to stop!

    Not entirely, of course. Storage is a specialty, and it definitely rewards having someone really know all about it.

    Think about it from an application programmer's point of view. What if files had the same rules as storage? Instead of just adding a "create file" statement to your program, you'd have to request one from the "file administrator," who may give you one after talking with you. You would agree on the name. You would talk about how big the file would get, and how often you intended to access it.

    If everything went well, you would each go off to your respective domains, enter exactly the same information into your respective systems, and the file would be available. It would almost certainly be created before you needed it (better that than after you needed it!), and be set up to give you ample space.

    What would be the net result?

    • You and the "file administrator" would spend more time setting up files.
    • There are more opportunities for mistakes and mis-communications.
    • The files would be set up longer than needed, and possibly larger than needed.
    • They wouldn't be deleted automatically by your program — the file administrator would need to be notified.

    In other words, the situation would be worse in just about every possible way. More time, more chances of error, less automation, more resources used longer than necessary.

    Fortunately, this was a hypothetical for files. We have language-specific versions of "create file" and "delete file," so that applications can control their own lives, and programmers can program and be done with it.

    Unfortunately, this is daily reality for storage! Bummer! Why shouldn't applications be able to control storage the way they do files? This may not matter much for persistent storage that doesn't change much, but it matters hugely for storage that is used on a temporary basis. Most places that I've looked at don't think about it any more. Why should they? They just over-provision like crazy and be done with it. 

    This situation changes the second you've got a simple set of API's you can call from scripts or applications that do for storage what we already know how to do for files. This is a capability all storage vendors should provide. Can you guess which one of my favorite storage vendors has a real story on this subject? Good, check it out…

    Actually, just going to their website isn't enough (for now), since the RESTful API for storage administration is still in advanced beta. But it's real, it's cool, I've seen it, and they're even starting to admit its existence, per Brian Reagan:

    CTO Steve Sicola detailed the Q1 and Q2 roadmap, including CorteX
    (coming in Q1) – Xiotech’s RESTful API that will allow developers
    simple yet powerful access to ISE.

  • Old Nerds at Xiotech

    Xiotech has some amazing old nerds. See descriptions here and pictures here.

    Most nerds who entered some computer-oriented field in the 1960's (like I did) gradually figured out that being a nerd was the best way to do most of the work and get the least of the credit, money and appreciation. So they wised up, dressed up, cleaned up and moved up the management hierarchy.

    Of the remaining nerds, most kind of drifted along being role players, fulfilling some incredibly narrow function.

    But some nerds were so strong, so self-motivated and so, well, nerdy, that they marched along actually doing real work, solving new problems, and creating products that changed industries. They didn't deny their inner nerdiness. They were proud of it during the days when it wasn't something to be proud of.

    One of the many amazing things about Xiotech's CTO, Steve Sicola, is that he is a world-class "nerd herder." Now that he is aided and abetted by the terrific Xiotech CEO, Alan Atkinson, he has been able to attract an amazing group of wildly productive and experienced nerds.The ones pictured on the web site aren't the only ones, either! Just to name a couple, there is Todd Burkey, Randy Roberson, Marshall Midden, Mark Rustad, and even that isn't everyone.

    This isn't just of historical or sociological interest. Old nerds of this kind, when given resources and focused on a juicy business problem, can conceive and build better products, more quickly and efficiently, than whole departments of well-qualified engineers working at established companies. It's like everyone else has a bunch of beginners at chess, organized and led by a bunch of chess "managers" (what a concept!), while Xiotech has an elite team of energized and focused chess grand masters. The rest of the world is hopelessly outclassed.

  • Icrossing and the Google Chess Game

    In your battle for consumer attention, can you afford to be
    lost in the crowd of losers? Probably not; probably you should be talking with these guys. But read on…

    If you’ve got a web site, the one thing you need, above all
    else, is attention.

    • ·        
      You may want attention for its own sake if
      you’re writing a blog.
    • ·        
      You may want attention because you’re a media or
      content site and page views translate into sellable inventory that translate
      into ad sales.
    • ·        
      You may want attention because you’re selling something,
      directly or indirectly.

     You may not think about “attention” much. Instead you may be
    focused on brand awareness, conversion rates, the plummeting rates for your
    run-of-network inventory or something else. But the root of everything you
    think about starts from attention.
    You get none of the things you want unless the people who look, type and click
    make their way to your site. Unless consumers pay attention to you, you’ve got
    nothing that you want.

     To make it simple, I’m going to assume that you already have
    some attention. If you’re a first-time blogger, you’ve given your URL to
    your mother, your friends, and as much of your e-mail address book as you have
    the nerve to. If you’re a brick-and-mortar kind of place, you’ve got a URL that
    is close to the name consumers already know, and a bunch of consumers who
    already know who you are and are well-disposed towards you have found their
    way. If you’re pure e-commerce, you’ve got some customers who have bookmarked
    you.

     Given this assumption, the question becomes, how do you get
    more attention than you already have?

    Chess

     This brings us to Chess.

     Perhaps you’re already familiar with this board game and its
    arcane rules; for example, pawns can only move forward one square at a
    time, except on the first move, when they can move either one or two, and
    except that they may capture an opponent’s piece that is one square diagonally
    ahead of them. The rules for the other pieces are similarly weird.

    Chess is a famously challenging game that takes piles and
    piles of excess brain power to avoid embarrassment when playing against
    precocious teen-agers. It is a classic nerd-magnet.

    One of the things that famous chess players seem to like to
    do to demonstrate how awesomely great they are is to play
    simultaneous matches against many opponents.
    The grand master walks from one board to the next, glances at the move the
    opponent spent agonized minutes of intense concentration figuring out, makes
    his counter-move in insultingly few seconds, moves on to the next victim, and
    keeps circling until everyone is vanquished. The great chess master, endowed
    with an overabundance of overactive brain cells, single-handedly defeats a
    large crowd of pretenders, each of whom is probably good enough at chess to dispatch
    normal mortals without breaking a sweat.

    The participants in this kind of one-sided battle know
    they’re going to lose, though a few naïve ones may hope otherwise. But the ones
    who are check-mated in six moves feel like crap, and the few who are still
    alive and giving the master a match at the end feel like winners.

     

    Google, the gate-keeper of consumer attention

     This brings us to Google.

     You’re an eager aspirant for consumer attention. How do you
    get attention on the web? When people are looking for what you’ve got, how do
    they discover that you’ve got it?

     In many cases, they go to Google (or Yahoo, etc.) and type
    in something like what they want. Suppose they want to buy a cool sweater. They
    might type in “buy sweater” or just “sweater” and get a list of millions
    of sites that have something to do with sweaters. If your site sells sweaters,
    where are you on that list? You might be in the second hundred thousand, which
    is actually pretty good, because that might put you in the top couple of
    percent of sites that are relevant to sweaters. 
    Being in the top percent or two or your graduating class is a great
    achievement. However, being in the top percent or two of Google results is
    abject failure. No consumer will ever spend hours scrolling down to your
    listing, picking your site above endless pages of results that are higher in
    the list than yours. In fact, if you’re not on the first couple of pages, you
    can forget about consumer attention. You’d be better off standing with a big
    sign at the busiest local intersection you can find.

     Awful. What can you do about it?

     

    Google the Chess Master

     When a consumer types “sweater” into the search box, it is
    entirely Google’s business who it selects to display in what order in the
    results; unlike chess, the rules are made by Google, they’re secret, and they
    change the rules any time they feel like it. Google is like the brainy,
    arrogant chess grand master who likes to show off by playing a host of
    opponents at the same time, and always winning. Your job isn’t to “beat”
    Google, but to come out on the first page or two of the results list; in other
    words, to be less of a loser than everyone else who “plays against” Google.

     You make your “move” for top search results by carefully
    crafting your website for maximum “searchability” for the search terms that
    interest you the most. Google examines your “board” (web site) and everyone
    else’s Google is “playing against,” applies the latest version of its secret
    rules of the game, and decides the outcome. For most of the players, the result
    is always the same: Check Mate, you lose!

     No one can “beat” Google. Google controls all the boards,
    makes up the rules and decides the outcome based on its made-up rules. It's like "blindfold chess," only with Google, it's everyone but Google who wears blindfolds.

     

    iCrossing and SEO

    This brings us to iCrossing’s search practice.

    Lots of our
    companies depend on the web for their business, so naturally they get pretty
    good at playing against chess-grand-master-Google. For some of them, this skill
    is life-or-death. But even the best of them only have their own web sites to
    learn from, and have to focus on their business. What’s fun about iCrossing is
    that optimizing web sites for search is their business (at least an
    important part of it). So not only are they real good at it, they’ve developed
    a bunch of tools to find and fix problems, and monitor the results.

     If this sounds like an ad, I guess it kind of is, but not
    intentionally. After one of my visits with their search gurus, I really got
    off on what an ever-changing problem they’re solving. Then when the chess
    metaphor popped into my head, I just couldn’t drop it – it’s just so
    appropriate to the challenge everyone faces in getting visible on the web.

  • Paleolithic Mainframes Discovered Alive in Data Center!

    A recent article on forbes.com quotes me on what many people find to be the surprising longevity of mainframe computers.

    Don’t things in computing just get better and better – not to mention faster, smaller and less expensive? Which implies that after a few years of use, it’s just not worth keeping the old stuff around anymore? So we throw out (oops, please excuse me, we meticulously recycle…) the useless old stuff and bring in the cool, cost-effective new stuff, right?

    Like most common wisdom in modern computing, this contains elements of truth, but isn’t quite right.

    The element of truth behind this thought is the astounding continued progress of Moore’s Law, which posits that electronics gets smaller and faster at a rate that boggles the mind. This is what gives us iPhones and portable computers that have more speed and capacity than the room-sized mainframes of the past.

    But there is more to computing than electronics. First of all, there is this little matter of physical devices that have mass and inertia, that no amount of Moore’s-Law-driven advance will free from the tyranny of the laws of physics. This leads to growing storage problems that Moore’s Law actually makes worse. See here for a description of the issue.

    Second of all, there is this thing called software. Yes, software, the invisible-to-the-human-eye “stuff” that makes all that amazing electronics actually do something. Software is really hard, complicated stuff, like most things that are essentially mental, conceptual and invisible (think math). Once some software actually gets working well enough, sensible people are loath to change it. Even worse, the amazing increases in speed and capacity of electronics mask simply awful problems in software.

    Building most real, practical production software tends to be a nightmare that rarely ends. Re-building software that more-or-less works is a nightmare in hell that visits all the circles of hell in round-robin. So if the credit card companies can process their transactions, and the software that gets the job done happens to be written in totally-out-of-fashion-squared COBOL that runs best on a mainframe – that’s a great reason for IBM to build a new implementation of the mainframe instruction set out of modern electronics (thus getting most of the benefits of all the advances), just so it can run the code. It’s kind of like a horse and buggy built out of modern materials and powered by a fuel cell – it looks funny, but it’s modern and efficient and gets the job done.

    So, yes, the electronic part of computers get faster, better and cheaper. And the software seems to get better because it’s along for the ride, but it actually tends to get worse, which is why Paleolithic mainframes have been discovered, alive and working, in otherwise modern data centers.

  • Computer Storage’s Performance Gap

    There is a growing "performance gap" in computer storage due to basic physics, and exacerbated by server virtualization. People who run data centers see the problem, and so do their customers. It gets worse every year.

    Most storage vendors have no real solution to this problem. Xiotech, with its ISE product, is a notable exception. For this reason (among several others), Xiotech is one of my favorite companies.

    The Performance Gap

    In a nutshell, the "performance gap" is that as disk-based storage gets less expensive, the rate at which any one customer or user can get at their data gets worse and worse. Capacity goes up and performance goes down, resulting in a performance problem. Seems strange, doesn't it? Doesn't everything about computers get faster, smaller and cheaper? What's this about something getting worse all the time? Well, it's true.

    As everyone knows, computers get faster and less expensive. As they get faster, they read and write data more quickly. Storage that is built out of the same electronic "stuff" as the computer (RAM, DRAM, the "main memory" of the computer) gets faster and less expensive at pretty much the same rate as the processors. No problem there. But what about the disks (hard disk drives, HDD's)? Do they get faster and less expensive too?

    Well, that's the problem. They do get less expensive — that's part of why we have iPod's and digitial cameras now. But they do not get much faster. Mostly what happens is they hold more data and get physically smaller.

    A dozen years ago, you could have bought a HDD that had 10GB on it. Today, you can buy HDD's that are physically smaller that hold over 1TB. For less money!

    Smaller, more capacity, less expensive. What's wrong with that? Imagine that you have 1TB of data. A dozen years ago, you would have stored it (ignoring details like overhead and extra space) on 100 HDD's, each holding 10GB. Today you would only need one HDD — a 100:1 advantage! That's the good news.

    The bad news is that a dozen years ago, you would have had 100 HDD's, each with a head and data channel to read and write your data, while today you would have a single read/write head to access the same amount of data. This is 100 times worse than it was a dozen years ago! It's like Yankee Stadium with the same number of fans, only locking all the entry gates except one; do think there would be a line at that single door?

    If banks operated the same way with ATM's the way drive vendors do with HDD's, here's what it would be like. The bankers figure they're going to put a certain amount of cash in ATM's for the citizens of, say, New York City. Each year, new ATM's become available that can magically store a lot more cash in less space at lower cost. Of course they go to town, replacing each couple of old ATM's with one double-capacity ATM. They're feeling good about themselves — they've made the same amount of cash available to their customers while lowering their costs.

    Suppose that ten years ago, their were 100 ATM's in NYC. With this incredible technical growth in ATM's, the bankers can make the same amount of cash available using just one ATM — isn't that great?!

    Of course, the problem is obvious. People don't care about how many ATM's it takes — they care whether there's an ATM near them when they need cash, and whether there's a line of people waiting for access to that ATM. Imagine the impact of reducing the number of ATM's by a factor of 100. The same amount of cash is in the remaining ATM's as before, so the capacity is not reduced, just the number of access points.

    That is the performance gap: the same amount of "stuff" is crammed into a tiny fraction of the number of "boxes," but the "doors" to the few remaining boxes are no larger.

    Fundamentals: Physics vs. Electronics

    Hard Disk Drives (HDD's) contain one or more little platters (kind of like small CD's) that spin in a sealed enclosure. The platters spin, and as they pass by the read/write heads, data may be written or read. Here is a basic summary of HDD technology, with diagrams.

    Every year, vendors manage to make the little platters even smaller, make the read/write heads able to handle "bits" that are smaller and smaller. More data gets crammed into less space.

    The trouble is that the platter can't spin much faster than it already does. If the data you want is on the other side of the platter from the head, you still have to wait for the platter to spin around — and that takes the same time as it did when there was 100 times less data on that platter.

    In addition, platters (like CD's) put data everywhere, from the small inner part to the larger outer edges. With a single head to read or write, the head still has to be moved to the right place (this is called "seek time"), and that movement isn't much faster than it was years ago. 

    Finally when you've got the "ATM" problem or the "Yankee Stadium" problem, the problem gets really bad — the chances that your data is on the same platter as someone else who wants their data at the same time has gotten 100 times worse.

    As the electronics of shrinking bits gets better, the performance gap gets worse, because physics doesn't let us move things or spin things way faster, and the simple arithmetic of cramming more data through the same-size door just blocks us.

    Server Virtualization

    Server virtualization is a major trend in data centers. It basically enables data centers to reduce the number of servers they use by using virtual machines instead of physical machines. With the help of software, applications that used to require dedicated machines can share a smaller number of physical machines. 

    This is generally a good idea and saves money. Except for this little problem about storage. By putting multiple applications on each server, the number of read/write requests coming out of each server has just gotten larger. Which makes the storage performance even worse. Ughh.

    Conclusion

    Advances in storage technology are paradoxically making storage performance worse. Server virtualization, while generally a good thing, exacerbates the problem. How do you solve the performance gap? There are several major ways.

    • Buy enough HDD's to give you the performance you need. This may mean that you are way over-capacity, but who cares? Your users aren't trying to wring your neck because they can't get their data. And the extra money all that costs? Well, that's life.
    • Pay 10 to 20 times more per GB and buy solid-state storage (SSD's) — and change your applications to take advantage of it. Problem solved! The extra money and trouble? Well, that's life.
    • Buy ISE's from Xiotech. Buy only the capacity you need. Your users will love you and you won't have to touch your applications to make it work. The extra money and trouble? There is no extra money or trouble. That's life — the good life, that is.

    Am I proud to be associated with Xiotech? You betcha.

  • Xiotech’s ISE: A Revolution in Storage

    Xiotech is leading a revolution in storage: all the leading vendors are selling the storage equivalent of mainframes, while Xiotech is building the storage equivalent of blade servers. I'm proud to be associated with them.

    Xiotech calls its "storage blade" an ISE — intelligent storage element. Insiders frequently call it "the brick." I’ve
    tried to come up with the simplest possible thoughts for explaining an
    all-brick approach to storage.


    All
    SAN’s today are built from two basic things:

    1. Drive bays or trays. These are the units that actually
      hold all the data, and where all the drives are. They include power
      supplies, data connectors and logic boards to perform basic functions.
    2. Controllers. This is where the “intelligence” of the
      SAN is. The controllers are connected on the “back” side to the drive
      bays, and on the “front” side to servers and/or switches. As far as the
      servers are concerned, the controllers are the storage. They get
      requests from the servers and satisfy those requests by getting data from
      various drive bays. Data goes from server to controller to disk when
      written, and from disk to controller to server when read.

     

    The
    Xiotech innovation with the ISE is to build a new-age drive bay that contains
    all the essential functions of the controller, which are:

      • read and write my data;
      • don’t make me worry about individual disks;
      • keep it safe, even if a disk breaks

     

    So
    first of all, the Xiotech ISE is a drive bay with built-in essential SAN
    functions. But why stop there? If you’re going to concentrate on basics, really
    do it right. So the ISE goes way beyond SAN controllers and drive bays
    by adding these functions:

     

      • read and write my data really quickly
      • don’t make me become an expert to get great results
      • don’t charge too much
      • it won’t break, so don’t charge me for maintenance.

     

    It’s
    important to understand is that the Xiotech ISE looks like a drive bay,
    except:

    • It fits in 30% less space
    • It uses less power
    • It requires no maintenance
    • It performs better than other systems because of novel,
      built-in virtualization

     

    This
    supercharged drive bay, the ISE, gives you everything you need from a SAN, without
    a controller. Why is this good? Because dropping the controller:

    • Saves money
    • Results in a linearly scalable system
    • Eliminates the drag of features you don’t need

     

    Do
    some people really need controllers? Yes. There are some functions that are
    performed very well in controllers, and for those functions Xiotech has an
    excellent controller, and others are available from third parties.

     

    Do
    you need a controller? Here are the reasons you may not:

    • You use a server-based LVM to perform higher level
      storage functions
    • You want your application to control its own storage
    • You use blade servers, and would like the equivalent of
      blades for storage
    • You’re very concerned about performance, and you need
      linear scaling
    • You want to avoid storage experts and getting into the
      whole world of storage; you just want fast, reliable storage
    • You know about server virtualization, and you want
      storage that “fits in” to that style of computing
    • You’re cost-conscious about storage, including
      operating costs and maintenance

     

    Xiotech
    is pioneering a new approach to storage – what server farms are for servers,
    Xiotech ISE’s are for storage.Obviously, I think it's a winner.

  • Xiotech Nerdfest

    I just attended the Xiotech Nerdfest — an unqualified success!

    What an idea — get all the nerds in the company together to geek out. Everyone, the field architects, the customer service people, everyone who wants or needs to know how the innovative Xiotech storage products work.

    It's a great model for tech-centric companies to follow.

    Here is an insider posting about it:

    http://pjselin.wordpress.com/2010/01/20/nerdfest-2010-day-3-2/

  • Introduction

    When I was in college, one of my passions was the music of Franz Liszt. Not that I could play it — way too many notes. But I still loved it. So I created and hosted a weekly radio show about his music called Black-Liszt. Pretty obvious, and pretty punny, which increased its appeal.

    All these years later, I'm still into computers, still into Liszt, Liszt still has way too many notes, I still use way too many words, and I still haven't "outgrown" puns — so what better title for a random list of thoughts by me?

Links

Recent Posts

Categories