I am sitting at McCarran airport waiting to board my flight back from TDWI, and am thinking about one trend I noticed at the vendor exhibits: there is a growing set of vendors selling high-performance columnar-based database systems. Interestingly, the common denominator is the positioning of the software as a means for virtualizing a data warehouse appliance.

Orienting the data in a columnar manner is nicely suited to analytic applications, so the clear opportunities for these kinds of products are partnered solution providers for specific types of analytics, or with data aggregators and providers to allow for data linkage and then analysis.

Some of the vendors (or vendor reps) I bumped into over the past few days include ParAccel, Vertica, Sybase IQ, Infobright. Kognitio, alternatively, is not columnar but through data distribution across parallel systems can also talk the virtual appliance talk.

One conclusion that can be drawn is an emerging market for providing high performance analytics platforms with a low barrier to entry points towards cracking open that small.medium business market. One interesting thing to watch is the ways these guys will partner with other BI vendors (e.g., OLAP, visualization, end-user analytics) to see who can put together a robust end-to-end BI solution suitably priced for the $50-$100 million company.

One of the hazards of advocating techniques intended to improve business through better customer insight is the occasional question of faith: does a good business intelligence strategy and program necessarily equate to greater profits? Sometimes I wonder: if customer analysis and predictive analytic techniques work so well, then one who is knowledgeable in the area should be able to apply the ideas directly to his/her own business, right? Isn't this just another example of eating one's own dog food?

Here is what I mean: using our business intelligence and data analysis and data mining and predictive analytics, we claim that we can increase response, reduce costs, extend customer lifetimes, improve lifetime values, etc. So as an experiment, I should be able to start a retail business and accumulate a bunch of customers who will always be satisfied, will never threaten to cancel their service, and will always be just about to buy the products I have already determined they need. They will each be at the center of a huge sphere of influence, and I will exploit the viral marketing opportunities by turning every satisfied customer into a walking advertisement for my products and services. I will have optimized my product and service offerings so that as one product becomes obsolete, the customer is dying to upgrade to the next level, and I will time their releases so that no follow-on product cannibalizes its predecessors' sales.

The idea intrigues me: pick a product or service to sell and then apply the performance improvement techniques driven by busiess intelligence. Some thoughts:

I would want to pick a business that is recession-proof (plumber? pest exterminator? funeral director?).

I would have to sell a product that needs updating or replacement within a relatively short cycle. Selling replacement windows is probably out. Selling office supplies is more like it.

You get the picture: a broad market where some knowledge of the customer community can drive repeatable sales, and where customer data is easy to get, maintain, enhance, analyze, and exploit.

There are a lot of success stories out there for applications of BI to business productivity improvement. Yet that is not true across the board, and that probably means that owning the software doesn't necessarily imply achieving the benefits without a little hard work. Ultimately, the successful organizations exploit BI by adapting their business processes to exploit the the knowledge discovered, and put practices in place to measure the value of each decision. Maybe that is what drives the belief in BI?

According to an article on a recent IDC report, small and medium businesses are rapidly making up a large piece of the business intelligence tools market. Adoption of BI tools by SMBs increased 40% in 2007.

Yet the increase in adoption of tools doesn't necessarily mean that there is an increase in value; it would be interesting to see some report documenting how SMBs adapt the way they do business as a result of integrating BI into their operations.