Blog: David Loshin Subscribe to this blog's RSS feed!

David Loshin

Welcome to my BeyeNETWORK Blog. This is going to be the place for us to exchange thoughts, ideas and opinions on all aspects of the information quality and data integration world. I intend this to be a forum for discussing changes in the industry, as well as how external forces influence the way we treat our information asset. The value of the blog will be greatly enhanced by your participation! I intend to introduce controversial topics here, and I fully expect that reader input will "spice it up." Here we will share ideas, vendor and client updates, problems, questions and, most importantly, your reactions. So keep coming back each week to see what is new on our Blog!

About the author >

David is the President of Knowledge Integrity, Inc., a consulting and development company focusing on customized information management solutions including information quality solutions consulting, information quality training and business rules solutions. Loshin is the author of The Practitioner's Guide to Data Quality Improvement, Master Data Management, Enterprise Knowledge Management: The Data Quality Approach and Business Intelligence: The Savvy Manager's Guide. He is a frequent speaker on maximizing the value of information. David can be reached at or at (301) 754-6350.

Editor's Note: More articles and resources are available in David's BeyeNETWORK Expert Channel. Be sure to visit today!

Recently in Cutting Edge Category

Strolling around the exhibit floor at the TDWI conference in Chicago the past few days provided an interesting look into a rapidly evolving trend in data warehousing applicances. Of the 30 or so vendors exhibiting, I counted at least 7 that would be considered appliance vendors:


I might throw Oracle, HP, and Sand Technology in there as well, but I think you see my point - there seems to be the perception that there is a market for high performance "plug-in" systems to deploy data warehouses. What is perhaps even more interesting is that half of these vendor offerings are not specifically hardware appliances, but rather software database systems that can be deployed on top of different hardware systems - in other words, they are "software appliances" (!?)

In essence, many of these approaches, along with some from other vendors as well (Vertica was notably absent from this crowd, but showed up at the previous Las Vegas TDWI) focus on structural optimizations (such as columnar-oriented databases) that are very well-suited for loading into core memory and providing very fast read access, making it especially nice for query/reporting clients. The realization that the database system can be optimized and parallelized in a way that is decoupled from the hardware makes these software-only approaches look very cost-effective, especially when considering sizing a warehouse to meet current needs while considering future growth. Not only that, these systems are finely tuned for performance, (see Mark Madsen's comments about ParAccel's TPC-H benchmark scores).

The common theme with the software appliance crowd is lowering the barrier of entry to Small/Medium businesses seeking to jump on the BI bandwagon. WIth a variety of operational modes that span full-blown deployments (with hardwre purchase and integration) down to a service-based hosted model, this platform enables data warehousing at a fraction of the cost. This concept in its own right is worth some more exploration, and I think I may try to address that in an upcoming column.

Posted May 15, 2008 11:40 AM
Permalink | No Comments |

We are currently updating our company web site, and I am extremely impressed with the ways that emerging blogging tools are able to solve certain "challenges" associated with managing a web site's content. I am planning to put together a new web site to accompany my MDM book and I am also thinking that blogging is the way to go.

Check out Movable Type and Wordpress - pretty impressive. The software is pretty cool, provides all sorts of widgets and plug ins, and makes life a lot easier for keeping a web site fresh.

Posted May 12, 2008 8:07 AM
Permalink | No Comments |

This past week I attended TDWI, and was lucky enough to conduct a podcast interview with Philip Russom, TDWI's Senior Manager of Research and Services. In our conversation, we hit upon a number of interesting topics regarding upcoming trends in business intelligence, but most interesting was his opinion about the integration of search technology into BI in a way that is likely to change the way we think about discovering actionable knowledge.

Posted May 18, 2007 6:21 AM
Permalink | No Comments |

Over the past few days, news items such as this one have told the story of Terry Wallis, a man who, 19 years after entering a comatose state of minimal consciousness, emerged from that coma as his brain spontaneously rewired itself back into awareness. Cells in undamaged areas had formed new axons, establishing new connectivity around the damaged parts of his brain.

Two quick questions to ponder: Is this an unique event, with little expectation of recurrence? And how does this natural occurrence compare to computational self-organization?

The answer to the first question is that while this rewiring is rare, there are other documented occurrences, which may introduce some hope that scientists can better understand the process by which this phenomenon occurs, and whether it can assisted externally.

The answer to the second question does hold some challenges computationally. The brain rewiring is reminiscent of self-organizing programs, and even neural networks, which are programs intended to recreate the network structure of intelligence as part of an Artificial Intelligence initiative. Understanding what the brain can do naturally may help in exploring other ways to lead to automated "thinking." Embedding these kinds of utilities/services/agents within operational applications could be just one more way to incorporate business intelligence into ongoing business processes.

Any comments?

Posted July 5, 2006 7:34 AM
Permalink | 1 Comment |

I had heard about this overloaded use of technology a few years back, but a few conversations last week at TDWI reminded me of an interesting exploitation of one technology to provide a service completely unrelated to the technology's original intent. This article from last October describes how one can use cell phone monitoring to track traffic patterns through highly-traveled routes.

Basically, the way mobile phones work is that they transmit and receive signals from particular antenna towers scattered across the region. As you are traveling, the mobile phone sends a message to establish its appearance within the nearest tower's range. As your phone leaves that tower's area, it will connect with another tower. As you can guess, one can calculate relative rates of speed by looking at the timestamps at which the same phone registers itself with a series of towers. When the reference space includes towers along an interstate highway, averaging those durations over a number of cell phones allows one to get an idea of how fast traffic is moving along different sections of the highway. That information can be routed back to subscribers (individuals or even news sites) to help in relieving congestion.

According to the article, a number of localities have, or are interested in deploying these kinds of systems. Is this a public service piggy-backed on harmless data collection, or a potential invasion of privacy... let me know what you think?

Posted May 22, 2006 11:39 AM
Permalink | No Comments |

Last week, Google announced that it was giving away its web analytics product as part of its online services. Today, they announced that they were freezing the application to new users, citing "high demand." Of course, any large scale demand for a free service is likely to overwhelm a system, but isn't it great that there is so much interest in an analytics application?

Think about how starved people are for quality quantitative information. This can only be good news for companies that develop more intriguing interfaces for analyzing data and presenting the results of that analysis in creative and insightful ways.

Posted November 30, 2005 7:41 PM
Permalink | No Comments |