The Information Quality Revolution--Tipping Point Just Around the Corner

Fixing Broken Election Processes: An Information Quality Mandate

After the 2006 primary elections in the United States, a local newspaper article headline read, ?Computer ballots in stage of ?trial and error.?? Although elections are critical in a democratic society, electronic voting that isn't transparent has been introduced with a trial-and-error approach instead of a proven process improvement approach.

After the 2000 U.S. presidential election, the ?Help America Vote Act? (HAVA) election reform was intended to improve the election processes. Its requirements to replace older voting technologies with information-age electronic voting machines have introduced negative side effects and increased known problems.

The 14 Points of Information Quality Transformation

Reversing the Broken Healthcare Medication Processes

DM Review, October 2006

When we go to our doctor or the hospital, we expect to receive medical care that makes us better.  But the physician?s mantra, ?Do no harm,? is fiction in our nation?s current healthcare system, with patients suffering 1.5 million injuries each year caused by ?preventable medication errors? with hospital patients subjected to one injurious medication error each day, according to the recently released Institute of Medicine?s study Preventing Medication Errors.[i]  These figures do not even count ?wrong-time? medication errors or omissions?the ?failure to prescribe medications for which there is an evidence base for the ability to reduce morbidity and [mortality].?  Medication errors in hospitals cost $3.5 billion per year in extra hospital time and treatment for adverse drug injuries, not counting lost productivity for extended hospital stay and recovery time.  The worldwide toll is far greater, at significant cost to healthcare providers as well as to the economy, but with worst-case consequences of injury or death and grief for families.

---

[i] Philip Aspden, et al, ed., ?Preventing Medication Errors.? Washington: The National Academies Press, 2006, p. 3.

Can We Learn From Our Mistakes?

DM Review, September 2006

First, I would like to thank DM Review for allowing me to take off a few months from my monthly column.  I would also like to thank my readers who noticed and emailed me.  I needed a brief sabbatical in a busy time.

My office is going through its summer office clean-up, and I have been reviewing files for what can be recycled and which still has value.  One such file was the ?Y2K? file.  As I was looking over the nearly 400 articles, I contemplated the process improvement cycle of ?Plan-Do-Study-Act.?  I reflected on the ?Study? phase which suggests that we study what we have learned from the global Y2K remediation initiatives.

Mea Culpa: What Do You Do When Mistakes Happen?

White Paper: Total Information Quality Management: A Complete Methodology for IQ Managment

DM Review, Cover Story

A colleague of mine recently told me he has gone to work for a consulting firm on an IQ project with a client where they were just ??playing? at data quality,? not really ?doing? it.  As he shared some details, we both agreed it was a sad state of affairs.  This article is about what is required to really ?DO? information quality and describes a methodology for how to do it.

The Economics of Information Quality

Information Quality Newsletter, B-Eye Network, May, 2006,

The Economics of Information Quality, explains that the real value proposition for information quality is when we prevent the creation of defective information that causes processes to fail. An order fulfillment process example is used to illustrate the economics of process improvement. This illustrates the costs of ownership of poor quality information compared with the cost of process improvement to eliminate IQ problems.

IQ and Data Warehousing: Trends and Best Practices

DM Review, April 2006

The February 15^th Anniversary issue of DM Review celebrated ?15 Years of Excellence? with a description of developments and challenges in data warehousing.  This article continues with a brief review of information quality in data warehousing and trends that are increasing business intelligence effectiveness through proactive information quality.

Information Quality in Reference Data

DM Review, March 2006

Reference data is one ?classification? of information that must be zero defect.  The term ?reference data? is actually a poor term relative to the importance of the information represented.  First the term represents a data-centric view rather than a business-centric view.  Reference data is data in one file that is ?referenced? by records in other files.  But the business significance is that reference data makes important classifications of objects and events of interest to the enterprise.

15 Years of Excellence

DM Review, February 2006

In honor of the 15th anniversary of DM Review magazine, the editorial staff interviewed several thought leaders who have history with DM Review to share their knowledge and insight about the technology industry in general and DM Review, specifically.  This is the full interview with Larry English that was not fully published in DM Review's issue.

Remembering Peter Drucker: Father of Modern Management

BI Report, Printed from DM Review.com, January 2006

Peter F. Drucker passed away in November at age 95.  I am so thankful I had the benefit of his education as a young manager.  His books and writings have taught me valuable insights from the man who invented management.

IAIDQ Principles Work Team: Call for Participation

DM Review, December, 2005

You are invited to join one of the most exciting opportunities in the development of Information Quality as a professional discipline.  After a year of administrative and foundational work, the International Association for Information and Data Quality (IAIDQ) has begun to formulate and develop a variety of important Work Teams to develop key resources for members.

When Is A Horse A House Boat? IQ And Data Overloading

DM Review, November 2005 

When is a Horse a House Boat?  Whenever you need it to be, of course, especially when you do not have a field defined for Horse in your database, but you need to know about it.

IQ and ?Muda:? Information Quality as Eliminating Waste

DM Review, September 2005

• One bank had to scrap a $29 million Data Warehouse project because their original design was faulty.  They had not designed the data acquisition process to capture data at the original sources, nor had they addressed the quality of information to support the Data Warehouse knowledge workers. 
• Another bank lost over $200 million in default loans where they approved credit based on faulty credit scores.
• Yet another bank lost $600 million as the result of misinterpretation of a risk code, when the company they invested in failed.

These examples of waste represent one of the primary objectives of Information Quality?to eliminate waste.  The other key objective of IQ, as with any valid quality system, is to increase customer satisfaction. 

 

IQ Problems are Pervasive--and Wasteful

DM Review, August 2005

Very few weeks go by during which I encounter NO information quality problems in the course of daily life.  Virtually all of them waste time and money.  Just this week alone, I had the following happen (the names are fictional; the facts are true)

Software Packages and Information Quality

DM Review, July 2005

 

Application software packages continue to be a ?standard approach? in many organizations for delivery of information system solutions (!?!?).  Ian Sinclair wrote the following reply to my October, 2003 column, ?How Best-of-Breed Software Selection Causes IQ Problems,? while he was with the UK Ministry of Defence [British spelling] (MoD).  I have only added clarifying wording with IQ observations at the end.

Business Intelligence Defined

Information Quality Newsletter / B-EYE-Network, 2nd Quarter 2005

Just what is ?Business Intelligence? anyway?  Among other things, I believe Business Intelligence has to do with using information to do the right things, such as giving a formulation of medication to a patient, and to make the right decisions, such as making an investment in a company.

However, in the first example, the patient died, because of misunderstanding of the procedure to formulate the medication mixture.  In the second example, a bank lost nearly 600 million dollars because of non-standardized ?risk? codes.

Not exactly examples of Business Intelligence, are they?

The Silent Scream of Numbers

DM Review, June 2005

In my March column I reported on the election reform movement that sprang out of problems observed in the 2004 presidential election.  As a result, a group called Gathering To Save Our Democracy organized a conference in Nashville called the ?National Conference on Election Reform? held in Nashville.  I was honored to be invited to speak on how quality principles must be applied to the election processes to produce viable and sustainable election reform.  Statisticians, computer scientists and election workers and concerned citizens shared analyses of data that revealed significant problems. 

Robert Koehler, an award-winning journalist and an editor at Tribune Media, and nationally syndicated writer, captured the essence of the conference in this Guest Editorial, ?The Silent Scream of Numbers.?  Robert has given me permission to publish it here. 

Don?t Speak in Pronouns: Avoiding Ambiguity

DM Review, April 2005

To assure high information quality, one must avoid ambiguity in both information collection and in information presentation.

Information quality is not just about quality of data in our databases, it is about quality of all of our communication, spoken, written and displayed.  It is a concept Diane and I call ?Don?t Speak in Pronouns.?  It comes from when we have analyzed the root cause of some of our moments of miscommunication.

IQ in the News: Lessons Learned

DM Review, April 2005

Hardly a day goes by that you cannot find at least one news story of problems caused by poor quality information.  Here are only two examples from late January and their lessons learned. 

?Some W-2?s go to wrong people.?  No problem, we?ll just re-mail 53,000 W-2 forms?after all, it?s just tax money.

Some Tennesseans must have thought they had won the lottery when they received their W-2 form with someone else?s data in it, while others simply gasped at the pay ?cut? they suddenly found in their W-2.  The State of Tennessee had a ?machinery error,? according to a State spokesperson.  She speculated it affected only a small number of state employees, saying the mix-ups were ?somewhat isolated because they occurred only between employees who live within the same ZIP codes.?

Isolated or not, the state had to resend new W-2 forms to every one of its 53,000 employees.  A board member of the Tennessee State Employees Association shrugged it off, saying, ?mistakes happen.  I?m sure this wasn?t intentional.?[i]

---

[i] Trent Seibert, ?Some W-2?s go to wrong people,? The Tennessean, January 26, 2005, p. 2A.

Election 2004: IQ is Now on the Map

DM Review, March 2005

History was made January 6, 2005.  I had the privilege to be there and be part of it. 

For the first time in modern history, the US House of Representatives and Senate joined together to ?object? to the outcome of a state Presidential Election result because of significant irregularities and questions about the integrity of the election process and vote results.  Representative Stephanie Tubbs Jones from Ohio and Senator Barbara Boxer from California raised this objection.

My study of the problems documented in numerous states revealed that the problems in this election were just as prevalent and just as significant as they were in the 2000 election.

Business Intelligence Without Information Quality??NOT!!!

Information Quality Newsletter / B-EYE-Network, 1st Quarter 2005

Welcome to this first edition of The High IQ Organization Newsletter.  I am pleased to partner with B-EYE Network to provide a quarterly update of tips and techniques for organizations to increase their effectiveness as ?intelligent learning organizations? through sound information quality management principles and practices.

The Intelligent Learning Organization

As we move further into the Information Age, or the ?Knowledge Worker Age,? as Stephen Covey describes it in his new book, The Eighth Habit, we information professionals must increase our enterprise?s ability to exploit its information and knowledge resources.  Covey describes how we move from effectiveness to greatness by applying the eighth habit, which is not a new habit. But a discipline of finding our voice and helping others find theirs.

The 7 Habits of Highly Effective Information Professionals: Habit 6: Synergize

DM Review, February 2005

The ninth, but last, in a series of 10 columns on Stephen Covey?s The 7 Habits of Highly Effective People and what they mean for information professionals.

Synergy, the 6^th Habit, is very much like the musical term ?ensemble,? which means to play together as a single entity.  In my early school life I was a musician, I playing trumpet in three different Texas symphony orchestras.  The first professional orchestra I played in, the San Angelo Symphony Orchestra, was a thrilling experience for me as a high school sophomore.  For our December concert we performed Tchaikovsky?s Nutcracker Suite.  During the first rehearsal I was the only trumpet player there, so I paid close attention as our Maestro described what he wanted in the music.  He raised his baton, gave the downbeat, and I started playing.  But before we reached the end of the first measure of the music, he stopped the orchestra.  He stepped off the podium, walked over to me, and with his baton he tapped a word on my music score.  That word was ?tacit.?  Tacit means ?silence.?  While everyone was playing the first movement of the Nutcracker Suite, I was playing the second movement!!!  Not exactly ?ensemble.? 

Nor was it ?synergy,? the 6^th Habit of highly effective people and the subject of this column.

The 7 Habits of Highly Effective Information Professionals: Habit 7: Sharpen the Saw

 

DM Review, January 2005

 

Tenth in a series of articles of how Stephen Covey?s The 7 Habits of Highly Effective People apply for information professionals.

We start this mid-point year of the first decade of the third millennium focusing on the seventh of The 7 Habits of Highly Effective Information Professionals, begun last January.  But this is not a ?conclusion,? but that it is a continuation of your?and my?journey to become more effective in accomplishing our respective goals, to add more value to our customers, to be more effective agents of change, and to better be able to bring our organizations into the realized Information Age.  If you have just joined us in this, I encourage you to go back to the previous issues or visit DM Review online for the archived issues.

The 7 Habits of Highly Effective Information Professionals: Habit 5: Seek First to Understand ? Then to Be Understood

 

DM Review, November, 2004

 

Eighth in a series of articles of Stephen Covey?s The 7 Habits of Highly Effective People and what they mean for information professionals.

 

One of the hazards information professionals face is the constant demand to keep up with the technology advances.  Major technology innovations occur at a rate of one each 18 months, much faster than can be learned, assimilated and exploited effectively.

But the real problem is that we can become so enamored with our technology, we can lose sight of the very knowledge workers we must serve with our technology.  It is easy to become more focused on ?debugging? the defects (nonquality) in the technology than in sitting down to seek to understand not just our knowledge workers? information requirements, but also to understand the problems they face in accomplishing their value work.

Six Sigma and Total Information Quality Management (TIQM)

 DM Review, Feature Article, October, 2004

 

Sigma has gained popularity recently as it has achieved dramatic benefits in organizations who have understood and implemented its rigorous processes.  Some have asked me what are the differences in a Six-Sigma system approach to Information Quality (IQ) and TIQM^®, the methodology described in Improving Data Warehouse and Business Information Quality (IDW&BIQ).

The 7 Habits of Highly Effective Information Professionals: Habit 4: Think Win/Win

DM Review, August and September, 2004

The sixth and seventh in a series of articles of Stephen Covey?s The 7 Habits of Highly Effective People and what they mean for information professionals.  Habit 4, ?Think Win/Win? is vital to every Information Professional who seeks enterprise information resource management, value chain optimization or information quality.

Mastery of the first three habits (1. Be Proactive, 2. Begin with the End in Mind, and 3. Put First Things First) gets you to independence, what Covey calls ?private victory.?  This puts us in control of ourselves and enables us to be effective with our inner life.  But we do not live life in a vacuum.  We need more to be effective in our inter-personal relationships.

The 7 Habits of Highly Effective Information Professionals: Habit 3: Put First Things First

DM Review, July, 2004

This is the fifth in a series of articles of Stephen Covey?s The 7 Habits of Highly Effective People.  Here we describe Habit 3, ?Put First Things First? and what that means for information professionals. 

Habit 3 is about time management; but it is much more.  Habit 3 is really about ?self-management.?  Effective self-management depends on Habits 1 (Be Proactive) and 2 (Begin with the End in Mind). 

In Memoriam: A Tribute to An Information Quality Innovator

DM Review, June, 2004

America has lost a true Patriot and an Information Quality innovator in the untimely death of Rev. Athan L. Gibbs, Sr. (August 30, 1946 ? March 12, 2004). 

The 7 Habits of Highly Effective Information Professionals: Habit 2: Begin with the End in Mind

 

DM Review, May 2004

Lewis Carroll?s marvelous story, Alice in Wonderland, contains an insightful truth in Alice?s dialog with the Cheshire Cat.  It goes like this:

?Would you tell me, please, which way I ought to walk from here?? asked Alice.

?That depends a good deal on where you want to get to,? said the Cat.

?I don?t much care where,? said Alice.

?Then it doesn?t matter which way you walk,? said the Cat.

The lesson here is that if we do not have a clear vision of the outcomes we desire from an information project or initiative, we may wander aimlessly getting to precisely where we end up.  Project plans may end up filled with activities that take our time, but do not produce results.  We may toil forever, without ever accomplishing our mission.

Habit 2, ?Begin with the end in mind,? is a required habit for information professionals to be able to visualize the end result for information excellence.

The 7 Habits of Highly Effective Information Professionals: Habit 1: Be Proactive

DM Review, March 2004

This is the third in a series of articles of Stephen Covey?s The 7 Habits of Highly Effective People.  In the last two columns we provided an introduction and an overview of the 7 Habits.  Here we describe the first Habit, ?Be Proactive? and what that means for information professionals.

Because Information Resource Management and Information Quality Management are disciplines that transform how business and systems activities are performed, being proactive is essential.  Both of these disciplines require change agents to lead them.  One cannot be a change agent without being proactive.

The 7 Habits of Highly Effective Information Professionals: Part Two

DM Review, February 2004

Last month we began a series in which I will explore Stephen Covey?s The 7 Habits of Highly Effective People.  In it I described the ?principles? behind effectiveness.  Here we give an overview of the 7 Habits as enabling traits of effective Information Professionals. 

The 7 Habits of Highly Effective Information Professionals: Part One

DM Review, January 2004

These habits are vital for any person to attain fulfillment in life, regardless of whether a CEO of a Fortune 500 company, a custodian of a public school or an information professional.  It is especially important of those of us whose ?end? products are not really ends or products in themselves.  Data models, database designs, application systems, IQ assessments or process improvements are but means to a greater end?business effectiveness. 

This month?s column introduces a multi-part series.  It may be punctuated from time-to-time with commentary on topics of temporal importance.

The Gift That Keeps On Giving

DM Review, December 2003

As we approach the holidays, our thoughts tend to focus on others.  We shop for gifts to give our loved ones.  I do not remember where the saying, ?the gift that keeps on giving? originated from.  But from the various gifts given during the holiday season, some are very transient, such as food that, once consumed, is gone.  Some are very meaningful and will be remembered for a lifetime because they touch an emotional or symbolic need of the recipient.  And still others ?keep on giving? because they grow in value, as in gifts of financial instruments that provide financial interest that continues to increase over time. 

How Best-of-Breed Software Selection Causes IQ Problems

DM Review, October 2003

Evaluating and Selection Software based on "Best-of-Breed" criteria has been given a lot of hype, and is touted as a best practice.

But is it, really? Best-of-breed is supposedly "what is the best product" in a given category of software. But, by whose standards? The popular belief is if we buy the "Best-of-breed" product in each category, then we have the best-of-all-worlds software environment.

Total Information Quality Management: A Complete Methodology for IQ Management

http://www.articles/2003-09 Methodology.pdf

DM Review, September 2003

A colleague of mine recently told me he has gone to work for a consulting firm on an IQ project with a client where they were just "'playing' at data quality," not really "doing" it. As he shared some details, we both agreed it was a sad state of affairs. This article is about what is required to really "DO" information quality and describes a methodology for how to do it.

IQ and Personal Integrity

DM Review, August 2003

In the wake of the separate collapses of Enron and Andersen and other corporate financial shenanigans, we are now caught up in other ethical scandals, such as Sam Waksal's conviction for insider trading, and former New York Times reporter Jayson Blair's "creative" reporting that contradicts the code of ethics for honest journalism.

All of these examples illustrate nonquality information and the ultimate costs of nonquality.

What does this have to say about those who aspire to a career in Information Quality?

Defining and Measuring Accuracy

DM Review, July 2003

Many people have an INACCURATE understanding of the IQ characteristic of "accuracy" and how to measure it. Not just practitioners, but also consultants, authors and teachers who write and teach about it. This article addresses how to: 1) define accuracy, 2) measure accuracy, 3) design accuracy measurement tests, and 4) solve accuracy problems.

How to Save $576,925,000 Through IQ Management

DM Review, June 2003

So your management is asking you:

• What is the ROI of Information Quality Management?
• Who is "doing" Information Quality well? (What benchmarks?)

Now you can tell them! Here I share: 1) the real savings leading edge organizations have achieved and documented through IQ management; 2) what these organizations did to accomplish them; 3) how you can measure costs of nonquality; and 4) the IQ Money-Back Guarantee.

Data Format Standards as an IQ Tool

DM Review, May 2003

Thanks to a reader for inspiring this month's column. She asked about the meaning of the terms "standardization" and "remediation" as applied to the work of creating data format standards for the data element "Organization Name."

Information Quality Professional Association Survey

DM Review, April 2003

As you know, most professionals are supported by professional societies. There is currently no such society that directly supports data/information quality professionals. Tom Redman and I have come to believe that such a society would be very beneficial, and together we are conducting a survey to determine interest, solicit interest, and seek out those who would help found such a professional society.

We are interested in your opinion. We do not wish for this organization to compete with any existing professional organization, but to establish a professional organization to support those interested in applying quality management principles to data and information to achieve business performance excellence.

Check Digits as a Tool for IQ

DM Review, March 2003

Last week I had my hearing checked as part of my routine medical check-up. My doctor had referred me to an audiologist, and his office called to make an appointment for me. Being my first visit to this office, I used my car navigation system to guide me and found the office without a problem.

However, on checking in I found they had no a record of my appointment. I confirmed the address; it was correct. Here I had a not uncommon information quality problem--"an appointment 'mix-up'." So I started my stopwatch that I use to measure wasted time due to nonquality information. The receptionist conferred with the audiologist about the problem. Six minutes and 23 seconds later they came out to notify me that they found "the problem."

The Information Quality Act: Mandate for IQ

DM Review, February 2003

Will Information Quality Be Legislated In?

Organizations must either provide quality information expected by their customers or run the risk of legislation that forces them to provide such quality. Legislation-forced quality to meet customers' needs usually costs producer organizations more and generally produces far less desirable results to the consumers.

New Year; New Name; New Resolve for High IQ

DM Review, January 2003

Welcome to the New Year. It is hard for me to believe that this issue begins the Seventh year of my column. We begin this Sabbatical Year with a new name, "Plain English About Information Quality." This issue I write of why we changed the name and to provide a brief "State of IQ."

How Process Management Improves Information Quality

DM Review, December 2002

Process management is a requirement for sustaining an environment that consistently produces quality information. Process management goes beyond process control, that is, consistently produces products whose variation is within defined specifications.

Process management requires a focus on the customers of the products produced by the process. It requires the definition of product specifications that meet end customer requirements, not just the specifications that may or may not meet customer needs.

Process as used here refers not just to single activity processes (input-process-output), but the entire value chain from the customer request (for a product) to the customer benefit (customer satisfaction with the product).

How High Is Your Customer IQ*?

DM Review, November 2002

The rise of CRM (Customer Relationship Management) software is a testament to the importance of Customer information. Unfortunately, two-thirds of CRM software implementations fail, according to the Gartner Group, mostly due to poor quality information. CRM software can be successful only if the organization first cares about its customers. If it's only interested in the software for "cross-selling," chances are the organization is only implementing crM (customer relationship "Manipulation").

Information Quality in e-Business:

DM Review, October 2002

Information quality is an essential ingredient to a successful web site, whether informational or e-business transactional. Consider the following:

• A UK retail company received and confirmed orders for thousands of television sets for a mis-priced £ 3 (US $5.00) instead of the actual price of £ 300 (US $500.00) before discovering the error.
• A European telecommunications company failed to charge their customers of Internet services an amount equivalent to 8 million Euros ($7.8 million) for 9 months before they discovered the "oversight."
• A US airline offered trans-Atlantic flights for $25.00 fares. One industrial supplier, whose typical orders were $1,000-$10,000, received an Internet order for $10 Billion of materials from a fictitious customer named "Dr. Evil."

Ten Essentials of Information Quality Management

DM Review, Feature Article, September 2002

Dangerous Trends Emerging

Information quality and data quality are becoming the new buzzwords.  Many organizations have IQ or DQ projects under way in some shape or form, whether in the context of data warehousing or CRM or e-business.  Software providers are springing up with new data quality products.  Consultants are jumping up to hang IQ or DQ on their list of services.  BUT I MUST RAISE A WARNING FLAG!!!  There are many attempts to apply the words ?data quality? or ?information quality? to practices that are, in fact, NOT ?quality? management.  There are practices that simply automate and institutionalize information scrap and rework.

 

Information Quality Newsletter: IQ Principles for Web Based Documents and Web Content and Presentation Quality

Information Quality Newsletter: September 4, 2002

Quality management principles must be applied to information, both web pages and database data, made available to e-Customers on the web site and disseminated electronically. This newsletter describes those principles.

Andersen and Enron: How Information Quality Problems Cause Businesses to Fail

DM Review, August 2002

This information quality (IQ) problem is one form of "information fraud," altering data to hide the truth from criminal investigation. But the larger IQ problem is the apparent failure of "knowledge stewardship" --not acting on knowledge it had as an independent watchdog to protect shareholders. Andersen appeared to give their audit approval on financial statements "inaccurate" according to generally accepted accounting practices.

Reply to Your Responses to My May Column on the Term "User"

DM Review, July 2002

My May Column, entitled "The Term 'User' Has No Place in the Information Age" generated the most reader replies of any article I have written. The replies have been split almost 50-50 in agreement-disagreement. I wish to share some of the comments and my responses here.

Mistakes to Avoid if Your Data Warehouse is to Deliver Quality Information

DM Review, June 2002

A major bank scrapped a $29 million data warehouse to start over from scratch. The reason? Failing to understand and avoid the pitfalls described here. The tragedy here was that three years ago they were warned that their data warehouse plan was flawed. But the lost opportunity caused by the nonquality data in the warehouse was the real tragedy.

The Term "User" Has No Place in the Information Age

DM Review, May 2002

With today's focuses on "business intelligence," "customer relationship management," and "knowledge management," the term "user" is an oxymoron. This omnipresent term is the predominate Information Technology (IT) label for business personnel who require information to perform their work and who perform their work on information systems and technologies. We must find a new term by which to refer to those people who, in fact, perform the real, value work of the enterprise.

Double-Dipping from Donors Not Data Quality--Case Study

DM Review, March 2002

When I was in Victoria, British Columbia recently providing Information Quality Improvement education for professionals from the various provincial government ministries, Bridget Hellyer, of the Ministry of Finance, shared her personal experience as a "victim" of poor quality information. Bridget has graciously allowed me to share her experience in this month's column. Bridget's experience provides an outstanding object lesson in the high costs of low quality information and how the principles of Information Quality Improvement can prevent aggravation to your customers and lost profits.

The Day Life Stood Still: A Time of Remembrance, Reflection and Resolve

DM Review, February 2002

September 11, 2001. The day the world was changed forever. The day America lost its innocence. The day that rearranged our priorities. That day causes us to rethink who we are, what our purpose is, and how we should live and work. We must not hoard information for our own selfish benefit, but we must share information appropriately for the good of all, and now in ways required for the security of all. The lessons that apply to national security have ramification for enterprises, large and small.

Information Quality Mandate for Election Reform

DM Review, October 2001

The purpose of the article is not to attempt to reopen the election results. The election is over. We have a duly and legally elected President of the United States. The purpose of this article is two-fold:

1. To analyze the data from various analyses of the disqualified votes and other problems of the 2000 Presidential election and to describe the information quality principles and process improvements that must be applied to the election processes. These improvements are imperative if we are to guarantee the most sacred right of a democratic society: the right to vote and to have our vote recorded and counted accurately.

2. To use the election process as a case study as to information quality improvements that must be incorporated into all-important processes where the costs of failure are high.

Pooped On In Pigeon Forge: How IQ Problems Hurt Business and Alienate Customers

DM Review, September 2001

This past Spring, I had the opportunity to speak to a group of quality managers, engineers and statisticians in Kingsport, Tennessee in the beautiful northeastern part of our state. Diane and I chose to drive and enjoy the Great Smoky Mountains scenery. On the way home we stopped at a new home accessory store near Pigeon Forge, typical of the outlets and shops you find along this scenic route. This store, however, was like a true distributed enterprise, only under one roof. This "store" had many individual and autonomous shops, but they all shared a single centralized sales checkout system.

Is There "Something Rotten in the State of Dataland?" How Effective is Today's Data Administration?

DM Review, August 2001

Well the data is in-- and it doesn't look good. Today's Data Administration, DRM or IRM (data or information resource management) appears to have failed to accomplish even the most basic objectives, according to a survey of 120 information professionals at the DAMA-I/Meta-Data Conference in Anaheim, March 2001. This column addresses a part of the survey results and analysis. The complete results and analysis may be found at http://www.infoimpact.com/ in the IQ Resources section in the Newsletters and Articles section. If you would like to participate in the ongoing survey, select the "IRM Survey" button on the home page.

Why You DON'T Need Management Commitment

DM Review, May - July 2001

For a successful information quality or information management initiative, like any other endeavor, one needs management commitment to providing the funding and resources to accomplish it, right? Wrong!!! These two initiatives are business transformation and culture change initiatives, and as such you need more than "buy-in;" you need active involvement and management transformation.

Data Definition Quality: Change Management

DM Review, March 2001

A common information quality problem exists when the meaning of data stored in historical databases such as in data warehouses changes over time. Correct handling of changed data definition is critical for archived data collections or for data warehouses and data marts that maintain historical data that may be compared over time. In this column I identify the types of data definition change, and describe guidelines for determining the proper method for handling.

Ten Years of Information Quality Advances: What Next?

DM Review, February 2001

Ten years ago, a new journal called DM Review made its debut. Let's take a moment to reflect on lessons learned in the past ten years of Information Management. How far we have come? What challenges remain, as we journey into the first decade of the third millennium AD?

What Is Your New Millennium's Resolution?

DM Review, January 2001

One year ago, many people celebrated the "arrival" of the new millennium. Now, twelve months later, that new millennium has finally arrived officially. In the Gregorian calendar, a new decade, new century, or new millennium begins with a year ending in "1," "01" or "001" respectively. There was no year 0 AD nor 0 BC. The year "1 AD" was the first year of the first decade, first century and first millennium AD. But it is easy to see how the year 2000 became the "popular" new millennium year, since the number "2000" is much more similar to "2001" than to "1999."

Lessons I Learned from the IQ Conference 2000

DM Review, December 2000

I write this column as I am flying to Budapest, Hungary to lead an Information Quality seminar and from there to the London IQ Conference 2000. Having just completed the IQ Conference Anaheim, I wish to share a few of the many things I personally learned from the rejuvenating Anaheim experience.

The Rubber Meets the Road

DM Review, November 2000

The rubber has met the road, and it has failed. This is not to make light of a human tragedy, and it is not to declare Firestone and Ford the villains. It is to remind all of us that we are probably managing (well, at least collecting!) information about people and products; that this information is hopefully being used to make decisions; and that these decisions can impact people's lives, either directly or indirectly, with from minor to very major consequences. If this human equation does not, in turn, remind us of our responsibilities to plan, collect, secure, maintain, measure, and share (appropriately) the information entrusted to us, then we are a hopeless bunch.

Which Information Quality Tool Is Best?

DM Review, October 2000

One of the frequently asked questions (FAQ) I get is "What is the best Information Quality (IQ) software product?" Before I answer the question, I address some misconceptions about software product evaluation in general.

What's In a Name: A Mandate for the IT Industry

DM Review, September 2000

There is a saying, "Say what you mean, and mean what you say." Words are important. They are the basis of communication between people. Without clear, precise definition of the words we use and clear names that we label things, we can have a "failure to communicate," as was so poignantly expressed in the movie "Cool Hand Luke."

In e-Commerce, It's e-Quality or e-Bust

DM Review, August 2000

The dot.com bubble of "build it and they will come" has burst with a loud truth. The reality is that without a plan for quality products, services and information that is focused on customers and meeting their real needs, a dot.com endeavor is doomed.

Knowledge Management? Not in IT Apparently

DM Review, July 2000

If the early indication holds true, the Knowledge Management revolution will not be led by the IT organization. The Knowledge Management revolution can only be led by those who can and do manage knowledge. Those who do not learn from their mistakes cannot teach the rest of the enterprise to do so. The fact of the matter is that more than six out of ten (62 percent of) information professionals believe that their organization and others have not learned important lessons from their Y2K experiences and change their data design processes.

Preventing Customer Alienation Through Quality Information

DM Review, June 2000

One of my colleagues, David Eddy, recently told me about his experience with credit reporting agencies (CRA) in which he discovered "preventable" information quality problems. One problem included a zip code that was correct in one place and incorrect in another (in one report). One CRA did not have updated zip codes that had changed in the greater Boston area from 021XX to 024XX.

Go to Gemba When Data Quality Problems Occur

DM Review, May 2000

Gemba, the Japanese word meaning the "real place," refers in business to the place where the work that adds value takes place. In manufacturing, Gemba is the shop floor. In the Information Age, Gemba is anywhere that processes are performed and information originates or knowledge is gained.

Information Quality Management: The Next Frontier

DM Review, April 2000

The Information Age was born in 1946 when the ENIAC came online. That was just before Deming went to Japan in what was to prove a turning point in the Industrial Age. The quality revolution ushered in the maturing of the Industrial Age through quality principles applied to manufacturing. The resulting elimination of waste and increased focus on customer needs has decreased the costs of doing business and has increased business effectiveness and customer satisfaction.

What Have We Learned from Y2K?

DM Review, March 2000

This is a column I hoped I would not have to write. I know we all are sick and tired of "Y2K." But as I am looking at the press commentary being communicated through January 9, 2000, I cannot stand idly by while people draw wrong conclusions. It appears that many do not understand the tragic waste caused by the defective data design problem that has been romanticized as "The Y2K Bug" or "The millenium Bug." Those who do not understand the root cause of this problem will congratulate themselves for responding to this crisis and go back to "business as usual" --not learning from this opportunity.

The Information Quality Revolution: Will America Be Left Out Again?

DM Review, February 2000

I am writing this month's column after returning from Tokyo, my first visit to Japan. My return was greeted by the headlines, "When Computers Fail: Y2K gets the headlines, but everyday glitches cost U.S. companies $100 billion a year" and "Software Hell: Glitches cost billions of dollars and jeopardize human lives. How can we kill the bugs?" and its related story, "Will Bugs Eat Up the U.S. Lead in Software?" These two articles highlight the phenomenal problems and costs caused by today's complex and nonquality software. Just to cite one example, Hershey Foods $115 million SAP computer system implementation had quality problems that crippled shipments during the Halloween season, and received the blame for Hershey's 12% decrease in third-quarter sales and 19% decrease in net income from the same period last year.

KAIZEN and Information Quality

DM Review, January 2000

This column shares some of the concepts and principles of kaizen as presented by Mr. Imai, and describes how they apply to information quality. The word itself comes from two Japanese words, KAI meaning "change," and ZEN meaning "good" (for the better). In business, kaizen means "continuous process improvement involving everyone in the organization."

Answers To Key Questions About Data Cleansing

DM Review, December 1999

Here are some straight answers to some important questions about data cleansing.  The questions came from students in my "Information Quality Assessment, Cleansing and Transformation" tutorial at a recent Data  Warehousing Institute Conference.   

Information Anarchy: The Strategic Challenge

DM Review, November 1999

The role of the data administrator is one of the most important roles in the intelligent learning organization. No organization can succeed in the Knowledge Age without exploiting its information and knowledge resources. No organization can successfully exploit its information and knowledge resources without an effective data and information resource management (IRM) function. As a working definition of IRM, let us use "(1) The application of generally accepted management principles to data and information as strategic business assets. (2) The function of managing data and information as enterprise resources." The data administrator is the only role in the enterprise that has the "view" of the enterprise to make this happen successfully. To be sure no information resource management organization or data administrator can accomplish this alone. They must facilitate and influence others in order to accomplish the goals of IRM.

Information Quality in the Knowledge Age

DM Review, October 1999

Information quality is not about the quality of data in databases and data warehouses!Information quality is about the quality of business communication in all forms  spoken, written, email, reports, internet, intranet, policy manuals, catalogs, newsletters, as well as databases and data warehouses. But more than that, information quality is about serving our customers by providing information products and services that "consistently meet end customer and internal knowledge worker expectations" so they can be efficient and effective in performing their work and successfully meet their goals.

Seven DEADLY Misconceptions about Information Quality

DM Review, June - September 1999

There are seven potentially fatal misconceptions about information quality that can hamper an information quality initiative. Worse yet, if these misconceptions are strongly held, they will hamper business effectiveness (best case) or result in business failure.

Information Quality Is Mightier Than the Pen

DM Review, May 1999

It is hard to believe that for nearly two years I have been discussing quality guru W. Edwards Deming's Fourteen Points of Quality and their ramifications for information quality. For anyone who may have missed any of the columns, they can be found all together in Chapter 11 (The Fourteen Points of Information Quality) in my book entitled Improving Data Warehouse and Business Information Quality, now available from John Wiley.

Having just proofed the last pages the week that I am writing this column, it seems fitting to share about lessons learned about information quality while writing a book on the subject.

DQ Point 3: Cease Dependence on Data Model Quality Assurance Inspections Alone

Information Impact Newletter, Summer 1998

Isn't it interesting that the practice of "quality assurance" to provide quality of products and services is actually a deterrent to quality? Quality assurance, as in inspection of products to assure conformance to specifications, actually increases costs and provides a false sense that one is providing quality. The goal of inspections, e.g., data model review or program testing, is to discover defective products and fix the defects or throw out defective products that are too costly to fix. The end result of inspection as a means of quality assurance is expensive, ineffective because it cannot assure 100% discovery of defects, and does not achieve real process improvement.

DQ Point 4.3: Single Data Sources Increase Data Quality

DM Review, February 1998

Earlier I examined the first two of three counter-productive IT practices. "1. Reward project development for on-time / within budget alone --without real measure of the quality of the product delivered." This practice has the impact of actually increasing costs while increasing defects. (see November 1997 column.) "2. Model and build databases based upon the application project requirements alone --without involvement of stakeholders outside of the project scope." This practice leads to fragmented, un-integratable data models and un-sharable databases that appears to be cheaper for the project. Experience now has disproved this with increased complexity and costs of redundancy maintenance and increased data quality problems. (See December, 1997 column.)

The High Costs of Low Quality Data

DM Review, January 1998

If the state of quality of your company's products and services was the same level of quality as the data in its databases, would your company survive or go out of business? Conduct a quick self-assessment test.

Transforming Business through Data Quality Improvement

Information Impact Newsletter, Winter 1998

So you say you organization wants to improve its data quality. Does it really? Data quality is not simply "scrubbing" data to put it into the data warehouse. Data quality is not simply auditing data to measure it. Data quality is fundamental changes in how the information systems organization defines and delivers its products and services. DQ is also fundamental changes in how the enterprise operates. These changes are systemic, throughout the entire organization.

Developing a Data Quality Mindset

DM Review, May 1997

Those of us who travel much have numerous travel experiences of a traumatic or humorous nature. The following incident describes the true mindset of an organization desiring to provide data quality with a "100% satisfaction guarantee" to its knowledge workers.

Information Stewardship: Accountability for Data Quality

Information Impact Newsletter, Summer 1997

Why is the state of data quality in the typical organization so abysmal? The plain truth is that data producers (the role in which business persons create or update data) produce data quality to the level for which they are trained, measured and held accountable. Real and sustainable data quality involvement can only be achieved by implementing accountability for information like accountability for other business products and resources.

The Year 2000 Tragedy: From Reactive to Proactive Quality Management

DM Review, February 1997

Organizations will fork over $400 - $600 BILLION worldwide converting software and databases for the year 2000, according to Gartner Group. Some  mostly consultants and software providers  will exploit this as an opportunity to sell products and services. Others will make money investing in them. Already "Year-2000 companies" are seeing their stock prices soar.

The question: what is the value added by this "investment" equivalent to ten times the assets in the Bank of Boston? The answer: "Zero!" The tragedy of the year-2000 dilemma is that the sole "benefit" is the ability to conduct business Monday, January 3, 2000 the same way as on Friday, December 31, 1999. Period!!! It will not contribute a single dollar to the gross national product. In fact, it will actually decrease business productivity and profitability. Every $1 million spent on conversion projects is $1 million not available for strategic, value-adding projects.

Data Quality in the Data Warehouse: A Key Critical Success Factor

Information Impact Newsletter, Winter 1997

Not even the most sophisticated data warehouse database technology, most comprehensive information directory, or the sexiest graphical interfaces can make a data warehouse succeed if the data is bad. Consider the following:

• A manufacturing company nearly scraped a $12 million data warehouse project due to inconsistently defined product data and poor data quality.
• A publishing company spent four years cleaning up customer data that came from a dozen disparate customer files.
• When an insurance company created a data warehouse to analyze medical diagnosis codes for which it was paying claims, it found an unusually high incidence of "hemorrhoid" diagnosis codes in one region. That region's claims manager confessed, "Oh! I didn't know anybody else was using that field. We use that code among ourselves to identify claimants that are 'a pain in the a _ _'."

The Golden Age of Information

DM Review, January 1997

Oxford Dictionary defines a golden age as a period when commerce, the arts, etc., flourished. It further defines flourish as successful, very active, or widespread. If that is the litmus test, we are indeed living in the Golden Age of Information. Data warehouses that now contain hundreds of Gigabytes of data have altered forever the information processing landscape. The explosion of information on the Web over the past two years confirms the importance and value of accessible information. Today's new information environments demand increased diligence in the management and stewardship of the information resource.

Information Quality: Meeting Customer Needs

DM Review, November 1996 and Information Impact Newsletter, Autumn 1996, revised June 8, 2004.

Data audits further reveal that the accuracy level of customer information in the typical customer database averages only around 85%. The cost of this nonquality information takes its toll on the business bottom line in the form of wasted communication costs to its customers. But the most significant real costs are lost customer lifetime value as a result of missed or late communication or the "aggravation factor". The aggravation factor is the nuisance caused to customers as a result of nonquality information such as incorrect invoices or having to change address information multiple times. Lost or missed customer lifetime value as the result of poor information quality can be significantly greater than the moneys wasted on duplicate and wrong address mailings. Wasted mailout costs of $10,000 may actually result in millions of dollars in lost customer lifetime value.

Redefining the Role of Information Management for the 21st Century: The Maturing of the Information Age

Information Management: Strategy, Systems, and Technologies, Auerbach, 1995 and Information Systems Management, Winter 1996

"Large organizations will have little choice but to become information-based" as they move into the next century, Peter Drucker states unequivocally. The approach of the 21st century challenges Information Systems (I/S) management and information professionals to rethink their mission, objectives and contributions to their enterprises. Ever-accelerating advances in information technologies with exponential increases in computing price-performance, challenge information professionals to rethink their paradigm of information management. Marginal improvements in information systems productivity and the inability to keep pace with the information demands of business challenges the information professional to rethink everything about the way they have been building applications and practicing data administration and data management.

INFORMATION STEWARDSHIP: A Human Solution to Information Integrity

Database and Programming Design, April 1993

This article explores Information Age management principles and discusses the increasingly popular notion of information stewardship --that is, accountability for the information resource. We will see why information stewardship is a required, not an optional component of information resource management (IRM). We will also consider how to implement and organize an information stewardship function.