Big data is already here and is changing the data management dramatically within the industry, your company and its core business processes. Big data is going to impact your business and your data management practices and is going to have a significant impact within your company. The continued declining costs of storage and computing power will make big data attractive. Dealing with big data requires your data management practices to understand and embrace the new reality. While most articles are only highlighting three Vs of big data, I believe there are truly “Five Vs” for big data.
V number one is Volume. The amount of data continues to explode. According to a recent IDC survey the volume of data that will be under management by 2020 will increase 44 times over 2009 levels. Think how big the systems are now and think about 44 times the volume. How are you improving your company’s archiving, and tiered data importance strategies to accommodate the new volumes?
Next is Velocity. With more big data comes the increased speed, number and frequency of transactions with your business. The velocity of big data needs to be understood, prioritized and synced up with your storage strategies. Just like anything that is moving quickly, the time for data designs, performance tuning and especially maintenance will be compressed. How automated are these processes and how are your data management practices spreading the knowledge within the organization?
Next is Variety. With all the storage capabilities available, the amount of structured and unstructured data and its diverse sources will continue to explode. New types of data will proliferate from all types of monitoring of highways, electricity and water utilities to new applications on mobile phones, tablets and other diverse devices. Integrating these diverse structured and unstructured data types quickly into useful information will require more video, audio and XML usage within your systems. How well are your data management practices at integrating and designing with these new data types?
Next is Verification. With all the big data there will be bad data and with diverse data there will be more diverse quality and security levels of users. Our data management practices are experiencing the overhead of extract, transform, and load processes that continue to struggle with bad data. How much data verification is being built into your application transactions? Also how are your security definition designs for the roles of all the new users with varied security levels? Security, governance and compliance will only become more time consuming with big data. Your data management department needs to automate and put in place processes and tools that will automatically verify the quality and compliance issues.
Finally, the most important item is Value. Most likely, your company has recently expanded its storage, the number of systems and their size has more than doubled. With the big data trend this will only accelerate as noted in the IDC survey mentioned earlier. How has the increased amount of data affected the value of the insights, benefits, and business processes within your company? How does your data management department support the extraction of value from your data? Are the right questions being asked from big data? Do your existing reports, web pages and user interfaces assist in helping find the value within the big data? Extracting value from big data is the toughest chore because of the factors I outlined earlier: volume, velocity, variety and verification. These factors, along with value make up the “Five Vs of Big Data.”
So get your data management department procedures and processes ready to work with the other thought leaders and start designing, mapping and planning your company’s journey into the “Five Vs of Big Data”.
I will be speaking at the International DB2 User Group (IDUG) European Conference in Prague. I will be presenting my “DB2 10 Temporal Database Designs for Performance” on Mon, November 14, 1:30 PM – 2:30 PM. For more details on my presentation click here. (http://www.idug.org/e/in/eid=2&req=info&s=97&all=1) I will also be involved in the Data Warehousing Special Interest Group (SIG) and the final discussion panel, so please join all the presentations, networking and discussions on DB2. Sign up now at www.idug.org
I will also be presenting at the Midwest DB2 Users groups in the 4th quarter. I will be doing Minneapolis on December 6; Milwaukee, Wisconsin (http://www.wdug.com/) on December 7; and Chicago, Illinois (http://www.mwdug.org/) on December 8. I look forward to seeing everyone at the meetings to discuss all their plans for the 2012 year.
Come see me in any of these venues and ask me your DB2 performance or data warehouse performance questions.
Leave a Reply