A Symphony of Big Data

Orchestra Playing Music

Imagine what a symphony and big data have in common. Forget, for a moment, about attending the symphony or using a data management system—and imagine being the creator, or the architect. To compose a symphony, there must be an overarching understanding of how the entire orchestra works together to create harmony, coupled with the intricacies of each instrument’s role.

Just like a symphony, effective data management must take into account enterprise business management skills along with the sophisticated technical skills required for data manipulation.

The Harmony of Data Management

The beauty of a symphony is that, although it is made up of multiple instruments and musical parts, the audience can often hear only the masterful blend of the finished product—a result of artists working together as a unit. If each player did as he pleased and ignored the conductor, there would be nothing beautiful about the resulting cacophony. Much like the players in that orchestra, business partners and IT must work together to stay in harmony where data management is concerned.

Try white boarding ideas with the IT department, rather than just having IT generate reports. Have IT and business users sit down together with the expectation that resulting ideas are to be evaluated—and that nothing is set in stone. Then, after everyone ideates, have the business users return to determine potential value to those ideas, and have IT do cost and gap analyses on the ideas versus the realities. This practice eliminates boundaries, and suddenly, the IT team and the business users are talking in concrete terms that are not limited to currently available data. Both sides are working together as a group—like an orchestra.

Business users often limit their thinking to that which they know exists right now. It is better that they think abstractly about an ideal world not constrained by their current system. Working in concert with the entire team can open up a whole new world.

Assemble an Ensemble of Talent: Big Data vs. Traditional Data

The classics don’t set the perfect tone for every occasion. Sometimes a newer approach is appropriate. The world of data has changed tenfold in recent years, thus the term “big data” and its revolution have created both opportunities as well as challenges in the way we capture, manage and store data.

Big data, defined as “massive amounts of unstructured data that are not traditionally stored in a relational form in enterprise databases,” is captured immediately, and in most cases, interpreted after the fact. It is easier to grab and store, and often remains in its original format—with most information still intact. Big data is ultimately more scalable than is traditional data.

Traditional data management identifies desired data first and then creates customized capture requirements. Traditional data is stored efficiently and in context. The limitation here is that any relationships not interpreted and specified as part of the fundamental structure of the storage, are generally lost, or reproducible only through great effort and with questionable fidelity.

Big data may seem like a win, however, it is not without its challenges. Big data talent can be extremely hard to find. And talent with domain knowledge is even harder to find. (It is difficult in nearly every IT capacity). Moreover, big data is new—which means there isn’t as much experience with it. Without having the track record that the “traditional data” approach offers, it is harder to tell whether a project is going astray solely based on experience.

The conductor of a symphony chooses talent and musical selections wisely—each artist boasts a particular talent and the group as a whole may perform some types of pieces better than others. The same concept must apply to enterprise. The right ensemble of business and data talent will guarantee that all facets of business and IT are properly covered. Business and data working together as a team to develop ideas and work through projects ensures business goals are met and exceeded.

As with a symphony, data management is complex and intricate, but when executed with an enterprise-wide strategy, its outcomes can be harmonious.

Image Source

Comments are closed.