Turning “Big Data” Into a Story

An interesting Harvard Business Review recently showcased the importance of storytelling in good data analytics. It is the data scientists job not just to sort through large amounts of data, but rather to make sense of the data and figure out the trends. These trends then form the basis of a broader story that can be turned into actionable insights for the leadership team. 

So, if you are sorting through a the web browsing data of a client, say in the online tablet retail business, you not only find out what products customers are most interested in, but use that data to forecast trends for the future. Additionally, when presenting the data to the client, it is important to not just dump the data with them, but show them how to make the best use of it. 

At Mazliah, we make this a strong focus of every project we get involved with, even if it is just a quick data pull. For larger, more strategic projects, we take the data points that we find through advanced analytic techniques, and turn that data into key takeaway points for the client.

QA Blog Series Post 4: Implementing a QA process

To wrap up our series on quality assurance, we wanted to discuss ways to implement a QA process. One of the important factors of a successful QA process, is to have buy in from the top. Senior management has to be involved in shaping the process and implementing a system. Although, management needs to be involved, it is also imperative to make sure development teams have a seat at the table. Since, for the most part, it will be the development teams that will be involved in the day to day QA process.

Additionally, a critical factor is to work out the detailed process or workflow involved in this process. It is important not to make the system bureaucratic to where cycle time increases. A way to overcome this issue is to divide most development projects into two categories: small and big projects. For big projects, it may be necessary to have an independent QA team that is not involved in the development process. However, for smaller projects, especially those involving just one person, it may not be necessary to physically separate the development and QA process. Essentially, the developer can take off his developer hat and put on his QA hat when reviewing the code and logic.

There are many nuances to developing and implementing a QA process. These processes have to be customized to fit your business needs. We hope this QA series initiates a discussion on the essential need to have a high level of confidence in your code and in the data analysis that follows.

QA Blog Series Post 3: Establishing filters to analyze data

As we continue our series on formalizing a QA process surrounding data analytics, we wanted to address the very critical step of determining the appropriate filters to narrow down a data set. In the hypothetical case we discussed, regarding a online music radio service trying to determine their most “popular” music, selecting the appropriate filters is very challenging. Numerous approaches can be taken to get the most “popular” music including most played track, most played artists, most “+1” or “Like’s”, most searched for songs or artists, and the list goes on. So, it is important to take a step back when approaching such a project and discuss with the stakeholder to ascertain the appropriate filters. Often, a mix of different factors can be used to determine a weighted average popularity score. Nevertheless, it is important to reach a consensus as to which factors should be weighted heavily versus those that are less important.

Such a situation can easily get messy if proper planning is not done at the start of the project.

QA Blog Series Post 2: Monitoring quality of code

In continuing our focus on quality assurance, we are focusing on strategies to ensure database query code is structured to pull data that answers the business question at hand. In our hypothetical scenario mentioned in the previous post, where an online radio service wants to understand their most popular music, the development of database query code is an integral part of getting to the answer and a number of questions have to be asked about the code. Is it pulling the right data? Does it have any potential bugs that may cause an issue? Does it take into account all factors? Is it efficient to reduce long run times?

To maintain a high quality level with code, it is important to initiate the QA process while writing the code itself and not after the fact. First, plan the major factors the code needs to address. Second, while developing code to address each factor, constantly question the code that is developed. Third, after developing the code, take off your developer “hat” and put on your QA “hat” and take a critical look at your own code. Lastly, a fresh set of eyes can really help spot any bugs that you may have overlooked. Try to have a colleague review your code and suggest improvements.

At Mazliah we are implementing these processes to ensure reliability in the query language we develop. What QA techniques do you use and find useful?

Stay tuned for our next post in our series focusing on QA.

QA Blog Series Post 1: Developing a Quality Assurance process for dealing with data

In today’s fast paced business world, where data insights are needed with a very short turn around time, it is important to not lose sight of the quality of the data. Also, with such a large amount of data available, big data can lead to big problems if quality assurance is not a strategic focus. Imagine this scenario: your client, an online radio service, tells you they need data driven support making a decision regarding which music licences should be acquired based on song popularity. The client also has a big meeting tomorrow with the record labels and needs a recommendation fast. In order to get these recommendations, you have to develop code to query the database of user listening data, establish filters to pull the data, then do an analysis, and finally make recommendations. It is easy to see how a very short turn around can lead to a number of issues during each of these steps.

In the next few weeks, we are going to address how to monitor quality at each of these steps, along with developing a comprehensive plan to deliver consistent and reliable analysis.

Hottest job of the 21st century? Data science, reports Yahoo Finance

“You should go into politics!” snapped my girlfriend, just as I had outwitted her again in the age-old argument of where to go for dinner. “You have exhausted every possible category of reasoning for why we should go there again. I don’t need to know that the drive takes less time! Or that it’s cheaper. Or that it’s less crowded. Sometimes I just want to go somewhere on a whim!”

Maybe a variable like “whimsicality” makes the age-old dinner question a tricky one to answer for a data scientist, but when businesses are overwhelmed by an apparently endless sea of data and have to make a decision- which customers to target, what market to enter, how much advertising is too much- a well planned approach to processing it all becomes a necessity. Buyers are becoming more informed. The world economy has become far more complex in the Internet age. Buyers have become more informed and also far more selective. Customer targeting and intelligent marketing are becoming cornerstones of a successful business. However, with the rapid growth in need for these types of services, manpower comes at a premium. The skillset and rarity has led one CEO to call data scientists “unicorns”. While we do believe we possess rare magical powers on par with the fabled unicorn, data consulting firms allow companies to reach out and implore these rare resources without having to go on a unicorn hunt.

After all, who wouldn’t want to work with a unicorn?
Source: http://finance.yahoo.com/blogs/daily-ticker/sexiest-job-21st-century-122238562.html

Back from crunching numbers

Hi folks, we have been busy crunching numbers for the last few months and haven't had time to update our blog. But, moving forward we are making this a priority. So be sure to check back for some great upcoming posts. We are kicking it off with a series on Quality Assurance processes in data analysis.