How to Build a Big Data and Analytics Team

by   |   April 7, 2016 5:30 am   |   2 Comments

Bernard Marr

Bernard Marr

As the data tsunami continues to surge and analytics tools become less expensive and easier to understand, big data is going mainstream. In fact, I would venture to say that businesses that aren’t considering their big data strategy and plans for the future are in very real danger of being left behind.

But what does that mean? Where does a business start to build a big data and analytics team? Here are some criteria to consider.

Hiring a great team doesn’t start with posting a job ad. It starts with the company taking a hard look at its goals and the talent it needs to achieve those goals.

As with anything surrounding data, the first step is to be clear on the questions that you want the data to answer and the challenges or goals you hope to address. No matter what size your business, don’t be afraid to start small and build your analytics as you go.

Start with the questions in mind and identify the key performance indicators that will allow you to accurately judge when the questions have been answered. Then – and only then – start considering which team members can help you answer the questions.

The three main roles any big data team should include are:

    • Business analyst. These people existed long before big data and continue to play an important role. They have intimate knowledge of your industry and company, and they analyze business-level data to produce actionable insights.


    • Machine-learning expert. This person is statistically minded, with experience in programming and building data models. This person develops algorithms and crunches numbers in order to help answer questions and make predictions.


  • Data engineer. The data engineer is concerned with the capture, storage, and processing of the data itself.


Related Stories

5 Tips for Finding Talent for Your Data Analytics Team.
Read the story »

A Data-Driven Solution to the Data Scientist Shortage.
Read the story »

Analytics Leaders Discuss Care and Feeding of a Successful Data Team.
Read the story »

Island of Misfit Toys: Building an Analytics Team from Within.
Read the story »

Together, these three roles make up the basis of any good analytics team. Occasionally, you can find one person who can fill multiple roles, but this is often referred to as a “unicorn” because such people are so rare.

Other team roles might include:

    • Executive sponsor. This is a senior-level person who understands the business needs, rallies support, and funds the solution.


    • Business user. This person defines and prioritizes the business requirements, and then translates them into high-level technical requirements.


    • Data scientist.  Ideally, this person has domain knowledge, a statistical analysis background, and basic understanding of computer science in order to extract information from the data to answer business questions.


    • Data journalist. Forbes predicts that storytelling will be the hot new job in big data analytics. This person serves as the translator between audience and data.


    • Platform/Systems architect. This person has a software engineering background in large-scale clustering/distributed processing systems and is responsible for technology selections and implementation process.


    • IT/Operations manager. This person operationalizes, deploys, manages, and monitors the systems.


Finding Talent

Once you have defined the goals of your big data project and decided which roles you need to fill, the next step is filling them.

And talent shortages are a critical issue. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills, according to the McKinsey Global Institute.

Companies are looking outside the box for new talent, recruiting people from fields as varied as physics and mathematics to language arts. Walmart even turned to crowdsourcing to fill its big data needs with a contest to seek out talent.

Depending on the roles you choose to fill for your organization, consider these questions when interviewing and hiring:

Does the candidate have solid programming skills? A data scientist needs the skills not only to view and analyze the data, but also to manipulate it. A statistician who reviews and interprets a set of data is very different from a data scientist, who can change the code that collects the data in the first place.

Does the candidate excel at producing analytics for computers or humans? (And which do you need?) There are two main types of big data analysts: those whose end user is solely a computer, and those whose end user is a computer. If your end result is a machine-learning algorithm to, for example, choose which ads to show on a website or make automatic stock trades, your analytics are for computers. If, on the other hand, a human will make a choice based on the analytics, your analyst needs a different set of skills – chiefly, being able to tell a story through data and providing good visualization of that data.

Can the candidate provide concrete examples of when she has improved a business process through her work? As with any position, you hope to see real-world examples of when candidates successfully implemented improvements to a business process.

Is the candidate a good communicator? Stereotypes would have us believe that it’s OK for scientists and techy types to be introverts with poor communication skills, but that’s not really an option with a data scientist. He or she needs to be able to communicate effectively with people who don’t “speak the same language,” tell a story through data, and use visual communications effectively.

Can the candidate be creative and open-minded? Big data is a rapidly changing and expanding field that requires a certain open-mindedness and creativity. To innovate, a good data scientist must be able to look beyond what came before. If a candidate has implemented the same processes or procedures at multiple companies, ask yourself seriously if he or she is able to innovate and try something new.

Does the candidate have solid business understanding? It’s one thing to understand the science and mathematics behind analyzing huge data sets. It’s another thing entirely to understand how that data affects profitability, user experience, and employee retention – or many other factors important to the business. Someone with a background in business will be better at spotting trends that will benefit your business.

Finally, because of the shortage of talent, recruiting for big data positions can be an HR nightmare. Keep these suggestions in mind:

Don’t get hung up on titles. One company’s “data analyst” may be another company’s “data scientist” or “data visualizer.” There is no standard definition for any job titles in the big data world, so don’t limit your search by title.

Focus on unique skills sets. Things like programming skills and math degrees may be important, but a candidate’s experience and insights may be the winning combination.

Look in other sectors. As I mentioned above, a person’s background doesn’t necessarily have to strictly include computer programming or data science. Recruiters are finding more and more quality candidates from diverse backgrounds, including philosophy, liberal arts, statistics, and others.

Get involved in the communities. Hadoop is an open-source platform, which means there is a thriving community using and improving it. Get involved, network, and get to know players in the community to make important connections.

Data analysis positions may be very different from the roles you or your HR department usually seek to fill, so the best strategy to make the best hires is to go into the process with a solid plan and a good understanding of exactly what it is that your company needs.

Bernard Marr is a bestselling author, keynote speaker, strategic performance consultant, and analytics, KPI, and big data guru. In addition, he is a member of the Data Informed Board of Advisers. He helps companies to better manage, measure, report, and analyze performance. His leading-edge work with major companies, organizations, and governments across the globe makes him an acclaimed and award-winning keynote speaker, researcher, consultant, and teacher.

Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.

Take a SMART Approach to Big Data Analytics

Tags: , , , , , , ,


  1. Sudhindra Arsikere
    Posted May 18, 2016 at 7:17 am | Permalink

    Awesome analysis.

  2. Jeff Morey
    Posted September 3, 2017 at 8:53 am | Permalink

    Good intro.
    Thanks for spelling out the roles, Data Journalist is something I knew we need but didn’t have a title for.

    I believe some roles are related of the 9 you defined.
    For example, these 3:
    1. Business analyst
    2. Business user
    3. Data Journalist
    And these 2:
    6. Data engineer
    7. Platform/Systems architect

    You may want to revise this sentence, if your intention is to communicate that Data analyst work can be consumed by both end users and computers and end user consumption requires presentation skills on top of analytical skill.
    “those whose end user is solely a computer, and those whose end user is a computer.”

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>