Uncertainty Drives Desire to Improve Big Data Skills

by   |   June 24, 2016 1:00 pm   |   0 Comments

PK Agarwal, Regional Dean and CEO, Northeastern University-Silicon Valley

PK Agarwal, Regional Dean and CEO, Northeastern University-Silicon Valley

Thousands of data science professionals descended on the San Jose Convention Center, in the heart of Silicon Valley, in late March to learn about the high-demand skills of the future. Some were at the STRATA convention to network; others were there to learn new skills to further their careers in data science. But even with data science being dubbed “the sexiest job of the 21st century,” many attendees shared an uneasy feeling of uncertainty about the future of this fast-moving field.

Northeastern University-Silicon Valley conducted a survey at STRATA, the largest annual gathering of statisticians. The results showed that a huge majority of those in attendance (96 percent) thought that acquiring new skills in data analytics would help them with career opportunities and growth. Even more interesting was the fact that more than two-thirds of that group (68 percent) was already employed in the data science field. In other words, many people already working this field don’t feel they are aptly prepared. Nearly two-thirds of respondents (63 percent) ranked data science-advanced analytics as their most coveted skill to learn. The rapidly evolving progression of the field, in combination with the massive amounts of data, explains this concern among professionals in the industry.

Today’s data scientists differ from their counterparts of just a few years ago. In the past, data scientists had to do it all. They had to be proficient in everything from complex back-end calculations to extracting data, analyzing it, visualizing it and, ultimately, presenting it. I liken the change we are seeing in the field now to that in the field of medicine. As the field swells, it’s getting to a critical-mass situation and subdividing into specialization.

Related Stories

Online Learning Helps Mitigate Big Data Skills Shortage.
Read the story »

Put Data Science Skills Before Big Data Infrastructure.
Read the story »

Build your Big Data Adventure Team with a Variety of Talent.
Read the story »

Island of Misfit Toys: Building an Analytics Team from Within.
Read the story »

Of course, each candidate must still have the skills in math, business technology, behavioral science, and design thinking. But the requirements don’t stop there. The data is demanding more and as the profession matures, and the expertise of the workers must evolve to meet these demands. The volume of data to be analyzed keeps increasing, making it more challenging to keep up with the skills needed to sort through all the information. This creates a near constant need for upskilling.

It’s not just important to have the skills to manage all that data. Once you have extracted what is crucial from the data, you have to be able to decide what to do with that information. Survey respondents believe that top employers see data management and data analysis traits as key to getting the job. Additionally, respondents felt their skills were lacking in the areas of advanced analytics, data mining, Hadoop, and statistical computing.

Acquiring the Skills

Respondents’ preferred method of learning was also a key component of the findings. Just as the skillsets are multifaceted, so is the approach to building them. For many of these professionals, returning to school full-time simply is not an option. Consider the constraints of professional obligations as well as personal commitments, and it becomes clear that the most convenient option is online learning.

However, the team of researchers from Northeastern University-Silicon Valley was surprised to discover that online learning has its limits. Students need at least some interactions that come with having a professor, fellow classmates, and hands-on learning. Those interactions can make the knowledge really come to life. A hybrid education strategy combines flexible course offerings with unique approaches to job training and mentoring programs. Northeastern, for example, works with 3,000 corporate partners to better match curricula and training programs to high-demand STEM jobs.

The new hybrid education model is more than just a novel approach to learning. It also can lead to a paradigm shift in the field. This shift involves an increase in emotional intelligence, which is the ability to understand your own emotions as well as the emotions of others. Employers often cite emotional intelligence as a quality that is severely lacking in business fields.

In the high-stakes war for talent, these soft skills have emerged as a new kind of “third space” that departs from the traditional business skills taught in most MBA programs and engineering schools. More employers are recognizing the importance of soft skills, which have been largely undervalued by academia and industry, according to a study by the USC Annenberg School of Communications and Journalism. It’s critical to have a workforce that can solve analytical problems and also work effectively with people across departments. Strong social skills will become increasingly important as more human jobs get automated because the ability to work well with other people is a skill that machines simply cannot replicate.

eBook: How to Build and Lead a Winning Data Team


Hybrid learning is the fastest way to gain emotional intelligence because students are thrust into collaborating on hands-on projects with people they have just met. For data scientists to get ahead in this fast-evolving field, they must continually focus on upskilling themselves through a combination of online courses, live classroom lectures, collaborative team projects, and experiential job training.

PK Agarwal is the Regional Dean of Northeastern University-Silicon Valley. His career in high tech spans the public, private, and nonprofit sectors. He was the CEO of TiE Global, a Silicon Valley-based nonprofit focused on fostering entrepreneurship, where he oversaw innovative programs that have helped more than 9,000  startups across the globe grow through mentoring, networking, education, incubating, and funding. He also advised policy makers worldwide on economic and job-creation issues. Prior to that, he was the California Chief Technology Officer (CTO) under Gov. Arnold Schwarzenegger, where was responsible for managing IT operations for the state.

Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.

Tags: , , , , ,

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>