You are currently viewing Why are so many people getting tired of the “sexiest job of the 21st century”?

Why are so many people getting tired of the “sexiest job of the 21st century”?

The role of “data scientist” experienced a meteoric rise in the early stages of the specialty around the turn of the millennium. With data becoming one of the key ingredients to business success – and becoming more accessible than ever – data workers updated their resumes with the new job title and saw an instant increase in the salary they could claim.

Database scientists, mathematicians, statisticians and physicists were among the first to turn to the data science industry. They wrote code from scratch and leveraged information from relatively limited data sources. One of the biggest challenges, however, was bringing together data owned by different departments, controlled by different people, and stored in hard-to-access data silos.

Since no formal data science training existed to enhance the skills of these specialists, they had to rely on their own coding knowledge to deliver data-driven insights. Their work had a high degree of freedom and required exceptionally high level of technological expertise to unlock business value – the perfect attributes for a boom in popularity. In 2012, Davenport and Patil called “Data Scientist” the “sexiest job of the 21st century” and that times were good.

20 years later: what has changed?

Record demand for data scientist jobs continues to rise in 2022. Today, LinkedIn has 44,000 data scientist job listings in the UK alone. With data-driven insights now a prerequisite for improving efficiency and performance, data science will continue to be one of the most in-demand specialties. An unsurprising statement given that the volume of data being created is growing at an exponential rate – expected to reach 180 zettabytes by 2025.

The need to mine unstructured wild data to generate insights has never been more critical. With the global supply chain disruption due to COVID, net zero initiatives across the western world, and the current energy crisis prompting an overhaul of efficiency standards, companies are asking more detailed questions than ever. These are questions that also require a greater degree of data literacy, experience, and domain knowledge to develop insights. Gartner confirms this, emphasizing that data literacy is an essential and necessary driver of business value.

This demand for data-driven competitive advantage has been growing for some time. Brandon Purcell, Forrester Research analyst, advised in 2019 this “The rise of AI and machine learning may also be a factor in the dramatic increase in demand for data scientists… Much of this is due to branding. Many companies see data scientists as the key to adopting AI or machine learning, which are the hottest technologies.

The data science exodus

The ever-increasing demand for data insights, combined with lack of data science skills, vacancies, and unsustainable workloads, is having a ripple effect. While companies often have access to data as a raw material, they also tend to lack a robust data infrastructure – a data-savvy workforce and people with advanced data skills – to actualize the value of data in innovative and efficient ways. The end result is one where existing data workers burn out and contemplate an exodus.

Despite a massive increase in demand for data scientists over the past decade, 97% of data professionals now report that they “experience burnout in their daily work”. According to a global study from a data science careers platform, data scientists now stay in their jobs for an average of just 1.7 years, well below the average software developer tenure of 4.2 years. Research has highlighted faulty data pipelines, finding and fixing data issues, unrealistic business expectations and a culture of ‘shame and blame’ as key drivers of burnout – a challenge present throughout the data skills specialty.

The discoveries of a IDC Infobrief commissioned by Alteryx supported this conclusion, pointing out that 100,000 human lives of data and analytical work hours are lost worldwide each year due to the use of legacy spreadsheet software among data native workers. The research noted that 91% of organizations reported “some areas of data and analytics skills gaps,” with a particular skills shortage involving predictive, prescriptive and machine learning.

Clearly, this skills gap is not limited to just one area of ​​data science specialization. We are not only seeing an extremely increased demand for – and therefore increased pressure on – highly skilled data scientists, but we are also seeing this reflected in data engineers, data workers, and data natives. Of the workers cited as suffering from burnout, the research noted that these data scientists were primarily proficient in Python (85%) and SQL (82%), with 56% holding a master’s degree.

Without this foundational benchmark of data talent and continuous skills upgrading across the data continuum – from data native to data engineer and data scientist – we are seeing a bullwhip effect. – one where the lack of capacity or experience of less experienced workers is taken into account by those who have more experience than them.

Retention of Data Scientists: the need for fundamental skills

The data science and analytics industry today stands at a crossroads similar to what we saw in IT teams in the mid-1900s – the fulcrum between tower specialists ivory and the global trade catalyst. These are challenges that have been addressed by embedding higher levels of sophistication across all businesses and moving away from the siled model of working.

IBM launched the world’s first consumer personal computer in 1981, helping to democratize the power of computing. The “Mark 1” of this innovation was widely used in 1944. This tool, however, was 50 feet long and eight feet high. Due to the cost involved and the experiential barrier to entry, the proprietary skills needed to access and use these machines were an absolute necessity. As this Mark 1 system evolved – becoming more accessible and user-friendly – ​​the productivity benefits became more apparent. Further democratization of technology and broadening of skills was clearly a next step towards success.

In hindsight, the levels of efficiency and productivity that we enjoy today through personal computer use would be impossible if responsibility for the Mark 1 remained siloed within the IT team. Had these teams maintained this siled approach, we would have witnessed similar burnout, mass exodus of staff, and myriad other challenges associated with an unsustainable workload and exponential demand on their time… as we are currently seeing in data science.

While the skills gap is certainly the most well-understood gap in business, one of the most pressing challenges in the data science industry today is the agreement difference. Simply put, the solution to solving the data science exodus is not for companies to hire more data scientists – the change must be fundamental. It is simply not an efficient use of resources for a master mechanic to perform oil changes.

The core strategy for mitigating data science burnout and retaining your staff is to develop a more robust data team based on a solid foundation of domain expert data analysts, data engineers and led by data scientists. More importantly, it must be facilitated by a strong set of fundamental data skills among the service’s data natives. Data teams with every level of experience honed to the point where they can enable the next step up the ladder can capitalize on what their data is telling them. Ensure that large volumes of complex and unstructured data are refined into standardized pipelines of relevant, timely and high quality data that delivers business value.

Leave a Reply