Times are uncertain, and experts argue if businesses will turn to automation to speed up the workings to match the pandemic's challenges. Pandemic has increased the speed of the transition towards automation due to social distancing measures and concerns about the spread of the infection, which has compelled the innovative use of technologies in business, education, medical, and various other sectors.
This has created a debate about whether automation will kill jobs, and the same argument is going on in the field of data science. "Will automation supplant the need for data scientists?" This is the question that often comes to the mind either to the wanna-be practitioners pondering an entry into the field or staff hoping to get data science benefits at a reduced cost.
Critics usually point out that automation in areas like data processing or data visualization will only make it easier for business experts to get what they need without human intervention. Gartner earlier forecasted that 40% of the data science task would be automated by 2020. Hence, critics debate that the need for data scientists might only decline.
Target, a big company, says that they can determine if the customer is pregnant from her purchase behavior and hope such robust tools for their data. Moreover, AI vendors promise to deliver what Target did without data scientists.
Despite this, some experts argue that automation will not reduce data scientist positions; it now needs them evermore. To justify these arguments, let's go through the article to get a clear idea of why automation won't kill jobs.
Automation is a way to do things quicker
IBM's AI vice president Alexander Gray portrayed automation as the mechanizations of boring and lengthy tasks. He said that automation tools are simply meant for time-saving and a benefit that data scientists enjoy in an interview. He says that automation is meant to make data scientists smarter and provide more robust tools to help them in their work. As much as digitalization has changed the way office workers work, data science tools will similarly change how data scientists work. This transformation will make them more valuable than ever, anticipating they keep up with the changes and not settle in the past.
Understanding Business Problems is Tough
The most crucial question in data science is the questions you must ask before even one line of code is scripted, such as what data you select & what set of questions you pick to ask of that data?
The thing missing from the popular imagination is creativity and the business know-how that goes into those tasks. Why care if our clients are pregnant? Target's data scientists had developed upon considerable work earlier to know why this was a profitable customer demographic prepared to switch vendors.
Target's data science team has baby record data tied to shopping history and understands how to relate that to customer spendings. How to measure success? Configuring non-technical needs into technical questions that can be responded with data is one of the most challenging data science tasks. Hence, experienced humans are required to formulate these questions to start on with the data science journey.
After forming a data science question, data scientists are required to draw their assumptions. This mostly displays itself in the form of data cleaning and feature engineering. Real-life data are filthy, and many speculations have to be made to connect the gap between the available data and policy questions seeking to address. These speculations are incredibly dependent on real-life knowledge and business context.
In the Target instance, data scientists had to assume the proxy variables for pregnancy, the realistic period of their study, and suitable control groups for precise comparisons. They certainly had to make real-life speculations that permitted them to throw out irrelevant data and correctly adapted features. All of this depends on human judgment. Eliminating human from the loop can be hazardous because it can create bias. Hence, it is not a fluke that many of them move around deep learning computations that make some of the strongest statements to do away with feature engineering.
While integral machine learning components are automated, the data munging and feature engineering cannot be safely automated away.
Weathering Automated Errors
As per Gray, automation offers the power to do things better and quicker and correct human errors if there is a flawed science behind it. It is way easier for this to happen than most people think.
Gray's personal experience says that even Ph.D. teams from top schools make errors in statistical computations that result in lousy data models. He adds that data scientists need a robust understanding of the hardcore principles that will not wither because human oversight will always be required for the essential applications.
In short, data scientists are needed to validate the correctness of the outcomes coming out of automated tools, and thus ensure that the models are working optimally.
Human Judgement Plays A Key Role
This article clearly suggests that no matter what, data science won't be automated away. There is another domain where extremely trained humans are creating code to make computers perform stunning tasks. These humans are paid a massive premium over those who are not trained in these fields, and there are education programs in special training for this skill. The economic pressure to automate the software engineering field is equally intense.
As software engineering has gotten easier, the need for programmers has only flourished. The saying that automation boosts efficiency, reducing the prices, and at last rising the demand is not new. This has been observed time and again in fields ranging from software engineering to accounting. Data science is not an exception, and robotization will probably propel the demand for this skillset.
Speaking of which, extremely trained and experienced data scientists will always be needed for their expertise to create data-handling code, select the correct data source, and create the optimal algorithms to remove the insights required by the organization.