This is the first part of a series by Levon Paradzhanyan that demystifies data science, machine learning, deep learning, and artificial intelligence down while explaining how they all tie into one another.
Artificial Intelligence emerged in our lives many years ago. First, as science fiction and today embedded in real products. It has since been followed by newer buzzwords such as data science, machine learning, and deep learning. Yet there are many misconceptions related to these terms. Most people think they mean the same thing. Even developers who are starting their journey in this field struggle to understand the difference between them.
In part one of this primer series, we will focus on data science, the discipline all these other methodologies belong to. Over the course of these articles, we’ll explore each technology and the fundamental asset they have in common — data.
Humanity’s First Attempts to Create Human-Like Abilities
For centuries, bright minds have invented new technologies, bringing automation to activities previously performed manually and enabling new possibilities. While these innovations lacked human-like intelligence, there are many examples in ancient Greek, Chinese, Jewish and other nation’s histories of the “automaton,” a self-operating “intelligent” machine designed to automatically perform a predetermined sequence of operations. Today, we call them “robots” and their capabilities are infinite.
But robots aren’t the only intelligent technology. Everything is getting smarter, for instance, social, audio or video streaming services, such as Facebook, Spotify and Netflix, recommend content that will resonate with the user’s preferences. Similarly, companies are increasingly processing data gathered about consumer behavior in order to improve the services they offer, detect, and prevent fraud or provide new services. And, finally, Tesla has developed self-driving, intelligent cars which are constantly improving autonomously.
Data is fueling all this.
Your Data, Your Goldmine
Think about all the data that companies and organizations are generating or collecting these days. The amount, and most importantly, the structure of it is sometimes unimaginably big and complex. Hence the term “Big Data.”
Big data claims to deliver business-changing analytics and insights. Unfortunately, much of that is mere hype. The bottom line is that big data is just another word for a large amount of data. In fact, at EastBanc Technologies, we’ve been advocating for years to start with small sets of data with what we’ve coined a Minimal Viable Prediction. The accuracy, quality and completeness of the data are what really matters, along with a good approach to analysis.
A few notable industries where this is relevant include healthcare, education, media and entertainment, government, transportation, banking, etc.
Using the power of data, these industries can uncover trends, make predictions, and improve decision-making. Data can help public health organizations understand what preventative measures they should take to minimize the risk of epidemic outbreaks. It can help guide a student towards the most suitable career based on his or her interests, strengths, and academic results. Data can help predict where traffic congestion will occur due to public events, holidays, accidents, or news alerts.
But data may also hide patterns and insights that have yet to be thought about, ones that could be potentially valuable for a company’s business intelligence/operations. For instance, data may reveal new patterns that fraudsters use to steal money from an online banking system. It can also show that if one person switches their cell phone provider, their friends are likely to follow suit or that men who buy diapers are more likely to buy beer. Or, perhaps, that individuals seeking a business loan who complete their application form in the correct care are more dependable debtors.
These are all real-life examples. So, what do we have here? Data is the foundation of any business. That’s why understanding it and finding hidden correlations, new patterns, and even discovering new insights can positively impact business success.
The challenge is that today most of an organization’s data is unstructured and segmented in different sources, such as analytical software, log files, legacy systems, cloud-based services, third-party enterprise solutions, and much more. There are no universal tools that can intelligently determine the relationships between these systems and process them in huge volumes.
This is where data science comes into play.
What Is Data Science?
Data science combines statistics, math, computer and information science, data analysis, and machine learning techniques to understand and discover hidden patterns in data and help build predictive analysis and decision-making tools.
How does data science differ from business intelligence (BI) tools? Typically, BI focuses on the analysis of historical data to interpret it, monitor the current state, and find the patterns of business trends. In contrast, data science is a more dynamic, progressive and forward-looking approach which focuses on predicting the future (predictive analytics) or providing the business with an informed decision-making capability (prescriptive analytics).
To achieve a better understanding of what data science is, it’s important to have a grasp of the role and duties of the data scientist.
Data Scientist
A data scientist is a person who can easily juggle many different fields and disciplines, a symbiosis of Sherlock Holmes, business analyst, and software developer. Indeed, a huge variety of skills and knowledge; plus the ability to dive into very different industries, analyze them and create predictive models are an essential prerequisite for any data scientist.
A data scientist is responsible for navigating the data science lifecycle, which looks like this:
Let’s break down each of these phases:
- Discovery. This involves understanding and analyzing business problems or objectives.
- Data Preparation and Mining. Here the data scientist collects data from different sources, cleans it, and transforms it to a format suitable for machine learning algorithms. He or she then creates a new feature (individual measurable property) by making assumptions or examining the hidden patterns in the data, and much more. This is probably the most important and time-consuming part of the work.
- Modeling. Now the data scientist trains machine learning models and evaluates and validates performance (accuracy).
- Visualize and Communicate the Results. Results are demonstrated using diagrams and other methods to clearly communicate and explain findings in a digestible way.
Data scientists can be involved in any manner of business use cases. For example, helping a marketing department improve the marketing techniques for retailers based on customers’ shopping and wish lists. A data scientist can also help optimize the process of price formation by analyzing consumer purchasing power, competitor price offers, sales history, popularity of the product, and more. They can also reveal hidden patterns in public trials by analyzing thousands of different factors and properties.
Data Scientist Versus Machine Learning Engineer
In some cases, the same person performs each step in the data science lifecycle, but it doesn’t have to be this way. In many cases, a Machine Learning Engineer (ML Engineer) will take over the steps of engineering the model. This typically occurs when the data scientist lacks the programming skills required for a full-stack development of the solution.
A data scientist and ML engineer have two distinct roles:
The data scientist must fully understand the business needs, the data, find the right approach to the problem, and help build and verify the results.
The ML engineer isn’t required to understand the algorithms and the overall science behind the solution, but may participate in building the Machine Learning Model (ML Model), using special technologies, programming frameworks and ensuring the data is gathered from data pipelines, cleansed and suited for the ML Model.
In this way, machine learning has enabled the function of traditional data analysis to grow into data science.
In summary, data has the huge potential to uncover unknown correlations and patterns which can improve decision-making and ultimately business outcomes. Most companies today have accumulated massive amounts of data, but it’s mostly unstructured and siloed. With no universal tool to determine relationships and effectively process the data, uncovering those trends is no easy task. That’s where data science comes in.
Data science can unify and process data to uncover hidden patterns and build predictive and prescriptive analytics tools for better decision-making. In our next article, we’ll examine machine learning and how it compares with data science.
Feature image by João Silas on Unsplash.
