DATA SCIENCE

Revision Notes

DATA SCIENCE
Data Science is a multidisciplinary field that integrates statistics, data analysis, machine
learning, and related techniques to analyse real-world data. It extracts insights and trends to
make informed decisions, enhancing the ability of machines to solve problems or perform tasks
autonomously. The core components of data science involve:
• Mathematics: Statistical models, probability theory, and algebra help understand and
predict data patterns.
• Statistics: Crucial for data summarization and analysis, providing tools for hypothesis
testing, regression analysis, etc.
• Computer Science: Implements algorithms to process and analyze large datasets
efficiently.
• Information Science: Deals with the management, retrieval, and storage of data.

Example: Playing with AI

To make the concept more accessible, the document uses a Rock, Paper, Scissors game
([Link] where users challenge an AI model.
This practical example emphasizes how AI learns from patterns and tries to anticipate user
choices. It challenges the player to win 20 games against the AI, encouraging reflection on:

• How AI responds based on data (user’s previous choices).

• Comparing human strategy vs AI strategy: Humans may strategize based on logic,
whereas AI learns patterns from data inputs.

This game helps explain how data drives the AI’s decision-making and pattern recognition,
demonstrating the power of data in making machines intelligent.

Core Domains of AI in Data Science:

Data Science is essential in different AI fields, each focusing on specific data types:
1. Data Science: Works with numeric and alpha-numeric data, essential for statistical
analysis and machine learning models.
• Example: A dataset containing sales figures, customer ages, and product prices
for predictive modelling.
2. Computer Vision (CV): Deals with image and visual data to enable machines to
understand and interpret visual information.
• Example: A self-driving car’s camera processing traffic signals and obstacles.
 3. Natural Language Processing (NLP): Focuses on textual and speech-based data,
helping machines understand and interact with human language.
• Example: Voice assistants like Siri and Alexa using NLP to understand spoken
commands.

Applications of Data Science:

Data Science has revolutionized industries by providing insights and driving decision-making in
many domains. Some notable applications include:
1. Fraud and Risk Detection (Finance):
In the early days, financial institutions struggled with defaults and losses due to bad debts. They
had extensive data about their customers’ financial history but needed effective tools to
leverage this information. Data Science algorithms helped analyze:
• Customer profiling: Identifying high-risk customers based on their past behavior.
• Predicting defaults: Using statistical models to predict which customers might default
on loans based on historical data.
Example: Banks now analyze transaction patterns and spending behavior to assess a loan
applicant’s risk level and offer customized banking products.
2. Genetics and Genomics (Healthcare):
Data Science plays a significant role in understanding genetic data and its impact on health. By
combining genomics with data analytics, researchers can:
• Personalize treatments based on an individual’s genetic makeup.
• Predict disease risk: Analyze the correlation between genetic variations and
susceptibility to certain diseases.
Example: Using genetic data to predict how a patient will respond to a specific drug, leading to
personalized medicine.
3. Internet Search Engines:
Search engines like Google use Data Science to handle vast amounts of data and deliver
relevant results within seconds. Algorithms analyze:
• User queries: Match them with indexed web pages.
• Click behavior: Improve ranking algorithms based on how users interact with search
results.
Example: Google processes over 20 petabytes of data daily. Without advanced data science
techniques, it would not be able to deliver accurate results at the speed it does.
4. Targeted Advertising (Digital Marketing):
Data Science has transformed the digital marketing landscape by enabling targeted
advertisements. Based on user data, algorithms predict:
• User preferences: Ads are tailored based on browsing history and behavior.
• Ad effectiveness: Measure and improve the click-through rate (CTR) by targeting ads at
users most likely to interact.
Example: Facebook and Instagram use past browsing behavior to serve ads relevant to the
user’s interests, resulting in higher engagement.
5. Website Recommendation Engines:
Companies like Amazon, Netflix, and YouTube rely heavily on recommendation engines powered
by Data Science. These systems analyze user behavior to suggest:
• Products on e-commerce sites based on browsing and purchase history.
• Movies or shows on streaming platforms based on watch history and ratings.
Example: Netflix’s recommendation engine suggests new shows based on what users have
previously watched, increasing user engagement and satisfaction.
6. Airline Route Planning:
Airlines face significant operational challenges, such as flight delays and optimizing routes for
fuel efficiency. Data Science helps them by:
• Predicting flight delays based on historical data (e.g., weather, traffic).
• Route optimization: Choosing whether to fly direct or via layovers to maximize efficiency.
Example: Using past data to predict the best flight routes, minimizing fuel costs, and improving
customer satisfaction.

Case Study: Predicting Food Waste in Restaurants

This section of the document presents a Data Science project example aimed at reducing food
waste in buffet restaurants. The challenge is that restaurants often overestimate the amount of
food needed, leading to waste and financial losses.
Problem Scoping:
1. Who: The primary stakeholders are restaurant owners and chefs.
2. What: The problem is that food is often left unconsumed at the end of the day, leading to
waste.
3. Where: Buffet-style restaurants where food is prepared in bulk.
4. Why: If restaurants could better predict customer turnout, they could prepare the right
amount of food, reducing waste.
Proposed Solution:
• Goal: To develop a predictive model that estimates the quantity of food to prepare daily.
• Data Required: Datasets related to daily customer numbers, dish prices, quantity
prepared, and unconsumed food over a period of 30 days.
Steps Involved:
1. Data Collection: Collect data on the number of customers, types of dishes, food
quantities prepared, and leftovers.
2. Data Exploration: Clean and preprocess the data to ensure accuracy, removing missing
values or outliers.
3. Modeling: Train a regression model on 30 days of data to predict the amount of food to
prepare based on historical consumption patterns.
4. Evaluation: Test the model’s accuracy by comparing its predictions with actual food
consumption.
Data Science Tools and Techniques:
Various tools and programming libraries are essential in Data Science, helping analysts and
developers process, analyze, and visualize data.
1. Data Collection Methods:
• Offline: Surveys, observations, and interviews conducted manually.
• Online: Data gathered from open-source websites (e.g., Kaggle) or government portals.
Examples of Data:
• Banking: Account holder details, transaction histories, and loan applications.
• Movie Theaters: Ticket sales, refreshment purchases, and customer demographics.
2. Data Storage Formats:
• CSV (Comma Separated Values): A simple text format where each data field is separated
by a comma.
• Spreadsheet: A grid format used for tabular data (e.g., Excel).
• SQL: Structured Query Language, used to manage and manipulate relational databases.
3. Python Libraries:
• NumPy: For numerical computing and working with arrays.
• Pandas: For data manipulation and handling tabular datasets (e.g., DataFrames).
• Matplotlib: For data visualization, including plotting graphs like bar charts, histograms,
and scatter plots.

Statistics in Data Science (with Python)

Basic statistics are fundamental to Data Science, providing tools to summarize and analyze
data:
1. Mean: The average value of a dataset, calculated by summing all values and dividing by
the number of values.
2. Median: The middle value of a sorted dataset, which is less sensitive to outliers than the
mean.
3. Mode: The most frequently occurring value in the dataset.
4. Standard Deviation: Measures how spread out the values are around the mean. A low
standard deviation means values are close to the mean; a high standard deviation means
they are spread out.
5. Variance: The square of the standard deviation, showing the variability of the data.

Data Visualization Techniques:

Data visualization is critical for interpreting large datasets. Some common visualizations
include:
1. Scatter Plots: Used for plotting discontinuous data, often showing relationships
between two variables (X and Y axes). Multiple parameters can be represented by color
and size of the points.
 • Example: Plotting customer age vs purchase amount with points representing
different product categories.
2. Bar Charts: Simple yet effective for visualizing categorical data, where each bar
represents a different category.
• Example: Comparing male and female participation in a survey.
3. Histograms: Show the frequency distribution of a continuous dataset, often used to
display data ranges.
• Example: Plotting the distribution of customer ages at a retail store.
4. Box Plots: Display the distribution of data across quartiles and highlight outliers, making
it useful for identifying skewness.
• Example: Analyzing salary ranges in a company and spotting outliers.

K-Nearest Neighbors (KNN) Algorithm

K-Nearest Neighbors (KNN) is a supervised learning algorithm used for both classification and
regression. It predicts outcomes by finding the ‘K’ nearest data points (neighbors) to a given point
and basing predictions on the majority class of those neighbors.
Example: Predicting Fruit Sweetness
Suppose you want to predict if a fruit is sweet or not, based on the surrounding data points
(known fruits).
• K=1: The closest point to the unknown fruit is used to predict sweetness.
• K=3: The three nearest neighbors are considered, and if two are sweet and one is not, the
model predicts the fruit is sweet.
The algorithm works on the principle that similar data points exist near each other.