AI & DIGITAL SKILLS MAGAZINE Learn from beginner to expert
Contact EDITION 2026
ISHRAQA7 — The practical digital learning magazine
ISHRAQA 7 ACADEMY · Excel & Data

Python & Data Science: from your first line of code to an analysis project

Twelve progressive chapters: Python basics, files and environments, NumPy, pandas, cleaning, exploratory analysis, statistics, Matplotlib and a first validated scikit-learn model, with scripts and Jupyter notebooks that were actually run.

  • LevelBeginner → Advanced
  • Course programme12 modules
  • Course filesPDF · TXT · PY · IPYNB · CSV · ZIP

Also available in Français العربية

This ISHRAQA7 Academy course teaches you to analyse data with Python, from the basics of the language to a first validated machine learning model. The twelve chapters work on the same case study, the sales activity of an office equipment distributor, and end with a complete project whose report is generated by the code.

Learning path

Level Chapters
Beginner 1. First steps · 2. Variables and structures · 3. Conditions, loops, functions · 4. Files and environments
Intermediate 5. NumPy · 6. pandas · 7. Cleaning · 8. Exploratory analysis · 9. Descriptive statistics · 10. Matplotlib
Advanced 11. Machine learning with scikit-learn · 12. Final project

Course files

The download section offers: the complete manual, the exercise workbook, the worked answers, the installation guide and the final project report as PDF; the CSV data; the twelve Python scripts; the Jupyter notebooks (executed solutions and exercises to complete); the requirements.txt file; and a complete practice kit (data, scripts, notebooks and requirements.txt, without the PDFs).

Level 1 · Beginner — Chapter 1

Python for data analysis: first steps

Python has become the reference language for data analysis: it is free, readable and surrounded by an ecosystem of specialised libraries. This chapter sets up the working environment and runs a first script that reads a real data file.

Why Python for data

A spreadsheet soon reaches its limits: calculations repeated by hand, large files, processing that is hard to replay. A Python script describes every step in writing; you run it again on next month’s data and it applies exactly the same processing. The course libraries form a consistent chain:

Library Role Chapters
csv, pathlib, json (standard) Read and write files 1 to 4
NumPy Numerical computing on arrays 5
pandas Data tables: reading, joins, aggregation, cleaning 6 to 9
Matplotlib Charts 10
scikit-learn Machine learning 11

Setting up the environment

  1. Install Python 3.13 from python.org (on Windows, tick “Add python.exe to PATH”). The scripts were run with Python 3.13.
  2. Unzip the course archive into a short folder, for example C:\ISHRAQA7\python\. It contains data/, scripts/, notebooks/ and requirements.txt.
  3. Create the environment and install the libraries; the detailed procedure is in chapter 4 and in the installation guide.
  4. Run jupyter lab from that folder: the browser shows the project tree.

A script (.py) runs in one go: python scripts/ch01.py. A Jupyter notebook (.ipynb) splits the same code into cells run one by one, with results displayed below each cell: the ideal tool for exploring. Each chapter is supplied in both forms, plus an exercise notebook to complete.

A first script

import csv
from pathlib import Path

DATA = Path('data')
with open(DATA / 'orders.csv', encoding='utf-8', newline='') as f:
    orders = list(csv.DictReader(f))

print('Number of orders:', len(orders))
print('First order:', orders[0])

csv.DictReader turns each row into a dictionary whose keys are the column headers: orders[0]['status'] gives the status of the first order. The encoding='utf-8' parameter guarantees that accented and Arabic characters are read correctly. The script shows 3,307 orders.

Counting by category

by_status = {}
for o in orders:
    by_status[o['status']] = by_status.get(o['status'], 0) + 1
print(by_status)

The get(key, 0) method returns 0 when the key does not exist yet: the dictionary fills up as the loop runs. Result: 3,108 delivered orders, 117 cancelled and 82 returned.

The course case study

Every chapter uses the same dataset: the activity of an office equipment distributor in 2024 and 2025 (orders, order lines, customers, products, regions, channels, targets), plus a deliberately imperfect extract for cleaning. The ISHRAQA7 Academy project in chapter 12 brings all the steps together.

Level 1 · Beginner — Chapter 2

Variables, types and data structures

Before analysing thousands of rows, you need to know how to represent a value, a list of values or a complete record. This chapter presents Python’s basic types and the four data structures used all the time.

Variables and types

product_id = 'P11'     # str: text
quantity = 4           # int: integer
unit_price = 2450.0    # float: decimal number
delivered = True       # bool: true or false
print(f'{quantity} × {unit_price:,.2f} = {quantity * unit_price:,.2f}')

A variable is a name attached to a value; Python infers the type from the value. An f-string (a string prefixed with f) inserts expressions between braces and formats them: :,.2f shows two decimals with a thousands separator.

Converting what you read

A CSV file contains only text. Before calculating, convert explicitly: int('4'), float('2450.00'). An impossible conversion (int('4 pcs')) raises a ValueError: this is useful, because unexpected data is flagged instead of being silently misread (chapter 3).

with open(DATA / 'products.csv', encoding='utf-8', newline='') as f:
    products = [
        {'id': r['product_id'], 'name': r['product_name'], 'cat': r['category_code'],
         'price': float(r['list_price']), 'cost': float(r['unit_cost'])}
        for r in csv.DictReader(f)
    ]

This block is a list comprehension: it builds a list in one expression from each row read. We get 30 products, with prices and costs already numeric.

Four structures, four uses

Structure Syntax Property Typical use
List [2450.0, 52.0] Ordered, mutable A series of values
Tuple (52.0, 'P15') Ordered, immutable A pair or a fixed record
Set {'FUR', 'ITE'} No duplicates, no order Distinct values, membership tests
Dictionary {'P11': 2450.0} Key → value Fast lookup by identifier
prices = [p['price'] for p in products]
cheapest = min((p['price'], p['id']) for p in products)
categories = {p['cat'] for p in products}
price_of = {p['id']: p['price'] for p in products}

The cheapest product is P15 (52.00 MAD): min compares tuples by their first element, the price. The set categories contains four codes; the dictionary price_of finds a price without scanning the list, like an instant VLOOKUP.

Sorting with a key

top5 = sorted(((p['id'], p['price']) for p in products),
              key=lambda t: t[1], reverse=True)[:5]

sorted accepts a key function that says what to sort on; lambda t: t[1] is a small anonymous function returning the price. The slice [:5] keeps the first five elements.

Level 1 · Beginner — Chapter 3

Conditions, loops, functions and modules

Data processing combines decisions (if… then…), repetition (for each row…) and reusable operations. This chapter teaches you to write clear functions and to rely on the modules of the standard library.

Functions: naming a business rule

def net_amount(quantity, unit_price, discount_rate=0.0):
    """Net amount of a line: quantity × price × (1 − discount)."""
    return quantity * unit_price * (1 - discount_rate)

print(net_amount(4, 2450.0, 0.05))   # 9310.0

A function has a name, receives parameters and returns a result with return. The parameter discount_rate=0.0 has a default value: it can be omitted. The string in triple quotes, the docstring, documents the rule. Writing the rule once avoids having three different versions of it in the code.

Loops and a lookup dictionary

orders = {o['order_id']: o for o in read_csv('orders.csv')}
total_by_status = {}
for l in lines:
    status = orders[l['order_id']]['status']
    amount = net_amount(int(l['quantity']), float(l['unit_price']), float(l['discount_rate']))
    total_by_status[status] = total_by_status.get(status, 0) + amount

Each order line must find the status of its order. The orders dictionary, keyed by order number, acts as a lookup table. Result: 28,654,029.30 MAD for delivered orders, 1,090,232.00 MAD for cancelled ones and 802,042.15 MAD for returned ones. Only the first amount corresponds to completed sales.

Conditions

def size_class(amount):
    if amount >= 10000:
        return 'L'
    elif amount >= 2000:
        return 'M'
    return 'S'

Conditions are tested in order; the first true one wins, so the descending order of the thresholds is essential. Out of 8,472 lines: 417 large, 4,949 medium and 3,106 small.

Modules

A module groups ready-made functions: import statistics gives access to statistics.mean, median, stdev. The standard library also contains csv, json, pathlib, datetime. Your own functions can be stored in a tools.py file and imported by several scripts.

Handling errors

try:
    q = int(text)
except ValueError:
    q = None   # flagged value, handled later

try … except catches an expected error. Catch only the expected error (ValueError), never all errors: a bare except: would also hide typos in the code.

Level 1 · Beginner — Chapter 4

Files, dependencies and environments

An analysis project must be movable, shareable and reinstallable without surprises. This chapter covers file paths, exchange formats, virtual environments and the requirements.txt file.

Portable paths with pathlib

from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent   # project folder
DATA = ROOT / 'data'
for p in sorted(DATA.glob('*.csv')):
    print(f'{p.name:<24} {p.stat().st_size:>9,} bytes')

The / operator joins paths with the right separator on Windows, macOS and Linux. Calculating ROOT from the script’s location (__file__) lets you run the script from any folder. In a notebook, where __file__ does not exist, the course scripts use the notebook’s parent folder. The data folder holds 11 CSV files.

Exchange formats

Format Strengths Watch out for
CSV Universal, readable, light Delimiter, encoding, decimals, everything is text
JSON Nested structures, simple types Less suited to large tables
Excel (.xlsx) Read by every department Formatting and merged cells disrupt reading
inventory = [{'file': p.name, 'bytes': p.stat().st_size, 'rows': rows[p.stem]} for p in files]
target = OUT / 'inventory.json'
target.write_text(json.dumps(inventory, ensure_ascii=False, indent=1), encoding='utf-8')

ensure_ascii=False keeps accents and Arabic readable in the file. Always write in UTF-8 and say so explicitly when reading.

Virtual environments and dependencies

Each project should have its own virtual environment: a folder holding an isolated copy of Python and the project’s libraries. Two projects can then use different versions of pandas without conflict.

python -m venv .venv
.venv\Scripts\activate          # Windows
source .venv/bin/activate       # macOS / Linux
python -m pip install -r requirements.txt

The course requirements.txt file pins the versions with which every script and notebook was run: numpy 2.5.3, pandas 3.0.5, matplotlib 3.11.2, scikit-learn 1.9.1, plus JupyterLab. Pinning versions ensures that the course results can be reproduced identically; other versions often work, but without guarantee.

Checking from a script

import importlib.metadata as md
for pkg in ['numpy', 'pandas', 'matplotlib', 'scikit-learn']:
    print(pkg, md.version(pkg))

This check, placed at the top of a project, saves a long search for the cause of a difference that comes from another version.

Level 2 · Intermediate — Chapter 5

Numerical computing with NumPy

NumPy is the foundation of scientific computing in Python: it handles arrays of numbers and applies an operation to all their values at once. pandas and scikit-learn are built on it. This chapter shows how to replace loops with vectorised calculation.

From dictionaries to arrays

import numpy as np
qty = np.array([int(r['quantity']) for r in rows])
price = np.array([float(r['unit_price']) for r in rows])
disc = np.array([float(r['discount_rate']) for r in rows])
print(qty.shape, qty.dtype, price.dtype)   # (8472,) int64 float64

A NumPy array holds values of a single type, stored compactly in memory. Its shape gives its dimensions: here a vector of 8,472 values, one per order line. The three arrays are aligned: position 0 of each describes the same line.

Vectorised calculation

net = qty * price * (1 - disc)
print(net[:5].round(2), round(float(net.sum()), 2))

Multiplication applies element by element, with no Python loop: this is vectorisation. The code is shorter and dozens of times faster than a loop over thousands of rows. The expression 1 - disc illustrates broadcasting: the number 1 is “stretched” to the whole length of the array. Total over all lines, all statuses: 30,546,303.45 MAD.

Boolean masks

big = net > 10_000            # array of True / False
print(int(big.sum()))         # number of lines concerned
share = net[big].sum() / net.sum()

A comparison produces an array of booleans, the mask. big.sum() counts the true values (417 lines) and net[big] selects the matching amounts. These large lines, under 5% of all lines, account for 24.3% of the total amount: a common concentration in sales analysis.

Aggregations and statistics

Function Result
a.sum(), a.mean(), a.std() Sum, mean, standard deviation
a.min(), a.max(), a.argmax() Extremes and position of the maximum
np.percentile(a, [25, 50, 75]) Quartiles
np.abs, np.round, np.where Element-wise functions

np.where(net > 10_000, 'L', 'S') applies a condition to each element and replaces a loop with if/else.

Reproducible randomness

rng = np.random.default_rng(2026)
sim = rng.uniform(0, 0.15, size=1000)

A generator created with a fixed seed always produces the same sequence of numbers: a simulation becomes reproducible and checkable. This is essential for sharing an analysis or a model.

Level 2 · Intermediate — Chapter 6

Working with tables in pandas

pandas is the analyst’s core Python tool: it reads files, selects, filters, joins and aggregates tables, like a programmable spreadsheet. This chapter builds the case-study sales table from five files.

Reading with controlled types

import pandas as pd
orders = pd.read_csv(DATA / 'orders.csv',
                     dtype={'order_id': str, 'customer_id': str},
                     parse_dates=['order_date', 'ship_date'])
orders.info()

A DataFrame is a table: named columns, each of one type (a Series). Declaring identifiers as text stops “00125” from becoming 125; parse_dates converts dates. info() shows the number of rows, the types and the missing values. Dimensions: orders (3307, 7), lines (8472, 6).

Selecting and filtering

delivered = orders.loc[orders['status'] == 'delivered',
                       ['order_id', 'order_date', 'customer_id', 'channel_code']]
orders.iloc[:3, :4]   # first 3 rows, first 4 columns

loc selects by label or boolean mask, iloc by position. The filter gives 3,108 delivered orders. To combine conditions, use & (and), | (or) and parentheses: (orders['status'] == 'delivered') & (orders['channel_code'] == 'WEB').

Joining tables

sales = (lines
    .merge(orders, on='order_id', how='left', validate='many_to_one')
    .merge(products[['product_id', 'product_name', 'category_code', 'unit_cost']],
           on='product_id', how='left', validate='many_to_one')
    .merge(customers[['customer_id', 'segment_code', 'region_code']],
           on='customer_id', how='left', validate='many_to_one')
    .merge(regions, on='region_code', how='left', validate='many_to_one'))
sales['net'] = sales['quantity'] * sales['unit_price'] * (1 - sales['discount_rate'])
assert len(sales) == len(lines)

merge is the equivalent of a VLOOKUP, only safer. The argument validate='many_to_one' checks that every key on the right is unique: a duplicated key would multiply rows and inflate sales without any error message. The final assert confirms that the sales table has as many rows as the order lines; it has 20 columns.

Aggregating

ok = sales[sales['status'] == 'delivered']
by_year = ok.groupby('year')['net'].sum().round(2)

groupby splits the table by key value, applies a function to each group and combines the results, like a PivotTable. Delivered net sales: 2024 = 13,556,438.85 MAD, 2025 = 15,097,590.45 MAD. For several indicators at once: .agg(net=('net', 'sum'), orders=('order_id', 'nunique')).

Good practice

  • Chain operations inside parentheses (method chaining) to read the processing from top to bottom.
  • Create a new variable at each important step (ok, y25) rather than modifying the original table.
  • Check every join: row count, validate, missing values after the join.

Level 2 · Intermediate — Chapter 7

Data cleaning and preparation

Real data contains duplicates, inconsistent formats and missing values. Reliable cleaning follows a method: diagnose, apply validated rules, verify with numeric checks. This chapter processes the case study’s imperfect extract.

Diagnose first

raw = pd.read_csv(DATA / 'orders_raw_export.csv', dtype=str, keep_default_na=False)
diag = {
    'duplicates': int(raw.duplicated().sum()),
    'dmy_dates': int(raw['order_date'].str.contains('/').sum()),
    'city_spaces_or_case': int((raw['city'] != raw['city'].str.strip().str.title()).sum()),
    'qty_not_digits': int((~raw['quantity'].str.fullmatch(r'\d+')).sum()),
    'comma_prices': int(raw['unit_price'].str.contains(',').sum()),
    'missing_customer': int((raw['customer_id'].str.strip() == '').sum()),
}

Read everything as text (dtype=str) without automatic conversion of empty cells (keep_default_na=False): conversion comes after diagnosis. Out of 1,174 rows: 28 exact duplicates, 61 dd/mm/yyyy dates, 34 badly entered cities, 35 non-numeric or negative quantities, 43 prices with a comma and 14 missing customers. The .str methods apply text functions to a whole column.

Rules validated by the business

Defect Rule pandas
Exact duplicates Remove drop_duplicates()
Two date formats ISO, otherwise dd/mm/yyyy to_datetime(..., errors='coerce') + fillna
Spaces and case Trim, capitalise str.strip().str.title()
Quantity “4 pcs” or “-2” Keep the digits str.replace(r'\D', '', regex=True)
Decimal comma Replace with a dot str.replace(',', '.')
Price entered ×100 Divide if > 10 × list price join + loc
Missing customer Take the order’s customer map
def clean(raw, products, orders):
    df = raw.drop_duplicates().copy()
    d1 = pd.to_datetime(df['order_date'], format='%Y-%m-%d', errors='coerce')
    d2 = pd.to_datetime(df['order_date'], format='%d/%m/%Y', errors='coerce')
    df['order_date'] = d1.fillna(d2)
    df['city'] = df['city'].str.strip().str.title()
    df['quantity'] = df['quantity'].str.replace(r'\D', '', regex=True).astype(int)
    df['unit_price'] = df['unit_price'].str.replace(',', '.', regex=False).astype(float)
    df['discount_rate'] = df['discount_rate'].astype(float)
    df = df.merge(products[['product_id', 'list_price']], on='product_id', how='left', validate='many_to_one')
    too_high = df['unit_price'] > 10 * df['list_price']
    df.loc[too_high, 'unit_price'] = df.loc[too_high, 'unit_price'] / 100
    missing = df['customer_id'].str.strip() == ''
    df.loc[missing, 'customer_id'] = df.loc[missing, 'order_id'].map(orders.set_index('order_id')['customer_id'])
    df['net'] = df['quantity'] * df['unit_price'] * (1 - df['discount_rate'])
    return df.drop(columns='list_price'), int(too_high.sum()), int(missing.sum())

Each rule is a decision by the department that knows the data; here, the sales department confirmed that negative quantities are sign errors and that the orders table is the reference for the customer. The function corrects 4 prices and fills in 14 customers. errors='coerce' turns an unreadable date into a missing value instead of stopping the script: we then check that none remain.

Missing values: decide, don’t guess

Faced with a missing value, there are four options: recover it from a reliable source (the customer from the order), calculate it with a documented rule, exclude the row and report it, or leave it missing if the analysis allows. Replacing it with an average “to make it work” distorts results without saying so.

Level 2 · Intermediate — Chapter 8

Exploratory data analysis (EDA)

Exploratory analysis means questioning the data before concluding: distributions, cross-tabulations, changes over time. It brings out useful questions and anomalies. This chapter builds an order-level table and explores it methodically.

Choosing the grain

The lines table answers “which products?” but not “how much is an order worth?”. So we build a table at order grain:

lines['net'] = lines['quantity'] * lines['unit_price'] * (1 - lines['discount_rate'])
per_order = lines.groupby('order_id').agg(net=('net', 'sum'), n_lines=('line_no', 'count')).reset_index()
od = (orders.merge(per_order, on='order_id', how='left', validate='one_to_one')
            .merge(customers[['customer_id', 'segment_code']], on='customer_id', how='left')
            .merge(segments, on='segment_code', how='left')
            .merge(channels, on='channel_code', how='left'))
od['delay'] = (od['ship_date'] - od['order_date']).dt.days

The delay is calculated by subtracting dates; .dt.days extracts the number of days. It is missing for cancelled orders, which were never shipped.

Describing

od.describe() summarises the numeric columns: count, mean, standard deviation, quartiles. value_counts(normalize=True) gives the proportions of a qualitative variable. Always read these summaries before going further: a negative minimum, an outsized maximum or an incomplete count signal a problem.

Crossing

pd.crosstab(od['channel_name'], od['status'], normalize='index').round(3)

crosstab counts the combinations of two variables; normalize='index' turns counts into proportions by row, to compare channels of different sizes. pivot_table does the same with an aggregated value (sum of sales by region and month, for example).

Time

dl = od[od['status'] == 'delivered'].set_index('order_date')
monthly = dl['net'].resample('MS').sum()

With a date index, resample('MS') groups by month (“MS”: month start); the series has 24 months. To neutralise seasonality, compare a period with the same period of the previous year, never with the previous month.

A method, not a list of charts

  1. Ask a precise question (“which channel cancels the most?”).
  2. Choose the right grain and measure (order, cancellation rate).
  3. Calculate, then check the order of magnitude.
  4. Note the finding and the next question it raises.

Level 2 · Intermediate — Chapter 9

Descriptive statistics

A mean alone does not describe data. Descriptive statistics summarise a distribution by its central tendency, spread, shape and relationships with other variables. This chapter applies them to order values and teaches you to interpret them with care.

Central tendency and spread

s = v['net']   # delivered orders, 2025
stats = {'mean': s.mean(), 'median': s.median(), 'std': s.std(),
         'q1': s.quantile(0.25), 'q3': s.quantile(0.75), 'skew': s.skew()}
Statistic Value (2025, 1,628 orders) Reading
Mean 9,274 MAD The “shared” value if all orders were equal
Median 7,132 MAD Half of the orders are below it
First / third quartile 3,739 MAD / 11,989 MAD The middle half of the orders lies between these bounds
Standard deviation 8,548 MAD Spread around the mean
Coefficient of variation 0.92 Standard deviation relative to the mean, unitless
Skewness 2.69 Positive: long tail towards large values

The mean is clearly above the median and the skewness is positive: a few very large orders pull the mean up. To describe a “typical” order, the median is more representative; for total revenue, it is the sum (and therefore the mean) that matters.

Outliers: the IQR rule

iqr = q3 - q1
upper = q3 + 1.5 * iqr
outliers = v[s > upper]

The interquartile range (IQR = 8,250 MAD) measures the spread of the middle half; it is not very sensitive to extremes. Above Q3 + 1.5 × IQR (24,363 MAD), a value is called an outlier: 87 orders, accounting for 20.6% of sales. An outlier is not necessarily an error: here they are real large orders, to be analysed rather than deleted.

Correlation

v['n_lines'].corr(v['net'])                      # Pearson
v['n_lines'].corr(v['net'], method='spearman')   # Spearman (ranks)

The Pearson coefficient measures a linear relationship, between −1 and 1; Spearman works on ranks and captures any increasing relationship, resisting extreme values better. A correlation, even a strong one, does not prove causation: two variables can both depend on a third.

Comparing groups

To compare channels or years, put mean, median and count side by side: v.groupby('channel_code')['net'].agg(['mean', 'median', 'count']). A difference in means driven by a few exceptional orders does not mean the same thing as a shift of the whole distribution.

Level 2 · Intermediate — Chapter 10

Visualising with Matplotlib

A good chart makes a result clear in a few seconds. Matplotlib is Python’s core visualisation library. This chapter builds the essential charts of sales analysis and the rules that make them readable.

Figure and axes

import matplotlib.pyplot as plt
m = sales.pivot_table(index='month', columns='year', values='net', aggfunc='sum') / 1000
fig, ax = plt.subplots(figsize=(8, 4))
ax.plot(m.index, m[2025], color='#10263A', marker='o', label='2025')
ax.plot(m.index, m[2024], color='#9AA8B5', linestyle='--', label='2024')
ax.set_title('Monthly net sales (thousand MAD)')
ax.set_xticks(range(1, 13))
ax.legend(frameon=False)
ax.spines[['top', 'right']].set_visible(False)
fig.savefig(OUT / 'fig_monthly.png', dpi=150)

The figure is the page; the axes (ax) are the plotting area. Working with ax rather than the global functions lets you control each chart in a panel. Dividing by 1,000 and saying so in the title makes the axes readable. The strong colour is reserved for the year under study; the comparison year stays grey.

Which chart for which question?

Question Chart Matplotlib
Change over time Line ax.plot
Comparing categories Sorted horizontal bars ax.barh
Distribution of a variable Histogram ax.hist
Relationship between two variables Scatter plot ax.scatter

Bars with a target marker

order = reg.sort_values().index
ax.barh(order, reg[order] / 1e6, color='#10263A')
ax.scatter(tgt[order] / 1e6, order, color='#C89D62', marker='|', s=400, linewidths=3, label='Target')

Sorting the bars makes comparison easier; the gold tick places each region’s annual target on the same scale. You immediately see which regions exceed their target and which fall short.

Panels and saving

plt.subplots(2, 2, figsize=(10, 7)) creates four axes in a grid; fig.tight_layout() prevents overlaps. Save as PNG at 150 dots per inch for a report, as SVG for a printed document. In a script, figures are only saved; in a notebook, they are also shown under the cell.

Readability rules

  • A title that says what the chart shows, units included.
  • An accent colour for what matters, grey for the rest.
  • No 3D effects and no pie chart with more than five slices.
  • Axes starting at zero for bars.

Level 3 · Advanced — Chapter 11

Introduction to machine learning with scikit-learn

Machine learning learns a relationship from examples to estimate an unknown value. Used well, it complements analysis; poorly validated, it produces misleading numbers. This chapter builds a first model and, above all, teaches you to measure its quality honestly.

The question

Can we estimate a customer’s 2025 purchases from their 2024 behaviour? We build a table of one row per customer (120 customers) with 2024 features (purchases, number of orders, average discount, share of online orders, segment, region) and the target: delivered purchases in 2025. The simple correlation between 2024 and 2025 purchases is 0.34.

A pipeline

X = data[['net24', 'orders24', 'disc24', 'web24', 'segment_code', 'region_code']]
y = data['net25']
prep = ColumnTransformer([('cat', OneHotEncoder(handle_unknown='ignore'),
                           ['segment_code', 'region_code'])], remainder='passthrough')
model = Pipeline([('prep', prep), ('reg', LinearRegression())])

A model reads only numbers: OneHotEncoder turns each segment and region into 0/1 columns. The pipeline chains preparation and model; it guarantees that preparation is learned from the training data only, with no information leaking from the test set.

Validating honestly

cv = KFold(n_splits=5, shuffle=True, random_state=2026)
scores = cross_validate(model, X, y, cv=cv, scoring=('neg_mean_absolute_error', 'r2'))

Cross-validation splits the customers into five groups; each group is used once for testing while the model learns from the other four. This gives a measure less dependent on the luck of a single split. Two metrics: the mean absolute error (MAE), in dirhams, and R², the share of variance explained (1 = perfect, 0 = no better than the mean, negative = worse than the mean).

Always compare with a baseline

The naive baseline estimates 2025 purchases as 2024 purchases × average growth, with growth calculated on the training folds only. Cross-validation results:

Approach MAE R²
Naive baseline 68,553 MAD -1.07
Linear regression 47,006 MAD 0.04

The model reduces the mean error by 31.4% compared with the baseline, but its R² stays close to zero: it does barely better than an estimate by the mean. Honest conclusion: with these features and two years of data, a customer’s 2025 purchases are hard to predict; the model helps avoid the large errors of the naive baseline, not to make fine forecasts. Calling the model “high-performing” would be misleading.

Level 3 · Advanced — Chapter 12

Final project: end-to-end sales analysis

The final project brings the whole chain together: import, check, prepare, compute indicators, visualise, analyse, model and report. It produces a report in which every figure is calculated by the code, which is what is expected of a data analyst at work.

The sponsor and the question

ISHRAQA7 Academy project: end-to-end sales analysis. The sales management of an office equipment distributor (case study, practice dataset) wants a 2025 review: how do sales compare with 2024 and with target, which regions and categories explain the gaps, how are orders distributed, and can customers’ future purchases be estimated?

The eight steps

Step Content Output
1. Import Ten tables, explicit types Dictionary of DataFrames
2. Quality Unique keys, foreign keys, missing values Blocking checks
3. Preparation Sales table: net amount, cost, delay One row per order line
4. Indicators Sales, growth, margin, basket, target, cancellations, delay kpi.csv
5. Charts Trend, regions, categories Three PNG files
6. Statistics Median, mean, IQR, outliers Summary
7. Model Regression validated by cross-validation MAE, R², baseline
8. Report Generated Markdown summary report.md

Checks that stop the process

q = {
    'dup_order_id': int(T['orders']['order_id'].duplicated().sum()),
    'orphan_lines': int((~T['order_lines']['order_id'].isin(T['orders']['order_id'])).sum()),
    'missing_ship_not_cancelled': int((T['orders']['ship_date'].isna() & (T['orders']['status'] != 'cancelled')).sum()),
}
assert all(v == 0 for v in q.values()), q

If a key is duplicated or orphaned, the script stops and shows the failing check: no report is better than a wrong report. All the project’s checks are 0 on the course data.

A generated report, not a copied one

report = f"""| Net sales (MAD) | {fmt(k4['net'], ',.0f')} | {fmt(k5['net'], ',.0f')} |
| Margin rate | {fmt(k4['margin_rate'], '.1%')} | {fmt(k5['margin_rate'], '.1%')} |"""
(P / 'report.md').write_text(report, encoding='utf-8')

The script’s fmt function adapts separators to the report language: comma for thousands and a decimal point in English.

Every value in the report is inserted by the code from the calculations: if the data changes, just run the script again. Markdown reads as is and converts easily to HTML or PDF.

Results obtained on the course data

2025 indicator Value
Net sales (2024) 15,097,590 MAD (13,556,439 MAD)
Growth 11.4%
Margin rate (2024) 38.3% (36.3%)
Target attainment 99.4%
Delivered orders / average order value 1,628 / 9,274 MAD
Cancellation rate / average delay 3.6% / 2.4 days

Assessment grid

Criterion Points
Code organisation: functions, names, comments, relative paths 20
Data quality: blocking, documented checks 20
Accuracy of the indicators (reference values) 20
Readable, relevant charts 15
Validated statistics and model, limits explained 15
Generated report, reasoned conclusions 10

Final project brief

  • Import the ten tables with explicit types.
  • Write blocking quality checks (keys, missing values).
  • Build the sales table with net amount, cost and delay.
  • Calculate the 2024 and 2025 indicators and save them to kpi.csv.
  • Produce three charts and a statistical analysis of orders.
  • Estimate customer purchases with cross-validation and a naive baseline.
  • Generate a Markdown report and write three recommendations.

Check your knowledge

1. Python for data analysis: first steps
1. Which course library is used to work with data tables?
2. What does csv.DictReader return for each row of the file?
3. What is the main benefit of a Jupyter notebook?
2. Variables, types and data structures
1. Which structure should you choose to find a product’s price quickly from its code?
2. What does the expression int('4 pcs') produce?
3. What does the set {p['cat'] for p in products} contain?
3. Conditions, loops, functions and modules
1. What happens if you call net_amount(4, 2450.0) without a discount?
2. In an if / elif chain, which branch runs?
3. Why avoid an except: with no error type?
4. Files, dependencies and environments
1. Why calculate ROOT from __file__?
2. What is a virtual environment for?
3. What does a requirements.txt with pinned versions guarantee?
5. Numerical computing with NumPy
1. What does qty * price do when qty and price are two NumPy arrays of the same length?
2. What does (net > 10_000).sum() return?
3. Why fix the seed of a random generator?
6. Working with tables in pandas
1. Why declare dtype={'order_id': str} when reading?
2. What does validate='many_to_one' check in a merge?
3. Which method counts the distinct orders in a table at line grain?
7. Data cleaning and preparation
1. Why read the raw extract with dtype=str?
2. What does errors='coerce' do in pd.to_datetime?
3. What is the best final check of a cleaning process?
8. Exploratory data analysis (EDA)
1. Why build an order-level table to study the basket?
2. What does normalize='index' do in pd.crosstab?
3. In a seasonal business, what should a quarter’s sales be compared with?
9. Descriptive statistics
1. To describe a “typical” order in a very skewed distribution, which indicator should you favour?
2. What does a value above Q3 + 1.5 × IQR mean?
3. Does a correlation of 0.8 between two variables prove that one causes the other?
10. Visualising with Matplotlib
1. Which chart should you choose to compare the sales of four regions?
2. Why work with the ax object rather than the global plt functions?
3. What does fig.tight_layout() do?
11. Introduction to machine learning with scikit-learn
1. Why use a pipeline that includes data preparation?
2. What does an R² close to 0 in cross-validation mean?
3. Why compare the model with a naive baseline?
12. Final project: end-to-end sales analysis
1. Why should a quality check stop the script when it fails?
2. What is the benefit of a report generated by the code?
3. How should you present a model whose R² is close to 0?

Glossary for this page

The key terms used in this content, each with a short definition.

Python
A readable programming language widely used for data analysis, automation and AI.
CSV
A plain text file with delimiter-separated values; the most common format for moving data between programs.
pivot table
An Excel tool that summarises thousands of rows in seconds: totals, averages and counts by the categories you choose.
key performance indicator
A figure tracked regularly to measure progress toward a specific goal, such as margin or delay rate.

Course files

Choose a language, then the format you need.

العربية

FileFormatSizeDownload
الدليل الكامل للدورة PDF 1.8 MB Download — الدليل الكامل للدورة (PDF)
دفتر التمارين PDF 618 KB Download — دفتر التمارين (PDF)
الحلول المشروحة PDF 690 KB Download — الحلول المشروحة (PDF)
دليل التثبيت PDF 145 KB Download — دليل التثبيت (PDF)
المشروع النهائي: التقرير والمنهجية PDF 262 KB Download — المشروع النهائي: التقرير والمنهجية (PDF)
سكربت المشروع النهائي ⁦(Python)⁩ Python 12 KB Download — سكربت المشروع النهائي ⁦(Python)⁩ (Python)
دفتر المشروع النهائي، منفَّذ Notebook 220 KB Download — دفتر المشروع النهائي، منفَّذ (Notebook)
بيانات ⁦CSV⁩ للحالة الدراسية (11 ملفًا) ZIP 83 KB Download — بيانات ⁦CSV⁩ للحالة الدراسية (11 ملفًا) (ZIP)
سكربتات ⁦Python⁩ للفصول الاثني عشر ZIP 23 KB Download — سكربتات ⁦Python⁩ للفصول الاثني عشر (ZIP)
دفاتر ⁦Jupyter⁩: حلول منفَّذة وتمارين ZIP 450 KB Download — دفاتر ⁦Jupyter⁩: حلول منفَّذة وتمارين (ZIP)
الحزمة التطبيقية الكاملة ZIP 557 KB Download — الحزمة التطبيقية الكاملة (ZIP)
Dependencies (requirements.txt) Shared TXT 1 KB Download — Dependencies (requirements.txt) (TXT)
Raw extract to clean (chapter 7) Shared CSV 57 KB Download — Raw extract to clean (chapter 7) (CSV)

Français

FileFormatSizeDownload
Manuel complet de la formation PDF 1.6 MB Download — Manuel complet de la formation (PDF)
Cahier d’exercices PDF 575 KB Download — Cahier d’exercices (PDF)
Corrigés commentés PDF 625 KB Download — Corrigés commentés (PDF)
Guide d’installation PDF 117 KB Download — Guide d’installation (PDF)
Projet final : rapport et méthode PDF 244 KB Download — Projet final : rapport et méthode (PDF)
Script du projet final (Python) Python 11 KB Download — Script du projet final (Python) (Python)
Carnet du projet final, exécuté Notebook 232 KB Download — Carnet du projet final, exécuté (Notebook)
Données CSV du cas pratique (11 fichiers) ZIP 83 KB Download — Données CSV du cas pratique (11 fichiers) (ZIP)
Les 12 scripts Python des chapitres ZIP 22 KB Download — Les 12 scripts Python des chapitres (ZIP)
Carnets Jupyter : corrigés exécutés et exercices ZIP 466 KB Download — Carnets Jupyter : corrigés exécutés et exercices (ZIP)
Kit pratique complet ZIP 572 KB Download — Kit pratique complet (ZIP)
Dependencies (requirements.txt) Shared TXT 1 KB Download — Dependencies (requirements.txt) (TXT)
Raw extract to clean (chapter 7) Shared CSV 57 KB Download — Raw extract to clean (chapter 7) (CSV)

English

FileFormatSizeDownload
Complete course manual PDF 1.6 MB Download — Complete course manual (PDF)
Exercise workbook PDF 556 KB Download — Exercise workbook (PDF)
Worked answers PDF 609 KB Download — Worked answers (PDF)
Installation guide PDF 116 KB Download — Installation guide (PDF)
Final project: report and method PDF 237 KB Download — Final project: report and method (PDF)
Final project script (Python) Python 10 KB Download — Final project script (Python) (Python)
Final project notebook, executed Notebook 226 KB Download — Final project notebook, executed (Notebook)
Case-study CSV data (11 files) ZIP 83 KB Download — Case-study CSV data (11 files) (ZIP)
The 12 chapter Python scripts ZIP 20 KB Download — The 12 chapter Python scripts (ZIP)
Jupyter notebooks: executed solutions and exercises ZIP 453 KB Download — Jupyter notebooks: executed solutions and exercises (ZIP)
Complete practice kit ZIP 557 KB Download — Complete practice kit (ZIP)
Dependencies (requirements.txt) Shared TXT 1 KB Download — Dependencies (requirements.txt) (TXT)
Raw extract to clean (chapter 7) Shared CSV 57 KB Download — Raw extract to clean (chapter 7) (CSV)