This ISHRAQA7 Academy course teaches you to analyse data with Python, from the basics of the language to a first validated machine learning model. The twelve chapters work on the same case study, the sales activity of an office equipment distributor, and end with a complete project whose report is generated by the code.
Learning path
| Level | Chapters |
|---|---|
| Beginner | 1. First steps · 2. Variables and structures · 3. Conditions, loops, functions · 4. Files and environments |
| Intermediate | 5. NumPy · 6. pandas · 7. Cleaning · 8. Exploratory analysis · 9. Descriptive statistics · 10. Matplotlib |
| Advanced | 11. Machine learning with scikit-learn · 12. Final project |
Course files
The download section offers: the complete manual, the exercise workbook, the worked answers, the installation guide and the final project report as PDF; the CSV data; the twelve Python scripts; the Jupyter notebooks (executed solutions and exercises to complete); the requirements.txt file; and a complete practice kit (data, scripts, notebooks and requirements.txt, without the PDFs).
Level 1 · Beginner — Chapter 1
Python for data analysis: first steps
Python has become the reference language for data analysis: it is free, readable and surrounded by an ecosystem of specialised libraries. This chapter sets up the working environment and runs a first script that reads a real data file.
Why Python for data
A spreadsheet soon reaches its limits: calculations repeated by hand, large files, processing that is hard to replay. A Python script describes every step in writing; you run it again on next month’s data and it applies exactly the same processing. The course libraries form a consistent chain:
| Library | Role | Chapters |
|---|---|---|
csv, pathlib, json (standard) |
Read and write files | 1 to 4 |
| NumPy | Numerical computing on arrays | 5 |
| pandas | Data tables: reading, joins, aggregation, cleaning | 6 to 9 |
| Matplotlib | Charts | 10 |
| scikit-learn | Machine learning | 11 |
Setting up the environment
- Install Python 3.13 from python.org (on Windows, tick “Add python.exe to PATH”). The scripts were run with Python 3.13.
- Unzip the course archive into a short folder, for example
C:\ISHRAQA7\python\. It containsdata/,scripts/,notebooks/andrequirements.txt. - Create the environment and install the libraries; the detailed procedure is in chapter 4 and in the installation guide.
- Run
jupyter labfrom that folder: the browser shows the project tree.
A script (.py) runs in one go: python scripts/ch01.py. A Jupyter notebook (.ipynb) splits the same code into cells run one by one, with results displayed below each cell: the ideal tool for exploring. Each chapter is supplied in both forms, plus an exercise notebook to complete.
A first script
import csv
from pathlib import Path
DATA = Path('data')
with open(DATA / 'orders.csv', encoding='utf-8', newline='') as f:
orders = list(csv.DictReader(f))
print('Number of orders:', len(orders))
print('First order:', orders[0])
csv.DictReader turns each row into a dictionary whose keys are the column headers: orders[0]['status'] gives the status of the first order. The encoding='utf-8' parameter guarantees that accented and Arabic characters are read correctly. The script shows 3,307 orders.
Counting by category
by_status = {}
for o in orders:
by_status[o['status']] = by_status.get(o['status'], 0) + 1
print(by_status)
The get(key, 0) method returns 0 when the key does not exist yet: the dictionary fills up as the loop runs. Result: 3,108 delivered orders, 117 cancelled and 82 returned.
The course case study
Every chapter uses the same dataset: the activity of an office equipment distributor in 2024 and 2025 (orders, order lines, customers, products, regions, channels, targets), plus a deliberately imperfect extract for cleaning. The ISHRAQA7 Academy project in chapter 12 brings all the steps together.
Level 1 · Beginner — Chapter 2
Variables, types and data structures
Before analysing thousands of rows, you need to know how to represent a value, a list of values or a complete record. This chapter presents Python’s basic types and the four data structures used all the time.
Variables and types
product_id = 'P11' # str: text
quantity = 4 # int: integer
unit_price = 2450.0 # float: decimal number
delivered = True # bool: true or false
print(f'{quantity} × {unit_price:,.2f} = {quantity * unit_price:,.2f}')
A variable is a name attached to a value; Python infers the type from the value. An f-string (a string prefixed with f) inserts expressions between braces and formats them: :,.2f shows two decimals with a thousands separator.
Converting what you read
A CSV file contains only text. Before calculating, convert explicitly: int('4'), float('2450.00'). An impossible conversion (int('4 pcs')) raises a ValueError: this is useful, because unexpected data is flagged instead of being silently misread (chapter 3).
with open(DATA / 'products.csv', encoding='utf-8', newline='') as f:
products = [
{'id': r['product_id'], 'name': r['product_name'], 'cat': r['category_code'],
'price': float(r['list_price']), 'cost': float(r['unit_cost'])}
for r in csv.DictReader(f)
]
This block is a list comprehension: it builds a list in one expression from each row read. We get 30 products, with prices and costs already numeric.
Four structures, four uses
| Structure | Syntax | Property | Typical use |
|---|---|---|---|
| List | [2450.0, 52.0] |
Ordered, mutable | A series of values |
| Tuple | (52.0, 'P15') |
Ordered, immutable | A pair or a fixed record |
| Set | {'FUR', 'ITE'} |
No duplicates, no order | Distinct values, membership tests |
| Dictionary | {'P11': 2450.0} |
Key → value | Fast lookup by identifier |
prices = [p['price'] for p in products]
cheapest = min((p['price'], p['id']) for p in products)
categories = {p['cat'] for p in products}
price_of = {p['id']: p['price'] for p in products}
The cheapest product is P15 (52.00 MAD): min compares tuples by their first element, the price. The set categories contains four codes; the dictionary price_of finds a price without scanning the list, like an instant VLOOKUP.
Sorting with a key
top5 = sorted(((p['id'], p['price']) for p in products),
key=lambda t: t[1], reverse=True)[:5]
sorted accepts a key function that says what to sort on; lambda t: t[1] is a small anonymous function returning the price. The slice [:5] keeps the first five elements.
Level 1 · Beginner — Chapter 3
Conditions, loops, functions and modules
Data processing combines decisions (if… then…), repetition (for each row…) and reusable operations. This chapter teaches you to write clear functions and to rely on the modules of the standard library.
Functions: naming a business rule
def net_amount(quantity, unit_price, discount_rate=0.0):
"""Net amount of a line: quantity × price × (1 − discount)."""
return quantity * unit_price * (1 - discount_rate)
print(net_amount(4, 2450.0, 0.05)) # 9310.0
A function has a name, receives parameters and returns a result with return. The parameter discount_rate=0.0 has a default value: it can be omitted. The string in triple quotes, the docstring, documents the rule. Writing the rule once avoids having three different versions of it in the code.
Loops and a lookup dictionary
orders = {o['order_id']: o for o in read_csv('orders.csv')}
total_by_status = {}
for l in lines:
status = orders[l['order_id']]['status']
amount = net_amount(int(l['quantity']), float(l['unit_price']), float(l['discount_rate']))
total_by_status[status] = total_by_status.get(status, 0) + amount
Each order line must find the status of its order. The orders dictionary, keyed by order number, acts as a lookup table. Result: 28,654,029.30 MAD for delivered orders, 1,090,232.00 MAD for cancelled ones and 802,042.15 MAD for returned ones. Only the first amount corresponds to completed sales.
Conditions
def size_class(amount):
if amount >= 10000:
return 'L'
elif amount >= 2000:
return 'M'
return 'S'
Conditions are tested in order; the first true one wins, so the descending order of the thresholds is essential. Out of 8,472 lines: 417 large, 4,949 medium and 3,106 small.
Modules
A module groups ready-made functions: import statistics gives access to statistics.mean, median, stdev. The standard library also contains csv, json, pathlib, datetime. Your own functions can be stored in a tools.py file and imported by several scripts.
Handling errors
try:
q = int(text)
except ValueError:
q = None # flagged value, handled later
try … except catches an expected error. Catch only the expected error (ValueError), never all errors: a bare except: would also hide typos in the code.
Level 1 · Beginner — Chapter 4
Files, dependencies and environments
An analysis project must be movable, shareable and reinstallable without surprises. This chapter covers file paths, exchange formats, virtual environments and the requirements.txt file.
Portable paths with pathlib
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent # project folder
DATA = ROOT / 'data'
for p in sorted(DATA.glob('*.csv')):
print(f'{p.name:<24} {p.stat().st_size:>9,} bytes')
The / operator joins paths with the right separator on Windows, macOS and Linux. Calculating ROOT from the script’s location (__file__) lets you run the script from any folder. In a notebook, where __file__ does not exist, the course scripts use the notebook’s parent folder. The data folder holds 11 CSV files.
Exchange formats
| Format | Strengths | Watch out for |
|---|---|---|
| CSV | Universal, readable, light | Delimiter, encoding, decimals, everything is text |
| JSON | Nested structures, simple types | Less suited to large tables |
| Excel (.xlsx) | Read by every department | Formatting and merged cells disrupt reading |
inventory = [{'file': p.name, 'bytes': p.stat().st_size, 'rows': rows[p.stem]} for p in files]
target = OUT / 'inventory.json'
target.write_text(json.dumps(inventory, ensure_ascii=False, indent=1), encoding='utf-8')
ensure_ascii=False keeps accents and Arabic readable in the file. Always write in UTF-8 and say so explicitly when reading.
Virtual environments and dependencies
Each project should have its own virtual environment: a folder holding an isolated copy of Python and the project’s libraries. Two projects can then use different versions of pandas without conflict.
python -m venv .venv
.venv\Scripts\activate # Windows
source .venv/bin/activate # macOS / Linux
python -m pip install -r requirements.txt
The course requirements.txt file pins the versions with which every script and notebook was run: numpy 2.5.3, pandas 3.0.5, matplotlib 3.11.2, scikit-learn 1.9.1, plus JupyterLab. Pinning versions ensures that the course results can be reproduced identically; other versions often work, but without guarantee.
Checking from a script
import importlib.metadata as md
for pkg in ['numpy', 'pandas', 'matplotlib', 'scikit-learn']:
print(pkg, md.version(pkg))
This check, placed at the top of a project, saves a long search for the cause of a difference that comes from another version.
Level 2 · Intermediate — Chapter 5
Numerical computing with NumPy
NumPy is the foundation of scientific computing in Python: it handles arrays of numbers and applies an operation to all their values at once. pandas and scikit-learn are built on it. This chapter shows how to replace loops with vectorised calculation.
From dictionaries to arrays
import numpy as np
qty = np.array([int(r['quantity']) for r in rows])
price = np.array([float(r['unit_price']) for r in rows])
disc = np.array([float(r['discount_rate']) for r in rows])
print(qty.shape, qty.dtype, price.dtype) # (8472,) int64 float64
A NumPy array holds values of a single type, stored compactly in memory. Its shape gives its dimensions: here a vector of 8,472 values, one per order line. The three arrays are aligned: position 0 of each describes the same line.
Vectorised calculation
net = qty * price * (1 - disc)
print(net[:5].round(2), round(float(net.sum()), 2))
Multiplication applies element by element, with no Python loop: this is vectorisation. The code is shorter and dozens of times faster than a loop over thousands of rows. The expression 1 - disc illustrates broadcasting: the number 1 is “stretched” to the whole length of the array. Total over all lines, all statuses: 30,546,303.45 MAD.
Boolean masks
big = net > 10_000 # array of True / False
print(int(big.sum())) # number of lines concerned
share = net[big].sum() / net.sum()
A comparison produces an array of booleans, the mask. big.sum() counts the true values (417 lines) and net[big] selects the matching amounts. These large lines, under 5% of all lines, account for 24.3% of the total amount: a common concentration in sales analysis.
Aggregations and statistics
| Function | Result |
|---|---|
a.sum(), a.mean(), a.std() |
Sum, mean, standard deviation |
a.min(), a.max(), a.argmax() |
Extremes and position of the maximum |
np.percentile(a, [25, 50, 75]) |
Quartiles |
np.abs, np.round, np.where |
Element-wise functions |
np.where(net > 10_000, 'L', 'S') applies a condition to each element and replaces a loop with if/else.
Reproducible randomness
rng = np.random.default_rng(2026)
sim = rng.uniform(0, 0.15, size=1000)
A generator created with a fixed seed always produces the same sequence of numbers: a simulation becomes reproducible and checkable. This is essential for sharing an analysis or a model.
Level 2 · Intermediate — Chapter 6
Working with tables in pandas
pandas is the analyst’s core Python tool: it reads files, selects, filters, joins and aggregates tables, like a programmable spreadsheet. This chapter builds the case-study sales table from five files.
Reading with controlled types
import pandas as pd
orders = pd.read_csv(DATA / 'orders.csv',
dtype={'order_id': str, 'customer_id': str},
parse_dates=['order_date', 'ship_date'])
orders.info()
A DataFrame is a table: named columns, each of one type (a Series). Declaring identifiers as text stops “00125” from becoming 125; parse_dates converts dates. info() shows the number of rows, the types and the missing values. Dimensions: orders (3307, 7), lines (8472, 6).
Selecting and filtering
delivered = orders.loc[orders['status'] == 'delivered',
['order_id', 'order_date', 'customer_id', 'channel_code']]
orders.iloc[:3, :4] # first 3 rows, first 4 columns
loc selects by label or boolean mask, iloc by position. The filter gives 3,108 delivered orders. To combine conditions, use & (and), | (or) and parentheses: (orders['status'] == 'delivered') & (orders['channel_code'] == 'WEB').
Joining tables
sales = (lines
.merge(orders, on='order_id', how='left', validate='many_to_one')
.merge(products[['product_id', 'product_name', 'category_code', 'unit_cost']],
on='product_id', how='left', validate='many_to_one')
.merge(customers[['customer_id', 'segment_code', 'region_code']],
on='customer_id', how='left', validate='many_to_one')
.merge(regions, on='region_code', how='left', validate='many_to_one'))
sales['net'] = sales['quantity'] * sales['unit_price'] * (1 - sales['discount_rate'])
assert len(sales) == len(lines)
merge is the equivalent of a VLOOKUP, only safer. The argument validate='many_to_one' checks that every key on the right is unique: a duplicated key would multiply rows and inflate sales without any error message. The final assert confirms that the sales table has as many rows as the order lines; it has 20 columns.
Aggregating
ok = sales[sales['status'] == 'delivered']
by_year = ok.groupby('year')['net'].sum().round(2)
groupby splits the table by key value, applies a function to each group and combines the results, like a PivotTable. Delivered net sales: 2024 = 13,556,438.85 MAD, 2025 = 15,097,590.45 MAD. For several indicators at once: .agg(net=('net', 'sum'), orders=('order_id', 'nunique')).
Good practice
- Chain operations inside parentheses (method chaining) to read the processing from top to bottom.
- Create a new variable at each important step (
ok,y25) rather than modifying the original table. - Check every join: row count,
validate, missing values after the join.
Level 2 · Intermediate — Chapter 7
Data cleaning and preparation
Real data contains duplicates, inconsistent formats and missing values. Reliable cleaning follows a method: diagnose, apply validated rules, verify with numeric checks. This chapter processes the case study’s imperfect extract.
Diagnose first
raw = pd.read_csv(DATA / 'orders_raw_export.csv', dtype=str, keep_default_na=False)
diag = {
'duplicates': int(raw.duplicated().sum()),
'dmy_dates': int(raw['order_date'].str.contains('/').sum()),
'city_spaces_or_case': int((raw['city'] != raw['city'].str.strip().str.title()).sum()),
'qty_not_digits': int((~raw['quantity'].str.fullmatch(r'\d+')).sum()),
'comma_prices': int(raw['unit_price'].str.contains(',').sum()),
'missing_customer': int((raw['customer_id'].str.strip() == '').sum()),
}
Read everything as text (dtype=str) without automatic conversion of empty cells (keep_default_na=False): conversion comes after diagnosis. Out of 1,174 rows: 28 exact duplicates, 61 dd/mm/yyyy dates, 34 badly entered cities, 35 non-numeric or negative quantities, 43 prices with a comma and 14 missing customers. The .str methods apply text functions to a whole column.
Rules validated by the business
| Defect | Rule | pandas |
|---|---|---|
| Exact duplicates | Remove | drop_duplicates() |
| Two date formats | ISO, otherwise dd/mm/yyyy | to_datetime(..., errors='coerce') + fillna |
| Spaces and case | Trim, capitalise | str.strip().str.title() |
| Quantity “4 pcs” or “-2” | Keep the digits | str.replace(r'\D', '', regex=True) |
| Decimal comma | Replace with a dot | str.replace(',', '.') |
| Price entered ×100 | Divide if > 10 × list price | join + loc |
| Missing customer | Take the order’s customer | map |
def clean(raw, products, orders):
df = raw.drop_duplicates().copy()
d1 = pd.to_datetime(df['order_date'], format='%Y-%m-%d', errors='coerce')
d2 = pd.to_datetime(df['order_date'], format='%d/%m/%Y', errors='coerce')
df['order_date'] = d1.fillna(d2)
df['city'] = df['city'].str.strip().str.title()
df['quantity'] = df['quantity'].str.replace(r'\D', '', regex=True).astype(int)
df['unit_price'] = df['unit_price'].str.replace(',', '.', regex=False).astype(float)
df['discount_rate'] = df['discount_rate'].astype(float)
df = df.merge(products[['product_id', 'list_price']], on='product_id', how='left', validate='many_to_one')
too_high = df['unit_price'] > 10 * df['list_price']
df.loc[too_high, 'unit_price'] = df.loc[too_high, 'unit_price'] / 100
missing = df['customer_id'].str.strip() == ''
df.loc[missing, 'customer_id'] = df.loc[missing, 'order_id'].map(orders.set_index('order_id')['customer_id'])
df['net'] = df['quantity'] * df['unit_price'] * (1 - df['discount_rate'])
return df.drop(columns='list_price'), int(too_high.sum()), int(missing.sum())
Each rule is a decision by the department that knows the data; here, the sales department confirmed that negative quantities are sign errors and that the orders table is the reference for the customer. The function corrects 4 prices and fills in 14 customers. errors='coerce' turns an unreadable date into a missing value instead of stopping the script: we then check that none remain.
Missing values: decide, don’t guess
Faced with a missing value, there are four options: recover it from a reliable source (the customer from the order), calculate it with a documented rule, exclude the row and report it, or leave it missing if the analysis allows. Replacing it with an average “to make it work” distorts results without saying so.
Level 2 · Intermediate — Chapter 8
Exploratory data analysis (EDA)
Exploratory analysis means questioning the data before concluding: distributions, cross-tabulations, changes over time. It brings out useful questions and anomalies. This chapter builds an order-level table and explores it methodically.
Choosing the grain
The lines table answers “which products?” but not “how much is an order worth?”. So we build a table at order grain:
lines['net'] = lines['quantity'] * lines['unit_price'] * (1 - lines['discount_rate'])
per_order = lines.groupby('order_id').agg(net=('net', 'sum'), n_lines=('line_no', 'count')).reset_index()
od = (orders.merge(per_order, on='order_id', how='left', validate='one_to_one')
.merge(customers[['customer_id', 'segment_code']], on='customer_id', how='left')
.merge(segments, on='segment_code', how='left')
.merge(channels, on='channel_code', how='left'))
od['delay'] = (od['ship_date'] - od['order_date']).dt.days
The delay is calculated by subtracting dates; .dt.days extracts the number of days. It is missing for cancelled orders, which were never shipped.
Describing
od.describe() summarises the numeric columns: count, mean, standard deviation, quartiles. value_counts(normalize=True) gives the proportions of a qualitative variable. Always read these summaries before going further: a negative minimum, an outsized maximum or an incomplete count signal a problem.
Crossing
pd.crosstab(od['channel_name'], od['status'], normalize='index').round(3)
crosstab counts the combinations of two variables; normalize='index' turns counts into proportions by row, to compare channels of different sizes. pivot_table does the same with an aggregated value (sum of sales by region and month, for example).
Time
dl = od[od['status'] == 'delivered'].set_index('order_date')
monthly = dl['net'].resample('MS').sum()
With a date index, resample('MS') groups by month (“MS”: month start); the series has 24 months. To neutralise seasonality, compare a period with the same period of the previous year, never with the previous month.
A method, not a list of charts
- Ask a precise question (“which channel cancels the most?”).
- Choose the right grain and measure (order, cancellation rate).
- Calculate, then check the order of magnitude.
- Note the finding and the next question it raises.
Level 2 · Intermediate — Chapter 9
Descriptive statistics
A mean alone does not describe data. Descriptive statistics summarise a distribution by its central tendency, spread, shape and relationships with other variables. This chapter applies them to order values and teaches you to interpret them with care.
Central tendency and spread
s = v['net'] # delivered orders, 2025
stats = {'mean': s.mean(), 'median': s.median(), 'std': s.std(),
'q1': s.quantile(0.25), 'q3': s.quantile(0.75), 'skew': s.skew()}
| Statistic | Value (2025, 1,628 orders) | Reading |
|---|---|---|
| Mean | 9,274 MAD | The “shared” value if all orders were equal |
| Median | 7,132 MAD | Half of the orders are below it |
| First / third quartile | 3,739 MAD / 11,989 MAD | The middle half of the orders lies between these bounds |
| Standard deviation | 8,548 MAD | Spread around the mean |
| Coefficient of variation | 0.92 | Standard deviation relative to the mean, unitless |
| Skewness | 2.69 | Positive: long tail towards large values |
The mean is clearly above the median and the skewness is positive: a few very large orders pull the mean up. To describe a “typical” order, the median is more representative; for total revenue, it is the sum (and therefore the mean) that matters.
Outliers: the IQR rule
iqr = q3 - q1
upper = q3 + 1.5 * iqr
outliers = v[s > upper]
The interquartile range (IQR = 8,250 MAD) measures the spread of the middle half; it is not very sensitive to extremes. Above Q3 + 1.5 × IQR (24,363 MAD), a value is called an outlier: 87 orders, accounting for 20.6% of sales. An outlier is not necessarily an error: here they are real large orders, to be analysed rather than deleted.
Correlation
v['n_lines'].corr(v['net']) # Pearson
v['n_lines'].corr(v['net'], method='spearman') # Spearman (ranks)
The Pearson coefficient measures a linear relationship, between −1 and 1; Spearman works on ranks and captures any increasing relationship, resisting extreme values better. A correlation, even a strong one, does not prove causation: two variables can both depend on a third.
Comparing groups
To compare channels or years, put mean, median and count side by side: v.groupby('channel_code')['net'].agg(['mean', 'median', 'count']). A difference in means driven by a few exceptional orders does not mean the same thing as a shift of the whole distribution.
Level 2 · Intermediate — Chapter 10
Visualising with Matplotlib
A good chart makes a result clear in a few seconds. Matplotlib is Python’s core visualisation library. This chapter builds the essential charts of sales analysis and the rules that make them readable.
Figure and axes
import matplotlib.pyplot as plt
m = sales.pivot_table(index='month', columns='year', values='net', aggfunc='sum') / 1000
fig, ax = plt.subplots(figsize=(8, 4))
ax.plot(m.index, m[2025], color='#10263A', marker='o', label='2025')
ax.plot(m.index, m[2024], color='#9AA8B5', linestyle='--', label='2024')
ax.set_title('Monthly net sales (thousand MAD)')
ax.set_xticks(range(1, 13))
ax.legend(frameon=False)
ax.spines[['top', 'right']].set_visible(False)
fig.savefig(OUT / 'fig_monthly.png', dpi=150)
The figure is the page; the axes (ax) are the plotting area. Working with ax rather than the global functions lets you control each chart in a panel. Dividing by 1,000 and saying so in the title makes the axes readable. The strong colour is reserved for the year under study; the comparison year stays grey.
Which chart for which question?
| Question | Chart | Matplotlib |
|---|---|---|
| Change over time | Line | ax.plot |
| Comparing categories | Sorted horizontal bars | ax.barh |
| Distribution of a variable | Histogram | ax.hist |
| Relationship between two variables | Scatter plot | ax.scatter |
Bars with a target marker
order = reg.sort_values().index
ax.barh(order, reg[order] / 1e6, color='#10263A')
ax.scatter(tgt[order] / 1e6, order, color='#C89D62', marker='|', s=400, linewidths=3, label='Target')
Sorting the bars makes comparison easier; the gold tick places each region’s annual target on the same scale. You immediately see which regions exceed their target and which fall short.
Panels and saving
plt.subplots(2, 2, figsize=(10, 7)) creates four axes in a grid; fig.tight_layout() prevents overlaps. Save as PNG at 150 dots per inch for a report, as SVG for a printed document. In a script, figures are only saved; in a notebook, they are also shown under the cell.
Readability rules
- A title that says what the chart shows, units included.
- An accent colour for what matters, grey for the rest.
- No 3D effects and no pie chart with more than five slices.
- Axes starting at zero for bars.
Level 3 · Advanced — Chapter 11
Introduction to machine learning with scikit-learn
Machine learning learns a relationship from examples to estimate an unknown value. Used well, it complements analysis; poorly validated, it produces misleading numbers. This chapter builds a first model and, above all, teaches you to measure its quality honestly.
The question
Can we estimate a customer’s 2025 purchases from their 2024 behaviour? We build a table of one row per customer (120 customers) with 2024 features (purchases, number of orders, average discount, share of online orders, segment, region) and the target: delivered purchases in 2025. The simple correlation between 2024 and 2025 purchases is 0.34.
A pipeline
X = data[['net24', 'orders24', 'disc24', 'web24', 'segment_code', 'region_code']]
y = data['net25']
prep = ColumnTransformer([('cat', OneHotEncoder(handle_unknown='ignore'),
['segment_code', 'region_code'])], remainder='passthrough')
model = Pipeline([('prep', prep), ('reg', LinearRegression())])
A model reads only numbers: OneHotEncoder turns each segment and region into 0/1 columns. The pipeline chains preparation and model; it guarantees that preparation is learned from the training data only, with no information leaking from the test set.
Validating honestly
cv = KFold(n_splits=5, shuffle=True, random_state=2026)
scores = cross_validate(model, X, y, cv=cv, scoring=('neg_mean_absolute_error', 'r2'))
Cross-validation splits the customers into five groups; each group is used once for testing while the model learns from the other four. This gives a measure less dependent on the luck of a single split. Two metrics: the mean absolute error (MAE), in dirhams, and R², the share of variance explained (1 = perfect, 0 = no better than the mean, negative = worse than the mean).
Always compare with a baseline
The naive baseline estimates 2025 purchases as 2024 purchases × average growth, with growth calculated on the training folds only. Cross-validation results:
| Approach | MAE | R² |
|---|---|---|
| Naive baseline | 68,553 MAD | -1.07 |
| Linear regression | 47,006 MAD | 0.04 |
The model reduces the mean error by 31.4% compared with the baseline, but its R² stays close to zero: it does barely better than an estimate by the mean. Honest conclusion: with these features and two years of data, a customer’s 2025 purchases are hard to predict; the model helps avoid the large errors of the naive baseline, not to make fine forecasts. Calling the model “high-performing” would be misleading.
Level 3 · Advanced — Chapter 12
Final project: end-to-end sales analysis
The final project brings the whole chain together: import, check, prepare, compute indicators, visualise, analyse, model and report. It produces a report in which every figure is calculated by the code, which is what is expected of a data analyst at work.
The sponsor and the question
ISHRAQA7 Academy project: end-to-end sales analysis. The sales management of an office equipment distributor (case study, practice dataset) wants a 2025 review: how do sales compare with 2024 and with target, which regions and categories explain the gaps, how are orders distributed, and can customers’ future purchases be estimated?
The eight steps
| Step | Content | Output |
|---|---|---|
| 1. Import | Ten tables, explicit types | Dictionary of DataFrames |
| 2. Quality | Unique keys, foreign keys, missing values | Blocking checks |
| 3. Preparation | Sales table: net amount, cost, delay | One row per order line |
| 4. Indicators | Sales, growth, margin, basket, target, cancellations, delay | kpi.csv |
| 5. Charts | Trend, regions, categories | Three PNG files |
| 6. Statistics | Median, mean, IQR, outliers | Summary |
| 7. Model | Regression validated by cross-validation | MAE, R², baseline |
| 8. Report | Generated Markdown summary | report.md |
Checks that stop the process
q = {
'dup_order_id': int(T['orders']['order_id'].duplicated().sum()),
'orphan_lines': int((~T['order_lines']['order_id'].isin(T['orders']['order_id'])).sum()),
'missing_ship_not_cancelled': int((T['orders']['ship_date'].isna() & (T['orders']['status'] != 'cancelled')).sum()),
}
assert all(v == 0 for v in q.values()), q
If a key is duplicated or orphaned, the script stops and shows the failing check: no report is better than a wrong report. All the project’s checks are 0 on the course data.
A generated report, not a copied one
report = f"""| Net sales (MAD) | {fmt(k4['net'], ',.0f')} | {fmt(k5['net'], ',.0f')} |
| Margin rate | {fmt(k4['margin_rate'], '.1%')} | {fmt(k5['margin_rate'], '.1%')} |"""
(P / 'report.md').write_text(report, encoding='utf-8')
The script’s fmt function adapts separators to the report language: comma for thousands and a decimal point in English.
Every value in the report is inserted by the code from the calculations: if the data changes, just run the script again. Markdown reads as is and converts easily to HTML or PDF.
Results obtained on the course data
| 2025 indicator | Value |
|---|---|
| Net sales (2024) | 15,097,590 MAD (13,556,439 MAD) |
| Growth | 11.4% |
| Margin rate (2024) | 38.3% (36.3%) |
| Target attainment | 99.4% |
| Delivered orders / average order value | 1,628 / 9,274 MAD |
| Cancellation rate / average delay | 3.6% / 2.4 days |
Assessment grid
| Criterion | Points |
|---|---|
| Code organisation: functions, names, comments, relative paths | 20 |
| Data quality: blocking, documented checks | 20 |
| Accuracy of the indicators (reference values) | 20 |
| Readable, relevant charts | 15 |
| Validated statistics and model, limits explained | 15 |
| Generated report, reasoned conclusions | 10 |
Final project brief
- Import the ten tables with explicit types.
- Write blocking quality checks (keys, missing values).
- Build the sales table with net amount, cost and delay.
- Calculate the 2024 and 2025 indicators and save them to kpi.csv.
- Produce three charts and a statistical analysis of orders.
- Estimate customer purchases with cross-validation and a naive baseline.
- Generate a Markdown report and write three recommendations.
