Learn AI Series (#152) - The AI Practitioner's Toolkit

avatar

Learn AI Series (#152) - The AI Practitioner's Toolkit

variant-c-12-green.png

What will I learn

  • the decision framework I actually use -- a set of blunt questions for picking the right approach before you write a single line of model code;
  • the concrete toolkit -- the three tiers of tools you reach for daily, weekly, and once in a blue moon, and why the boring ones matter most;
  • building your own ML library -- the little bag of tricks you write once, debug yourself, and reuse forever;
  • staying current without drowning -- how to keep a mental map of a field that ships 500 papers a day without reading any of them at 3am;
  • the skill tree -- where to specialise after this series, and the one meta-skill that separates "dangerous" from "careful".

Requirements

  • A working modern computer running macOS, Windows or Ubuntu;
  • Python 3.10+ with PyTorch and scikit-learn installed (pip install torch scikit-learn pandas);
  • Familiarity with the whole series, because this one is a consolidation -- it reaches back across nearly every arc we have built, from the classical models (#16-33) to production (#117-136).

Difficulty

  • Beginner

Curriculum (of the Learn AI Series):

Learn AI Series (#152) - The AI Practitioner's Toolkit

151 episodes of theory, code and projects behind us. At this point you have the knowledge -- honestly more of it than most people who put "ML" on their resume. But knowledge scattered across 151 episodes is not the same thing as a toolkit you can reach for when a real problem lands on your desk on a Monday morning and someone wants an answer by Friday ;-)

So today we consolidate. Not a rehash of what we covered -- I promise not to make you sit through linear regression again -- but a working framework for what to use WHEN, and how to keep it all organised in your head (and on your hard drive, wich is where most of it quietly rots if you are not deliberate about it). This one is different from the mini project last time: no big build, just the stuff I wish someone had handed me on a single page years ago.

The decision tree: what approach for what problem

After 151 episodes covering dozens of techniques, the single most practical skill is not knowing how attention works or how to derive backprop. It is knowing which tool to reach for. A senior practitioner and a junior one often know the same algorithms -- the senior one just wastes far less time on approaches that were never going to work for the data in front of them.

Here is the decision framework I actually use. It is not a flowchart with pretty boxes; it is a set of honest questions I ask before writing any model code:

def choose_approach(problem):
    """
    The practitioner's decision tree.
    Not a flowchart - a set of honest questions you answer out loud.
    """
    questions = {
        # Step 1: Do you even need ML?
        "can_solve_with_rules": {
            True: "Write rules. Don't use ML. Seriously.",
            False: "Continue to step 2.",
        },

        # Step 2: What kind of data do you have?
        "data_type": {
            "tabular": "gradient_boosting_first",
            "text": "llm_or_embeddings_first",
            "images": "pretrained_cnn_or_vit_first",
            "audio": "pretrained_whisper_or_spectrograms",
            "time_series": "feature_engineering_plus_boosting",
            "graph": "gnn_if_structure_matters",
            "mixed": "separate_pipelines_then_combine",
        },

        # Step 3: How much labeled data do you have?
        "labeled_data_amount": {
            "zero": "unsupervised_or_pretrained",
            "tens": "few_shot_or_transfer",
            "hundreds": "fine_tune_pretrained",
            "thousands": "train_from_scratch_possible",
            "millions": "scale_aggressively",
        },
    }
    return questions

Step 1 is the one everyone skips and it is the most important. Do you even need machine learning? A shocking amount of "AI projects" are a regular expression and three if-statements wearing a lab coat. ML brings a training pipeline, a data-drift problem, a monitoring burden (#123) and a whole category of failure modes that plain code simply does not have. If a handful of rules solves it, write the rules and go home early. You can always add ML later, in stead of the other way around.

Let me turn that dictionary into something you would actually run, because a framework you can call is harder to hand-wave past than one you just nod along to:

def decide(data_type, n_labeled, solvable_with_rules=False):
    """Pick a starting point. A STARTING point - not the final answer."""
    if solvable_with_rules:
        return "Write rules. Save the ML budget for a problem that needs it."

    starting_points = {
        "tabular":     "XGBoost / LightGBM (#19), a strong baseline in minutes",
        "text":        "An LLM API (#66) or embeddings + vector search (#63)",
        "images":      "Transfer-learn a pretrained ViT/ResNet (#46, #54)",
        "audio":       "Pretrained Whisper or log-mel spectrograms (#92-93)",
        "time_series": "Lag/rolling features + gradient boosting (#28-29)",
    }
    base = starting_points.get(data_type, "Unknown data type - go back to step 2")

    if n_labeled == 0:
        base += " | no labels: lean unsupervised or pretrained (#22-26, #90)"
    elif n_labeled < 100:
        base += " | tiny data: few-shot / transfer, do NOT train from scratch (#144)"
    return base

print(decide("tabular", n_labeled=800))
print(decide("images", n_labeled=40))

Let me expand on the cases that come up the most, because "gradient_boosting_first" is a slogan until you understand WHY.

Tabular data: start with gradient boosting (#19). Always. XGBoost or LightGBM will beat neural networks on tabular data in maybe 90% of real cases, train in seconds, and barely care about preprocessing. If you need interpretability, drop to a single decision tree (#17) or plain logistic regression (#12) and read the coefficients. Only if boosting genuinely is not good enough do you try stacking (#33), and only after THAT do you reach for deep learning on tables (#132). People invert this order constantly and pay for it in weeks.

Text data in 2026: the whole landscape shifted under our feet since the early NLP episodes. For classification, sentiment or extraction, reach for an LLM API (#66) or fine-tune a small language model (#69). For search and retrieval, embeddings plus vector search (#63). For generation, prompt a large model well (#62). The old bag-of-words / TF-IDF pipeline (#30) still works, and it is fast and interpretable, so it stays useful when you need speed on a boring classification task -- but it is no longer the default first move.

Image data: transfer learning from a pretrained model, nearly every time (#46). Fine-tune the last few layers of a ResNet or a ViT (#54) on your data and stop there. Do not train from scratch unless you have millions of labelled images and a good reason. Detection wants YOLO-family models (#79), segmentation reaches for SAM (#80), generation means diffusion (#84-85).

Time series: this is where people reach for neural networks far too early, because sequences FEEL like they need something fancy. Most of the time, feature engineering -- lags, rolling statistics, calendar features -- fed into gradient boosting beats an LSTM on tabular time series (#28-29). Neural approaches earn their keep when sequences are very long or the data is raw signal, like audio or sensor streams.

The toolkit: the tools you actually reach for

Here is the practical stack, sorted not by hype but by how often your hands land on it:

# Tier 1: use daily - the stuff under your fingers without thinking
essential_tools = {
    "python":       "3.10+ - the lingua franca of ML",
    "numpy":        "arrays, linear algebra, the foundation of everything",
    "pandas":       "tabular data manipulation",
    "scikit-learn": "classical ML, preprocessing, metrics (#16)",
    "pytorch":      "deep learning, GPU compute, autograd (#42)",
    "matplotlib":   "plots - your primary debugging tool, honestly",
    "jupyter":      "exploration, prototyping, sharing a result",
}

# Tier 2: use weekly
regular_tools = {
    "huggingface_transformers": "pretrained models, tokenizers, pipelines (#74)",
    "xgboost_or_lightgbm":      "gradient boosting for tabular data (#19)",
    "fastapi":                  "serving a model behind HTTP (#121)",
    "docker":                   "reproducible environments, deployment",
    "wandb_or_mlflow":          "experiment tracking (#119)",
    "opencv":                   "image and video processing (#77)",
}

# Tier 3: reach for when the specific need shows up
specialized_tools = {
    "onnx":            "model export across frameworks (#120)",
    "tensorrt":        "squeezing GPU inference latency",
    "ray":             "distributed compute + hyperparameter search (#126)",
    "dvc":             "data versioning (#118)",
    "gymnasium":       "reinforcement learning environments (#102)",
    "librosa":         "audio features (#92)",
    "networkx_or_pyg": "graphs and GNNs (#131)",
    "ollama":          "local LLM inference (#70)",
}

You do not need to master everything in Tier 3, and anyone who claims they have is either lying or unemployed with a lot of free time. What you DO need is to know these tools exist and roughly what they are for, so that when a problem shows up you know which drawer to open. That is the whole point of breadth: not memorising APIs, but building an index in your head.

Notice how boring Tier 1 is. NumPy, pandas, matplotlib -- tools that have barely changed in years. That is not a weakness, it is the signal. The tools you use every single day are the stable, unglamorous ones, and the flashy thing that trended on X last month is almost never one of them. When in doubt, bet on boring.

Building your personal ML library

Over time you accumulate code you keep re-writing across projects. The junior move is to copy-paste it out of an old notebook every time (and re-introduce the same bug every time). The senior move is to collect it deliberately into your own little library -- not a framework, not something to publish, just YOUR bag of tricks that you wrote and debugged yourself.

Start with the data helpers, because you touch data before anything else:

def quick_eda(df):
    """One-call exploratory data analysis for any dataframe."""
    print(f"Shape: {df.shape}")
    print(f"\nDtypes:\n{df.dtypes.value_counts()}")
    missing = df.isnull().sum()
    print(f"\nMissing:\n{missing[missing > 0]}")
    print(f"\nNumeric stats:\n{df.describe()}")

    # Cardinality of categoricals - catches ID columns masquerading as features
    cat_cols = df.select_dtypes(include=['object', 'category']).columns
    if len(cat_cols):
        print("\nCategorical cardinality:")
        for col in cat_cols:
            print(f"  {col}: {df[col].nunique()} unique")

import numpy as np

def train_val_test_split(df, val_frac=0.15, test_frac=0.15, seed=42):
    """Three-way split that is harder to mess up than chaining two splits."""
    n = len(df)
    rng = np.random.RandomState(seed)
    idx = rng.permutation(n)

    n_test = int(n * test_frac)
    n_val = int(n * val_frac)

    test_idx = idx[:n_test]
    val_idx = idx[n_test:n_test + n_val]
    train_idx = idx[n_test + n_val:]
    return df.iloc[train_idx], df.iloc[val_idx], df.iloc[test_idx]

Then the training helpers. Early stopping is the one you re-implement in every project, so write it once and never think about it again:

class EarlyStopping:
    """Stop training when the validation metric stops improving."""
    def __init__(self, patience=5, min_delta=1e-4, mode='min'):
        self.patience = patience
        self.min_delta = min_delta
        self.mode = mode
        self.best = float('inf') if mode == 'min' else float('-inf')
        self.counter = 0

    def __call__(self, metric):
        improved = (metric < self.best - self.min_delta if self.mode == 'min'
                    else metric > self.best + self.min_delta)
        if improved:
            self.best, self.counter = metric, 0
            return False
        self.counter += 1
        return self.counter >= self.patience   # True => stop now

The single highest-value utility in the whole library is the boring one nobody brags about: seeding everything. Half of "my results are not reproducible" (#119) is really "I forgot to seed something." So make it impossible to forget:

import os, random
import numpy as np

def set_seed(seed=42):
    """Seed every RNG that can bite you. Call this first, every run."""
    random.seed(seed)
    np.random.seed(seed)
    os.environ['PYTHONHASHSEED'] = str(seed)
    try:
        import torch
        torch.manual_seed(seed)
        torch.cuda.manual_seed_all(seed)
        # deterministic cuDNN: slower, but the run repeats exactly
        torch.backends.cudnn.deterministic = True
        torch.backends.cudnn.benchmark = False
    except ImportError:
        pass

Now the evaluation side. sklearn gives you the pieces; you just want them assembled the way you always want them, so you stop re-typing the same three imports:

def classification_report_plus(y_true, y_pred, y_prob=None):
    """The classification report I always end up wanting, in one call."""
    from sklearn.metrics import (
        classification_report, confusion_matrix, roc_auc_score)
    print(classification_report(y_true, y_pred))
    print(f"Confusion matrix:\n{confusion_matrix(y_true, y_pred)}")
    if y_prob is not None:
        try:
            auc = roc_auc_score(y_true, y_prob, multi_class='ovr')
            print(f"\nAUC-ROC: {auc:.4f}")
        except ValueError:
            pass   # AUC undefined for a single-class slice - skip, do not crash

And finally the PyTorch odds-and-ends you type on autopilot -- so stop typing them and import them in stead:

import time
from functools import wraps

def count_parameters(model):
    """How big is this thing, really? Trainable vs total."""
    total = sum(p.numel() for p in model.parameters())
    trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
    print(f"Total: {total:,} | Trainable: {trainable:,}")
    return trainable

def get_device():
    """Best available device, without the three-line dance every time."""
    import torch
    if torch.cuda.is_available():
        return torch.device('cuda')
    if getattr(torch.backends, 'mps', None) and torch.backends.mps.is_available():
        return torch.device('mps')       # Apple Silicon
    return torch.device('cpu')

def timed(fn):
    """Decorator: print how long a step took. Cheap profiling that pays off."""
    @wraps(fn)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = fn(*args, **kwargs)
        print(f"{fn.__name__} took {time.perf_counter() - start:.2f}s")
        return result
    return wrapper

The key thing is not the specific functions -- yours will look different from mine and that is exactly right. The key thing is that these are functions YOU wrote and debugged, so you understand every line. When something breaks at 2am, you can fix your own five-line helper. You cannot easily fix a mystery buried three layers deep in someone else's utility package. That ownership is the whole value.

Staying current without drowning

The field publishes roughly 500 papers a day on arXiv. You cannot read them. You should not try, and anyone who tells you they do is padding. So here is the system that actually works, and it costs about half an hour a week.

Weekly, about 30 minutes total:

  • Papers With Code style newsletters -- curated, and crucially they link the code, so you can tell the difference between a result and a press release;
  • The Batch (deeplearning.ai) -- Andrew Ng's weekly digest, honest and accessible summaries;
  • The programming/ML front page of a place like Hacker News -- filter for the posts that pulled real, sceptical discussion, in stead of just upvotes.

Monthly, a couple of hours:

  • Pick ONE paper that touches what you are actually working on right now;
  • Read it properly -- we will get into HOW to read a paper without it reading you, a little later in this series;
  • Reproduce the key result, even a shrunken version of it. You learn ten times more re-running a thing than re-reading it;
  • Write two paragraphs of notes for your future self, who will have forgotten all of this by spring.

What to skip without guilt:

  • Papers claiming state-of-the-art on a benchmark you do not care about (that is a leaderboard sport, not your job);
  • The "everything changes now" hype threads (it almost never does, and the genuinely important stuff is still there next month);
  • Tutorials that teach a tool without teaching the concept underneath -- you are well past that now;
  • Any conference talk that could have been a blog post. You know the ones.

The goal is not to know everything -- it is to keep a mental MAP of what exists, so you can find the right thing when a problem needs it. "I remember there was a technique for X" is genuinely enough. You look up the details when the day comes. Trying to hold all the details in your head at once is how you burn out by March.

The skill tree: where to specialise

AI is far too broad for one human to master all of it -- don't let anyone's LinkedIn convince you otherwise. After this series you have BREADTH, which is exactly the right thing to have first. Now you pick depth in one or two areas. Here is how I picture the main paths, kept deliberately simple:

                        AI Practitioner
                              |
        +---------------------+---------------------+
        |                     |                     |
   ML Engineer         Research / Science      Domain Expert
        |                     |                     |
   +----+----+           +----+----+          +-----+-----+
   |    |    |           |         |          |           |
 MLOps Sys  Data        NLP     Vision     Finance     Health
   |    |    |           |         |          |           |
 #117 #120 #118        #57-      #77-       #21,        #89,
 -136 -126             #76       #91        #28-29      #140

The ML Engineer path: you build and ship ML systems that stay up. Your depth is production (#117-136) -- serving, monitoring, optimisation, CI/CD. You want solid breadth across model types, but your real value is making models work reliably at scale, at 3am, when you are asleep. This is where most of the actual jobs are, quietly.

The Research path: you push the frontier in one specific area -- NLP, vision, RL, whatever genuinely pulls you. You read papers weekly, reproduce results, and eventually publish your own. The math episodes (#8-9) and the from-scratch builds (#10, #37-39) are not optional trivia for you -- they are your foundation, the thing you stand on when a paper's equations do not match its code.

The Domain Expert path: you apply AI to a specific field -- medical imaging, markets, climate, legal documents. Your value is NOT the ML itself; it is understanding the domain deeply enough to know which problems AI can actually solve and what data even exists. I argue this is the most impactful path of the three, because in most industries the bottleneck was never model capability -- it is knowing where to point it.

None of these is "better." They pay differently, they feel differently day to day, and plenty of good careers wander between them. Pick the one that makes you want to open the laptop on a Sunday, and go deep there for a year or two.

The meta-skill: knowing what you don't know

Here is the thing I keep circling back to after years of doing this.. the most dangerous state in this whole field is "I know enough to be dangerous but not enough to be careful." You can train a model that scores 95% on your test set and deploy it straight into a catastrophe, because the test set quietly was not representative of the world you deployed into. Enthusiasm ships the bug; only doubt catches it.

The antidote is not more knowledge -- you can always learn more and still miss this. The antidote is a boring checklist habit. Before you ever call a model "done":

preflight_checklist = [
    "Did I check for data leakage between train and test?",
    "Did I evaluate on a TRULY held-out set (not the one I tuned on)?",
    "Did I check performance across subgroups, not just the overall number?",
    "Did I test with out-of-distribution and plain garbage inputs?",
    "Did I measure calibration (does 80% confidence mean 80% correct)?",
    "Did I report confidence intervals, not just a single point estimate?",
    "Did I compare against a trivial baseline (majority class, mean)?",
    "Did I write down, in advance, what would make me say 'this does NOT work'?",
    "Would I bet my own money on these results?",
]

That last one is the real test, and I mean it literally. If you would not put your own money on the result, you do not actually believe it yet -- you are hoping. Hope is not a deployment strategy. Go do the work the checklist just exposed.

Where this leaves us

We are genuinely near the end of the road now. What is left is not more techniques -- you have plenty. What is left is turning all of it into something that lasts: how a model becomes a product that real people pay for and rely on, how to read the research so it sharpens you in stead of drowning you, and finally how all 150-plus pieces we have assembled click together into one coherent stack you carry with you. Bring the toolkit you just built -- we are going to put it to work.

The important bits

  • decide before you build -- ask whether you need ML at all, then let data type and label count pick your STARTING point; for tabular start with gradient boosting, for text use LLMs or embeddings, for images transfer-learn, for time series try features + boosting before neural nets;
  • your toolkit has three tiers -- daily (NumPy, pandas, sklearn, PyTorch, matplotlib), weekly (Hugging Face, FastAPI, Docker, experiment tracking), and specialised tools you only need to KNOW exist until the day you reach for them;
  • build your own little library -- EDA, splits, early stopping, set_seed, evaluation reports, device/param helpers -- code you wrote and debugged yourself, so you can fix it at 2am;
  • stay current on 30 minutes a week -- curated sources plus one reproduced paper a month; skip the hype and keep a mental map, not a memorised pile;
  • specialise after breadth -- ML Engineer (production), Research (frontier), or Domain Expert (applying AI where it matters), and pick the one you would open the laptop for on a Sunday;
  • the real meta-skill is systematic doubt -- a preflight checklist catches the failures enthusiasm ships, and "would I bet my own money on this?" is the only question that never lies.

That is your toolkit on one page -- now go build yourself a utils.py you are proud of and never copy-paste from an old notebook again. Bedankt en tot de volgende keer! ;-)

@scipio

stem stemsocial steemstem python programming
0
0
0.000
0 comments