Selected work

MLOps · Streaming ML · Anomaly Detection

SentinelML

Project purpose

A bank transaction anomaly detection project connecting offline Isolation Forest training and MLflow model management with Kafka-based streaming inference.

  • Kafka
  • Python
  • MLflow
  • Isolation Forest
  • Docker
  • Jenkins

Bank Transaction Anomaly Detection

01 / Offline

Prepare transaction features, train and evaluate an Isolation Forest, then record a versioned model in MLflow.

02 / Online

Replay transactions through Kafka, transform each event, load a configured registry version, and publish prediction events.

The engineering problem

Model development and transaction scoring run on different data paths. The offline path needs repeatable experiments and a model version; the streaming path needs a compatible feature vector, decision threshold, and reliable event handoff.

SentinelML separates these responsibilities across two repositories. The architecture below follows the checked source. Its model handoff is configured, but the published defaults and metadata do not yet establish a verified end-to-end run.

Conceptual flow

One project, two connected pipelines

Offline code trains Isolation Forest and registers a model in MLflow. Online Python workers replay transactions through Kafka, process feature events, and publish predictions. The dashed model lookup is configured in source but unverified end to end: the default model names differ and serving also requires local metadata.

Conceptual execution sequence

  1. The offline pipeline prepares transaction features from CSV data.
  2. Isolation Forest fits on normal rows and is evaluated on a mixed validation split.
  3. MLflow records the run and registers a model version.
  4. At startup, inference requests a configured model version and reads local feature metadata. The checked defaults do not yet align.
  5. A dataset row is replayed to Kafka raw_events.
  6. The processing worker publishes a transformed event to Kafka features.
  7. The inference worker scores the feature event and publishes to Kafka predictions.

Offline Training & Model Lifecycle

The MLOps repository samples a transaction CSV, derives time and transaction features, and fits an Isolation Forest on normal training rows. Its training script uses a stratified 80/20 train and validation split after feature preparation, then selects a decision threshold against the mixed validation set.

The script logs model parameters and validation metrics to an MLflow run, saves a local joblib file, and registers a scikit-learn model version. A separate validation script can tag a candidate and set a champion alias. That script is not invoked by the checked Jenkins pipeline, and its comparison of an existing champion reads f1 while training logs f1_score.

The feature transform fits a scaler and category encoding before the split. No fitted scaler, encoder, or feature-order manifest is logged with the registered model. The validation metrics therefore need a leakage-safe rerun before they can be treated as held-out generalization results.

Online Transaction Inference

The SentinelML repository implements a CSV event generator that publishes to Kafka raw_events. A Python processing worker consumes those events, transforms fields, and publishes feature events to features. A separate inference worker consumes features, scores them, and publishes prediction events to predictions. It also writes a score to PostgreSQL when a transaction ID is present.

In the inspected runtime, these are Kafka consumers and producers. The worker does not call a FastAPI endpoint, and no FastAPI service was found in the checked source. The CV documents a FastAPI-based pipeline, but this public revision does not substantiate that detail.

Model handoff & feature contract

At worker startup, IsolationForestModel.__init__ loads models:/<name>/<version> from MLflow and reads a local JSON file for feature order and threshold. It keeps that model in memory for the consumer loop; the configured alias is unused, and the worker has no model reload path.

The training script registers FraudDetectionModel_Champion, while serving defaults to fraud-isolation-forest version 1. Both Compose stacks configure a shared MLflow network, but the names differ and the required metadata file is absent from the checked runtime repository. Model selection and deployment still require configuration and integration validation.

The streaming processor also builds features differently from the batch transform. Serving zero-fills missing named features using local metadata; it does not reuse the offline fitted scaler or category mapping. This prevents a claim that training and serving have a guaranteed matching feature contract.

Saved notebook output · historical

Evaluation evidence, with its limits

The awab branch notebook stores results from an Isolation Forest experiment. These are the CV figures, but the dataset was unavailable for an independent rerun and the saved notebook does not support a clean generalization claim.

Historical notebook results and evaluation context
MeasureSaved valueContext
Test rows1,496,754Saved awab notebook classification report
ROC-AUC0.875Saved test scores from a normal-only Isolation Forest
Average precision0.678Reported as PR-AUC in the notebook
Anomaly recall72%Class-1 recall at the notebook's model threshold

The same saved classification report shows anomaly-class precision of 54% and F1 of 0.62. Its displayed feature selection includes is_fraud in the input matrix, and scaling happens before the train/test split. The 1,496,754 rows are the notebook’s reported test set, not a verified deployment result. The current training script uses a different train/validation procedure, so these values should not be attributed to its registered model.

Inspect the saved evaluation notebook

Engineering decisions

Two repositories, one model lifecycle

MLOps contains feature engineering, training, MLflow tracking, and a separate registry-validation script. SentinelML contains transaction replay, Kafka workers, and inference. Both Compose stacks name the same MLflow network, but source alone does not verify a successful combined run.

Explicit model lookup

The inference worker loads a fixed registry name and version once when it starts, then reads feature order and threshold from local JSON. The promotion script can set a champion alias, but serving does not resolve that alias or reload a model during its loop.

Kafka service boundaries

The generator publishes raw_events, the processor publishes features, and inference publishes predictions. The three topics separate replay, transformation, and scoring without making FastAPI a hidden step in the event path.

Feature contract remains open

Training fits scaling and categorical encoding on a batch before the split; the streaming processor transforms individual records differently. No fitted transform or feature-order artifact is published with the registered model in the inspected source.

Automation scope

The public Jenkinsfile checks out source and installs worker dependencies. Training, validation, registry promotion, image deployment, and service restart are not Jenkins stages in the inspected repositories.

Implementation evidence

From a feature vector to a prediction event

These source excerpts show the scoring operation and the worker entry point. They describe implementation, not a captured end-to-end execution.

Score and threshold

raw_score = self.model.decision_function(x)[0]
anomaly_score = float(-raw_score)
is_anomaly = anomaly_score >= self.threshold

The model orders inputs with local metadata before scoring. The prediction schema requires a nonnegative anomaly score, although the negated decision function can be negative; this path needs runtime validation.

Read the scoring implementation

Worker entry point

CMD ["python","main.py"]

The container starts a Kafka consumer. Its loop publishes prediction events and conditionally persists scores associated with transaction IDs.

Read the Dockerfile

Automation & validation boundary

The public Jenkinsfile performs checkout and installs the three worker services’ Python dependencies. It does not train, evaluate, register, promote, deploy, or restart a model. The CV documents retraining automation, but those steps cannot be verified from the inspected Jenkinsfiles and scripts.

The repositories establish training, model registration, registry lookup, and Kafka scoring components. A successful combined execution, a matching deployed feature schema, and an automatically promoted or reloaded model remain unverified.

Technology stack

Offline training
Python · scikit-learn Isolation Forest · pandas
Model management
MLflow runs, metrics, model versions · PostgreSQL + object storage
Streaming inference
Kafka · Python workers · Pydantic events · PostgreSQL
Packaging & automation
Docker Compose · Jenkins checkout and dependency installation

Implementation references