MLOps · Streaming ML · Anomaly Detection
SentinelML
Project purpose
A bank transaction anomaly detection project connecting offline Isolation Forest training and MLflow model management with Kafka-based streaming inference.
- Kafka
- Python
- MLflow
- Isolation Forest
- Docker
- Jenkins
Bank Transaction Anomaly Detection
Prepare transaction features, train and evaluate an Isolation Forest, then record a versioned model in MLflow.
Replay transactions through Kafka, transform each event, load a configured registry version, and publish prediction events.
The engineering problem
Model development and transaction scoring run on different data paths. The offline path needs repeatable experiments and a model version; the streaming path needs a compatible feature vector, decision threshold, and reliable event handoff.
SentinelML separates these responsibilities across two repositories. The architecture below follows the checked source. Its model handoff is configured, but the published defaults and metadata do not yet establish a verified end-to-end run.
Conceptual flow
One project, two connected pipelines
Offline code trains Isolation Forest and registers a model in MLflow. Online Python workers replay transactions through Kafka, process feature events, and publish predictions. The dashed model lookup is configured in source but unverified end to end: the default model names differ and serving also requires local metadata.
- Training data / artifact
- Configured model lookup
- Transaction event
- Feature vector
- Prediction event
Conceptual execution sequence
- The offline pipeline prepares transaction features from CSV data.
- Isolation Forest fits on normal rows and is evaluated on a mixed validation split.
- MLflow records the run and registers a model version.
- At startup, inference requests a configured model version and reads local feature metadata. The checked defaults do not yet align.
- A dataset row is replayed to Kafka raw_events.
- The processing worker publishes a transformed event to Kafka features.
- The inference worker scores the feature event and publishes to Kafka predictions.
Offline Training & Model Lifecycle
The MLOps repository samples a transaction CSV, derives time and transaction features, and fits an Isolation Forest on normal training rows. Its training script uses a stratified 80/20 train and validation split after feature preparation, then selects a decision threshold against the mixed validation set.
The script logs model parameters and validation metrics to an MLflow run, saves a local joblib file, and registers a scikit-learn model version. A separate validation script can tag a candidate and set a champion alias. That script is not invoked by the checked Jenkins pipeline, and its comparison of an existing champion reads f1 while training logs f1_score.
The feature transform fits a scaler and category encoding before the split. No fitted scaler, encoder, or feature-order manifest is logged with the registered model. The validation metrics therefore need a leakage-safe rerun before they can be treated as held-out generalization results.
Online Transaction Inference
The SentinelML repository implements a CSV event generator that publishes to Kafka raw_events. A Python processing worker consumes those events, transforms fields, and publishes feature events to features. A separate inference worker consumes features, scores them, and publishes prediction events to predictions. It also writes a score to PostgreSQL when a transaction ID is present.
In the inspected runtime, these are Kafka consumers and producers. The worker does not call a FastAPI endpoint, and no FastAPI service was found in the checked source. The CV documents a FastAPI-based pipeline, but this public revision does not substantiate that detail.
Model handoff & feature contract
At worker startup, IsolationForestModel.__init__ loads models:/<name>/<version> from MLflow and reads a local JSON file for feature order and threshold. It keeps that model in memory for the consumer loop; the configured alias is unused, and the worker has no model reload path.
The training script registers FraudDetectionModel_Champion, while serving defaults to fraud-isolation-forest version 1. Both Compose stacks configure a shared MLflow network, but the names differ and the required metadata file is absent from the checked runtime repository. Model selection and deployment still require configuration and integration validation.
The streaming processor also builds features differently from the batch transform. Serving zero-fills missing named features using local metadata; it does not reuse the offline fitted scaler or category mapping. This prevents a claim that training and serving have a guaranteed matching feature contract.
Saved notebook output · historical
Evaluation evidence, with its limits
The awab branch notebook stores results from an Isolation Forest experiment. These are the CV figures, but the dataset was unavailable for an independent rerun and the saved notebook does not support a clean generalization claim.
| Measure | Saved value | Context |
|---|---|---|
| Test rows | 1,496,754 | Saved awab notebook classification report |
| ROC-AUC | 0.875 | Saved test scores from a normal-only Isolation Forest |
| Average precision | 0.678 | Reported as PR-AUC in the notebook |
| Anomaly recall | 72% | Class-1 recall at the notebook's model threshold |
The same saved classification report shows anomaly-class precision of 54% and F1 of 0.62. Its displayed feature selection includes is_fraud in the input matrix, and scaling happens before the train/test split. The 1,496,754 rows are the notebook’s reported test set, not a verified deployment result. The current training script uses a different train/validation procedure, so these values should not be attributed to its registered model.
Engineering decisions
Two repositories, one model lifecycle
MLOps contains feature engineering, training, MLflow tracking, and a separate registry-validation script. SentinelML contains transaction replay, Kafka workers, and inference. Both Compose stacks name the same MLflow network, but source alone does not verify a successful combined run.
Explicit model lookup
The inference worker loads a fixed registry name and version once when it starts, then reads feature order and threshold from local JSON. The promotion script can set a champion alias, but serving does not resolve that alias or reload a model during its loop.
Kafka service boundaries
The generator publishes raw_events, the processor publishes features, and inference publishes predictions. The three topics separate replay, transformation, and scoring without making FastAPI a hidden step in the event path.
Feature contract remains open
Training fits scaling and categorical encoding on a batch before the split; the streaming processor transforms individual records differently. No fitted transform or feature-order artifact is published with the registered model in the inspected source.
Automation scope
The public Jenkinsfile checks out source and installs worker dependencies. Training, validation, registry promotion, image deployment, and service restart are not Jenkins stages in the inspected repositories.
Implementation evidence
From a feature vector to a prediction event
These source excerpts show the scoring operation and the worker entry point. They describe implementation, not a captured end-to-end execution.
Score and threshold
raw_score = self.model.decision_function(x)[0]
anomaly_score = float(-raw_score)
is_anomaly = anomaly_score >= self.thresholdThe model orders inputs with local metadata before scoring. The prediction schema requires a nonnegative anomaly score, although the negated decision function can be negative; this path needs runtime validation.
Read the scoring implementationWorker entry point
CMD ["python","main.py"]The container starts a Kafka consumer. Its loop publishes prediction events and conditionally persists scores associated with transaction IDs.
Read the DockerfileAutomation & validation boundary
The public Jenkinsfile performs checkout and installs the three worker services’ Python dependencies. It does not train, evaluate, register, promote, deploy, or restart a model. The CV documents retraining automation, but those steps cannot be verified from the inspected Jenkinsfiles and scripts.
The repositories establish training, model registration, registry lookup, and Kafka scoring components. A successful combined execution, a matching deployed feature schema, and an automatically promoted or reloaded model remain unverified.
Technology stack
- Offline training
- Python · scikit-learn Isolation Forest · pandas
- Model management
- MLflow runs, metrics, model versions · PostgreSQL + object storage
- Streaming inference
- Kafka · Python workers · Pydantic events · PostgreSQL
- Packaging & automation
- Docker Compose · Jenkins checkout and dependency installation
Implementation references
- Saved historical evaluation
MLOps · awab/train.ipynb - Training and validation
MLOps · tracking/train_eval.py - Offline feature engineering
MLOps · tracking/feature_eng.py - MLflow model registration
MLOps · tracking/utils.py - Registry validation and alias
MLOps · validation/validate_promote.py - MLflow service and artifact store
MLOps · infra/mlflow/docker-compose/docker-compose.yml - Transaction replay
SentinelML · services/event_generator_service/generator.py - Feature processing
SentinelML · services/data_process_service/main.py - Streaming transformation
SentinelML · services/data_process_service/data_processing.py - Inference consumer and output
SentinelML · services/inference_service/main.py - Registry model lookup
SentinelML · services/inference_service/model.py - Serving configuration
SentinelML · services/inference_service/config.py - Jenkins stages
SentinelML · Jenkinsfile - Runtime topology
SentinelML · infra/docker-compose.yaml