AeroRUL – Turbofan Engine Remaining Useful Life Prediction
An end-to-end predictive-maintenance system for jet engines: raw sensor telemetry in, a maintenance recommendation out. Five genuinely different models — XGBoost, LSTM, TCN, Transformer, and Weibull AFT survival analysis — are compared consistently across NASA's four CMAPSS subsets using RMSE, MAE, and the asymmetric PHM08 scoring function, with split conformal prediction intervals for calibrated uncertainty. A real evaluation bug — scoring against the capped training label instead of true RUL, which understated XGBoost's FD002 RMSE by 45% (15.2 vs. the real 27.7) — is caught, explained, and fixed with before/after numbers across four files. Served by a FastAPI backend behind a React/TypeScript fleet-health dashboard that runs backend-free from static snapshots or against a live API, containerized with Docker, and walked through across five fully-executed Jupyter notebooks.
Proof points
- 5 Models × 4 Subsets Compared
- Split Conformal Intervals
- Caught & Fixed Eval Bug
Technologies
Python, XGBoost, PyTorch, FastAPI, React, TypeScript, MLflow, Docker, uv
Links
Problem
A single RMSE number doesn't tell a maintenance planner anything actionable — they need a point estimate and a sense of how much to trust it, with a business-legible risk tier, served somewhere a monitoring dashboard can consume it. Most RUL projects stop at model.predict() in a notebook.
Approach
Built condition-aware feature engineering, compared five genuinely different models (XGBoost, LSTM, TCN, Transformer, Weibull AFT survival) per CMAPSS subset with a per-subset champion selection, added split conformal prediction intervals for calibrated uncertainty, and served the result through a FastAPI backend behind a React fleet-health dashboard, containerized with Docker.
Results
Selected per-subset champions scored RMSE 16.0-28.3 across FD001-FD004. Along the way, caught and fixed a real evaluation bug that scored predictions against the capped training label instead of true RUL — understating XGBoost's FD002 RMSE by 45% (15.2 vs. the real 27.7) — and documented the fix with before/after numbers across four files and a dedicated notebook.
AeroRUL: Turbofan Engine RUL Prediction — Engineering Report
1. Executive Summary
AeroRUL is a full-pipeline predictive-maintenance system for jet engines built on NASA's CMAPSS turbofan degradation dataset. Rather than reporting a single RMSE, it builds the whole system a production MLE would be asked to ship: condition-aware feature engineering, a five-model comparison with a per-subset champion, calibrated uncertainty, a served API, and a real dashboard — with an honest account of a real evaluation bug found and fixed along the way.
2. Feature Engineering
- Six distinct operating conditions are clustered and normalized per-condition, so models spend their capacity distinguishing wear from altitude instead of conflating the two
- Rolling-window features and a hand-designed, sign-aligned health indicator summarize sensor degradation trends
- Training targets RUL capped at 125 cycles (a standard trick — an engine ~300 cycles from failure shows negligible wear)
3. Model Comparison
Five architectures evaluated identically per CMAPSS subset (FD001-FD004): XGBoost, LSTM, TCN (dilated causal convolutions), a Transformer-encoder regressor (all three deep models built from scratch in PyTorch), and a Weibull AFT survival model via lifelines. The champion is selected per-subset, not assumed in advance, and written to models_store/champion.json for the API to serve.
| Subset | Champion | RMSE | MAE | NASA score |
|---|---|---|---|---|
| FD001 | Transformer | 16.0 | 12.2 | 439 |
| FD002 | TCN | 28.3 | 19.4 | 10,486 |
| FD003 | Transformer | 16.0 | 11.7 | 549 |
| FD004 | Transformer | 26.6 | 19.5 | 5,647 |
FD002/FD004 (six operating conditions, multiple fault modes) are genuinely harder — every model's error roughly doubles versus FD001/FD003. Weibull AFT loses on NASA score everywhere despite a solid ~0.80-0.82 concordance index; it's kept deliberately because it's the only model that natively produces a full failure-time distribution rather than one number.
4. The Bug: Evaluating Against the Wrong Target
Partway through, the evaluation code turned out to be scoring predictions against the capped training label instead of the true, uncapped RUL from RUL_*.txt. That hid the error almost entirely for engines with a lot of life left: 57 of FD002's 259 test engines have true RUL above the 125-cycle cap, and evaluating against the cap made XGBoost's FD002 RMSE look like 15.2 instead of its real 27.7. The fix touched four places — train.py, train_sequence.py, compare_models.py, and the conformal-calibration diagnostic — plus a follow-up: the conformal calibration set had the identical contamination and needed the same filter. Everything is walked through with real before/after numbers in notebook 04 and the git history.
5. Uncertainty Quantification
Split conformal prediction intervals give a finite-sample coverage guarantee on top of each point estimate. Coverage lands at 89-96% against a 90% target on FD001/FD003, but under-covers at 76-80% on FD002/FD004 — a gap that's reported rather than smoothed over, since residual-distribution shift is larger on the harder, multi-condition subsets.
6. Served System
- FastAPI backend (
/predict,/fleet,/engine,/models,/health) whoseModelRegistrylazily loads each subset's champion model and calibrated interval, dispatching every prediction through the sameFittedPipeline.transformused at training time - React + TypeScript fleet-health dashboard that runs two ways from one codebase: backend-free against static JSON snapshots (
scripts/export_static_data.py), or live against the FastAPI service, selected by a single build-time environment variable - MLflow experiment tracking, Docker + docker-compose for the full stack, and five Jupyter notebooks (preprocessing, EDA, modeling, uncertainty/survival, API deployment) executed end-to-end with real output
7. Dataset
NASA CMAPSS (Commercial Modular Aero-Propulsion System Simulation): simulated turbofan degradation runs across four subsets varying in operating conditions and fault modes, each engine described by 3 operational settings and 21 sensor channels.