Track experiments with MLflow
Track experiments with MLflow. In this video you'll record every training run of a small model, and compare the runs side by side. A training script logs its parameters, metrics and model to the MLflow tracking server. The server keeps the runs in SQLite and the files in a folder, and its web interface shows them. In this lesson you will start a tracking server, log three training runs, and compare them. Before you start, you need the starter repository's virtual environment: make check shows its versions. The repository's virtual environment has MLflow three point eighteen, scikit-learn, and TrustyAI with its own Java runtime. Docker and kind are here for later. That starts the MLflow tracking server on one two seven dot zero dot zero dot one, port five zero five zero. It keeps the runs in a SQLite file, and the model files in a folder next to it. Every script in the repository talks to this address. This is the tracked part of a training run. It logs the parameters and the training data, trains a random forest on scikit-learn's diabetes dataset, then logs three metrics and the model itself. The question is yes or no: will this patient need a follow-up visit? One run: a depth of four and a hundred trees. It reports the area under the ROC curve, the receiver operating characteristic, and the accuracy, both on patients it has not seen. Now two more depths, in one loop. Three runs, best first. The shallow trees win here: depth two scores highest on held-out patients, and depth six is the lowest. That comparison is what experiment tracking is for. This is the MLflow web interface, on port five zero five zero. The experiment has the three runs, each with its name, the dataset it was trained on, and how long it took. A run keeps its metrics, its parameters, and the model it logged. The data set is recorded too, with a digest, so you can tell later exactly which data trained this model. Under artifacts is the model itself: the MLmodel file that describes it, its environment, and the weights, saved in the skops format. On OpenShift AI, training runs happen in a workbench of a data science project, and pipeline runs are grouped in experiments. In this lesson you learned to start an MLflow tracking server, to log parameters, data, metrics and a model in a run, and to compare runs in the terminal and in the MLflow web interface.