Running toy datasets through scikit-learn, with Claude's assistance, and confusion matrices and the classification report came up.
uv run python .\main.py
Confusion Matrix:
[[14 0 0]
[ 0 14 0]
[ 0 0 8]]
Classification Report:
precision recall f1-score support
class_0 1.00 1.00 1.00 14
class_1 1.00 1.00 1.00 14
class_2 1.00 1.00 1.00 8
accuracy 1.00 36
macro avg 1.00 1.00 1.00 36
weighted avg 1.00 1.00 1.00 36
I did some digging through the documentation and learned:
The precision is intuitively the ability of the classifier not to label a negative sample as positive. The recall is intuitively the ability of the classifier to find all the positive samples.
My rough understanding here is that if false positives are costly, you want your model to be more precise, and if false negatives are costly, you want to ensure your model has good recall. The F1-score balances precision and recall equally, which is tunable using F-beta: beta > 1 favors recall, beta < 1 favors precision.
The F-beta score weights recall more than precision by a factor ofbeta.beta == 1.0means recall and precision are equally important.