You have deployed multiple versions of an image classification model on AI Platform. You want to monitor the performance of the model versions over time. How should you perform this comparison?
You have deployed multiple versions of an image classification model on AI Platform. You want to monitor the performance of the model versions over time. How should you perform this comparison?
To monitor the performance of multiple versions of an image classification model over time, it is important to use a continuous evaluation method that can provide ongoing metrics. The Continuous Evaluation feature allows for measuring and comparing metrics such as mean average precision across different model versions, making it suitable for tracking performance over a period. This approach ensures that performance is monitored in a systematic and continuous manner, which is critical for understanding how models perform over time and in changing conditions.
Answer is D
D is correct. Choose the feature / capability GCP provides is always a good bet. :)
The best option to monitor the performance of multiple versions of an image classification model on AI Platform over time is to compare the loss performance for each model on the validation data. Option B is the best approach because comparing the loss performance of each model on the validation data is a common method to monitor machine learning model performance over time. The validation data is a subset of the data that is not used for model training, but is used to evaluate its performance during training and to compare different versions of the model. By comparing the loss performance of each model on the same validation data, you can determine which version of the model has better performance.
D - because you are using a Google provided feature. remember in this exam its important to always choose the google services over anything else
Guys, I not sure about the answer D ... And maybe you could help me in my arguments. I think choose loss to compare the model performance is better than see for metrics. For example, when can build an image model classification that has good precision metrics, because the class in unbalanced, but the loss could be terrible because of kind of loss choose that penalizes classes. so, losses are better than metrics to available models, and the answer is in A or B. I thought that the A could be the answer because I see validation as a part of the training process. So, If we want to test the model performance over time, we have to use new data, which I suppose to be the held-out data.
In the official study guide, this was the explanation given for answer B : "The image classification model is a deep learning model. You minimize the loss of deep learning models to get the best model. So comparing loss performance for each model on validation data is the correct answer."
The answer is A. I am not sure why people choose B vs A as you may overfit your validation set. And you are using your held-out set really rare == no option to overfit.
ans: D
If you have multiple model versions in a single model and have created an evaluation job for each one, you can view a chart comparing the mean average precision of the model versions over time
o monitor the performance of the model versions over time, you should compare the loss performance for each model on the validation data. Therefore, option B is the correct answer.
Please, How? B is not monitoring. It is a validation. The definition of monitoring states: "observe and check the progress or quality of (something) over a period of time" So it is a continuous process. Each option A,B,C are just one time check, not monitoring.
I go for B. Option D is good when we are already in production
Went with D
D. Compare the mean average precision across the models using the Continuous Evaluation feature https://cloud.google.com/vertex-ai/docs/evaluation/introduction Vertex AI provides model evaluation metrics, such as precision and recall, to help you determine the performance of your models... Vertex AI supports evaluation of the following model types: AuPRC: The area under the precision-recall (PR) curve, also referred to as average precision. This value ranges from zero to one, where a higher value indicates a higher-quality model.
I choose by myself D. But as I read the post here https://www.v7labs.com/blog/mean-average-precision, I was not sure about D. It wrote mAP is commonly used for object detection or instance segmentation tasks. Validation Dataset in GCP context: not trained dataset and not seen dataset
Went with D, using continuous evaluation feature seems correct to me.
mAP is for object detection, so the answer should be B
Continuous Evaluation feature is deprecated.
so it looks like that B is the best answer
so is the what if tool