Your team needs to build a model that predicts whether images contain a driver's license, passport, or credit card. The data engineering team already built the pipeline and generated a dataset composed of 10,000 images with driver's licenses, 1,000 images with passports, and 1,000 images with credit cards. You now have to train a model with the following label map: [`˜drivers_license', `˜passport', `˜credit_card']. Which loss function should you use?
For this problem, where each image can belong to one of three classes (driver's license, passport, or credit card), the appropriate loss function is categorical cross-entropy. This loss function is specifically designed for multi-class classification tasks with mutually exclusive classes and measures the dissimilarity between the predicted class probabilities and the true class labels. Using categorical cross-entropy ensures that the model is trained to produce probability distributions over the three classes, which helps in achieving accurate predictions.
Answer is C
Use sparse categorical crossentropy when your classes are mutually exclusive (e.g. when each sample belongs exactly to one class) and categorical crossentropy when one sample can have multiple classes or labels are soft probabilities (like [0.5, 0.3, 0.2]).
Definitely C - the target variable label formulated in the question requires a categorical cross entropy loss function i.e. 3 columns 'drivers_license' , 'passport', 'credit_card' that can take values 1, 0. Meanwhile sparse categorical cross entropy would require the labels to be integer encoded in a single vector, for example, 'drivers_license' = 1, 'passport' = 2, 'credit_card' = 3.
Actually it is exactly the opposite. Your label map has 3 options which are mutually exclusive. A document cannot be both - a driver license and a passport. There is a SPARSE vector as output - only one of the categorical outputs is valid for a one example.
No, I'm sorry, I wrote it before checking - You were right. We use sparse categorical cross entropy when we have just an index (integer) as a label. The only difference is that it decodes the integer into one hot representation that suites to out DNN output.
answer is D https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/
Use sparse categorical crossentropy when your classes are mutually exclusive (e.g. when each sample belongs exactly to one class) and categorical crossentropy when one sample can have multiple classes or labels are soft probabilities (like [0.5, 0.3, 0.2]).
Literally from the link you posted: "A possible cause of frustration when using cross-entropy with classification problems with a large number of labels is the one hot encoding process. [...] This can mean that the target element of each training example may require a one hot encoded vector with tens or hundreds of thousands of zero values, requiring significant memory. Sparse cross-entropy addresses this by performing the same cross-entropy calculation of error, without requiring that the target variable be one hot encoded prior to training". Here we have 3 categories...No problem doing one-hot encoding. Answer: C
In this case, we have a multi-class classification problem with three classes: driver's license, passport, and credit card. Therefore, we should use the categorical cross-entropy loss function to train our model. Sparse categorical cross-entropy is used for multi-class classification problems where the labels are represented in a sparse matrix format. This is not the case in this problem.
You now HAVE TO to train a model with the following label map: [`˜drivers_license', `˜passport', `˜credit_card'].
Use sparse categorical crossentropy when your classes are mutually exclusive (e.g. when each sample belongs exactly to one class) and categorical crossentropy when one sample can have multiple classes or labels are soft probabilities (like [0.5, 0.3, 0.2]).
A. Categorical hinge : Mainly for SVM soft margins B. Binary cross-entropy : for 2 class only C. Categorical cross-entropy: Multi class but not necessarily Mutually exclusive D. Sparse categorical cross-entropy : Multi class + Mutually exclusive only , saves memory too
https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy https://www.tensorflow.org/api_docs/python/tf/keras/metrics/sparse_categorical_crossentropy
it's C
Req : Multi class + mutually exclusive labels A. Categorical hinge : Mainly for SVM soft margins B. Binary cross-entropy : for 2 class only C. Categorical cross-entropy: Multi class but not necessarily Mutually exclusive D. Sparse categorical cross-entropy : Multi class + Mutually exclusive only , saves memory too
The correct answer is: C. Categorical cross-entropy. you are dealing with a multi-class classification problem where each image can belong to one of three classes: "driver's license," "passport," or "credit card." Categorical cross-entropy is the appropriate loss function for multi-class classification tasks. It measures the dissimilarity between the predicted class probabilities and the true class labels. It's designed to penalize larger errors in predicted probabilities and help the model converge towards more accurate predictions.
https://fmorenovr.medium.com/sparse-categorical-cross-entropy-vs-categorical-cross-entropy-ea01d0392d28
categorical_crossentropy (cce) produces a one-hot array containing the probable match for each category, sparse_categorical_crossentropy (scce) produces a category index of the most likely matching category.
Only 3 categories of values being either T or F. They don't really need to be integer encoded, which differs sparse cross-entropy from categorical.
mutually exclusive classes
If you are wondering between C & D - think about what "sparse" means It is used when dealing with hundreds of categories
It depends on how the labels are encoded. If onehot use CCE. If its a single integer representing the class use SCCE (Source: same as in the official (wrong) answer) From the question it's not clear how the labels are encoded. But for just 3 classes there is no doubt it's better to go with one-hot encoding. Memory restrictions or a huge number of classes might point to SCCE
C D is for integer value instead of one-hot encoded vectors, in our question, it is 'drivers_license', 'passport', 'credit_card' one-hot.
I'd go with C. Categorical cross entropy is used when classes are mutually exclusive. If the number of classes was very high, then we could use sparse categorical cross entropy.
C) Multi-Class Classification (Three or More Classes): Since you have three classes, you should use a multi-class loss function. The most common choice for multi-class image classification is categorical cross-entropy2. Categorical cross-entropy is designed for scenarios where each input belongs to exactly one class (i.e., mutually exclusive classes). Therefore, the correct answer is C. Categorical cross-entropy. It’s well-suited for multi-class classification tasks like this one. References: How to Choose Loss Functions When Training Deep Learning Neural Networks (https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/) Stack Exchange: How to know which loss function is suitable for image classification? (https://datascience.stackexchange.com/questions/58138/how-to-know-which-loss-function-is-suitable-for-image-classification)
C needs the target to be One hot encoded already. Since it is not, the answer is D