You are working on a Neural Network-based project. The dataset provided to you has columns with different ranges. While preparing the data for model training, you discover that gradient optimization is having difficulty moving weights to a good solution. What should you do?
When working with a neural network and encountering difficulties in gradient optimization, especially in the presence of features with different ranges, the appropriate step is to normalize the data. Normalization scales the dataset's numeric fields to a common range, typically between 0 and 1. This process ensures that features contribute equally to the gradient descent optimization, facilitating better and faster convergence towards an optimal solution. Therefore, using the representation transformation (normalization) technique is the appropriate approach in this scenario.
Vote for B. We could impute instead of remove the column to avoid loss of information
I also think it is B: "The presence of feature value X in the formula will affect the step size of the gradient descent. The difference in ranges of features will cause different step sizes for each feature. To ensure that the gradient descent moves smoothly towards the minima and that the steps for gradient descent are updated at the same rate for all the features, we scale the data before feeding it to the model."
B. The problem does not mention anything about missing values. It needs to normalize the features with different ranges.
normalization https://developers.google.com/machine-learning/data-prep/transform/transform-numeric
B - remove the outliers?
Normalization is more complicated than that. Normalization changes the values of dataset's numeric fields to be in a common scale, without impacting differences in the ranges of values. Normalization is required only when features have different ranges.
(B) - NN models needs features with close ranges - SGD converges well using features in [0, 1] scale - The question specifically mention "different ranges" Documentation - https://developers.google.com/machine-learning/data-prep/transform/transform-numeric
When gradient descent fails, it's out of the lacking of a powerful feature. Using normalization would make it worse. Instead, using either A or C would increase the strength of certain feature. But, C should come first since A is only feasible after at least 1 meaningful training. So C.
Normalization is the word.
B - The key phrase is "different ranges", therefore we need to normalize the values.
Because the range needs to normalize
B) Option B (Use the representation transformation technique) is the most relevant choice. Normalizing the features will help gradient descent converge efficiently, leading to better weight updates and improved model performance. Remember that feature scaling is crucial for gradient optimization, especially when dealing with features that have different ranges. By ensuring consistent scales, you’ll enhance the effectiveness of your Neural Network training process.
Looking at explanation I would choose C as well
B "Normalization" is the keyword
Normalization is the word.
Normalization
Went with B