I am having trouble of understanding the concept of Gridsearch. Could someone explain it for me that how it works in tuning the hyperparamter and how the parameter of the grid is determined(the hidden layer size)? Thanks.
1 Like
Every model has their own hyperparameters, if you are using layers I assume you are trying out some neural network model (MLP regression perhaps).
A neural network might have a hyperparameter called hidden layers size, you have to search for the best value for this parameter.
One way to search for the best value is to perform GridsearchCV which tests all the possible combinations of hyperparameters and gives you the best model back based on how well it performs.
You have to specify which parameters you have to search for and what values you want to search across. Grid search will test out every combination that you give it.
For example if you are using a SVC classification model, you know the parameters are the kernel, gamma, and C.
tuned_parameters = [
{"kernel": ["rbf"], "gamma": [1e-3, 1e-4], "C": [1, 10, 100, 1000]},
{"kernel": ["linear"], "C": [1, 10, 100, 1000]},
]
It is your job to prespecify them above, and then to perform the search below.
clf = GridSearchCV(SVC(), tuned_parameters, scoring="%s_macro" % score)
clf.fit(X_train, y_train)
print("Best parameters set found on development set:")
print(clf.best_params_)
print("Grid scores on development set:")
means = clf.cv_results_["mean_test_score"]
stds = clf.cv_results_["std_test_score"]
1 Like