Feature selection in machine learning using Lasso regression

 Лассо-регрессия — это линейная модель, в которой используется эта функция стоимости:

a(j) — коэффициент j-го признака

Последний член называется штрафом l1, а α является гиперпараметром, который настраивает интенсивность этого штрафного члена. Чем выше коэффициент признака, тем выше значение функции стоимости. Идея регрессии Лассо состоит в том, чтобы оптимизировать функцию стоимости, уменьшая абсолютные значения коэффициентов. Очевидно, это работает, если функции были предварительно масштабированы, например, с использованием стандартизации или других методов масштабирования. Значение гиперпараметра α должно быть найдено с использованием подхода перекрестной проверки.

Загружаемые данные

(.env) boris@boris-All-Series:~/LASSO$ cat showDiabetes.py

import pandas as pd

from sklearn.datasets import load_diabetes

X, y = load_diabetes(return_X_y=True, as_frame=True)

n_samples = X.shape[0]

X.head()

print(X.head())

(.env) boris@boris-All-Series:~/LASSO$ python3 showDiabetes.py

        age       sex       bmi        bp  ...        s3        s4        s5        s6

0  0.038076  0.050680  0.061696  0.021872  ... -0.043401 -0.002592  0.019907 -0.017646

1 -0.001882 -0.044642 -0.051474 -0.026328  ...  0.074412 -0.039493 -0.068332 -0.092204

2  0.085299  0.050680  0.044451 -0.005670  ... -0.032356 -0.002592  0.002861 -0.025930

3 -0.089063 -0.044642 -0.011595 -0.036656  ... -0.036038  0.034309  0.022688 -0.009362

4  0.005383 -0.044642 -0.036385  0.021872  ...  0.008142 -0.002592 -0.031988 -0.046641

[5 rows x 10 columns]

*************************************

Выбор функций в машинном обучении с 

использованием регрессии Лассо

*************************************

(.env) boris@boris-All-Series:~/LASSO$ cat lassloEvaluate.py

import numpy as np

from sklearn.preprocessing import StandardScaler

from sklearn.pipeline import Pipeline

from sklearn.model_selection import train_test_split, GridSearchCV

from sklearn.linear_model import Lasso

from numpy import nan

from sklearn.datasets import load_diabetes

X,y = load_diabetes(return_X_y=True)

features = load_diabetes()['feature_names']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

# Подгоняем регрессию Лассо к масштабированной версии нашего набора данных 

# и рассматриваем только те признаки, коэффициент которых отличен от 0. 

# сначала нам нужно настроить α гиперпараметр, 

# чтобы иметь правильный вид регрессии Лассо.

pipeline = Pipeline([

                     ('scaler',StandardScaler()),

                     ('model',Lasso())

])

# рассчитываем среднее значение среднеквадратичной ошибки в 5-кратной 

# перекрестной проверке и выбираем значение α, которое минимизирует 

# средние показатели производительности. Используем объект GridSearchCV.

search = GridSearchCV(pipeline,

                      {'model__alpha':np.arange(0.1,10,0.1)},

                      cv = 5, scoring="neg_mean_squared_error",verbose=3

                      )

search.fit(X_train,y_train)

search.best_params_

coefficients = search.best_estimator_.named_steps['model'].coef_

# Важность признака – это абсолютное значение его коэффициента

importance = np.abs(coefficients)

print(importance)

# признаки, пережившие регрессию Лассо.

print(np.array(features)[importance > 0])

# в то время как 3 отброшенные функции

print(np.array(features)[importance == 0])

(.env) boris@boris-All-Series:~/LASSO$ python3 lassloEvaluate.py

Fitting 5 folds for each of 99 candidates, totalling 495 fits

[CV 1/5] END ..............model__alpha=0.1;, score=-3051.708 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.1;, score=-3405.374 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.1;, score=-2593.337 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.1;, score=-3309.628 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.1;, score=-3512.110 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.2;, score=-3057.672 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.2;, score=-3413.561 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.2;, score=-2606.353 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.2;, score=-3317.427 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.2;, score=-3502.696 total time=   0.0s

[CV 1/5] END model__alpha=0.30000000000000004;, score=-3057.759 total time=   0.0s

[CV 2/5] END model__alpha=0.30000000000000004;, score=-3421.294 total time=   0.0s

[CV 3/5] END model__alpha=0.30000000000000004;, score=-2604.960 total time=   0.0s

[CV 4/5] END model__alpha=0.30000000000000004;, score=-3320.206 total time=   0.0s

[CV 5/5] END model__alpha=0.30000000000000004;, score=-3497.555 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.4;, score=-3054.995 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.4;, score=-3423.267 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.4;, score=-2600.535 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.4;, score=-3317.535 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.4;, score=-3490.310 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.5;, score=-3052.664 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.5;, score=-3425.377 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.5;, score=-2596.304 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.5;, score=-3315.477 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.5;, score=-3483.496 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.6;, score=-3050.765 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.6;, score=-3427.624 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.6;, score=-2592.268 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.6;, score=-3314.126 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.6;, score=-3477.421 total time=   0.0s

[CV 1/5] END model__alpha=0.7000000000000001;, score=-3049.299 total time=   0.0s

[CV 2/5] END model__alpha=0.7000000000000001;, score=-3430.010 total time=   0.0s

[CV 3/5] END model__alpha=0.7000000000000001;, score=-2588.427 total time=   0.0s

[CV 4/5] END model__alpha=0.7000000000000001;, score=-3313.166 total time=   0.0s

[CV 5/5] END model__alpha=0.7000000000000001;, score=-3471.671 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.8;, score=-3048.266 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.8;, score=-3432.533 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.8;, score=-2585.202 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.8;, score=-3312.524 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.8;, score=-3466.204 total time=   0.0s

[CV 1/5] END ..............model__alpha=0.9;, score=-3047.657 total time=   0.0s

[CV 2/5] END ..............model__alpha=0.9;, score=-3435.193 total time=   0.0s

[CV 3/5] END ..............model__alpha=0.9;, score=-2582.412 total time=   0.0s

[CV 4/5] END ..............model__alpha=0.9;, score=-3312.175 total time=   0.0s

[CV 5/5] END ..............model__alpha=0.9;, score=-3459.595 total time=   0.0s

[CV 1/5] END ..............model__alpha=1.0;, score=-3047.490 total time=   0.0s

[CV 2/5] END ..............model__alpha=1.0;, score=-3437.991 total time=   0.0s

[CV 3/5] END ..............model__alpha=1.0;, score=-2579.756 total time=   0.0s

[CV 4/5] END ..............model__alpha=1.0;, score=-3312.138 total time=   0.0s

[CV 5/5] END ..............model__alpha=1.0;, score=-3452.790 total time=   0.0s

[CV 1/5] END ..............model__alpha=1.1;, score=-3050.246 total time=   0.0s

[CV 2/5] END ..............model__alpha=1.1;, score=-3440.927 total time=   0.0s

[CV 3/5] END ..............model__alpha=1.1;, score=-2577.251 total time=   0.0s

[CV 4/5] END ..............model__alpha=1.1;, score=-3312.405 total time=   0.0s

[CV 5/5] END ..............model__alpha=1.1;, score=-3446.137 total time=   0.0s

[CV 1/5] END model__alpha=1.2000000000000002;, score=-3053.802 total time=   0.0s

[CV 2/5] END model__alpha=1.2000000000000002;, score=-3443.990 total time=   0.0s

[CV 3/5] END model__alpha=1.2000000000000002;, score=-2574.880 total time=   0.0s

[CV 4/5] END model__alpha=1.2000000000000002;, score=-3312.977 total time=   0.0s

[CV 5/5] END model__alpha=1.2000000000000002;, score=-3439.590 total time=   0.0s

[CV 1/5] END model__alpha=1.3000000000000003;, score=-3058.377 total time=   0.0s

[CV 2/5] END model__alpha=1.3000000000000003;, score=-3447.207 total time=   0.0s

[CV 3/5] END model__alpha=1.3000000000000003;, score=-2572.643 total time=   0.0s

[CV 4/5] END model__alpha=1.3000000000000003;, score=-3313.854 total time=   0.0s

[CV 5/5] END model__alpha=1.3000000000000003;, score=-3433.196 total time=   0.0s

[CV 1/5] END model__alpha=1.4000000000000001;, score=-3063.172 total time=   0.0s

[CV 2/5] END model__alpha=1.4000000000000001;, score=-3452.612 total time=   0.0s

[CV 3/5] END model__alpha=1.4000000000000001;, score=-2570.352 total time=   0.0s

[CV 4/5] END model__alpha=1.4000000000000001;, score=-3315.036 total time=   0.0s

[CV 5/5] END model__alpha=1.4000000000000001;, score=-3426.910 total time=   0.0s

[CV 1/5] END model__alpha=1.5000000000000002;, score=-3068.177 total time=   0.0s

[CV 2/5] END model__alpha=1.5000000000000002;, score=-3458.469 total time=   0.0s

[CV 3/5] END model__alpha=1.5000000000000002;, score=-2568.037 total time=   0.0s

[CV 4/5] END model__alpha=1.5000000000000002;, score=-3316.522 total time=   0.0s

[CV 5/5] END model__alpha=1.5000000000000002;, score=-3420.777 total time=   0.0s

[CV 1/5] END ..............model__alpha=1.6;, score=-3073.409 total time=   0.0s

[CV 2/5] END ..............model__alpha=1.6;, score=-3464.458 total time=   0.0s

[CV 3/5] END ..............model__alpha=1.6;, score=-2565.841 total time=   0.0s

[CV 4/5] END ..............model__alpha=1.6;, score=-3318.312 total time=   0.0s

[CV 5/5] END ..............model__alpha=1.6;, score=-3414.748 total time=   0.0s

[CV 1/5] END model__alpha=1.7000000000000002;, score=-3078.862 total time=   0.0s

[CV 2/5] END model__alpha=1.7000000000000002;, score=-3472.823 total time=   0.0s

[CV 3/5] END model__alpha=1.7000000000000002;, score=-2563.757 total time=   0.0s

[CV 4/5] END model__alpha=1.7000000000000002;, score=-3320.408 total time=   0.0s

[CV 5/5] END model__alpha=1.7000000000000002;, score=-3408.853 total time=   0.0s

[CV 1/5] END model__alpha=1.8000000000000003;, score=-3084.532 total time=   0.0s

[CV 2/5] END model__alpha=1.8000000000000003;, score=-3481.395 total time=   0.0s

[CV 3/5] END model__alpha=1.8000000000000003;, score=-2561.806 total time=   0.0s

[CV 4/5] END model__alpha=1.8000000000000003;, score=-3322.808 total time=   0.0s

[CV 5/5] END model__alpha=1.8000000000000003;, score=-3403.089 total time=   0.0s

[CV 1/5] END model__alpha=1.9000000000000001;, score=-3090.430 total time=   0.0s

[CV 2/5] END model__alpha=1.9000000000000001;, score=-3490.066 total time=   0.0s

[CV 3/5] END model__alpha=1.9000000000000001;, score=-2559.969 total time=   0.0s

[CV 4/5] END model__alpha=1.9000000000000001;, score=-3325.513 total time=   0.0s

[CV 5/5] END model__alpha=1.9000000000000001;, score=-3397.455 total time=   0.0s

[CV 1/5] END ..............model__alpha=2.0;, score=-3096.542 total time=   0.0s

[CV 2/5] END ..............model__alpha=2.0;, score=-3498.843 total time=   0.0s

[CV 3/5] END ..............model__alpha=2.0;, score=-2558.260 total time=   0.0s

[CV 4/5] END ..............model__alpha=2.0;, score=-3328.526 total time=   0.0s

[CV 5/5] END ..............model__alpha=2.0;, score=-3391.956 total time=   0.0s

[CV 1/5] END ..............model__alpha=2.1;, score=-3102.868 total time=   0.0s

[CV 2/5] END ..............model__alpha=2.1;, score=-3507.722 total time=   0.0s

[CV 3/5] END ..............model__alpha=2.1;, score=-2557.360 total time=   0.0s

[CV 4/5] END ..............model__alpha=2.1;, score=-3332.364 total time=   0.0s

[CV 5/5] END ..............model__alpha=2.1;, score=-3386.566 total time=   0.0s

[CV 1/5] END ..............model__alpha=2.2;, score=-3109.422 total time=   0.0s

[CV 2/5] END ..............model__alpha=2.2;, score=-3514.595 total time=   0.0s

[CV 3/5] END ..............model__alpha=2.2;, score=-2557.369 total time=   0.0s

[CV 4/5] END ..............model__alpha=2.2;, score=-3336.518 total time=   0.0s

[CV 5/5] END ..............model__alpha=2.2;, score=-3381.331 total time=   0.0s

[CV 1/5] END model__alpha=2.3000000000000003;, score=-3116.208 total time=   0.0s

[CV 2/5] END model__alpha=2.3000000000000003;, score=-3518.052 total time=   0.0s

[CV 3/5] END model__alpha=2.3000000000000003;, score=-2557.541 total time=   0.0s

[CV 4/5] END model__alpha=2.3000000000000003;, score=-3340.954 total time=   0.0s

[CV 5/5] END model__alpha=2.3000000000000003;, score=-3376.208 total time=   0.0s

[CV 1/5] END model__alpha=2.4000000000000004;, score=-3124.331 total time=   0.0s

[CV 2/5] END model__alpha=2.4000000000000004;, score=-3521.579 total time=   0.0s

[CV 3/5] END model__alpha=2.4000000000000004;, score=-2557.871 total time=   0.0s

[CV 4/5] END model__alpha=2.4000000000000004;, score=-3345.654 total time=   0.0s

[CV 5/5] END model__alpha=2.4000000000000004;, score=-3371.222 total time=   0.0s

[CV 1/5] END model__alpha=2.5000000000000004;, score=-3131.134 total time=   0.0s

[CV 2/5] END model__alpha=2.5000000000000004;, score=-3525.166 total time=   0.0s

[CV 3/5] END model__alpha=2.5000000000000004;, score=-2558.360 total time=   0.0s

[CV 4/5] END model__alpha=2.5000000000000004;, score=-3350.646 total time=   0.0s

[CV 5/5] END model__alpha=2.5000000000000004;, score=-3366.353 total time=   0.0s

[CV 1/5] END ..............model__alpha=2.6;, score=-3137.608 total time=   0.0s

[CV 2/5] END ..............model__alpha=2.6;, score=-3528.818 total time=   0.0s

[CV 3/5] END ..............model__alpha=2.6;, score=-2559.007 total time=   0.0s

[CV 4/5] END ..............model__alpha=2.6;, score=-3355.907 total time=   0.0s

[CV 5/5] END ..............model__alpha=2.6;, score=-3361.627 total time=   0.0s

[CV 1/5] END ..............model__alpha=2.7;, score=-3144.345 total time=   0.0s

[CV 2/5] END ..............model__alpha=2.7;, score=-3532.533 total time=   0.0s

[CV 3/5] END ..............model__alpha=2.7;, score=-2559.412 total time=   0.0s

[CV 4/5] END ..............model__alpha=2.7;, score=-3361.452 total time=   0.0s

[CV 5/5] END ..............model__alpha=2.7;, score=-3357.025 total time=   0.0s

[CV 1/5] END model__alpha=2.8000000000000003;, score=-3151.172 total time=   0.0s

[CV 2/5] END model__alpha=2.8000000000000003;, score=-3536.311 total time=   0.0s

[CV 3/5] END model__alpha=2.8000000000000003;, score=-2560.217 total time=   0.0s

[CV 4/5] END model__alpha=2.8000000000000003;, score=-3367.268 total time=   0.0s

[CV 5/5] END model__alpha=2.8000000000000003;, score=-3352.553 total time=   0.0s

[CV 1/5] END model__alpha=2.9000000000000004;, score=-3154.902 total time=   0.0s

[CV 2/5] END model__alpha=2.9000000000000004;, score=-3540.152 total time=   0.0s

[CV 3/5] END model__alpha=2.9000000000000004;, score=-2559.684 total time=   0.0s

[CV 4/5] END model__alpha=2.9000000000000004;, score=-3373.353 total time=   0.0s

[CV 5/5] END model__alpha=2.9000000000000004;, score=-3348.204 total time=   0.0s

[CV 1/5] END model__alpha=3.0000000000000004;, score=-3158.734 total time=   0.0s

[CV 2/5] END model__alpha=3.0000000000000004;, score=-3544.056 total time=   0.0s

[CV 3/5] END model__alpha=3.0000000000000004;, score=-2559.268 total time=   0.0s

[CV 4/5] END model__alpha=3.0000000000000004;, score=-3379.726 total time=   0.0s

[CV 5/5] END model__alpha=3.0000000000000004;, score=-3343.984 total time=   0.0s

[CV 1/5] END ..............model__alpha=3.1;, score=-3163.488 total time=   0.0s

[CV 2/5] END ..............model__alpha=3.1;, score=-3548.024 total time=   0.0s

[CV 3/5] END ..............model__alpha=3.1;, score=-2558.965 total time=   0.0s

[CV 4/5] END ..............model__alpha=3.1;, score=-3386.376 total time=   0.0s

[CV 5/5] END ..............model__alpha=3.1;, score=-3339.909 total time=   0.0s

[CV 1/5] END ..............model__alpha=3.2;, score=-3168.444 total time=   0.0s

[CV 2/5] END ..............model__alpha=3.2;, score=-3552.054 total time=   0.0s

[CV 3/5] END ..............model__alpha=3.2;, score=-2558.776 total time=   0.0s

[CV 4/5] END ..............model__alpha=3.2;, score=-3393.512 total time=   0.0s

[CV 5/5] END ..............model__alpha=3.2;, score=-3337.058 total time=   0.0s

[CV 1/5] END model__alpha=3.3000000000000003;, score=-3173.489 total time=   0.0s

[CV 2/5] END model__alpha=3.3000000000000003;, score=-3556.148 total time=   0.0s

[CV 3/5] END model__alpha=3.3000000000000003;, score=-2558.704 total time=   0.0s

[CV 4/5] END model__alpha=3.3000000000000003;, score=-3400.620 total time=   0.0s

[CV 5/5] END model__alpha=3.3000000000000003;, score=-3334.866 total time=   0.0s

[CV 1/5] END model__alpha=3.4000000000000004;, score=-3178.625 total time=   0.0s

[CV 2/5] END model__alpha=3.4000000000000004;, score=-3560.305 total time=   0.0s

[CV 3/5] END model__alpha=3.4000000000000004;, score=-2558.747 total time=   0.0s

[CV 4/5] END model__alpha=3.4000000000000004;, score=-3407.495 total time=   0.0s

[CV 5/5] END model__alpha=3.4000000000000004;, score=-3332.878 total time=   0.0s

[CV 1/5] END model__alpha=3.5000000000000004;, score=-3183.850 total time=   0.0s

[CV 2/5] END model__alpha=3.5000000000000004;, score=-3564.526 total time=   0.0s

[CV 3/5] END model__alpha=3.5000000000000004;, score=-2558.905 total time=   0.0s

[CV 4/5] END model__alpha=3.5000000000000004;, score=-3414.613 total time=   0.0s

[CV 5/5] END model__alpha=3.5000000000000004;, score=-3331.004 total time=   0.0s

[CV 1/5] END ..............model__alpha=3.6;, score=-3189.165 total time=   0.0s

[CV 2/5] END ..............model__alpha=3.6;, score=-3568.812 total time=   0.0s

[CV 3/5] END ..............model__alpha=3.6;, score=-2559.179 total time=   0.0s

[CV 4/5] END ..............model__alpha=3.6;, score=-3421.976 total time=   0.0s

[CV 5/5] END ..............model__alpha=3.6;, score=-3329.244 total time=   0.0s

[CV 1/5] END ..............model__alpha=3.7;, score=-3194.569 total time=   0.0s

[CV 2/5] END ..............model__alpha=3.7;, score=-3573.159 total time=   0.0s

[CV 3/5] END ..............model__alpha=3.7;, score=-2559.568 total time=   0.0s

[CV 4/5] END ..............model__alpha=3.7;, score=-3428.570 total time=   0.0s

[CV 5/5] END ..............model__alpha=3.7;, score=-3327.419 total time=   0.0s

[CV 1/5] END model__alpha=3.8000000000000003;, score=-3200.063 total time=   0.0s

[CV 2/5] END model__alpha=3.8000000000000003;, score=-3577.569 total time=   0.0s

[CV 3/5] END model__alpha=3.8000000000000003;, score=-2560.072 total time=   0.0s

[CV 4/5] END model__alpha=3.8000000000000003;, score=-3433.264 total time=   0.0s

[CV 5/5] END model__alpha=3.8000000000000003;, score=-3325.700 total time=   0.0s

[CV 1/5] END model__alpha=3.9000000000000004;, score=-3205.647 total time=   0.0s

[CV 2/5] END model__alpha=3.9000000000000004;, score=-3582.042 total time=   0.0s

[CV 3/5] END model__alpha=3.9000000000000004;, score=-2560.692 total time=   0.0s

[CV 4/5] END model__alpha=3.9000000000000004;, score=-3438.085 total time=   0.0s

[CV 5/5] END model__alpha=3.9000000000000004;, score=-3324.064 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.0;, score=-3211.321 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.0;, score=-3586.579 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.0;, score=-2561.427 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.0;, score=-3443.033 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.0;, score=-3322.511 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.1;, score=-3217.084 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.1;, score=-3591.178 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.1;, score=-2562.277 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.1;, score=-3448.107 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.1;, score=-3321.043 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.2;, score=-3222.937 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.2;, score=-3595.841 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.2;, score=-2563.242 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.2;, score=-3453.307 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.2;, score=-3319.658 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.3;, score=-3228.879 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.3;, score=-3600.568 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.3;, score=-2564.322 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.3;, score=-3458.634 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.3;, score=-3318.357 total time=   0.0s

[CV 1/5] END model__alpha=4.3999999999999995;, score=-3234.946 total time=   0.0s

[CV 2/5] END model__alpha=4.3999999999999995;, score=-3605.358 total time=   0.0s

[CV 3/5] END model__alpha=4.3999999999999995;, score=-2565.518 total time=   0.0s

[CV 4/5] END model__alpha=4.3999999999999995;, score=-3464.088 total time=   0.0s

[CV 5/5] END model__alpha=4.3999999999999995;, score=-3317.139 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.5;, score=-3241.068 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.5;, score=-3610.211 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.5;, score=-2566.829 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.5;, score=-3469.667 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.5;, score=-3316.005 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.6;, score=-3247.280 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.6;, score=-3615.127 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.6;, score=-2568.256 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.6;, score=-3475.373 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.6;, score=-3314.955 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.7;, score=-3253.582 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.7;, score=-3620.106 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.7;, score=-2569.798 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.7;, score=-3481.206 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.7;, score=-3313.988 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.8;, score=-3259.974 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.8;, score=-3625.149 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.8;, score=-2571.455 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.8;, score=-3487.165 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.8;, score=-3313.106 total time=   0.0s

[CV 1/5] END ..............model__alpha=4.9;, score=-3266.455 total time=   0.0s

[CV 2/5] END ..............model__alpha=4.9;, score=-3630.257 total time=   0.0s

[CV 3/5] END ..............model__alpha=4.9;, score=-2573.227 total time=   0.0s

[CV 4/5] END ..............model__alpha=4.9;, score=-3493.250 total time=   0.0s

[CV 5/5] END ..............model__alpha=4.9;, score=-3312.306 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.0;, score=-3273.026 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.0;, score=-3632.048 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.0;, score=-2575.114 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.0;, score=-3499.462 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.0;, score=-3311.591 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.1;, score=-3279.686 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.1;, score=-3630.754 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.1;, score=-2577.117 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.1;, score=-3505.800 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.1;, score=-3310.959 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.2;, score=-3286.436 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.2;, score=-3629.495 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.2;, score=-2579.235 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.2;, score=-3512.265 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.2;, score=-3310.411 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.3;, score=-3293.276 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.3;, score=-3628.270 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.3;, score=-2581.468 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.3;, score=-3518.856 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.3;, score=-3309.947 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.4;, score=-3300.206 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.4;, score=-3627.080 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.4;, score=-2583.816 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.4;, score=-3525.573 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.4;, score=-3309.567 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.5;, score=-3307.225 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.5;, score=-3625.924 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.5;, score=-2586.280 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.5;, score=-3532.417 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.5;, score=-3309.986 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.6;, score=-3314.334 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.6;, score=-3624.802 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.6;, score=-2588.859 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.6;, score=-3539.388 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.6;, score=-3310.577 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.7;, score=-3321.532 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.7;, score=-3623.715 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.7;, score=-2591.553 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.7;, score=-3546.484 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.7;, score=-3311.242 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.8;, score=-3328.820 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.8;, score=-3622.662 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.8;, score=-2594.363 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.8;, score=-3553.707 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.8;, score=-3311.982 total time=   0.0s

[CV 1/5] END ..............model__alpha=5.9;, score=-3336.198 total time=   0.0s

[CV 2/5] END ..............model__alpha=5.9;, score=-3621.643 total time=   0.0s

[CV 3/5] END ..............model__alpha=5.9;, score=-2597.287 total time=   0.0s

[CV 4/5] END ..............model__alpha=5.9;, score=-3561.057 total time=   0.0s

[CV 5/5] END ..............model__alpha=5.9;, score=-3312.795 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.0;, score=-3343.666 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.0;, score=-3620.659 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.0;, score=-2600.327 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.0;, score=-3568.533 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.0;, score=-3313.682 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.1;, score=-3351.223 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.1;, score=-3619.709 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.1;, score=-2603.482 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.1;, score=-3576.135 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.1;, score=-3314.643 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.2;, score=-3358.870 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.2;, score=-3618.794 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.2;, score=-2606.752 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.2;, score=-3583.864 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.2;, score=-3315.678 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.3;, score=-3366.607 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.3;, score=-3617.913 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.3;, score=-2610.138 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.3;, score=-3591.719 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.3;, score=-3316.788 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.4;, score=-3374.433 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.4;, score=-3617.066 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.4;, score=-2614.965 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.4;, score=-3599.701 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.4;, score=-3317.971 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.5;, score=-3382.349 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.5;, score=-3616.254 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.5;, score=-2620.516 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.5;, score=-3607.809 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.5;, score=-3319.228 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.6;, score=-3390.354 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.6;, score=-3615.476 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.6;, score=-2626.181 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.6;, score=-3616.044 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.6;, score=-3320.559 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.7;, score=-3398.450 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.7;, score=-3614.733 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.7;, score=-2631.961 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.7;, score=-3624.419 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.7;, score=-3321.964 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.8;, score=-3403.532 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.8;, score=-3614.024 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.8;, score=-2637.856 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.8;, score=-3632.907 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.8;, score=-3323.443 total time=   0.0s

[CV 1/5] END ..............model__alpha=6.9;, score=-3408.526 total time=   0.0s

[CV 2/5] END ..............model__alpha=6.9;, score=-3613.349 total time=   0.0s

[CV 3/5] END ..............model__alpha=6.9;, score=-2643.865 total time=   0.0s

[CV 4/5] END ..............model__alpha=6.9;, score=-3641.520 total time=   0.0s

[CV 5/5] END ..............model__alpha=6.9;, score=-3324.995 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.0;, score=-3413.551 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.0;, score=-3612.709 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.0;, score=-2649.988 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.0;, score=-3650.261 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.0;, score=-3326.622 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.1;, score=-3418.607 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.1;, score=-3612.103 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.1;, score=-2655.955 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.1;, score=-3656.930 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.1;, score=-3328.323 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.2;, score=-3423.692 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.2;, score=-3611.531 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.2;, score=-2659.540 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.2;, score=-3662.882 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.2;, score=-3330.098 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.3;, score=-3428.807 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.3;, score=-3610.994 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.3;, score=-2663.181 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.3;, score=-3668.875 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.3;, score=-3331.947 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.4;, score=-3433.951 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.4;, score=-3610.492 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.4;, score=-2666.878 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.4;, score=-3674.927 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.4;, score=-3333.869 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.5;, score=-3439.124 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.5;, score=-3610.023 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.5;, score=-2670.632 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.5;, score=-3681.039 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.5;, score=-3335.866 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.6;, score=-3444.327 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.6;, score=-3609.589 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.6;, score=-2674.443 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.6;, score=-3687.210 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.6;, score=-3337.937 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.7;, score=-3449.559 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.7;, score=-3609.190 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.7;, score=-2678.311 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.7;, score=-3693.440 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.7;, score=-3340.081 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.8;, score=-3454.820 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.8;, score=-3608.825 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.8;, score=-2682.235 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.8;, score=-3699.729 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.8;, score=-3342.300 total time=   0.0s

[CV 1/5] END ..............model__alpha=7.9;, score=-3460.111 total time=   0.0s

[CV 2/5] END ..............model__alpha=7.9;, score=-3608.494 total time=   0.0s

[CV 3/5] END ..............model__alpha=7.9;, score=-2686.216 total time=   0.0s

[CV 4/5] END ..............model__alpha=7.9;, score=-3706.077 total time=   0.0s

[CV 5/5] END ..............model__alpha=7.9;, score=-3343.384 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.0;, score=-3465.431 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.0;, score=-3608.198 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.0;, score=-2690.253 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.0;, score=-3712.484 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.0;, score=-3344.281 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.1;, score=-3470.780 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.1;, score=-3607.936 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.1;, score=-2694.347 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.1;, score=-3718.951 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.1;, score=-3345.212 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.2;, score=-3476.191 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.2;, score=-3607.708 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.2;, score=-2698.498 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.2;, score=-3725.476 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.2;, score=-3346.177 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.3;, score=-3481.599 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.3;, score=-3607.515 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.3;, score=-2702.706 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.3;, score=-3732.061 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.3;, score=-3347.176 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.4;, score=-3487.035 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.4;, score=-3607.357 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.4;, score=-2706.970 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.4;, score=-3738.704 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.4;, score=-3348.209 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.5;, score=-3492.501 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.5;, score=-3607.232 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.5;, score=-2711.290 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.5;, score=-3745.407 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.5;, score=-3349.278 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.6;, score=-3497.996 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.6;, score=-3607.142 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.6;, score=-2715.668 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.6;, score=-3752.169 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.6;, score=-3350.381 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.7;, score=-3503.521 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.7;, score=-3607.087 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.7;, score=-2720.102 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.7;, score=-3758.990 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.7;, score=-3351.518 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.8;, score=-3509.074 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.8;, score=-3607.066 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.8;, score=-2724.592 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.8;, score=-3765.870 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.8;, score=-3352.691 total time=   0.0s

[CV 1/5] END ..............model__alpha=8.9;, score=-3514.657 total time=   0.0s

[CV 2/5] END ..............model__alpha=8.9;, score=-3607.079 total time=   0.0s

[CV 3/5] END ..............model__alpha=8.9;, score=-2729.139 total time=   0.0s

[CV 4/5] END ..............model__alpha=8.9;, score=-3772.809 total time=   0.0s

[CV 5/5] END ..............model__alpha=8.9;, score=-3353.898 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.0;, score=-3520.269 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.0;, score=-3607.126 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.0;, score=-2733.743 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.0;, score=-3779.808 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.0;, score=-3355.139 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.1;, score=-3525.910 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.1;, score=-3607.209 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.1;, score=-2738.404 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.1;, score=-3786.865 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.1;, score=-3356.416 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.2;, score=-3531.581 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.2;, score=-3607.325 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.2;, score=-2743.121 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.2;, score=-3793.982 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.2;, score=-3357.727 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.3;, score=-3537.281 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.3;, score=-3607.476 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.3;, score=-2747.895 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.3;, score=-3801.157 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.3;, score=-3359.073 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.4;, score=-3543.010 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.4;, score=-3607.661 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.4;, score=-2752.725 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.4;, score=-3808.392 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.4;, score=-3360.454 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.5;, score=-3548.768 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.5;, score=-3607.881 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.5;, score=-2757.612 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.5;, score=-3815.686 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.5;, score=-3361.869 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.6;, score=-3554.555 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.6;, score=-3608.135 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.6;, score=-2762.556 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.6;, score=-3823.039 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.6;, score=-3363.320 total time=   0.0s

[CV 1/5] END model__alpha=9.700000000000001;, score=-3560.372 total time=   0.0s

[CV 2/5] END model__alpha=9.700000000000001;, score=-3608.423 total time=   0.0s

[CV 3/5] END model__alpha=9.700000000000001;, score=-2767.557 total time=   0.0s

[CV 4/5] END model__alpha=9.700000000000001;, score=-3830.451 total time=   0.0s

[CV 5/5] END model__alpha=9.700000000000001;, score=-3364.804 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.8;, score=-3566.218 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.8;, score=-3608.746 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.8;, score=-2772.614 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.8;, score=-3837.922 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.8;, score=-3366.324 total time=   0.0s

[CV 1/5] END ..............model__alpha=9.9;, score=-3572.093 total time=   0.0s

[CV 2/5] END ..............model__alpha=9.9;, score=-3609.103 total time=   0.0s

[CV 3/5] END ..............model__alpha=9.9;, score=-2777.727 total time=   0.0s

[CV 4/5] END ..............model__alpha=9.9;, score=-3845.452 total time=   0.0s

[CV 5/5] END ..............model__alpha=9.9;, score=-3367.878 total time=   0.0s

{'model__alpha': 1.2000000000000002}

[ 0.15109046  9.00504871 26.90196877 18.04852682  5.41859386  0.

 12.27906268  0.         19.48909411  0.        ]

['age' 'sex' 'bmi' 'bp' 's1' 's3' 's5']

['s2' 's4' 's6']