标签归档：hyperparameter-optimization

github

Tpot-使用遗传编程优化机器学习管道的Python自动机器学习工具

TPOT代表T基于REE的PipelineO优化T哦哦。将TPOT视为您的数据科学助理TPOT是一种Python自动机器学习工具，可使用遗传编程优化机器学习管道

TPOT将通过智能地探索数千个可能的管道来找到最适合您数据的管道，从而自动化机器学习中最繁琐的部分

一个机器学习流水线示例

一旦TPOT完成搜索(或者您厌倦了等待)，它就会为您提供它找到的最佳管道的Python代码，这样您就可以从那里修补管道了

TPOT构建在SCRICKIT-LEARN之上，因此它生成的所有代码看起来都应该很熟悉。如果你熟悉SCRICKIT-不管怎样，还是要学

TPOT仍在积极发展中我们鼓励您定期检查此存储库是否有更新

有关TPOT的更多信息，请参阅project documentation

许可证

请参阅repository license有关TPOT的许可和使用信息

通常，我们已经授权TPOT使其尽可能广泛使用

安装

我们坚持TPOT installation instructions在文档中。TPOT需要Python的正常安装

用法

可以使用TPOTon the command line或with Python code

单击相应的链接以在文档中查找有关TPOT用法的更多信息

示例

分类

以下是光学识别手写数字数据集的最小工作示例

from tpot import TPOTClassifier
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

digits = load_digits()
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target,
                                                    train_size=0.75, test_size=0.25, random_state=42)

tpot = TPOTClassifier(generations=5, population_size=50, verbosity=2, random_state=42)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('tpot_digits_pipeline.py')

运行此代码应该会发现达到约98%测试准确率的管道，并且相应的Python代码应该导出到tpot_digits_pipeline.py文件，如下所示：

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline, make_union
from sklearn.preprocessing import PolynomialFeatures
from tpot.builtins import StackingEstimator
from tpot.export_utils import set_param_recursive

# NOTE: Make sure that the outcome column is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1)
training_features, testing_features, training_target, testing_target = \
            train_test_split(features, tpot_data['target'], random_state=42)

# Average CV score on the training set was: 0.9799428471757372
exported_pipeline = make_pipeline(
    PolynomialFeatures(degree=2, include_bias=False, interaction_only=False),
    StackingEstimator(estimator=LogisticRegression(C=0.1, dual=False, penalty="l1")),
    RandomForestClassifier(bootstrap=True, criterion="entropy", max_features=0.35000000000000003, min_samples_leaf=20, min_samples_split=19, n_estimators=100)
)
# Fix random state for all the steps in exported pipeline
set_param_recursive(exported_pipeline.steps, 'random_state', 42)

exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

回归

同样，TPOT可以针对回归问题优化管道。下面是使用Practice波士顿房价数据集的最小工作示例

from tpot import TPOTRegressor
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split

housing = load_boston()
X_train, X_test, y_train, y_test = train_test_split(housing.data, housing.target,
                                                    train_size=0.75, test_size=0.25, random_state=42)

tpot = TPOTRegressor(generations=5, population_size=50, verbosity=2, random_state=42)
tpot.fit(X_train, y_train)
print(tpot.score(X_test, y_test))
tpot.export('tpot_boston_pipeline.py')

这将导致管道达到约12.77的均方误差(MSE)，并且中的Python代码tpot_boston_pipeline.py应与以下内容类似：

import numpy as np
import pandas as pd
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from tpot.export_utils import set_param_recursive

# NOTE: Make sure that the outcome column is labeled 'target' in the data file
tpot_data = pd.read_csv('PATH/TO/DATA/FILE', sep='COLUMN_SEPARATOR', dtype=np.float64)
features = tpot_data.drop('target', axis=1)
training_features, testing_features, training_target, testing_target = \
            train_test_split(features, tpot_data['target'], random_state=42)

# Average CV score on the training set was: -10.812040755234403
exported_pipeline = make_pipeline(
    PolynomialFeatures(degree=2, include_bias=False, interaction_only=False),
    ExtraTreesRegressor(bootstrap=False, max_features=0.5, min_samples_leaf=2, min_samples_split=3, n_estimators=100)
)
# Fix random state for all the steps in exported pipeline
set_param_recursive(exported_pipeline.steps, 'random_state', 42)

exported_pipeline.fit(training_features, training_target)
results = exported_pipeline.predict(testing_features)

请查看文档以了解more examples and tutorials

对TPOT的贡献

我们欢迎您的光临check the existing issues以获取要处理的错误或增强功能。如果您有扩展TPOT的想法，请file a new issue这样我们就可以讨论一下了

在提交任何投稿之前，请审阅我们的contribution guidelines

对TPOT有问题或有疑问吗？

请check the existing open and closed issues看看您的问题是否已经得到处理。如果没有，file a new issue在此存储库上，以便我们可以检查您的问题

引用TPOT

如果您在科学出版物中使用TPOT，请考虑至少引用以下一篇论文：

陈天乐，傅维轩，杰森·H·摩尔(2020)。Scaling tree-based automated machine learning to biomedical big data with a feature set selector生物信息学36(1)：250-256

BibTeX条目：

@article{le2020scaling,
  title={Scaling tree-based automated machine learning to biomedical big data with a feature set selector},
  author={Le, Trang T and Fu, Weixuan and Moore, Jason H},
  journal={Bioinformatics},
  volume={36},
  number={1},
  pages={250--256},
  year={2020},
  publisher={Oxford University Press}
}

兰德尔·S·奥尔森、瑞安·J·厄巴诺维茨、彼得·C·安德鲁斯、妮可·A·拉文德、拉克里斯·基德和杰森·H·摩尔(2016)。Automating biomedical data science through tree-based pipeline optimization进化计算的应用，第123-137页

BibTeX条目：

@inbook{Olson2016EvoBio,
    author={Olson, Randal S. and Urbanowicz, Ryan J. and Andrews, Peter C. and Lavender, Nicole A. and Kidd, La Creis and Moore, Jason H.},
    editor={Squillero, Giovanni and Burelli, Paolo},
    chapter={Automating Biomedical Data Science Through Tree-Based Pipeline Optimization},
    title={Applications of Evolutionary Computation: 19th European Conference, EvoApplications 2016, Porto, Portugal, March 30 -- April 1, 2016, Proceedings, Part I},
    year={2016},
    publisher={Springer International Publishing},
    pages={123--137},
    isbn={978-3-319-31204-0},
    doi={10.1007/978-3-319-31204-0_9},
    url={http://dx.doi.org/10.1007/978-3-319-31204-0_9}
}

兰德尔·S·奥尔森、内森·巴特利、瑞安·J·厄巴诺维奇和杰森·H·摩尔(2016)。Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data ScienceGECCO 2016论文集，第485-492页

BibTeX条目：

@inproceedings{OlsonGECCO2016,
    author = {Olson, Randal S. and Bartley, Nathan and Urbanowicz, Ryan J. and Moore, Jason H.},
    title = {Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science},
    booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference 2016},
    series = {GECCO '16},
    year = {2016},
    isbn = {978-1-4503-4206-3},
    location = {Denver, Colorado, USA},
    pages = {485--492},
    numpages = {8},
    url = {http://doi.acm.org/10.1145/2908812.2908918},
    doi = {10.1145/2908812.2908918},
    acmid = {2908918},
    publisher = {ACM},
    address = {New York, NY, USA},
}

或者，您也可以使用以下DOI直接引用存储库：

支持TPOT

TPOT是在Computational Genetics Lab在University of Pennsylvania有了来自NIH在赠款R01 AI117694项下。我们非常感谢美国国立卫生研究院和宾夕法尼亚大学在这个项目的发展过程中给予的支持

TPOT标志是由托德·纽穆伊斯(Todd Newmuis)设计的，他慷慨地为该项目贡献了时间

github

Ray 一个开放源码框架，为构建分布式应用程序提供简单、通用的API

Ray为构建分布式应用程序提供了简单、通用的API，为构建分布式应用程序提供简单、通用的API。Ray与RLlib(一个可伸缩的强化学习库)和Tune(一个可伸缩的超参数调整库)可以打包在一起。

Ray附带以下库，用于加速机器学习工作负载：

  • Tune：可伸缩的超参数调整
  • RLlib：可扩展强化学习
  • RaySGD：分布式培训包装器
  • Ray Serve：可扩展、可编程的服务

也有很多community integrations和Ray在一起，包括Dask，MARS，Modin，Horovod，Hugging Face，Scikit-learn，以及其他。请查看full list of Ray distributed libraries here

使用以下选项安装Ray：pip install ray有关夜间车轮的信息，请参阅Installation page

快速入门

并行执行Python函数

import ray
ray.init()

@ray.remote
def f(x):
    return x * x

futures = [f.remote(i) for i in range(4)]
print(ray.get(futures))

要使用Ray的演员模型，请执行以下操作：

import ray
ray.init()

@ray.remote
class Counter(object):
    def __init__(self):
        self.n = 0

    def increment(self):
        self.n += 1

    def read(self):
        return self.n

counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
print(ray.get(futures))

Ray程序可以在一台计算机上运行，也可以无缝扩展到大型群集。要在云中执行上述Ray脚本，只需下载this configuration file，然后运行：

ray submit [CLUSTER.YAML] example.py --start

阅读有关以下内容的更多信息launching clusters

调整快速入门

Tune是一个用于任何规模的超参数调优的库

要运行此示例，您需要安装以下软件：

$ pip install "ray[tune]"

此示例运行并行格网搜索以优化示例目标函数

from ray import tune


def objective(step, alpha, beta):
    return (0.1 + alpha * step / 100)**(-1) + beta * 0.1


def training_function(config):
    # Hyperparameters
    alpha, beta = config["alpha"], config["beta"]
    for step in range(10):
        # Iterative training function - can be any arbitrary training procedure.
        intermediate_score = objective(step, alpha, beta)
        # Feed the score back back to Tune.
        tune.report(mean_loss=intermediate_score)


analysis = tune.run(
    training_function,
    config={
        "alpha": tune.grid_search([0.001, 0.01, 0.1]),
        "beta": tune.choice([1, 2, 3])
    })

print("Best config: ", analysis.get_best_config(metric="mean_loss", mode="min"))

# Get a dataframe for analyzing trial results.
df = analysis.results_df

如果安装了TensorBoard，则自动可视化所有试验结果：

tensorboard --logdir ~/ray_results

RLlib快速入门

RLlib是构建在Ray之上的用于强化学习的开源库，它为各种应用程序提供了高可伸缩性和统一的API

pip install tensorflow  # or tensorflow-gpu
pip install "ray[rllib]"
import gym
from gym.spaces import Discrete, Box
from ray import tune

class SimpleCorridor(gym.Env):
    def __init__(self, config):
        self.end_pos = config["corridor_length"]
        self.cur_pos = 0
        self.action_space = Discrete(2)
        self.observation_space = Box(0.0, self.end_pos, shape=(1, ))

    def reset(self):
        self.cur_pos = 0
        return [self.cur_pos]

    def step(self, action):
        if action == 0 and self.cur_pos > 0:
            self.cur_pos -= 1
        elif action == 1:
            self.cur_pos += 1
        done = self.cur_pos >= self.end_pos
        return [self.cur_pos], 1 if done else 0, done, {}

tune.run(
    "PPO",
    config={
        "env": SimpleCorridor,
        "num_workers": 4,
        "env_config": {"corridor_length": 5}})

Ray Serve快速入门

Ray Serve是一个构建在Ray之上的可伸缩的模型服务库。它是：

  • 框架不可知性：使用相同的工具包提供各种服务，从使用PyTorch或TensorFlow&Kera等框架构建的深度学习模型到Scikit-Learning模型或任意业务逻辑
  • Python优先：在纯Python中配置声明性服务的模型，不需要YAML或JSON配置
  • 以性能为导向：启用批处理、流水线和GPU加速以提高模型的吞吐量
  • 原生合成：允许您通过将多个模型组合在一起来驱动单个预测来创建“模型管道”
  • 水平可扩展：随着您添加更多的机器，Serve可以线性扩展。使您的ML支持的服务能够处理不断增长的流量

要运行此示例，您需要安装以下软件：

$ pip install scikit-learn
$ pip install "ray[serve]"

此示例Run服务于一个SCRICKIT-LEARN梯度增强分类器

from ray import serve
import pickle
import requests
from sklearn.datasets import load_iris
from sklearn.ensemble import GradientBoostingClassifier

# Train model
iris_dataset = load_iris()
model = GradientBoostingClassifier()
model.fit(iris_dataset["data"], iris_dataset["target"])

# Define Ray Serve model,
class BoostingModel:
    def __init__(self):
        self.model = model
        self.label_list = iris_dataset["target_names"].tolist()

    def __call__(self, flask_request):
        payload = flask_request.json["vector"]
        print("Worker: received flask request with data", payload)

        prediction = self.model.predict([payload])[0]
        human_name = self.label_list[prediction]
        return {"result": human_name}


# Deploy model
client = serve.start()
client.create_backend("iris:v1", BoostingModel)
client.create_endpoint("iris_classifier", backend="iris:v1", route="/iris")

# Query it!
sample_request_input = {"vector": [1.2, 1.0, 1.1, 0.9]}
response = requests.get("http://localhost:8000/iris", json=sample_request_input)
print(response.text)
# Result:
# {
#  "result": "versicolor"
# }

更多信息

较旧的文档：

参与其中