We are happy to finally announce the first release of mlrMBO on cran after a quite long development time. For the theoretical background and a nearly complete overview of mlrMBOs capabilities you can check our paper onmlrMBO that we presubmitted to arxiv.

The key features of mlrMBO are:

  • Global optimization of expensive Black-Box functions.
  • Mulit-Criteria Optimization.
  • Parallelization through multi-point proposals.
  • Support for optimization over categorical variables using random forests as a surrogate.

For examples covering different scenarios we have Vignettes that are also available as an online documentation. For mlr users mlrMBO is especially interesting for hyperparameter optimization.

mlrMBO for mlr hyperparameter tuning was already used in an earlier blog post. Nonetheless we want to provide a small toy example to demonstrate the work flow of mlrMBO in this post.

Example

First, we define an objective function that we are going to minimize:

set.seed(1)
library(mlrMBO)
fun = makeSingleObjectiveFunction(
name = "SineMixture",
fn = function(x) sin(x[1])*cos(x[2])/2 + 0.04 * sum(x^2),
par.set = makeNumericParamSet(id = "x", len = 2, lower = -5, upper = 5)
)

To define the objective function we use makeSingleObjectiveFunction from the neat package smoof, which gives us the benefit amongst others to be able to directly visualize the function. If you happen to be in need of functions to optimize and benchmark your optimization algorithm I recommend you to have a look at the package!

library(plot3D)
plot3D(fun, contour = TRUE, lightning = TRUE)

Let’s start with the configuration of the optimization:

# In this simple example we construct the control object with the defaults:
ctrl = makeMBOControl()
# For this numeric optimization we are going to use the Expected Improvement as infill criterion:
ctrl = setMBOControlInfill(ctrl, crit = crit.ei)
# We will allow for exactly 25 evaluations of the objective function:
ctrl = setMBOControlTermination(ctrl, max.evals = 25L)

The optimization has to so start with an initial design. mlrMBO can automatically create one but here we are going to use a randomly sampled LHS design of our own:

library(ggplot2)
des = generateDesign(n = 8L, par.set = getParamSet(fun), fun = lhs::randomLHS)
autoplot(fun, render.levels = TRUE) + geom_point(data = des)
## Warning: Ignoring unknown aesthetics: fill

The points demonstrate how the initial design already covers the search space but is missing the area of the global minimum. Before we can start the Bayesian optimization we have to set the surrogate learner to Kriging. Therefore we use an mlr regression learner. In fact, with mlrMBO you can use any regression learner integrated inmlr as a surrogate allowing for many special optimization applications.

sur.lrn = makeLearner("regr.km", predict.type = "se", config = list(show.learner.output = FALSE))

Note: mlrMBO can automatically determine a good surrogate learner based on the search space defined for the objective function. For a purely numeric domain it would have chosen Kriging as well with some slight modifications to make it a bit more stable against numerical problems that can occur during optimization.

Finally, we can start the optimization run:

res = mbo(fun = fun, design = des, learner = sur.lrn, control = ctrl, show.info = TRUE)
## Computing y column(s) for design. Not provided.
## [mbo] 0: x=-0.0101,-4.52 : y = 0.817 : 0.0 secs : initdesign
## [mbo] 0: x=-4.52,-2.48 : y = 0.677 : 0.0 secs : initdesign
## [mbo] 0: x=-2.78,-3.27 : y = 0.913 : 0.0 secs : initdesign
## [mbo] 0: x=4.92,1.09 : y = 0.787 : 0.0 secs : initdesign
## [mbo] 0: x=2.77,2.93 : y = 0.469 : 0.0 secs : initdesign
## [mbo] 0: x=0.815,-0.647 : y = 0.333 : 0.0 secs : initdesign
## [mbo] 0: x=-2.34,4.5 : y = 1.11 : 0.0 secs : initdesign
## [mbo] 0: x=1.58,1.87 : y = 0.0939 : 0.0 secs : initdesign
## [mbo] 1: x=1.48,5 : y = 1.23 : 0.0 secs : infill_ei
## [mbo] 2: x=-3.77,2.2 : y = 0.589 : 0.0 secs : infill_ei
## [mbo] 3: x=0.429,1.49 : y = 0.113 : 0.0 secs : infill_ei
## [mbo] 4: x=0.776,1.98 : y = 0.0413 : 0.0 secs : infill_ei
## [mbo] 5: x=0.126,1.93 : y = 0.127 : 0.0 secs : infill_ei
## [mbo] 6: x=1.01,2.15 : y = -0.00662 : 0.0 secs : infill_ei
## [mbo] 7: x=0.963,2.36 : y = -0.0317 : 0.0 secs : infill_ei
## [mbo] 8: x=0.922,0.539 : y = 0.388 : 0.0 secs : infill_ei
## [mbo] 9: x=-2.7,-0.524 : y = 0.119 : 0.0 secs : infill_ei
## [mbo] 10: x=-5,-0.253 : y = 1.47 : 0.0 secs : infill_ei
## [mbo] 11: x=-1.46,-0.613 : y = -0.306 : 0.0 secs : infill_ei
## [mbo] 12: x=-1.39,-1.1 : y = -0.098 : 0.0 secs : infill_ei
## [mbo] 13: x=-1.29,-0.228 : y = -0.4 : 0.0 secs : infill_ei
## [mbo] 14: x=-1.57,0.256 : y = -0.382 : 0.0 secs : infill_ei
## [mbo] 15: x=-1.43,-0.0423 : y = -0.413 : 0.0 secs : infill_ei
## [mbo] 16: x=-1.27,0.0745 : y = -0.412 : 0.0 secs : infill_ei
## [mbo] 17: x=5,-3.84 : y = 1.96 : 0.0 secs : infill_ei
res$x
## $x
## [1] -1.42836803 -0.04234841
res$y
## [1] -0.4128122

We can see that we have found the global optimum of y=−0.414964y=−0.414964 at x=(−1.35265,0)x=(−1.35265,0) quite sufficiently. Let’s have a look at the points mlrMBO evaluated. Therefore we can use the OptPath which stores all information about all evaluations during the optimization run:

opdf = as.data.frame(res$opt.path)
autoplot(fun, render.levels = TRUE, render.contours = FALSE) + geom_text(data = opdf, aes(label = dob))

It is interesting to see, that for this run the algorithm first went to the local minimum on the top right in the 6th and 7th iteration but later, thanks to the explorative character of the Expected Improvement, found the real global minimum.

Comparison

That is all good, but how do other optimization strategies perform?

Grid Search

Grid search is seldom a good idea. But especially for hyperparameter tuning it is still used. Probably because it kind of gives you the feeling that you know what is going on and have not left out any important area of the search space. In reality the grid is usually so sparse that it leaves important areas untouched as you can see in this example:

grid.des = generateGridDesign(par.set = getParamSet(fun), resolution = 5)
grid.des$y = apply(grid.des, 1, fun)
grid.des[which.min(grid.des$y),]
##      x1 x2           y
## 12 -2.5 0 -0.04923607
autoplot(fun, render.levels = TRUE, render.contours = FALSE) + geom_point(data = grid.des)

It is no surprise, that the grid search could not cover the search space well enough and we only reach a bad result.

What about a simple random search?

random.des = generateRandomDesign(par.set = getParamSet(fun), n = 25L)
random.des$y = apply(random.des, 1, fun)
random.des[which.min(random.des$y),]
##           x1         x2          y
## 20 -1.784371 -0.9802194 -0.1063019
autoplot(fun, render.levels = TRUE, render.contours = FALSE) + geom_point(data = random.des)

With the random search you could always be lucky but in average the optimum is not reached if smarter optimization strategies work well.

A fair comarison

… for stochastic optimization algorithms can only be achieved by repeating the runs. mlrMBO is stochastic as the initial design is generated randomly and the fit of the Kriging surrogate is also not deterministic. Furthermore we should include other optimization strategies like a genetic algorithm and direct competitors like rBayesOpt. An extensive benchmark is available in our mlrMBO paper. The examples here are just meant to demonstrate the package.

Engage

If you want to contribute to mlrMBO we ware always open to suggestions and pull requests on github. You are also invited to fork the repository and build and extend your own optimizer based on our toolbox.

转自:https://mlr-org.github.io/First_release_of_mlrMBO_the_toolbox_for_Bayesian_Black_Box_Optimization/

First release of mlrMBO - the toolbox for (Bayesian) Black-Box Optimization的更多相关文章

  1. matlab toolboxes 大全

    MATLAB Toolboxes top (Top) Audio - Astronomy - BiomedicalInformatics - Chemometrics  - Chaos - Chemi ...

  2. plain framework 商业版 开发总结2 项目管理器

    任何事情都有三个阶段,分析.制作.质检的过程.在程序中就分为设计.编码.调试(测试)三个阶段,其中设计最为重要,设计的不好会导致编码和调试重复,甚至最后又回到了设计的过程.为了不会重复返工,所以设计的 ...

  3. 【转】Application.mk 文件语法规范

    原文网址:http://blog.sina.com.cn/s/blog_4c451e0e0100s6q4.html Application.mk file syntax specification A ...

  4. Android NDK开发指南---Application.mk文件和android.mk文件

    https://android.googlesource.com/platform/development/+/donut-release/ndk/docs/OVERVIEW.TXT https:// ...

  5. mysql 源码编绎修改 FLAGS,调试MYSQL

    http://dev.mysql.com/doc/refman/5.7/en/source-configuration-options.html#option_cmake_cmake_c_flags ...

  6. Windows 10 安装 Docker for Windows

    Docker for Windows是Docker社区版(CE)应用程序. Docker for Windows安装包包括在Windows系统上运行Docker所需的一切. 本主题介绍了预安装注意事项 ...

  7. Docker for Windows 使用 VMware WorkStation

    一.前言 Docker for Windows 不同于 Docker Toolbox.Docker for Windows 对系统的要求至少为Windows 10专业版,因为它需要Hyper-V的支持 ...

  8. [zz] MATLAB工具箱介绍

    http://blog.sina.com.cn/s/blog_57235cc701012kfb.html Toolbox工具箱 序号 工具箱 备注   数学.统计与优化   1 Symbolic Ma ...

  9. Sphinx 2.2.11-release reference manual

    1. Introduction 1.1. About 1.2. Sphinx features 1.3. Where to get Sphinx 1.4. License 1.5. Credits 1 ...

随机推荐

  1. C#委托冒泡

    委托的实现,就是编译器自行定义了一个类:有三个重要参数1.制定操作对象,2.指定委托方法3.委托链 看如下一个列子: class DelegatePratice { public static voi ...

  2. java基础--动态代理实现与原理详细分析

    关于Java中的动态代理,我们首先需要了解的是一种常用的设计模式--代理模式,而对于代理,根据创建代理类的时间点,又可以分为静态代理和动态代理. 一.代理模式                     ...

  3. java 基础知识六 字符串2

    java  基础知识六  字符串2 1.String类是final类,也即意味着String类不能被继承,并且它的成员方法都默认为final方法 . String 是不可变对象,也就是一旦创建,那么整 ...

  4. oracle定时执行一个存储过程

    首先需要新建存储过程 一 存储过程: create or replace procedure Insertdata is begin INSERT INTO tab_dayta select * fr ...

  5. Nginx反向代理以及负载均衡配置

    项目地址:http://git.oschina.net/miki-long/nginx 前提:最近在研究nginx的用法,在windows上小试了一下,由于windows下不支持nginx缓存配置,所 ...

  6. cookie和session的区别异同

    1.用于保存页面信息:如自动登录,记住用户名 2.对于同一个网站只有一套cookie,它是以域名为单位的,一个域名就是一套,数量大小有限4k-10k,同时会具有过期时间 3.JS中通过document ...

  7. 软件工程工具学习(1)---Visio

    要给15级软件工程上机了.开个系列记录软件工程开发过程中所会用到的一些工具的学习. 第一篇---软件分析与设计工具 Microsoft Visio Visio 介绍 1.Visio是一款矢量图形与图标 ...

  8. ASP.NET MVC5请求管道和生命周期

    请求处理管道 请求管道是一些用于处理HTTP请求的模块组合,在ASP.NET中,请求管道有两个核心组件:IHttpModule和IHttpHandler.所有的HTTP请求都会进入IHttpHandl ...

  9. [.NET] 《C# 高效编程》(一) - C# 语言习惯

    C# 语言习惯 目录 一.使用属性而不是可访问的数据成员 二.使用运行时常量(readonly)而不是编译时常量(const) 三.推荐使用 is 或 as 操作符而不是强制类型转换 四.使用 Con ...

  10. 用Entity Framework往数据库插数据时,出现异常,怎么查看异常的详细信息呢?

    做项目时,在用Entity Framework往数据库插数据时,程序报异常,但是通过报的异常死活没法查看异常的详细信息.这让人很是烦恼.本着自己动手丰衣足食的原则,通过查看资料终于找到了显示异常详细信 ...