SkillAgentSearch skills...

LauraeCE

Laurae's Cross-Entropy Optimization for R

GenerateConvertImprove

Install / Use

/learn @Laurae2/LauraeCE
About this skill

Quality Score

0/100

Supported Platforms

Universal

README

LauraeCE: Laurae's R package for Parallel Cross-Entropy Optimization

This R pacakge is meant to be used for Cross-Entropy optimization, which is a global optimization method for both continuous and discrete parameters. It tends to outperform Differential Evolution in my local tests.

It also uses LauraeParallel load balancing for parallelization, which makes it suitable for long and dynamic optimization tasks.

Cross-Entropy optimization earned 3rd place (thanks to Laurae) in 2017 at the Ecole Nationale Supérieure metachallenge, earning several 1st and 2nd places in several challenges of the metachallenge (did you know Laurae did not receive any gift for such feat because the organizers ran out of gifts? now you know!).

Installation:

devtools::install_github("Laurae2/LauraeParallel")
devtools::install_github("Laurae2/LauraeCE")

Original source: https://cran.r-project.org/web/packages/CEoptim/index.html

TO-DO:

  • [x] add parallelism
  • [x] add load balancing
  • [x] ~~add hot loading (use previous optimization)~~ (this one was stupid because one could just use the previous mean/sd/probs)
  • [ ] add interrupt on the fly while saving data (tcltk?)
  • [x] add maximum computation time before cancelling (while returning cleanly)

Example

This is how it currently looks and you will notice it is absurdly SLOW on very small tasks:

> suppressMessages(library(LauraeCE))
> suppressMessages(library(parallel))
> 
> 
> # Continuous Testing
> 
> fun <- function(x){
+   return(3 * (1 - x[1]) ^ 2 * exp(-x[1] ^ 2 - (x[2] + 1) ^ 2) - 10 * (x[1] / 5 - x[1] ^ 3 - x[2] ^ 5) * exp(-x[1] ^ 2 - x[2] ^ 2) - 1 / 3 * exp(-(x[1] + 1) ^ 2 - x[2] ^ 2))
+ }
> 
> mu0 <- c(-3, -3)
> sigma0 <- c(10, 10)
> 
> system.time({
+   set.seed(11111)
+   res1 <- CEoptim::CEoptim(fun,
+                            continuous = list(mean = mu0,
+                                              sd = sigma0),
+                            maximize = TRUE)
+ })
   user  system elapsed 
   0.03    0.00    0.03 
> 
> system.time({
+   set.seed(11111)
+   res2 <- CEoptim(fun,
+                   continuous = list(mean = mu0,
+                                     sd = sigma0),
+                   maximize = TRUE)
+ })
   user  system elapsed 
   0.42    0.00    0.44 
> 
> cl <- makeCluster(2)
> system.time({
+   set.seed(11111)
+   res3 <- CEoptim(fun,
+                   continuous = list(mean = mu0,
+                                     sd = sigma0),
+                   maximize = TRUE,
+                   parallelize = TRUE,
+                   cl = cl)
+ })
   user  system elapsed 
   0.09    0.02    0.14 
> stopCluster(cl)
> closeAllConnections()
> 
> all.equal(res1$optimum, res2$optimum)
[1] TRUE
> all.equal(res1$optimum, res3$optimum)
[1] TRUE
> 
> 
> # Discrete Testing
> 
> data(lesmis)
> fmaxcut <- function(x,costs) {
+   v1 <- which(x == 1)
+   v2 <- which(x == 0)
+   return(sum(costs[v1, v2]))
+ }
> 
> p0 <- list()
> for (i in 1:77) {
+   p0 <- c(p0, list(rep(0.5, 2)))
+ }
> p0[[1]] <- c(0, 1)
> 
> system.time({
+   set.seed(11111)
+   res1 <- CEoptim::CEoptim(fmaxcut,
+                            f.arg = list(costs = lesmis),
+                            maximize = TRUE,
+                            verbose = TRUE,
+                            discrete = list(probs = p0),
+                            N = 3000L)
+ })
Number of continuous variables: 0  
Number of discrete variables: 77 
conMat= 
NULL
conVec= 
NULL
smoothMean: 1 smoothSd: 1 smoothProb: 1 
N: 3000 rho: 0.1 iterThr: 10000 sdThr: 0.001 probThr 0.001 
iter: 0  opt: 494 maxProbs: 0.5
iter: 1  opt: 501 maxProbs: 0.5
iter: 2  opt: 501 maxProbs: 0.5
iter: 3  opt: 501 maxProbs: 0.4966667
iter: 4  opt: 506 maxProbs: 0.5
iter: 5  opt: 510 maxProbs: 0.5
iter: 6  opt: 514 maxProbs: 0.5
iter: 7  opt: 515 maxProbs: 0.5
iter: 8  opt: 519 maxProbs: 0.4966667
iter: 9  opt: 523 maxProbs: 0.4933333
iter: 10  opt: 526 maxProbs: 0.4966667
iter: 11  opt: 528 maxProbs: 0.4933333
iter: 12  opt: 528 maxProbs: 0.4866667
iter: 13  opt: 530 maxProbs: 0.4966667
iter: 14  opt: 532 maxProbs: 0.49
iter: 15  opt: 532 maxProbs: 0.4733333
iter: 16  opt: 532 maxProbs: 0.4533333
iter: 17  opt: 533 maxProbs: 0.49
iter: 18  opt: 533 maxProbs: 0.4533333
iter: 19  opt: 533 maxProbs: 0.5
iter: 20  opt: 533 maxProbs: 0.4366667
iter: 21  opt: 533 maxProbs: 0.3766667
iter: 22  opt: 533 maxProbs: 0.3633333
   user  system elapsed 
   2.40    0.00    2.41 
> 
> system.time({
+   set.seed(11111)
+   res2 <- CEoptim(fmaxcut,
+                   f.arg = list(costs = lesmis),
+                   maximize = TRUE,
+                   verbose = TRUE,
+                   discrete = list(probs = p0),
+                   N = 3000L)
+ })
Number of continuous variables: 0  
Number of discrete variables: 77 
conMat= 
NULL
conVec= 
NULL
smoothMean: 1 smoothSd: 1 smoothProb: 1 
N: 3000 rho: 0.1 iterThr: 10000 sdThr: 0.001 probThr 0.001 
Sat Dec 23 2017 02:36:45 PM - iter: 00001 (00s082ms, 36287.07 samples/s) - opt: 494 - maxProbs: 0.5
Sat Dec 23 2017 02:36:45 PM - iter: 00002 (00s066ms, 44985.61 samples/s) - opt: 501 - maxProbs: 0.5
Sat Dec 23 2017 02:36:45 PM - iter: 00003 (00s073ms, 40941.17 samples/s) - opt: 501 - maxProbs: 0.5
Sat Dec 23 2017 02:36:46 PM - iter: 00004 (00s068ms, 43923.99 samples/s) - opt: 501 - maxProbs: 0.4966667
Sat Dec 23 2017 02:36:46 PM - iter: 00005 (00s076ms, 39105.11 samples/s) - opt: 506 - maxProbs: 0.5
Sat Dec 23 2017 02:36:46 PM - iter: 00006 (00s143ms, 20878.71 samples/s) - opt: 510 - maxProbs: 0.5
Sat Dec 23 2017 02:36:46 PM - iter: 00007 (00s065ms, 45805.62 samples/s) - opt: 514 - maxProbs: 0.5
Sat Dec 23 2017 02:36:46 PM - iter: 00008 (00s070ms, 42742.95 samples/s) - opt: 515 - maxProbs: 0.5
Sat Dec 23 2017 02:36:46 PM - iter: 00009 (00s076ms, 39368.59 samples/s) - opt: 519 - maxProbs: 0.4966667
Sat Dec 23 2017 02:36:46 PM - iter: 00010 (00s081ms, 36701.63 samples/s) - opt: 523 - maxProbs: 0.4933333
Sat Dec 23 2017 02:36:46 PM - iter: 00011 (00s068ms, 43615.30 samples/s) - opt: 526 - maxProbs: 0.4966667
Sat Dec 23 2017 02:36:46 PM - iter: 00012 (00s086ms, 34672.88 samples/s) - opt: 528 - maxProbs: 0.4933333
Sat Dec 23 2017 02:36:47 PM - iter: 00013 (00s170ms, 17563.39 samples/s) - opt: 528 - maxProbs: 0.4866667
Sat Dec 23 2017 02:36:47 PM - iter: 00014 (00s060ms, 49344.64 samples/s) - opt: 530 - maxProbs: 0.4966667
Sat Dec 23 2017 02:36:47 PM - iter: 00015 (00s065ms, 45481.46 samples/s) - opt: 532 - maxProbs: 0.49
Sat Dec 23 2017 02:36:47 PM - iter: 00016 (00s157ms, 19045.67 samples/s) - opt: 532 - maxProbs: 0.4733333
Sat Dec 23 2017 02:36:47 PM - iter: 00017 (00s079ms, 37972.81 samples/s) - opt: 532 - maxProbs: 0.4533333
Sat Dec 23 2017 02:36:47 PM - iter: 00018 (00s069ms, 42964.58 samples/s) - opt: 533 - maxProbs: 0.49
Sat Dec 23 2017 02:36:47 PM - iter: 00019 (00s072ms, 41309.76 samples/s) - opt: 533 - maxProbs: 0.4533333
Sat Dec 23 2017 02:36:47 PM - iter: 00020 (00s066ms, 45445.80 samples/s) - opt: 533 - maxProbs: 0.5
Sat Dec 23 2017 02:36:47 PM - iter: 00021 (00s081ms, 36698.12 samples/s) - opt: 533 - maxProbs: 0.4366667
Sat Dec 23 2017 02:36:48 PM - iter: 00022 (00s065ms, 45617.55 samples/s) - opt: 533 - maxProbs: 0.3766667
Sat Dec 23 2017 02:36:48 PM - iter: 00023 (00s072ms, 41595.58 samples/s) - opt: 533 - maxProbs: 0.3633333
Sat Dec 23 2017 02:36:48 PM - iter: 00024 (00s128ms, 23352.47 samples/s) - opt: 533 - maxProbs: 0.3466667
   user  system elapsed 
   2.77    0.01    2.77 
> 
> cl <- makeCluster(2)
> system.time({
+   set.seed(11111)
+   res3 <- CEoptim(fmaxcut,
+                   f.arg = list(costs = lesmis),
+                   maximize = TRUE,
+                   verbose = TRUE,
+                   discrete = list(probs = p0),
+                   N = 3000L,
+                   parallelize = TRUE,
+                   cl = cl)
+ })
Number of continuous variables: 0  
Number of discrete variables: 77 
conMat= 
NULL
conVec= 
NULL
smoothMean: 1 smoothSd: 1 smoothProb: 1 
N: 3000 rho: 0.1 iterThr: 10000 sdThr: 0.001 probThr 0.001 
Sat Dec 23 2017 02:36:50 PM - iter: 00001 (02s014ms, 1489.11 samples/s, 744.55 s/s/thread) - opt: 494 - maxProbs: 0.5
Sat Dec 23 2017 02:36:51 PM - iter: 00002 (00s682ms, 4395.58 samples/s, 2197.79 s/s/thread) - opt: 501 - maxProbs: 0.5
Sat Dec 23 2017 02:36:52 PM - iter: 00003 (01s691ms, 1773.06 samples/s, 886.53 s/s/thread) - opt: 501 - maxProbs: 0.5
Sat Dec 23 2017 02:36:54 PM - iter: 00004 (00s440ms, 6811.68 samples/s, 3405.84 s/s/thread) - opt: 501 - maxProbs: 0.4966667
Sat Dec 23 2017 02:36:55 PM - iter: 00005 (01s532ms, 1957.99 samples/s, 978.99 s/s/thread) - opt: 506 - maxProbs: 0.5
Sat Dec 23 2017 02:36:56 PM - iter: 00006 (01s619ms, 1852.11 samples/s, 926.05 s/s/thread) - opt: 510 - maxProbs: 0.5
Sat Dec 23 2017 02:36:57 PM - iter: 00007 (00s571ms, 5247.09 samples/s, 2623.54 s/s/thread) - opt: 514 - maxProbs: 0.5
Sat Dec 23 2017 02:36:59 PM - iter: 00008 (01s551ms, 1933.35 samples/s, 966.68 s/s/thread) - opt: 515 - maxProbs: 0.5
Sat Dec 23 2017 02:37:00 PM - iter: 00009 (01s431ms, 2095.47 samples/s, 1047.74 s/s/thread) - opt: 519 - maxProbs: 0.4966667
Sat Dec 23 2017 02:37:01 PM - iter: 00010 (00s544ms, 5514.17 samples/s, 2757.08 s/s/thread) - opt: 523 - maxProbs: 0.4933333
Sat Dec 23 2017 02:37:02 PM - iter: 00011 (01s449ms, 2070.38 samples/s, 1035.19 s/s/thread) - opt: 526 - maxProbs: 0.4966667
Sat Dec 23 2017 02:37:03 PM - iter: 00012 (01s438ms, 2085.91 samples/s, 1042.95 s/s/thread) - opt: 528 - maxProbs: 0.4933333
Sat Dec 23 2017 02:37:05 PM - iter: 00013 (00s497ms, 6034.04 samples/s, 3017.02 s/s/thread) - opt: 528 - maxProbs: 0.4866667
Sat Dec 23 2017 02:37:06 PM - iter: 00014 (01s495ms, 2006.57 samples/s, 1003.29 s/s/thread) - opt: 530 - maxProbs: 0.4966667
Sat Dec 23 2017 02:37:07 PM - iter: 00015 (01s589ms, 1887.85 samples/s, 943.93 s/s/thread) - opt: 532 - maxProbs: 0.49
Sat Dec 23 2017 02:37:08 PM - iter: 00016 (01s465ms, 2047.09 samples/s, 1023.55 s/s/

Laurae2

View profile

View on GitHub
GitHub Stars5
CategoryEducation
Updated4y ago
Forks0
Laurae2/LauraeCE

Languages

R

Security Score

60/100

Audited on Jun 29, 2021

No findings