Preliminary Survey on LASSO Convex optimization

LASSO is short for least absolute shrinkage and selection operator, which can be seen in wiki. The followings are some optimization python package, showing whether support LASSO and gpu.

Sklearn

• support LASSO in linear regression
• support Poisson regressor
• NOT support LASSO in Poisson regressor, some related issues
  • Add Lasso regularization to PoissonRegressor
  • Add L1 penalty and coordinate descent solver to TweedieRegression
    • benchmarks
• NOT support GPU

Statsmodel

• support LASSO in Poisson regressor
• support sqrt link function?
• NOT support GPU

H2O

• support LASSO in Poisson regressor
  • support lambda search

H2O4GPU

• using sklearn api
  • import h2o4gpu as sklearn
  • install trouble, only support python3.6?

CVXPY

• support LASSO
• construct Poisson regression following DCP rules

  def loss_fn(X, Y, beta):
      lbd = X @ beta
      return cp.sum(cp.exp(lbd) - cp.multiply(Y, lbd))
  
  def regularizer(beta):
      return cp.norm1(beta)
  
  def objective_fn(X, Y, beta, lambd):
      return loss_fn(X, Y, beta) + lambd * regularizer(beta)
  
  def mse(X, Y, beta):
      return (1.0 / X.shape[0]) * loss_fn(X, Y, beta).value

• on GPU depend on the solver

glmnet

• support LASSO in Poisson Models
• gpu on R

Not support LASSO?

• PYGMO
• NLOPT
  • robust optimization hint