---
name: "mlpack_sparse_coding_SparseCoding"
section: "3"
platform: "linux"
package: "mlpack-doc"
version: "1.0.10-1"
release: "Debian 8 (Jessie)"
platforms: ["linux"]
url: "https://new.carta.tech/mlpack_sparse_coding_SparseCoding"
---

# mlpack\_sparse\_coding\_SparseCoding(3)

An implementation of Sparse Coding with Dictionary Learning that achieves sparsity via an l1-norm regularizer on the codes (LASSO) or an (l1+l2)-norm regularizer on the codes (the Elastic Net).

## NAME

mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \> -

An implementation of Sparse Coding with Dictionary Learning that achieves sparsity via an l1-norm regularizer on the codes (LASSO) or an (l1+l2)-norm regularizer on the codes (the Elastic Net).

## SYNOPSIS

### Public Member Functions

**SparseCoding** (const arma::mat &**data**, const size\_t **atoms**, const double **lambda1**, const double **lambda2**=0)\
 *Set the parameters to* **SparseCoding***.*  const arma::mat & **Codes** () const \
 *Access the sparse codes.*  arma::mat & **Codes** ()\
 *Modify the sparse codes.*  const arma::mat & **Data** () const \
 *Access the data.*  const arma::mat & **Dictionary** () const \
 *Access the dictionary.*  arma::mat & **Dictionary** ()\
 *Modify the dictionary.*  void **Encode** (const size\_t maxIterations=0, const double objTolerance=0.01, const double newtonTolerance=1e-6)\
 *Run Sparse Coding with Dictionary Learning.*  double **Objective** () const \
 *Compute the objective function.*  void **OptimizeCode** ()\
 *Sparse code each point via LARS.*  double **OptimizeDictionary** (const arma::uvec &adjacencies, const double newtonTolerance=1e-6)\
 *Learn dictionary via Newton method based on Lagrange dual.*  void **ProjectDictionary** ()\
 *Project each atom of the dictionary back onto the unit ball, if necessary.*  std::string **ToString** () const \

### Private Attributes

size\_t **atoms**\
 *Number of atoms.*  arma::mat **codes**\
 *Sparse codes (columns are points).*  const arma::mat & **data**\
 *Data matrix (columns are points).*  arma::mat **dictionary**\
 *Dictionary (columns are atoms).*  double **lambda1**\
 *l1 regularization term.*  double **lambda2**\
 *l2 regularization term.*

## Detailed Description

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\>class mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>

An implementation of Sparse Coding with Dictionary Learning that achieves sparsity via an l1-norm regularizer on the codes (LASSO) or an (l1+l2)-norm regularizer on the codes (the Elastic Net).

Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective:

.PP subject to $ \|\|D\_j\|\|\_2 \<= 1 $ for $ 1 \<= j \<= k $ where typically $ lambda\_1 \> 0 $ and $ lambda\_2 = 0 $.

This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D using a Newton method based on the Lagrange dual (see the paper below for details). The sparse coding step involves solving a large number of sparse linear regression problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO or the Elastic Net (papers below).

Here are those papers:

```
@incollection{lee2007efficient,
  title = {Efficient sparse coding algorithms},
  author = {Honglak Lee and Alexis Battle and Rajat Raina and Andrew Y. Ng},
  booktitle = {Advances in Neural Information Processing Systems 19},
  editor = {B. Sch
  publisher = {MIT Press},
  address = {Cambridge, MA},
  pages = {801--808},
  year = {2007}
}
```

```
@article{efron2004least,
  title={Least angle regression},
  author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
  journal={The Annals of statistics},
  volume={32},
  number={2},
  pages={407--499},
  year={2004},
  publisher={Institute of Mathematical Statistics}
}
```

```
@article{zou2005regularization,
  title={Regularization and variable selection via the elastic net},
  author={Zou, H. and Hastie, T.},
  journal={Journal of the Royal Statistical Society Series B},
  volume={67},
  number={2},
  pages={301--320},
  year={2005},
  publisher={Royal Statistical Society}
}
```

Before the method is run, the dictionary is initialized using the DictionaryInitializationPolicy class. Possible choices include the **RandomInitializer**, which provides an entirely random dictionary, the **DataDependentRandomInitializer**, which provides a random dictionary based loosely on characteristics of the dataset, and the **NothingInitializer**, which does not initialize the dictionary -- instead, the user should set the dictionary using the **Dictionary()** mutator method.

**Template Parameters:**

*DictionaryInitializationPolicy* The class to use to initialize the dictionary; must have 'void Initialize(const arma::mat& data, arma::mat& dictionary)' function.

Definition at line 119 of file sparse\_coding.hpp.

## Constructor & Destructor Documentation

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::SparseCoding (const arma::mat &data, const size\_tatoms, const doublelambda1, const doublelambda2 = `0`)

Set the parameters to **SparseCoding**. lambda2 defaults to 0.

**Parameters:**

*data* Data matrix \
 *atoms* Number of atoms in dictionary \
 *lambda1* Regularization parameter for l1-norm penalty \
 *lambda2* Regularization parameter for l2-norm penalty

## Member Function Documentation

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> const arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Codes () const` [inline]`

Access the sparse codes.

Definition at line 187 of file sparse\_coding.hpp.

References mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::codes.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Codes ()` [inline]`

Modify the sparse codes.

Definition at line 189 of file sparse\_coding.hpp.

References mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::codes.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> const arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Data () const` [inline]`

Access the data.

Definition at line 179 of file sparse\_coding.hpp.

References mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::data.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> const arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Dictionary () const` [inline]`

Access the dictionary.

Definition at line 182 of file sparse\_coding.hpp.

References mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::dictionary.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Dictionary ()` [inline]`

Modify the dictionary.

Definition at line 184 of file sparse\_coding.hpp.

References mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::dictionary.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> void mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Encode (const size\_tmaxIterations = `0`, const doubleobjTolerance = `0.01`, const doublenewtonTolerance = `1e-6`)

Run Sparse Coding with Dictionary Learning.

**Parameters:**

*maxIterations* Maximum number of iterations to run algorithm. If 0, the algorithm will run until convergence (or forever). \
 *objTolerance* Tolerance for objective function. When an iteration of the algorithm produces an improvement smaller than this, the algorithm will terminate. \
 *newtonTolerance* Tolerance for the Newton's method dictionary optimization step.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> double mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Objective () const

Compute the objective function.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> void mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::OptimizeCode ()

Sparse code each point via LARS.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> double mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::OptimizeDictionary (const arma::uvec &adjacencies, const doublenewtonTolerance = `1e-6`)

Learn dictionary via Newton method based on Lagrange dual.

**Parameters:**

*adjacencies* Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms). \
 *newtonTolerance* Tolerance of the Newton's method optimizer.

**Returns:**

the norm of the gradient of the Lagrange dual with respect to the dual variables

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> void mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::ProjectDictionary ()

Project each atom of the dictionary back onto the unit ball, if necessary.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> std::string mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::ToString () const

## Member Data Documentation

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> size\_t mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::atoms` [private]`

Number of atoms.

Definition at line 196 of file sparse\_coding.hpp.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> arma::mat mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::codes` [private]`

Sparse codes (columns are points).

Definition at line 205 of file sparse\_coding.hpp.

Referenced by mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Codes().

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> const arma::mat& mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::data` [private]`

Data matrix (columns are points).

Definition at line 199 of file sparse\_coding.hpp.

Referenced by mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Data().

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> arma::mat mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::dictionary` [private]`

Dictionary (columns are atoms).

Definition at line 202 of file sparse\_coding.hpp.

Referenced by mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::Dictionary().

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> double mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::lambda1` [private]`

l1 regularization term.

Definition at line 208 of file sparse\_coding.hpp.

### template\<typename DictionaryInitializer = DataDependentRandomInitializer\> double mlpack::sparse\_coding::SparseCoding\< DictionaryInitializer \>::lambda2` [private]`

l2 regularization term.

Definition at line 211 of file sparse\_coding.hpp.

## Author

Generated automatically by Doxygen for MLPACK from the source code.