44 lines
1.5 KiB
Markdown
44 lines
1.5 KiB
Markdown
|
---
|
||
|
|
title: Deeplearning4j Iterators
|
||
|
|
short_title: Iterators
|
||
|
|
description: Data iteration tools for loading into neural networks.
|
||
|
|
category: Models
|
||
|
|
weight: 5
|
||
|
|
---
|
||
|
|
|
||
|
|
## What is an iterator?
|
||
|
|
|
||
|
|
A dataset iterator allows for easy loading of data into neural networks and help organize batching, conversion, and masking. The iterators included in Eclipse Deeplearning4j help with either user-provided data, or automatic loading of common benchmarking datasets such as MNIST and IRIS.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
For most use cases, initializing an iterator and passing a reference to a `MultiLayerNetwork` or `ComputationGraph` `fit()` method is all you need to begin a task for training:
|
||
|
|
|
||
|
|
```java
|
||
|
|
MultiLayerNetwork model = new MultiLayerNetwork(conf);
|
||
|
|
model.init();
|
||
|
|
|
||
|
|
// pass an MNIST data iterator that automatically fetches data
|
||
|
|
DataSetIterator mnistTrain = new MnistDataSetIterator(batchSize, true, rngSeed);
|
||
|
|
net.fit(mnistTrain);
|
||
|
|
```
|
||
|
|
|
||
|
|
Many other methods also accept iterators for tasks such as evaluation:
|
||
|
|
|
||
|
|
```java
|
||
|
|
// passing directly to the neural network
|
||
|
|
DataSetIterator mnistTest = new MnistDataSetIterator(batchSize, false, rngSeed);
|
||
|
|
net.eval(mnistTest);
|
||
|
|
|
||
|
|
// using an evaluation class
|
||
|
|
Evaluation eval = new Evaluation(10); //create an evaluation object with 10 possible classes
|
||
|
|
while(mnistTest.hasNext()){
|
||
|
|
DataSet next = mnistTest.next();
|
||
|
|
INDArray output = model.output(next.getFeatureMatrix()); //get the networks prediction
|
||
|
|
eval.eval(next.getLabels(), output); //check the prediction against the true class
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
## Available iterators
|
||
|
|
|
||
|
|
{{autogenerated}}
|