cavis/deeplearning4j
Alex Black 3f0b4a2d4c
SameDiff execution, TF and memory management overhaul (#10)
* SameDiff execution memory management improvements, round 1

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Round 2

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Round 3

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Clear node outputs closed array references; Slight change to OpValidation internals to not rely on cached op outputs

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Next step

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More polish

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add WeakIdentityHashmap

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Session fixes for control ops and next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* First steps for training session + in-line updating

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix losses and history during training

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* BiasAdd and other fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Don't use SDVariable.getArr() in TFGraphTestAllHelper (import tests)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* First steps for new dependency tracking approach

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Start integrating dependency tracking for memory management

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Non-control op dependency tracking works/passes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Switch/merge

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup and next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix issue dependency tracking for initial variables/constants

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add check for aliases when determining if safe to close array

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* First pass on new TF graph import class

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Import fixes, op fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup and fixes for new TF import mapper

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Partial implementation of new dependency tracker

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* AbstractDependencyTracker for shared code

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Overhaul SameDiff graph execution (dependency tracking)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More fixes, cleanup, next steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Ad no-op memory manager, cleanup, fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix switch dependency tracking

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* INDArray.toString: no exception on closed arrays, just note closed

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix enter and exit dependency tracking

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* TensorArray memory management fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add unique ID for INDArray instances

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix memory management for NextIteration outputs in multi-iteration loops

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Remove (now unnecessary) special case handling for nested enters

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Handle control dependencies during execution; javadoc for memory managers

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup, polish, code comments, javadoc

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup and more javadoc

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add memory validation for all TF import tests - ensure all arrays (except outputs) are released

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Clean up arrays waiting on unexecuted ops at the end of execution

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fixes for enter op memory managent in the context of multiple non-nested loops/frames

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix order of operation issues for dependency tracker

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Always clear op fields after execution to avoid leaks or unintended array reuse

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Re-implement dtype conversion

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix for control dependencies execution (dependency tracking)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix TF import overrides and filtering

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix for constant enter array dependency tracking

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* DL4J Fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More DL4J fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup and polish

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More polish and javadoc

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More logging level tweaks, small DL4J fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix to DL4J SameDiffLayer

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix empty array deserialization, add extra deserialization checks

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* FlatBuffers control dep serialization fixes; test serialization as part of all TF import tests

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Variable control dependencies serialization fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix issue with removing inputs for ops

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* FlatBuffers NDArray deserialization fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* FlatBuffers NDArray deserialization fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Final cleanup/polish

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-10-23 21:19:50 +11:00
..
ci Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
contrib Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
deeplearning4j-common Version upgrades (#199) 2019-08-30 14:35:27 +10:00
deeplearning4j-core SameDiff execution, TF and memory management overhaul (#10) 2019-10-23 21:19:50 +11:00
deeplearning4j-cuda change pointer reference for cudnn (#220) 2019-09-02 12:40:32 +03:00
deeplearning4j-data Version upgrades (#199) 2019-08-30 14:35:27 +10:00
deeplearning4j-dataimport-solrj Fix backend dependencies for tests (#189) 2019-08-29 12:54:48 +09:00
deeplearning4j-graph First round of runtime test improvements (#7875) 2019-06-13 20:40:40 +10:00
deeplearning4j-manifold Build fix - tsne random method signature (#259) 2019-09-13 13:40:46 +10:00
deeplearning4j-modelexport-solr Fix backend dependencies for tests (#189) 2019-08-29 12:54:48 +09:00
deeplearning4j-modelimport Version upgrades (#199) 2019-08-30 14:35:27 +10:00
deeplearning4j-nearestneighbors-parent Merge pull request #7 from KonduitAI/asto_nd4s_10172019 2019-10-23 12:11:25 +03:00
deeplearning4j-nlp-parent Fixed shape for muli 2019-10-16 12:59:25 +03:00
deeplearning4j-nn SameDiff execution, TF and memory management overhaul (#10) 2019-10-23 21:19:50 +11:00
deeplearning4j-remote Fix backend dependencies for tests (#189) 2019-08-29 12:54:48 +09:00
deeplearning4j-scaleout Remove old datavec.spark.version tag (#8232) 2019-09-12 23:43:01 +10:00
deeplearning4j-ui-parent Ensure UI overview page is refreshed when loading saved net data (#243) 2019-09-05 18:30:51 +10:00
deeplearning4j-util Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
deeplearning4j-zoo Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
dl4j-integration-tests Version upgrades (#199) 2019-08-30 14:35:27 +10:00
dl4j-perf Merge master to upstream (#7945) 2019-06-27 18:37:04 +03:00
.codeclimate.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.travis.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
CONTRIBUTORS.md Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
GITTER_GUIDELINES.md Update links to eclipse repos (#252) 2019-09-10 19:09:46 +10:00
LICENSE.txt Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
README.md Update links to eclipse repos (#252) 2019-09-10 19:09:46 +10:00
buildmultiplescalaversions.sh Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
pom.xml Version upgrades (#199) 2019-08-30 14:35:27 +10:00

README.md

Eclipse Deeplearning4J: Neural Networks for Java/JVM

Join the chat at https://gitter.im/deeplearning4j/deeplearning4j Maven Central Javadoc

Eclipse Deeplearning4J is part of the Skymind Intelligence Layer, along with ND4J, DataVec, Arbiter and RL4J. It is an Apache 2.0-licensed, open-source, distributed neural net library written in Java and Scala. By contributing code to this repository, you agree to make your contribution available under an Apache 2.0 license.

Deeplearning4J integrates with Hadoop and Spark and runs on several backends that enable use of CPUs and GPUs. The aim is to create a plug-and-play solution that is more convention than configuration, and which allows for fast prototyping.

The most recent stable release in Maven Central is 0.9.1, and the current master on Github can be built from source.

For more info, see: https://docs.skymind.ai/docs


Using Eclipse Deeplearning4j

To get started using Deeplearning4j, please go to our Quickstart. You'll need to be familiar with a Java automated build tool such as Maven and an IDE such as IntelliJ.

Main Features

  • Versatile n-dimensional array class
  • GPU integration (supports devices starting from Kepler, cc3.0. You can check your device's compute compatibility here.)

Modules

  • datavec = Library for converting images, text and CSV data into format suitable for Deep Learning
  • nn = core neural net structures MultiLayer Network and Computation graph for designing Neural Net structures
  • core = additional functionality building on deeplearning4j-nn
  • modelimport = functionality to import models from Keras
  • nlp = natural language processing components including vectorizers, models, sample datasets and renderers
  • scaleout = integrations
    • spark = integration with Apache Spark versions 1.3 to 1.6 (Spark 2.0 coming soon)
    • parallel-wraper = Single machine model parallelism (for multi-GPU systems, etc)
    • aws = loading data to and from aws resources EC2 and S3
  • ui = provides visual interfaces for tuning models. Details here

Documentation

Documentation is available at deeplearning4j.org and JavaDocs. Open-source contributors can help us improve our documentation for Deeplearning4j by sending pull requests for the DL4J website here

Support

We are not supporting Stackoverflow right now. Github issues should focus on bug reports and feature requests. Please join the community on Gitter, where we field questions about how to install the software and work with neural nets. For support from Skymind, please see our contact page.

Installation

To install Deeplearning4J, see our Quickstart and below. More information can be found on the ND4J web site as well as here.

Use Maven Central Repository

Search Maven Central for deeplearning4j to get a list of dependencies.

Add the dependency information to your pom.xml file. We highly recommend downloading via Maven unless you plan to help us develop DL4J. An easy way to get up-to-date dependencies is to use the ones listed in our dl4j-examples POM.


Contribute

  1. Check for open issues or open a fresh one to start a discussion around a feature idea or a bug.
  2. If you feel uncomfortable or uncertain about an issue or your changes, don't hesitate to contact us on Gitter using the link above.
  3. Fork the repository on GitHub to start making your changes (branch off of the master branch).
  4. Write a test that shows the bug was fixed or the feature works as expected.
  5. Note the repository follows the Google Java style with two modifications: 120-char column wrap and 4-spaces indentation. You can format your code to this format by typing mvn formatter:format in the subproject you work on, by using the contrib/formatter.xml at the root of the repository to configure the Eclipse formatter, or by using the Intellij plugin.
  6. Send a pull request and bug us on Gitter until it gets merged and published. :)
  7. Add technical documentation on the Deeplearning4j website and fix any typos you see.