Go to file
raver119 5d69069177
[WIP] Memory limits (#167)
* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* one more initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* additional initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* subsequent initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit testing

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit per device

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit per group

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit for cuda

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit for cuda + few missed lines

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit for cuda + missed includes

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit for cuda + one more missed include

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit shouldn't count host mem as dev0 in cuda

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit that tracks HOST group limits for CUDA

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit with some Environment changes

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit with more Environment changes

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit with maxMasterThreads fix

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit with maxMasterThreads fix

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit without maxMasterThreads exception

Signed-off-by: raver119 <raver119@gmail.com>

* initial commit without Nd4jULong in Environment

Signed-off-by: raver119 <raver119@gmail.com>

* add sleep and more iterations for OOM cases

Signed-off-by: raver119 <raver119@gmail.com>

* limits propagation from java side

Signed-off-by: raver119 <raver119@gmail.com>

* - consume ErrorCode every time
- one test for memory limits

Signed-off-by: raver119 <raver119@gmail.com>

* unordered_map

Signed-off-by: raver119 <raver119@gmail.com>

* unordered_map

Signed-off-by: raver119 <raver119@gmail.com>

* unordered_map

Signed-off-by: raver119 <raver119@gmail.com>

* RSub op mapping fixed

Signed-off-by: raver119 <raver119@gmail.com>

* typo fixed

Signed-off-by: raver119 <raver119@gmail.com>

* one bad test fixed

Signed-off-by: raver119 <raver119@gmail.com>
2020-01-24 10:11:09 +03:00
.github Update contributing and issue/PR templates (#7934) 2019-06-22 16:21:27 +10:00
arbiter Unit/integration test split + test speedup (#166) 2020-01-22 22:27:01 +11:00
datavec Various fixes (#143) 2020-01-04 13:45:07 +11:00
deeplearning4j Unit/integration test split + test speedup (#166) 2020-01-22 22:27:01 +11:00
docs Mention the new % unit for maxBytes and maxPhysicalBytes in Memory management documentation (#8435) (#8461) 2019-12-05 12:47:53 +09:00
gym-java-client RL4J: Make a few fixes (#8303) 2019-10-31 13:41:52 +09:00
jumpy Update links to eclipse repos (#252) 2019-09-10 19:09:46 +10:00
libnd4j [WIP] Memory limits (#167) 2020-01-24 10:11:09 +03:00
nd4j [WIP] Memory limits (#167) 2020-01-24 10:11:09 +03:00
nd4s String changes (#3) 2020-01-04 13:27:50 +03:00
pydatavec Minor edits to README for pydatavec and pydl4j (#8336) 2019-12-06 08:10:38 +01:00
pydl4j Minor edits to README for pydatavec and pydl4j (#8336) 2019-12-06 08:10:38 +01:00
rl4j Merge pull request #8495 from KonduitAI/master 2019-12-05 11:05:44 +11:00
scalnet Add support for CUDA 10.2 (#89) 2019-11-29 16:31:03 +11:00
.gitignore fix pydatavec for python 3... and python2 install problems (#8422) 2019-11-20 08:20:04 +01:00
CONTRIBUTING.md Various fixes (#43) 2019-11-14 19:38:20 +11:00
Jenkinsfile Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
LICENSE Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
README.md Update links to eclipse repos (#252) 2019-09-10 19:09:46 +10:00
change-cuda-versions.sh Add support for CUDA 10.2 (#89) 2019-11-29 16:31:03 +11:00
change-scala-versions.sh Version upgrades (#199) 2019-08-30 14:35:27 +10:00
perform-release.sh Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
pom.xml Unit/integration test split + test speedup (#166) 2020-01-22 22:27:01 +11:00

README.md

Monorepo of Deeplearning4j

Welcome to the new monorepo of Deeplearning4j that contains the source code for all the following projects, in addition to the original repository of Deeplearning4j moved to deeplearning4j:

To build everything, we can use commands like

./change-cuda-versions.sh x.x
./change-scala-versions.sh 2.xx
./change-spark-versions.sh x
mvn clean install -Dmaven.test.skip -Dlibnd4j.cuda=x.x -Dlibnd4j.compute=xx

or

mvn -B -V -U clean install -pl '!jumpy,!pydatavec,!pydl4j' -Dlibnd4j.platform=linux-x86_64 -Dlibnd4j.chip=cuda -Dlibnd4j.cuda=9.2 -Dlibnd4j.compute=<your GPU CC> -Djavacpp.platform=linux-x86_64 -Dmaven.test.skip=true

An example of GPU "CC" or compute capability is 61 for Titan X Pascal.

Want some examples?

We have separate repository with various examples available: https://github.com/eclipse/deeplearning4j-examples

In the examples repo, you'll also find a tutorial series in Zeppelin: https://github.com/eclipse/deeplearning4j-examples/tree/master/tutorials