Commit Graph

1114 Commits (1f6cb95c67701c14eb93301f6ea9470814935f07)

Author SHA1 Message Date
Andrii T 58b11bfecc
Map C++ ops to Java (#392)
* MergeMaxIndex, ReverseBp, Tri, Triu and TriuBp added

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Upsamling3d draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fix (upsampling3dBp inputDatatype.size=2)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* polished testcases

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* matching of Upsampling3d input format according to cpp iArg

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ops generated from codegen

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* requested changes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added super() for Triu

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* everything passes except TriuOp

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Tri op dtype arg (output datatype config support) + default float32

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* temporary commit with manually edited sd/nd ops

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Cannot use 'val' here because initializer expression does not have a representable type: Type cannot be resolved

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* all tests passed

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* few requested changes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore reverse_bp test due to logged issue

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix reverse op

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix MergeMaxIndex dtype -> iarg

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-25 23:56:02 +10:00
Andrii T bd376ca993
GRU and GRUBp (#410)
* GRU and GRUBp ops added and tested

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor polishing

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* few requested changes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* regenerated namespace + small fix in RnnOpValidation

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fix bad character in RnnOpValidation

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-25 01:12:46 +10:00
raver119 5e779574cb
Few Workspace tweaks (#409)
* Allow to destroy workspaces on demand

Signed-off-by: raver119 <raver119@gmail.com>

* MMAP'd workspace can't have LearningPolicy

Signed-off-by: raver119 <raver119@gmail.com>

* throw an exception on CUDA

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-23 15:50:30 +03:00
Alex Black f5f77df846
Dependency version upgrades + small SameDiff fix (#405)
* #8861 Training evaluation on variables not required for loss fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Dependency version updates flagged by dependabot

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-23 15:38:42 +10:00
raver119 8f765c80ff
- do not use GetShortName approach for mmap on windows (#408)
- additional test for long file names on windows

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-23 08:24:49 +03:00
Fariz Rahman 2ecabde500
DL4J NWC support for RNNs (#379)
* merge conf

* merge conf

* conf fix

* NWC initial

* revert pom.xml

* revert pom.xml

* default NCW

* bidirectional+some tests

* RNNOutputLayer, RNNLossLayer, Graves + tests

* rnn tests

* LastTimeStep + tests

* masking + tests

* graves, rnnoutput, rnnloss

* nwc timeseries reverse

* more tests

* bi-gravelstm test

* fixes

* rnn df tests basic

* bug fix: cudnn fallback

* bug fix

* misc

* gravelstm tests

* preprocessor fixes

* TimeDistributed

* more tests

* RnnLossLayer builder def val

* copyright headers

* Remove debug println

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fix + test naming

Signed-off-by: Alex Black <blacka101@gmail.com>

* Parameterized test name

Signed-off-by: Alex Black <blacka101@gmail.com>

* fix LastTimeStep masked

* Fix MaskZero mask datatype issue

Signed-off-by: Alex Black <blacka101@gmail.com>

* rem println

* javadoc

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-23 12:16:44 +10:00
Chris Bamford 032b97912e
RL4J: Sanitize Observation (#404)
* working on ALE image pipelines that appear to lose data

* transformation pipeline for ALE has been broken for a while and needed some cleanup to make sure that openCV tooling for scene transforms was actually working.

* allowing history length to be set and passed through to history merge transforms

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* native image loader is not thread-safe so should not be static

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* make sure the transformer for encoding observations that are not pixels converts corectly

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Test fixes for ALE pixel observation shape

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Fix compilation errors

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>

* fixing some post-merge issues, and comments from PR

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

Co-authored-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-23 10:47:26 +09:00
Alex Black 75cc6e2ed7
Increase default timeouts [WIP] (#407)
* Increase default test timeout to 60 sec due to slow PPC CI machines

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix flaky NLP test (randomly fails due to no RNG seed set)

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-23 10:46:42 +10:00
Alex Black 0a1f75a5d9 Merge & fix small test conflict
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-23 10:31:15 +10:00
Tamás Fenyvesi 722d5a052a
Fix UIServer features in multi-session mode, synchronous start and stop (#8856)
* fix Freemarker version mismatch: change version requested in TrainModule to 2.3.23 (freemarker.version in pom.xml)

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix checking multi-session mode in VertxUIServer.getInstance. Tested multiple calls in TestVertxUIMultiSession.

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix UIServer.getInstance() to return existing instance

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* extend timeout for manual UI tests from 30 to 600 seconds

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* start and stop UI server synchronously (wait until complete), tests

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix for auto-attaching StatsStorage given in VertxUIServer#getInstance(Integer, boolean, Function<String,StatsStorage>), test improvements

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* exception handling, test improvements

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* add asynchronous method to start UI server

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix UIServer.getInstance() to return existing instance

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix UI server language setting in multi-session mode

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix UI server system tab not loading data in multi-session mode

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* undo added InterruptedException in UIServer.getInstance()

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* fix async stopping of UIServer.stopAsync(Promise<Void>), added test

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* restore the daemon thread style behaviour of UIServer: don't keep the process alive just because the UI is running

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>

* speed up and don't @Ignore tests in TestVertxUI and TestVertxUIMultiSession, put longer tests to separate class TestVertxUIManual

Signed-off-by: Tamás Fenyvesi <tamas.fenyvesi@doknet.hu>
2020-04-23 10:26:51 +10:00
Alex Black 3c6cb6d1a3
#8503 Add timeouts for downloads (test resources, Downloader utility) (#406)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-23 08:39:13 +10:00
Alexander Stoyakin ccb216a3ed
Fixing issues from Sonar report (#391)
* Fixing issues from Sonar report

* Proper logger of exceptions

* Coding style fixes

* Use dup parameter

* Cleanup, minor issues

* Cuda compilation fixed and some minor fixes
2020-04-23 08:36:49 +10:00
Alex Black 2a488efb1b
DL4J CNN2D layers NHWC support (#376)
* First steps for DL4J NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d NHWC forward pass works

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d NHWC backprop

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d backprop + fixes; subsampling fwd/bwd; improve tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Zero padding layer NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Cropping2D NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Deconv2d NHWC + clean up NHWC test framework code duplication

Signed-off-by: Alex Black <blacka101@gmail.com>

* CnnLossLayer NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling and batchnorm NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Space to depth

Signed-off-by: Alex Black <blacka101@gmail.com>

* Depthwise pt1

Signed-off-by: Alex Black <blacka101@gmail.com>

* Depthwise pt2 and LRN

Signed-off-by: Alex Black <blacka101@gmail.com>

* SpaceToBatch

Signed-off-by: Alex Black <blacka101@gmail.com>

* LocallyConnected2D

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix depthwise nhwc support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling NHWC - workaround for #8857

Signed-off-by: Alex Black <blacka101@gmail.com>

* Workaround for #8859 - SpaceToDepth

Signed-off-by: Alex Black <blacka101@gmail.com>

* Batch normalization workaround - #8860

Signed-off-by: Alex Black <blacka101@gmail.com>

* cuDNN fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Switch cudnn conv2d to permute based impl due to 'true' NHWC not working

Signed-off-by: Alex Black <blacka101@gmail.com>

* cuDNN subsampling helper NHWC fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling/batchnorm fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* CNN2D NHWC gradient checks (make CNNGradientCheckTest parameterized)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Gradient checks, SConv2d, bunch of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Global pooling NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Also test both float and double for cuDNN NHWC tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Javadoc

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore failing keras import test until next PR

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-22 22:54:29 +10:00
raver119 2c80b18f1d disable mmap test for cuda
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-22 12:57:26 +03:00
raver119 1b8077f66a Merge remote-tracking branch 'origin/master' 2020-04-22 12:38:37 +03:00
raver119 78260efe54 fix overallocation for memory-mapped workspaces
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-22 12:38:22 +03:00
shugeo fc3f5d4ffb
Shugeo exponential distribution infinities fix (#403)
* Fixed bound problem with Exponential distribution implementation.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added test for Exponential distribution to avoid infinities.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added a test for exponential distribution with 1M elements.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Cosmetical changes only and tests.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Modified test and implementation for exponential_distribution op.

Signed-off-by: shugeo <sgazeos@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-22 12:12:00 +03:00
Alexandre Boulanger 550e84ef43
RL4J: Add Agent and Environment (#358)
* Added Agent and Environment

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Added headers

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Fix compilation errors

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-22 09:13:08 +09:00
raver119 a10fd4524a
Fix for certain non-ews cases (#402)
* BtS/StB/StD/DtS dup for views

Signed-off-by: raver119 <raver119@gmail.com>

* batchnorm_bp dup for views

Signed-off-by: raver119 <raver119@gmail.com>

* two java tests for bad strides

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-21 12:41:30 +03:00
raver119 5ee37a22eb
Workspace ID fix (#400)
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-21 07:31:57 +03:00
Alex Black 76aab75174
Fix uberjar dependencies (#401)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-21 10:51:26 +10:00
Alex Black b2f86ab8a8
#8849 Add ignore for randomly failing test (#399)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 23:59:30 +10:00
Alex Black 8eccc170ec
Merge pull request #8854 from KonduitAI/master
Merge latest development work
2020-04-20 22:36:36 +10:00
Alex Black fe516ae6cf
nd4j-jackson dependency fix (#398)
* Remove old nd4j-jackson dependencies

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix use of old/deprecated JSON serializer

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix deserialization

Signed-off-by: Alex Black <blacka101@gmail.com>

* Delete test using deleted ser/de classes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Delete another copy of old test

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 22:33:11 +10:00
Alex Black 191bda3228
Base namespace (#287)
* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to assign operation.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix Imax, IMin.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* concat.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dynamicPartition

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* new ops up to gte.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updated review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to matchCondition.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to OneHot.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to permute.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to rank.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to scatterMul.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* resolving code review issues.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. inclides UnsortedSegment ops.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to stridedSlice.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix stridedSlice.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* first pass of SDBaseops.kt complete.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* put branch in compilable state.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* add NDBaseTest. fix dynamicPartition signature. failed fix of assign.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* make tests public.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* adds tests up to invertedPermutation.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix ScalarEquals, Assign.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updates NDBaseTest.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updates 'check' comments based on test pass/fail.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix scalar ops. Update tests,

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dev-tools review items. wip.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dev-tools code review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* complete review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Comment for logged issue; fix test case

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* undo changes to Nd4jCpu.java

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* update tests.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Fixes and regenerate

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small test fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* small fixes to tests.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Cleanup

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small CUDAExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Robert Altena <Rob@Ra-ai.com>
2020-04-20 16:57:00 +10:00
shugeo a5db0e33be
Shugeo segment fix4 (#385)
* Added test and fixed error message for unsorted_segment_sqrt_n op.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Fixed error message for unsorted_segment_* ops when 1 segment is given.

Signed-off-by: shugeo <sgazeos@gmail.com>
2020-04-20 09:04:35 +03:00
raver119 1bec9a4f61 mkldnn version upgrade: v1.4
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-20 08:19:03 +03:00
Chris Bamford 74420bca31
RL4J: Sanitize async learner (#327)
* refactoring global async to use a much simpler update procedure with a single global lock

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* simplification of async learning algorithms, stabilization + better hyperparameters

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* started to play with using mockito for tests

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Working on refactoring tests for async classes and trying to make async simpler

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* more work on mockito tests and making some tests much less complex and more explicit in what they are testing

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* some fixes from merging

* do not allow copying of the current network to worker threads, fixing debug line

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* adding some more tests around PR review

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Adding more tests after review comments

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* few more tests and fixes from PR review

* remove rename of maxEpochStep to maxStepsPerEpisode as we agreed to review this in a seperate PR

* 2019 instead of 2018 on copyright header

* adding konduit copyright to files

* some more copyright headers

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

Co-authored-by: Alexandre Boulanger <aboulang2002@yahoo.com>
2020-04-20 11:21:01 +09:00
Alexander Stoyakin 455a5d112d
Fixes for codegen generated classes and build improvements (#367)
* Input format extended

* Deleted redundant code

* Added weights format to conv2d config

* Refactoring

* dl4j base test functionality

* Different tests base class per module

* Check base class for dl4j-graph subproject tests

* Check if test classes extend BaseDL4JTest

* Use nd4j-common-tests as transient dependency

* Enums and tests added

* Added codegenerated methods

* Use namespace methods

* Replace DifferentialFunctionFactory  with codegenerated classes

* Fixed linspace

* Namespaces regenerated

* Namespaces used instead of factory

* Regenerated base classes

* Input format extended

* Added weights format to conv2d config

* Refactoring

* dl4j base test functionality

* Different tests base class per module

* Check base class for dl4j-graph subproject tests

* Check if test classes extend BaseDL4JTest

* Use nd4j-common-tests as transient dependency

* Enums and tests added

* Added codegenerated methods

* Use namespace methods

* Replace DifferentialFunctionFactory  with codegenerated classes

* Fixed linspace

* Namespaces regenerated

* Regenerated base classes

* Regenerated namespaces

* Generate nd4j namespaces

* INDArrays accepting constructors

* Generated some ops

* Some fixes

* SameDiff ops regenerated

* Regenerated nd4j ops

* externalErrors moved

* Compilation fixes

* SquaredDifference - strict number of args

* Deprecated code cleanup. Proper base class for tests.

* Extend test classes with BaseND4JTest

* Extend test classes with BaseDL4JTest

* Legacy code

* DL4J cleanup

* Exclude test utils from base class check

* Tests fixed

* Arbiter tests fix

* Test dependency scope fix + pom.xml formatting

Signed-off-by: Alex Black <blacka101@gmail.com>

* Significant number of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another round of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another round of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Few additional fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* DataVec missing test scope dependencies

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-20 10:27:13 +10:00
Alex Black 73aa760c0f
Timeouts and scala 2.12 for deeplearning4j-nlp-korean workaround (#380)
* Increase default timeout on Spark tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* #8840 disable deeplearning4j-nlp-korean module for scala 2.12

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix for change-scala-versions.sh

Signed-off-by: Alex Black <blacka101@gmail.com>

* CUDA test fixes + more timeout issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* More CUDA

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fix for cuDNN subsampling + same mode

Signed-off-by: Alex Black <blacka101@gmail.com>

* Flaky test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Reduce memory requirements for ValidateCuDNN BN test

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix slow/ineffirient scalnet tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Increase timeouts to avoid failures if CI machines are slower than expected

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore flaky test (issue #8849) and increase timeout for slow CI downloads

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 10:26:00 +10:00
raver119 163222e853 tick device write in RandomLauncher
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-19 13:34:26 +03:00
shugeo 0eca33ad94
Shugeo cuda solver fix (#383)
* Refactored cuSolver handle usage to handle LaunchContext instance properly.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Refactored svd solver usage with LaunchContext instance singleton.

Signed-off-by: shugeo <sgazeos@gmail.com>

* add device locks for cuSolver uses

Signed-off-by: raver119 <raver119@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-17 16:52:08 +03:00
Alex Black 18d4eaa68d
DL4J SameDiff loss function (#251)
* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* SameDiffLoss draft

* very very draft

* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* temporary commit for clarification

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v2

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* very very draft

* temporary commit for clarification

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v2

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* SDLoss after requested changes but with questions in comments

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added requested changes

* small fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Javadoc

Signed-off-by: Alex Black <blacka101@gmail.com>

* Test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Andrii Tuzhykov <andrew@unrealists.com>
Co-authored-by: atuzhykov <andrewtuzhukov@gmail.com>
Co-authored-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>
2020-04-17 19:47:57 +10:00
Andrii T 5fbb04531d
At cpp ops (#378)
* crelu op added

* crelu op added

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* crelu(bp)+transformOpValidation op

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added ClipByAvgNorm and DepthwiseConv2DBp

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ClipByAvgNorm passes forward check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* EmbeddingLookup draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* DepthwiseConv2DB gradient check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* EmbeddingLookup and DepthwiseConv2dBp finished + tests added

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ImageResize draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* DepthwiseConv2DB gradient check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ImageResize passed tests except helper::resizeFunctor:Non implemented

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* replaced ImageResizeMethods enum by codegen

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* polished checkpoint (OPValidationSuite passed and mvn install build succesfull after codegen)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* manually merged LSTMLayerTestCases from master
Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MaximumBp added and tested

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MergeAddBp draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MergeMaxBp and MergeAvgBP added and tests passed

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fix

* draft LSTMLayerBp (big relative layer in gradient check)

* LSTMLayerBp check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayerBp check v2

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* requested changes (test passes)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer testcases passed gradientcheck

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small LSTMLayer testcase1 improvement (cLast, yLast)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Warnings issue solved

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fixes for MKLDNN LSTM layer helper

Signed-off-by: Alex Black <blacka101@gmail.com>

* stable version

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-17 15:16:14 +10:00
Alex Black 3967e039a5
Switch Java-based updater implementations to C++ ops (#384)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-17 14:41:49 +10:00
Paul Dubs bb9cdb251e
Small ConvolutionalIterationListener improvements (#382)
* Add a message to the runtime exception

Signed-off-by: Paul Dubs <paul.dubs@gmail.com>

* Output Convolutions as PNG instead of JPG

A lossless encoding is useful in this case, as it allows small details to be preserved

Signed-off-by: Paul Dubs <paul.dubs@gmail.com>
2020-04-17 10:19:21 +10:00
Oleh 3d15706ffa
Lin_space operation improve (#373)
* libnd4j update linspace op

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j #8513 update lin_space op, tests added

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* - minor linspace tweaks (num_elements now iArg)
- java linspace updates
- couple of additional tests for linspace

Signed-off-by: raver119 <raver119@gmail.com>

* roll back timeout change

Signed-off-by: raver119 <raver119@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-16 14:53:56 +03:00
raver119 12ba1fa406
Few minor fixes (#381)
* - 1D indexing fix
- couple of new tests for 1D indexing

Signed-off-by: raver119 <raver119@gmail.com>

* percentile fix + test

Signed-off-by: raver119 <raver119@gmail.com>

* wrong signature used in test

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-16 13:25:13 +03:00
Yurii Shyrma 4247718f61
Shyrma gru bp (#377)
* - update gru ff op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implementation and testing gru_bp op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - neglect dependencies between dLdh/dLdhLast/dLdcLast in lstmLayer backprop

Signed-off-by: Yurii <iuriish@yahoo.com>
2020-04-16 08:09:04 +03:00
raver119 217fa433d4 mkldnn repo url updated
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-16 07:39:45 +03:00
raver119 75af392671
- memcpy fix + validation for CUDA: skip memcpu if length < 1 (#375)
- Reset cached context after device affinity change

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-14 18:47:46 +03:00
Alex Black 09f4c21059
Merge pull request #8835 from KonduitAI/master
Merge recent development work
2020-04-14 22:35:49 +10:00
Alex Black 133223e865 Merge remote-tracking branch 'eclipse/master' 2020-04-14 22:24:17 +10:00
Samuel Audet 6fcd078c5e
Update dependencies to just released JavaCPP and JavaCV 1.5.3 (#374)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-14 19:36:14 +10:00
Yurii Shyrma 23e4aa99ad
Shyrma lstm layer bp (#370)
* - start working on bp for lstm

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further working on bp for lstmLayer

Signed-off-by: Yurii <iuriish@yahoo.com>

* - minor change

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 2

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 3

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 4

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 5

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 6

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 7

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 8

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 9

Signed-off-by: Yurii <iuriish@yahoo.com>

* - provide lstmLayerCell and lstmLayerCellBp as separate CUSTOM_OPs

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing lstmLayerCellBp helper

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implement lstmLayerCellBp as separate op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implement lstmLayerBp as separate op (not tested)

Signed-off-by: Yurii <iuriish@yahoo.com>

* - fixing calculations of dLdWp and dLdb in lstmLayerCellBp

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 10

Signed-off-by: Yurii <iuriish@yahoo.com>

* - fixing typo in lstmLayerTimeLoop

Signed-off-by: Yurii <iuriish@yahoo.com>

* - forgot to perform clipping of c array and calculate corresponding derivative in lstmLayerCellBp

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 10

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing bugs in lstmLayer_bp op 1

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing bugs in lstmLayer_bp op 2

Signed-off-by: Yurii <iuriish@yahoo.com>

* - turn off heavy tests for cuda for lstmLayer_bp op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - forgot to nullify gradients at eliminated time steps (when sequnce length array is present )

Signed-off-by: Yurii <iuriish@yahoo.com>
2020-04-13 13:21:51 +03:00
Alexandre Boulanger f1debe8c07
RL4J: Add ExperienceHandler (#369)
* Added ExperienceHandler

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Added getTrainingBatchSize()

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>
2020-04-11 08:50:40 +09:00
raver119 3e2dbc65dd
MatMul for gemm/gemv calls (#365)
* libnd4j added optional alpha and beta support to matmul

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j typos fixes

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j add optional alpha and beta to matmul_bp

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j one more typo fix

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j added optional alpha and beta to mkl implementation

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* MatMul alpha/beta on java side

Signed-off-by: raver119 <raver119@gmail.com>

* alpha/beta fix in libnd4j

Signed-off-by: raver119 <raver119@gmail.com>

* alpha/beta fix in matmul_bp

Signed-off-by: raver119 <raver119@gmail.com>

* restored view validation

Signed-off-by: raver119 <raver119@gmail.com>

* gemv/gemm now use MatMul op

Signed-off-by: raver119 <raver119@gmail.com>

* few tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* additional INDArray.mmul signature

Signed-off-by: raver119 <raver119@gmail.com>

* make C order default for INDArray.mmul, unless both A/B have F order

Signed-off-by: raver119 <raver119@gmail.com>

* Nd4j.gemm validation fix

Signed-off-by: raver119 <raver119@gmail.com>

* disable mkldnn matmul for xxf with beta != 0 case

Signed-off-by: raver119 <raver119@gmail.com>

* SimpleRnn workspace fix + timeouts

Signed-off-by: Alex Black <blacka101@gmail.com>

* two more tests + minor fix in matmul platform check

Signed-off-by: raver119 <raver119@gmail.com>

* Flaky test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* propagate testresources profile

Signed-off-by: raver119 <raver119@gmail.com>

* Resources fix + flaky test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Oleg <oleg.semeniv@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-10 17:57:02 +03:00
Alex Black 99c727f15b
Add javacpp classifier dependency to nd4j-native and nd4j-cuda (#366)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-09 10:00:27 +10:00
Andrii T d86dd5b131
DL4J and SameDiff integration tests + LSTMLayer java op class (#353)
* init in this branch

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Lenetet Mnist workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small fix for calculations

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* for Alex to check placeholder null pointer issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* CNN3D workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* state for launching on dxg to regenterate dl4j examples

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* SD RNN test case workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* checkpoint at lstmBlock: Input array 1 (x) rank must be got input with rank 2 issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fix LSTMLayer inputs order

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* lstm mismatch with c++ op issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer config draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer config draft v2

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* have doubt I had to do this

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* NDRNN generated by codegen

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayerTestCases draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes again

* added LSTMLayer testcases to nd4j-tests + setted Preconditions in LSTMLayer constructors

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added lost SDCNNtestcases

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* overrided getNumOutputs from DynamicCustomOp in LSTMLayer and reorganized LSTMLayerOutputs according to cpp op

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished with LSTMLayerOutputs

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fix MKLDNN platform checks (i.e., when MKLDNN can be used vs. not)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix LSTMLayerWeights input order

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* fixed LSTMLayer testcases

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished SameDiffRNNTestCase

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished all testcases + minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Multiple generation-related fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix multiple issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* LSTM fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Regenerate ND4J namespaces and fix multiple issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* changed SameDiffRNNTestCase

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Small fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* added  Nd4j.getRandom().setSeed(12345) where needed

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* #8828 Fix ND4J profiler NaN/Inf checks when using OpContext

Signed-off-by: Alex Black <blacka101@gmail.com>

* #8828 Fix ND4J profiler NaN/Inf checks when using OpContext

Signed-off-by: Alex Black <blacka101@gmail.com>

* Tweak to weight init for SameDiff CNN test case

Signed-off-by: Alex Black <blacka101@gmail.com>

* Tweaks for test cases

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore failing tests until fixed

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-09 00:20:48 +10:00
Samuel Audet ab083b9167
Update versions of JavaCPP Presets for OpenCV and MKL (#363)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-08 22:09:45 +10:00