Commit Graph

1253 Commits (b3805a132f8de6d6db80607ba46e7f8b8847cca6)

Author SHA1 Message Date
Alexander Stoyakin ccb216a3ed
Fixing issues from Sonar report (#391)
* Fixing issues from Sonar report

* Proper logger of exceptions

* Coding style fixes

* Use dup parameter

* Cleanup, minor issues

* Cuda compilation fixed and some minor fixes
2020-04-23 08:36:49 +10:00
Alex Black 2a488efb1b
DL4J CNN2D layers NHWC support (#376)
* First steps for DL4J NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d NHWC forward pass works

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d NHWC backprop

Signed-off-by: Alex Black <blacka101@gmail.com>

* Conv2d backprop + fixes; subsampling fwd/bwd; improve tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Zero padding layer NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Cropping2D NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Deconv2d NHWC + clean up NHWC test framework code duplication

Signed-off-by: Alex Black <blacka101@gmail.com>

* CnnLossLayer NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling and batchnorm NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Space to depth

Signed-off-by: Alex Black <blacka101@gmail.com>

* Depthwise pt1

Signed-off-by: Alex Black <blacka101@gmail.com>

* Depthwise pt2 and LRN

Signed-off-by: Alex Black <blacka101@gmail.com>

* SpaceToBatch

Signed-off-by: Alex Black <blacka101@gmail.com>

* LocallyConnected2D

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix depthwise nhwc support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling NHWC - workaround for #8857

Signed-off-by: Alex Black <blacka101@gmail.com>

* Workaround for #8859 - SpaceToDepth

Signed-off-by: Alex Black <blacka101@gmail.com>

* Batch normalization workaround - #8860

Signed-off-by: Alex Black <blacka101@gmail.com>

* cuDNN fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Switch cudnn conv2d to permute based impl due to 'true' NHWC not working

Signed-off-by: Alex Black <blacka101@gmail.com>

* cuDNN subsampling helper NHWC fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Upsampling/batchnorm fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* CNN2D NHWC gradient checks (make CNNGradientCheckTest parameterized)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Gradient checks, SConv2d, bunch of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Global pooling NHWC support

Signed-off-by: Alex Black <blacka101@gmail.com>

* Also test both float and double for cuDNN NHWC tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Javadoc

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore failing keras import test until next PR

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-22 22:54:29 +10:00
raver119 2c80b18f1d disable mmap test for cuda
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-22 12:57:26 +03:00
raver119 1b8077f66a Merge remote-tracking branch 'origin/master' 2020-04-22 12:38:37 +03:00
raver119 78260efe54 fix overallocation for memory-mapped workspaces
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-22 12:38:22 +03:00
shugeo fc3f5d4ffb
Shugeo exponential distribution infinities fix (#403)
* Fixed bound problem with Exponential distribution implementation.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added test for Exponential distribution to avoid infinities.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added a test for exponential distribution with 1M elements.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Cosmetical changes only and tests.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Modified test and implementation for exponential_distribution op.

Signed-off-by: shugeo <sgazeos@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-22 12:12:00 +03:00
Alexandre Boulanger 550e84ef43
RL4J: Add Agent and Environment (#358)
* Added Agent and Environment

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Added headers

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Fix compilation errors

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-22 09:13:08 +09:00
raver119 a10fd4524a
Fix for certain non-ews cases (#402)
* BtS/StB/StD/DtS dup for views

Signed-off-by: raver119 <raver119@gmail.com>

* batchnorm_bp dup for views

Signed-off-by: raver119 <raver119@gmail.com>

* two java tests for bad strides

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-21 12:41:30 +03:00
raver119 5ee37a22eb
Workspace ID fix (#400)
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-21 07:31:57 +03:00
Alex Black 76aab75174
Fix uberjar dependencies (#401)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-21 10:51:26 +10:00
Alex Black b2f86ab8a8
#8849 Add ignore for randomly failing test (#399)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 23:59:30 +10:00
Alex Black 8eccc170ec
Merge pull request #8854 from KonduitAI/master
Merge latest development work
2020-04-20 22:36:36 +10:00
Alex Black fe516ae6cf
nd4j-jackson dependency fix (#398)
* Remove old nd4j-jackson dependencies

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix use of old/deprecated JSON serializer

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix deserialization

Signed-off-by: Alex Black <blacka101@gmail.com>

* Delete test using deleted ser/de classes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Delete another copy of old test

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 22:33:11 +10:00
Alex Black 191bda3228
Base namespace (#287)
* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to assign operation.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix Imax, IMin.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* concat.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dynamicPartition

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* new ops up to gte.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updated review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to matchCondition.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to OneHot.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to permute.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to rank.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to scatterMul.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* resolving code review issues.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. inclides UnsortedSegment ops.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip. up to stridedSlice.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix stridedSlice.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* first pass of SDBaseops.kt complete.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* put branch in compilable state.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* add NDBaseTest. fix dynamicPartition signature. failed fix of assign.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* make tests public.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* adds tests up to invertedPermutation.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix ScalarEquals, Assign.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updates NDBaseTest.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* updates 'check' comments based on test pass/fail.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix scalar ops. Update tests,

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dev-tools review items. wip.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* dev-tools code review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* complete review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Comment for logged issue; fix test case

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* undo changes to Nd4jCpu.java

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* update tests.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Fixes and regenerate

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small test fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* small fixes to tests.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Cleanup

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small CUDAExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another small CudaExecutioner fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Robert Altena <Rob@Ra-ai.com>
2020-04-20 16:57:00 +10:00
shugeo a5db0e33be
Shugeo segment fix4 (#385)
* Added test and fixed error message for unsorted_segment_sqrt_n op.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Fixed error message for unsorted_segment_* ops when 1 segment is given.

Signed-off-by: shugeo <sgazeos@gmail.com>
2020-04-20 09:04:35 +03:00
raver119 1bec9a4f61 mkldnn version upgrade: v1.4
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-20 08:19:03 +03:00
Chris Bamford 74420bca31
RL4J: Sanitize async learner (#327)
* refactoring global async to use a much simpler update procedure with a single global lock

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* simplification of async learning algorithms, stabilization + better hyperparameters

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* started to play with using mockito for tests

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Working on refactoring tests for async classes and trying to make async simpler

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* more work on mockito tests and making some tests much less complex and more explicit in what they are testing

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* some fixes from merging

* do not allow copying of the current network to worker threads, fixing debug line

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* adding some more tests around PR review

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* Adding more tests after review comments

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* few more tests and fixes from PR review

* remove rename of maxEpochStep to maxStepsPerEpisode as we agreed to review this in a seperate PR

* 2019 instead of 2018 on copyright header

* adding konduit copyright to files

* some more copyright headers

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

Co-authored-by: Alexandre Boulanger <aboulang2002@yahoo.com>
2020-04-20 11:21:01 +09:00
Alexander Stoyakin 455a5d112d
Fixes for codegen generated classes and build improvements (#367)
* Input format extended

* Deleted redundant code

* Added weights format to conv2d config

* Refactoring

* dl4j base test functionality

* Different tests base class per module

* Check base class for dl4j-graph subproject tests

* Check if test classes extend BaseDL4JTest

* Use nd4j-common-tests as transient dependency

* Enums and tests added

* Added codegenerated methods

* Use namespace methods

* Replace DifferentialFunctionFactory  with codegenerated classes

* Fixed linspace

* Namespaces regenerated

* Namespaces used instead of factory

* Regenerated base classes

* Input format extended

* Added weights format to conv2d config

* Refactoring

* dl4j base test functionality

* Different tests base class per module

* Check base class for dl4j-graph subproject tests

* Check if test classes extend BaseDL4JTest

* Use nd4j-common-tests as transient dependency

* Enums and tests added

* Added codegenerated methods

* Use namespace methods

* Replace DifferentialFunctionFactory  with codegenerated classes

* Fixed linspace

* Namespaces regenerated

* Regenerated base classes

* Regenerated namespaces

* Generate nd4j namespaces

* INDArrays accepting constructors

* Generated some ops

* Some fixes

* SameDiff ops regenerated

* Regenerated nd4j ops

* externalErrors moved

* Compilation fixes

* SquaredDifference - strict number of args

* Deprecated code cleanup. Proper base class for tests.

* Extend test classes with BaseND4JTest

* Extend test classes with BaseDL4JTest

* Legacy code

* DL4J cleanup

* Exclude test utils from base class check

* Tests fixed

* Arbiter tests fix

* Test dependency scope fix + pom.xml formatting

Signed-off-by: Alex Black <blacka101@gmail.com>

* Significant number of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another round of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another round of fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Few additional fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* DataVec missing test scope dependencies

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-20 10:27:13 +10:00
Alex Black 73aa760c0f
Timeouts and scala 2.12 for deeplearning4j-nlp-korean workaround (#380)
* Increase default timeout on Spark tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* #8840 disable deeplearning4j-nlp-korean module for scala 2.12

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix for change-scala-versions.sh

Signed-off-by: Alex Black <blacka101@gmail.com>

* CUDA test fixes + more timeout issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* More CUDA

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fix for cuDNN subsampling + same mode

Signed-off-by: Alex Black <blacka101@gmail.com>

* Flaky test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Reduce memory requirements for ValidateCuDNN BN test

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix slow/ineffirient scalnet tests

Signed-off-by: Alex Black <blacka101@gmail.com>

* Increase timeouts to avoid failures if CI machines are slower than expected

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore flaky test (issue #8849) and increase timeout for slow CI downloads

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-20 10:26:00 +10:00
raver119 163222e853 tick device write in RandomLauncher
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-19 13:34:26 +03:00
shugeo 0eca33ad94
Shugeo cuda solver fix (#383)
* Refactored cuSolver handle usage to handle LaunchContext instance properly.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Refactored svd solver usage with LaunchContext instance singleton.

Signed-off-by: shugeo <sgazeos@gmail.com>

* add device locks for cuSolver uses

Signed-off-by: raver119 <raver119@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-17 16:52:08 +03:00
Alex Black 18d4eaa68d
DL4J SameDiff loss function (#251)
* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* SameDiffLoss draft

* very very draft

* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* temporary commit for clarification

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v2

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* Copied and pasted RegressionTest100b4.java to RegressionTest100b6.java with renamed b4->b6

* very very draft

* temporary commit for clarification

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v2

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* temporary commit for clarification v3

Signed-off-by: atuzhykov <andrewtuzhukov@gmail.com>

* SDLoss after requested changes but with questions in comments

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added requested changes

* small fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Javadoc

Signed-off-by: Alex Black <blacka101@gmail.com>

* Test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Andrii Tuzhykov <andrew@unrealists.com>
Co-authored-by: atuzhykov <andrewtuzhukov@gmail.com>
Co-authored-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>
2020-04-17 19:47:57 +10:00
Andrii T 5fbb04531d
At cpp ops (#378)
* crelu op added

* crelu op added

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* crelu(bp)+transformOpValidation op

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added ClipByAvgNorm and DepthwiseConv2DBp

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ClipByAvgNorm passes forward check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* EmbeddingLookup draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* DepthwiseConv2DB gradient check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* EmbeddingLookup and DepthwiseConv2dBp finished + tests added

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ImageResize draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* DepthwiseConv2DB gradient check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* ImageResize passed tests except helper::resizeFunctor:Non implemented

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* replaced ImageResizeMethods enum by codegen

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* polished checkpoint (OPValidationSuite passed and mvn install build succesfull after codegen)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* manually merged LSTMLayerTestCases from master
Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MaximumBp added and tested

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MergeAddBp draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* MergeMaxBp and MergeAvgBP added and tests passed

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fix

* draft LSTMLayerBp (big relative layer in gradient check)

* LSTMLayerBp check

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayerBp check v2

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* requested changes (test passes)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer testcases passed gradientcheck

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small LSTMLayer testcase1 improvement (cLast, yLast)

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Warnings issue solved

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fixes for MKLDNN LSTM layer helper

Signed-off-by: Alex Black <blacka101@gmail.com>

* stable version

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-17 15:16:14 +10:00
Alex Black 3967e039a5
Switch Java-based updater implementations to C++ ops (#384)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-17 14:41:49 +10:00
Paul Dubs bb9cdb251e
Small ConvolutionalIterationListener improvements (#382)
* Add a message to the runtime exception

Signed-off-by: Paul Dubs <paul.dubs@gmail.com>

* Output Convolutions as PNG instead of JPG

A lossless encoding is useful in this case, as it allows small details to be preserved

Signed-off-by: Paul Dubs <paul.dubs@gmail.com>
2020-04-17 10:19:21 +10:00
Oleh 3d15706ffa
Lin_space operation improve (#373)
* libnd4j update linspace op

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j #8513 update lin_space op, tests added

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* - minor linspace tweaks (num_elements now iArg)
- java linspace updates
- couple of additional tests for linspace

Signed-off-by: raver119 <raver119@gmail.com>

* roll back timeout change

Signed-off-by: raver119 <raver119@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-04-16 14:53:56 +03:00
raver119 12ba1fa406
Few minor fixes (#381)
* - 1D indexing fix
- couple of new tests for 1D indexing

Signed-off-by: raver119 <raver119@gmail.com>

* percentile fix + test

Signed-off-by: raver119 <raver119@gmail.com>

* wrong signature used in test

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-16 13:25:13 +03:00
Yurii Shyrma 4247718f61
Shyrma gru bp (#377)
* - update gru ff op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implementation and testing gru_bp op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - neglect dependencies between dLdh/dLdhLast/dLdcLast in lstmLayer backprop

Signed-off-by: Yurii <iuriish@yahoo.com>
2020-04-16 08:09:04 +03:00
raver119 217fa433d4 mkldnn repo url updated
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-16 07:39:45 +03:00
raver119 75af392671
- memcpy fix + validation for CUDA: skip memcpu if length < 1 (#375)
- Reset cached context after device affinity change

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-14 18:47:46 +03:00
Alex Black 09f4c21059
Merge pull request #8835 from KonduitAI/master
Merge recent development work
2020-04-14 22:35:49 +10:00
Alex Black 133223e865 Merge remote-tracking branch 'eclipse/master' 2020-04-14 22:24:17 +10:00
Samuel Audet 6fcd078c5e
Update dependencies to just released JavaCPP and JavaCV 1.5.3 (#374)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-14 19:36:14 +10:00
Yurii Shyrma 23e4aa99ad
Shyrma lstm layer bp (#370)
* - start working on bp for lstm

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further working on bp for lstmLayer

Signed-off-by: Yurii <iuriish@yahoo.com>

* - minor change

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 2

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 3

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 4

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 5

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 6

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 7

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 8

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 9

Signed-off-by: Yurii <iuriish@yahoo.com>

* - provide lstmLayerCell and lstmLayerCellBp as separate CUSTOM_OPs

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing lstmLayerCellBp helper

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implement lstmLayerCellBp as separate op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - implement lstmLayerBp as separate op (not tested)

Signed-off-by: Yurii <iuriish@yahoo.com>

* - fixing calculations of dLdWp and dLdb in lstmLayerCellBp

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 10

Signed-off-by: Yurii <iuriish@yahoo.com>

* - fixing typo in lstmLayerTimeLoop

Signed-off-by: Yurii <iuriish@yahoo.com>

* - forgot to perform clipping of c array and calculate corresponding derivative in lstmLayerCellBp

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on bp for lstmLayer 10

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing bugs in lstmLayer_bp op 1

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing bugs in lstmLayer_bp op 2

Signed-off-by: Yurii <iuriish@yahoo.com>

* - turn off heavy tests for cuda for lstmLayer_bp op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - forgot to nullify gradients at eliminated time steps (when sequnce length array is present )

Signed-off-by: Yurii <iuriish@yahoo.com>
2020-04-13 13:21:51 +03:00
Alexandre Boulanger f1debe8c07
RL4J: Add ExperienceHandler (#369)
* Added ExperienceHandler

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Added getTrainingBatchSize()

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>
2020-04-11 08:50:40 +09:00
raver119 3e2dbc65dd
MatMul for gemm/gemv calls (#365)
* libnd4j added optional alpha and beta support to matmul

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j typos fixes

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j add optional alpha and beta to matmul_bp

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j one more typo fix

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j added optional alpha and beta to mkl implementation

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* MatMul alpha/beta on java side

Signed-off-by: raver119 <raver119@gmail.com>

* alpha/beta fix in libnd4j

Signed-off-by: raver119 <raver119@gmail.com>

* alpha/beta fix in matmul_bp

Signed-off-by: raver119 <raver119@gmail.com>

* restored view validation

Signed-off-by: raver119 <raver119@gmail.com>

* gemv/gemm now use MatMul op

Signed-off-by: raver119 <raver119@gmail.com>

* few tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* additional INDArray.mmul signature

Signed-off-by: raver119 <raver119@gmail.com>

* make C order default for INDArray.mmul, unless both A/B have F order

Signed-off-by: raver119 <raver119@gmail.com>

* Nd4j.gemm validation fix

Signed-off-by: raver119 <raver119@gmail.com>

* disable mkldnn matmul for xxf with beta != 0 case

Signed-off-by: raver119 <raver119@gmail.com>

* SimpleRnn workspace fix + timeouts

Signed-off-by: Alex Black <blacka101@gmail.com>

* two more tests + minor fix in matmul platform check

Signed-off-by: raver119 <raver119@gmail.com>

* Flaky test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* propagate testresources profile

Signed-off-by: raver119 <raver119@gmail.com>

* Resources fix + flaky test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Oleg <oleg.semeniv@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-10 17:57:02 +03:00
Alex Black 99c727f15b
Add javacpp classifier dependency to nd4j-native and nd4j-cuda (#366)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-09 10:00:27 +10:00
Andrii T d86dd5b131
DL4J and SameDiff integration tests + LSTMLayer java op class (#353)
* init in this branch

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Lenetet Mnist workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small fix for calculations

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* for Alex to check placeholder null pointer issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* CNN3D workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* state for launching on dxg to regenterate dl4j examples

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* SD RNN test case workflow

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* small fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* checkpoint at lstmBlock: Input array 1 (x) rank must be got input with rank 2 issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fix LSTMLayer inputs order

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* lstm mismatch with c++ op issue

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer config draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayer config draft v2

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* have doubt I had to do this

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* NDRNN generated by codegen

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* LSTMLayerTestCases draft

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* minor fixes again

* added LSTMLayer testcases to nd4j-tests + setted Preconditions in LSTMLayer constructors

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* added lost SDCNNtestcases

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* overrided getNumOutputs from DynamicCustomOp in LSTMLayer and reorganized LSTMLayerOutputs according to cpp op

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished with LSTMLayerOutputs

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Fix MKLDNN platform checks (i.e., when MKLDNN can be used vs. not)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix LSTMLayerWeights input order

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* fixed LSTMLayer testcases

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished SameDiffRNNTestCase

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* finished all testcases + minor fixes

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Multiple generation-related fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix multiple issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* More fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* LSTM fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Regenerate ND4J namespaces and fix multiple issues

Signed-off-by: Alex Black <blacka101@gmail.com>

* changed SameDiffRNNTestCase

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* Small fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* added  Nd4j.getRandom().setSeed(12345) where needed

Signed-off-by: Andrii Tuzhykov <andrewtuzhykov@gmail.com>

* #8828 Fix ND4J profiler NaN/Inf checks when using OpContext

Signed-off-by: Alex Black <blacka101@gmail.com>

* #8828 Fix ND4J profiler NaN/Inf checks when using OpContext

Signed-off-by: Alex Black <blacka101@gmail.com>

* Tweak to weight init for SameDiff CNN test case

Signed-off-by: Alex Black <blacka101@gmail.com>

* Tweaks for test cases

Signed-off-by: Alex Black <blacka101@gmail.com>

* Ignore failing tests until fixed

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-09 00:20:48 +10:00
Samuel Audet ab083b9167
Update versions of JavaCPP Presets for OpenCV and MKL (#363)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2020-04-08 22:09:45 +10:00
raver119 e57f35c2e4 mkldnn version bump
Signed-off-by: raver119 <raver119@gmail.com>
2020-04-07 13:14:43 +03:00
raver119 04b2b4f9b6
Few fixes (#361)
* INDArray.close() fix for CPU

Signed-off-by: raver119 <raver119@gmail.com>

* - BroadcastableBoolOp introduced
- ConfusionMatrix now supports explicit DataType argument

Signed-off-by: raver119 <raver119@gmail.com>

* confusion_matrix: dtype is still optional

Signed-off-by: raver119 <raver119@gmail.com>

* disable bert tests in debug builds

Signed-off-by: raver119 <raver119@gmail.com>

* Affinity fix

Signed-off-by: raver119 <raver119@gmail.com>

* minor workspace tweak to allow close() on scoped out borrowed workspace

Signed-off-by: raver119 <raver119@gmail.com>
2020-04-06 21:01:59 +03:00
Alex Black 986ec4b51a
Add test from reported issue (confirmed fixed) (#359)
Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-06 15:02:09 +10:00
Chris Bamford 1a35ebec2e
RL4J: Add Backwardly Compatible Builder patterns (#326)
* Starting to switch configs of RL algorithms to use more fluent builder patterns. Many parameter choices in different algorithms default to SOTA and only be changed in specific cases

Signed-off-by: Bam4d <chris.bam4d@gmail.com>

* remove personal gpu-build file

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* refactored out configurations so they are heirarchical and re-usable, this is a step towards having a plug-and-play framework for different algorithms

* backwardly compatible configurations

* adding documentation to new configuration classes

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* private access modifiers are better suited here

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* RL4j does not compile without java 8 due to previous updates

fixing null pointers when listener arrays are empty

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* fixing copyright headers

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* uncomment logging line

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* fixing default value for learningUpdateFrequency

fixing test failure due to #352

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

Co-authored-by: Bam4d <chris.bam4d@gmail.com>
2020-04-06 12:36:12 +09:00
Alexander Stoyakin fb1c41c512
Build fix (#357) 2020-04-01 19:09:48 +11:00
Chris Bamford 8ac89aeb19
RL4J: Force shape fix (#352)
* fix edge case where input to network needs to have shape > 1

Signed-off-by: Bam4d <chrisbam4d@gmail.com>

* adding test for single dimension

Signed-off-by: Bam4d <chrisbam4d@gmail.com>
2020-04-01 14:28:01 +09:00
Yurii Shyrma 48102c61d0
- correct reshape op for empty shapes (#354)
* - correct reshape op for empty shape in case of -1 at the end

Signed-off-by: Yurii <iuriish@yahoo.com>

* Fix test + new reshape op constructor

Signed-off-by: Alex Black <blacka101@gmail.com>

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-01 15:13:34 +11:00
Alex Black 81ebfeead1
Small fixes (#355)
* #8787 DataVec test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* New nd4j test + fix bad datavec test

Signed-off-by: Alex Black <blacka101@gmail.com>

* #8745 Fix flaky arbiter test

Signed-off-by: Alex Black <blacka101@gmail.com>

* One more test

Signed-off-by: Alex Black <blacka101@gmail.com>
2020-04-01 15:11:39 +11:00
Alex Black 2dadbed861
Merge pull request #8815 from eclipse/ab_horschi_modelserializer
Model serializer - don't write temp file
2020-04-01 12:54:20 +11:00
Alexander Stoyakin 0a27e9f41d
Fix incompatibilities with generated code (#303)
* Cholesky fixed

* Constructors added

* MatMul wrapper

* Constructor added

* Missing wrappers added

* Generate Linalg namespace added

* Output data types

* Unit tests

* Added mmul

* Code generation

* Code generated

* Build fixed

* Fixing signatures

* Tests fixed

* Tests fixed

* Added enum

* Fix tests

* Some fixes

* Eye test fixed

* SameDiff: small fix for renameVariable - also replace variable name in lossVariable list if necessary

Signed-off-by: Alex Black <blacka101@gmail.com>

* Some fixes

* Tests fixed

* Revert wrong fix

* Some fixes

* Some fixes

* Extending base test class

* Added pad

* Fixed for generated signatures

* Fixes due to nd4j codegen

* Backwards compatibility fixes

* Fixed errors in tests, reverted wrong changes

* Test fixed

* Added missing operations used for nd4s operators

* Compilation fixed

* Added meshgrid

* Fixed constructors

* fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix bad commit (incorrectly reverted change from master)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fixed test

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-04-01 12:00:38 +11:00
Oleh 1d004b542a
xw_plus_b mkldnn implementation (#247)
* libnd4j first step of mkldnn for xw_plus_b and test of aurora crash in imageHelper

* libnd4j sync folders with master

* libnd4j merge master, raw implementation of xw_plus_b on mkldnn, clean up, need testing and adding checks for corresponded input shapes

* libnd4j corrections and checks added to xw_plus_b mkl

* libnd4j corrected dataType description based on mkl operation description, need more investigation

* libnd4j fixe xw_blus_b mkl implementation, need testing

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j two unit tests added

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j fixed check input dimensions bug

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libndj4 one more test added to cover different order handling

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j added optional int arg support to define weights format, if arg == 1, mkldnn (do not need transpose in mkldnn implementation), else mmul weights format, corrected check points, added unit test

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j merge master

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j some improvements to avoid NDArray transpose in xw_plus_b operation

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j fixed issues connected with weights rank, also added support of one case based on tf (for mkldnn, cpu, cuda), test case added

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j added proper handling of empty inputs (all implementations)

* libnd4j fixed compilation error

* libnd4j several more corrections after conflict solve and fixed typos

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j removed unsupported data types

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j merge master and fixed issues

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j added propagation implementation for xw_plus_b, fixed issue connected with mkl weights data format, avoided data copy in transpose mode, test cases added, manually tested with gradCheck

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j one minor fix of double operation declaration

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j code clean up

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j minor tests fixes

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

* libnd4j fixed build problem, integrate helpers changes

Signed-off-by: Oleg <oleg.semeniv@gmail.com>

Co-authored-by: raver119 <raver119@gmail.com>
2020-03-31 13:03:10 +03:00