Commit Graph

190 Commits (b85238a6df8ce72021b33f40e9beb99d9d3661a0)

Author SHA1 Message Date
Alex Black cb6654bebb Add libnd4j benchmarks (#3)
This PR adds 2 libnd4j benchmarking suits
2019-07-20 22:54:44 +10:00
Ryan Nett 62c6a73f9d Include TF Import tests as forward pass tests in OpValidation (#53)
* ignore multinomial

Signed-off-by: Ryan Nett <rnett@skymind.io>

* fix for @NonNull varargs

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:24:11 +10:00
Ryan Nett daf3950d8d SameDiff If, While, and Misc changes (#52)
* softmax and logSoftmax w/ dimension

Signed-off-by: Ryan Nett <rnett@skymind.io>

* start of while

Signed-off-by: Ryan Nett <rnett@skymind.io>

* if, start of javadocs

Signed-off-by: Ryan Nett <rnett@skymind.io>

* while foreward pass working, backprop WIP

Signed-off-by: Ryan Nett <rnett@skymind.io>

* no backprop

Signed-off-by: Ryan Nett <rnett@skymind.io>

* Tensorflow style if/while (& tests), name scope fixes (and test), argument interceptor (for if/while), use '_' in op names instead of ':'

Signed-off-by: Ryan Nett <rnett@skymind.io>

* javadoc

Signed-off-by: Ryan Nett <rnett@skymind.io>

* many fixes

Signed-off-by: Ryan Nett <rnett@skymind.io>

* many fixes

Signed-off-by: Ryan Nett <rnett@skymind.io>

* Some fixes

Signed-off-by: Ryan Nett <rnett@skymind.io>

* cleanup if condition doesn't return boolean

Signed-off-by: Ryan Nett <rnett@skymind.io>

* serialization fix

Signed-off-by: Ryan Nett <rnett@skymind.io>

* use constants instead of magic numbers

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:23:40 +10:00
Ryan Nett 2d991f5445 LeakyReLU fix (#55)
* LeakyReLU: Use serScalar to set alpha correctly in TF import
LogX: remove incorrect TF mapping
Pow: remove TF import method (no mapping)
BaseOp: remove duplicate extraArgs

Signed-off-by: Ryan Nett <rnett@skymind.io>

* un-ignore cifar-10 gan, as it is now passing

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:23:19 +10:00
Ryan Nett 027d4d2a47 Add XLNet to zoo model ignores (#54)
* ignore xlnet

Signed-off-by: Ryan Nett <rnett@skymind.io>

* comment

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:22:57 +10:00
Ryan Nett d7c261ec40 added ability to run as test, and comments (#44)
Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:22:34 +10:00
raver119 5708fc087a [WIP] INDArray hashCode() impl (#50)
* initial commit

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* one more initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* parallel hashCode prototype

Signed-off-by: raver119 <raver119@gmail.com>

* longBytes for hashCode

Signed-off-by: raver119 <raver119@gmail.com>

* INDArray hashCode java side

Signed-off-by: raver119 <raver119@gmail.com>

* few tests fixed for MSVC

Signed-off-by: raver119 <raver119@gmail.com>

* Small gradcheck validation util fix - hash names not SDVariables

Signed-off-by: Alex Black <blacka101@gmail.com>

* Small fix + ignore for logged issue

Signed-off-by: Alex Black <blacka101@gmail.com>

* - scrollable iterator fix
- sptree hashset replaced with collection

Signed-off-by: raver119 <raver119@gmail.com>

* hashcode exception removed

* int hashCode for java
2019-07-20 22:22:11 +10:00
Alex Black cc65c01118 Op validation improvements + encoding fix (#49)
* Op validation updates

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small logging fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #7986 Fix minor character encoding issues

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small ignore fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-07-20 22:21:46 +10:00
Alex Black 88ea9a49eb TF optional properties resolution fix (#48)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-07-20 22:21:20 +10:00
Ryan Nett 4b23dc01fd fix for #7976, update test comment (to OOM) (#45)
Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:20:58 +10:00
Alexander Stoyakin 9f401dc020 ND4S build fixed (#47)
Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>
2019-07-20 22:20:36 +10:00
raver119 15e7984392 Small fixes (#46)
* - fix for eclipse/#7959

* createFromNpy scalar fix

Signed-off-by: raver119 <raver119@gmail.com>

* remove spam

Signed-off-by: raver119 <raver119@gmail.com>

* rng custom ops tests

Signed-off-by: raver119 <raver119@gmail.com>

* Numpy headers validation + tests

Signed-off-by: raver119 <raver119@gmail.com>

* fix for scalar string flat serde

Signed-off-by: raver119 <raver119@gmail.com>

* Where empty shape test

Signed-off-by: raver119 <raver119@gmail.com>
2019-07-20 22:20:14 +10:00
Ryan Nett c135883162 [WIP] Ignored TF OP Import test updates (#37)
* new issue for b_d_s (old is closed), new issue for boolean_mask (old is fixed)

Signed-off-by: Ryan Nett <rnett@skymind.io>

* date update

Signed-off-by: Ryan Nett <rnett@skymind.io>

* "embedding_lookup/.*multiple.*" is passing

Signed-off-by: Ryan Nett <rnett@skymind.io>

* new bincount issue (old was fixed)

Signed-off-by: Ryan Nett <rnett@skymind.io>

* date update

Signed-off-by: Ryan Nett <rnett@skymind.io>

* where op comment

Signed-off-by: Ryan Nett <rnett@skymind.io>

* where passing

Signed-off-by: Ryan Nett <rnett@skymind.io>

* ignore gan

Signed-off-by: Ryan Nett <rnett@skymind.io>

* scatter_nd/rank2shape_2indices passing

Signed-off-by: Ryan Nett <rnett@skymind.io>

* topk update, test fix (need to verify)

Signed-off-by: Ryan Nett <rnett@skymind.io>

* batch_to_space issue update

Signed-off-by: Ryan Nett <rnett@skymind.io>

* updates, no change

Signed-off-by: Ryan Nett <rnett@skymind.io>

* date updates

Signed-off-by: Ryan Nett <rnett@skymind.io>

* batch_to_space is fixed

Signed-off-by: Ryan Nett <rnett@skymind.io>

* equality function helper

Signed-off-by: Ryan Nett <rnett@skymind.io>

* topk passing

Signed-off-by: Ryan Nett <rnett@skymind.io>

* dropout equality check

Signed-off-by: Ryan Nett <rnett@skymind.io>

* adding ignores for zoo models

Signed-off-by: Ryan Nett <rnett@skymind.io>

* ignore libnd4j rnn tests

Signed-off-by: Ryan Nett <rnett@skymind.io>

* added batch to space test

Signed-off-by: Ryan Nett <rnett@skymind.io>

* issue comments

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:19:48 +10:00
raver119 dde50ee570 Small fixes (#43)
* minor cmake changes to make macos happy

* space_to_batch/batch_to_space validation fix

* - choose op tweaks
- tests updated to match appleid tweaks

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - get rid of bad import

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - choose now uses shape function
- choose test updated
2019-07-20 22:19:20 +10:00
Alex Black 8b4b8977a0 Batch Mmul v2 import fix (#41)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-07-20 22:18:52 +10:00
Ryan Nett d854e28b34 System info export for debugging and bug reporting (#34)
* System info export for debugging and bug reporting

Signed-off-by: Ryan Nett <rnett@skymind.io>

* class name fix

Signed-off-by: Ryan Nett <rnett@skymind.io>

* add version information, pointer memory info

Signed-off-by: Ryan Nett <rnett@skymind.io>

* add nvidia-smi and nvcc info

Signed-off-by: Ryan Nett <rnett@skymind.io>

* line cleanup

Signed-off-by: Ryan Nett <rnett@skymind.io>

* nvidia-smi run works

Signed-off-by: Ryan Nett <rnett@skymind.io>

* add oshi dependency

Signed-off-by: Ryan Nett <rnett@skymind.io>

* use OS info, add workspaces info

Signed-off-by: Ryan Nett <rnett@skymind.io>

* use ServiceLoader to load GPU information

Signed-off-by: Ryan Nett <rnett@skymind.io>

* register service

Signed-off-by: Ryan Nett <rnett@skymind.io>

* moved service out of NativeOpsHolder (private constructor)

Signed-off-by: Ryan Nett <rnett@skymind.io>

* added newline

Signed-off-by: Ryan Nett <rnett@skymind.io>

* added license

Signed-off-by: Ryan Nett <rnett@skymind.io>

* and one more

Signed-off-by: Ryan Nett <rnett@skymind.io>

* copyright update

Signed-off-by: Ryan Nett <rnett@skymind.io>

* fixes

Signed-off-by: Ryan Nett <rnett@skymind.io>

* removed unused imports

Signed-off-by: Ryan Nett <rnett@skymind.io>

* removed more unused imports

Signed-off-by: Ryan Nett <rnett@skymind.io>

* close streams

Signed-off-by: Ryan Nett <rnett@skymind.io>

* and another one

Signed-off-by: Ryan Nett <rnett@skymind.io>

* use method

Signed-off-by: Ryan Nett <rnett@skymind.io>

* one more copyright

Signed-off-by: Ryan Nett <rnett@skymind.io>

* remove double license

Signed-off-by: Ryan Nett <rnett@skymind.io>

* moved test to correct package

Signed-off-by: Ryan Nett <rnett@skymind.io>

* classpath update

Signed-off-by: Ryan Nett <rnett@skymind.io>

* classpath for java >8 fix

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:18:24 +10:00
Alex Black e67e25ba30 libnd4j: use buildnativeoperations.sh parallel build by default in maven (#38)
* Libnd4j: parallel build in maven by default

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* build-helper-maven-plugin - version property not hardcoded version

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-07-20 22:17:57 +10:00
Alex Black 79d1f02ee1 DL4J BERT iterator: Add option to prepend token (#39)
* Add option to prepend token

Signed-off-by: Alex Black <blacka101@gmail.com>

* Javadoc

Signed-off-by: Alex Black <blacka101@gmail.com>
2019-07-20 22:17:08 +10:00
Ryan Nett ef8cb55b7b Nd4j: Change array args to vararg and add toString options (#36)
* changed [] to ...

Signed-off-by: Ryan Nett <rnett@skymind.io>

* added randn(long seed, int... shape)

Signed-off-by: Ryan Nett <rnett@skymind.io>

* Fixed a couple of methods

Signed-off-by: Ryan Nett <rnett@skymind.io>

* ToString methods w/ options

Signed-off-by: Ryan Nett <rnett@skymind.io>

* fixes, less toString methods, and a few ops I missed

Signed-off-by: Ryan Nett <rnett@skymind.io>

* some javadocs, change int... to long... where possible

Signed-off-by: Ryan Nett <rnett@skymind.io>

* another javadoc

Signed-off-by: Ryan Nett <rnett@skymind.io>

* javadoc fix

Signed-off-by: Ryan Nett <rnett@skymind.io>

* just javadoc in INDArray

Signed-off-by: Ryan Nett <rnett@skymind.io>

* local/static fix

Signed-off-by: Ryan Nett <rnett@skymind.io>

* Add @NonNull to options

Signed-off-by: Ryan Nett <rnett@skymind.io>

* javadoc updates/fixes

Signed-off-by: Ryan Nett <rnett@skymind.io>

* more @NonNulls

Signed-off-by: Ryan Nett <rnett@skymind.io>

* even more @NonNulls, this time on varargs

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:16:13 +10:00
Ryan Nett 366d850f5e Add placeholder value for keras learning phase if required (#30)
* automatically add placeholders for keras_learning_phase if required

Signed-off-by: Ryan Nett <rnett@skymind.io>

* comment

Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 22:03:31 +10:00
Alex Black c28372cb49 BERT tokenization fixes (#35)
* Add composite token preprocessor

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix case issue with bert tokenization

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-07-20 22:00:40 +10:00
Ryan Nett cc6063402e added keepDims to INDArray methods (#33)
Signed-off-by: Ryan Nett <rnett@skymind.io>
2019-07-20 21:58:06 +10:00
Robert Altena 0ef373fe45 Nd4j.java javadoc (#7997)
* up to line 500.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 1120.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

*  up to 1286.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 2400.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

*  up to line 2500.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 3600.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

*  up to line 4000.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 4500.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 5000.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* up to line 6000.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-07-20 12:05:46 +10:00
Alexandre Boulanger ee6aae268f RL4J refac: Added some observation transform classes (#7958)
* Added observation classes and tests

Signed-off-by: unknown <aboulang2002@yahoo.com>

* Now uses DataSetPreProcessors

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* CompositeDataSetPreProcessor can now stop processing on empty dataset; Some DataSetPreProcessors moving from RL4J to ND4J

Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>

* Did requested minor changes

Signed-off-by: Alexandre Boulanger <Alexandre.Boulanger@ia.ca>
Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com>
2019-07-20 10:28:20 +10:00
Samuel Audet 9bb11d5b06
Fix OpenMP by replacing pragmas with macros (#8026)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2019-07-18 20:13:56 +09:00
jxtps 5a0904cb4c Fix getMemoryReport ArrayIndexOutOfBoundsException (#8015)
Fix Upsampling.getMemoryReport ArrayIndexOutOfBoundsException, see https://github.com/eclipse/deeplearning4j/issues/8014
2019-07-17 12:23:37 +10:00
Samuel Audet deba1e0df9 Update dependencies to just released JavaCPP and JavaCV 1.5.1 (#8004)
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2019-07-14 21:07:33 +03:00
raver119 85e212fece
exclude memory tracker for android/ios/macos platforms (#8005)
Signed-off-by: raver119 <raver119@gmail.com>
2019-07-11 18:28:19 +03:00
Robert Altena 0c48e55f91 Remove duplicate code for append() and prepend() Fix #7983 (#7987)
* fix #7983

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* small fix.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix javadoc.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-07-09 13:58:50 +10:00
Robert Altena 595656d01e fix #7947 (#7985)
Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-07-08 16:35:21 +10:00
Robert Altena 66b7c3d6e3 Quick start ND4J (#7916)
* New clean and signed attempt.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* WIP.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP.'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP.'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'WIP'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* 'Ready.'

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* small edits.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-07-02 13:43:07 +10:00
Nikolay Ivankov c8c98e1af2 SameDiff variables documentation (#7949)
* SameDiff variables documentation

Signed-off-by: Nikolay Ivankov <kolya.ivankov@gmail.com>

* Minor changes in variables.md

Signed-off-by: Nikolay Ivankov <kolya.ivankov@gmail.com>
2019-07-01 13:17:54 +10:00
Alex Black 1170827c18 Merge master to upstream (#7945)
* Shugeo strided slice zeros (#14)

* Modified strided_slice op to properly work with empty-like shapes.

* Fixed test for reduce_mean with empty-like input.

* [WIP] Last merge (#15)

* correct logsoftmax looss (#2)

* Small SameDiff listener fix (#4)

* Various fixes (#6)

* #7839 Fix for asXMatrix and tests

* #7866 EmbeddingSequenceLayer dtype fix + test

* #7856 SameDiff save/load stream methods

* #7859 RegressionEvaluation rank 4 fix + tests + axis configuration

* EvaluationBinary 3d/4d

* More evaluation 3d/4d tests

* #7847 Evaluation empty checks

* Small test ifx

* #7848 Fix median edge case

* Improve DL4J samediff layer tests

* [WIP] FastText wrapper implemented (#8)

* FastText implemented

* Some fixes

* Fix shapes for wordsNearest

* Validation of input vectors

* Fixes

* Fixed test

* Thread tagged

* Some tweaks

* setContextClassLoader for DeallocatorServiceThread

* Numpy format tests (#1)

* Various fixes (#11)

* #7852 SameDiff gather fix

* #7892 SameDiff placeholder to constant conversion

* #7890 validate input rank for MLN/CG init methods

* Fix broken permute shape calculation

* Permute and gather fixes

* Tests

* #7850 LogSumExp fix + test

* Handful of test fixes

* Empty arrays with non-scalar shapes (#10)

* minor rearrangements for lambdas

* empty tensors with non-scalar shapes

* numpy empty tensors with non-scalar shapes

* few more empty tweaks

* Small fixes

* conv3d signature update

* micro fix in batchnorm mkldnn

* Import fixes

* Fix

* MKL-DNN update

* Small fill fix

* fill with empty input + test

* Fixes

* Small error improvement

* Fix

* one special test

* couple of fixes for lstm

* Rewrite TFGraphMapper.getNDArrayFromTensor to be maintainable and less error prone

* Fixes

* FP16

* Unsigned

* BFloat16

* Fill op - empty tweaks

* - couple of fixes for empty arrays construction
- stack updated

* strided slice fix

* one transform test

* provide method for reducing shapeInfo in case of input array is empty

* Fixed reduceAlongDimensions to use empty input properly.

* couple of broadcast tests

* couple of tests broadcast tests + tweak to make them pass

* add check of non-empty to methods producing sub-arrays

* Fixed reshapeC with zeros in shape.

* complete empty check in reduce_... legacy ops

* Concat and cumsum/prod

* Tweak to empty shape inference on import

* add empty check to the rest of reduce legacy ops

* one more test

* correct typo in evalReduceShapeInfoEmpty

* Added tests for reduce_* ops to tests with zero shapes.

* few more tests for empty reductions

* Fixed strided_slice op with empty case and tests.

* one more empty reduction test

* Fixed strided_slice test.

* add empty check to NDArray::reshapei

* infOrMax

* empty min/max with infinity tests

* made unstack working correctly with empty arrays

* few IndexReduce tests + tweaks for empty shapes

* add test for empty concat

* few tests fixed

* Validation fix for reductions on empty shapes

* Reverse fix

* Reduction shape calc fixes

* SameDiff.generateOutputVariable: don't use shape function to determine number of outputs

* Range fix

* - NDArray constructor updated for scalars/empty arrays
- few tests fixed

* More fixes

* Empty creator fixes

* concat fix

* concat fix

* TF import tests: allow 'both all NaN' and 'both all inf' to pass

* Slice, zero fraction, and reshape fixes

* transpose, gather

* Zero fraction

* scalar cast fix

* Empty reduction axis support

* few more tests fixed

* Fixed input checks conforming with TF for concat op and tests.

* few tests fixed

* matmul scalar shape fix

* Fixed checkout for data type and scalarity with concat to allow non-empty scalars with vector concats.

* broadcast bool fix

* few more tests

* few more tests

* correct evalReduceShapeInfoEmpty

* argmax/argmin + tests

* one more empty edge case + one more test

* argmax/argmin/realdiv_bp tweaks

* empty reshape test + fix

* Helper fixes

* Small fixes

* Gather test fix

* Gather test fix

* Small fixes

* reduce scalar zero values

* scalar mean workaround

* Remove debug code

* along dim mean workaround

* one more test

* - equalsTo() tweak for empty arrays
- one more test

* broadcast tweaks

* [WIP] Fixing outstanding issues for NLP (#9)

* Avoid using not-inited objects

* Test fixed.

* Redundant method avoided for models like FastText

* KMeans++ implementation

* KMeans++ implementation

* Disable parallel execution

* KMeans++

* Tests

* Dev branch merge (#16)

* SameDiff: convertDataType and gradient check util improvements (#12)

* GradCheck util improvements

* StopGradient constructor + test

* SameDiff: Add datatype conversion

* Javadoc and add DataType.isNumerical()

* Small fix

* Fix SameDiff TF import test cases intermediate naming (workaround for bad default)

* TFGraphTestAllHelper: check intermediates in execution order

* Add missing debug listener

* [WIP] lstmBlock fix + other changes (#13)

- fixes lstmBlock issue
- changes NDArray method reshape(), permute(), transpose() by making them return instance instead of pointer
- CheckNumerics op
- fixes for ReduceBool IsInfOrNan & IsFinite

* Small test fix

* CheckNumerics op wrapper

* Fix some issues on master (#17)

* Fix DataVec test issue

* Fix issue with dl4j SameDiff output layer

* Dtype fix for lambda layers

* #7912 BertIterator dtype fix (use float32 not global default)

* [WIP] Next set of CUDA stuff (#7)

New CUDA implementations and improvements

* bad file

* Dev branch master merge (#23)

* SameDiff: convertDataType and gradient check util improvements (#12)

* GradCheck util improvements

* StopGradient constructor + test

* SameDiff: Add datatype conversion

* Javadoc and add DataType.isNumerical()

* Small fix

* Fix SameDiff TF import test cases intermediate naming (workaround for bad default)

* TFGraphTestAllHelper: check intermediates in execution order

* Add missing debug listener

* [WIP] lstmBlock fix + other changes (#13)

- fixes lstmBlock issue
- changes NDArray method reshape(), permute(), transpose() by making them return instance instead of pointer
- CheckNumerics op
- fixes for ReduceBool IsInfOrNan & IsFinite

* Small test fix

* CheckNumerics op wrapper

* Compatibility of deserialization (#18)

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* SameDiff: add activation gradient checking support for debugging (#19)

* SameDiff gradient checker: first pass on activation gradient checks

* Fixes + tests for activation gradient checking

* Javadoc

* [WIP] Some nd4j data type corrections (#20)

* Adjust data type

* Set correct Data type.

* Size of proper data type.

* fix averaged cpu load (#22)

* SameDiff ops, TF import and fixes (#24)

* CheckNumerics tests + fixes + misc fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fake quant

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* FakeQuantWithMinMaxArgs

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* CheckNumerics fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix libnd4j ALL_INTS and ALL_FLOATS declaration (uint and bfloat types)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Javadoc

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Exception tweak

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix for out of scope stack allocated var use

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Ignores

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Ignore for known failing test (already logged issue)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Merge upstream to fork (#25)

* Add thousand-separator commas to TotalParams (#7915)

* Add thousand-separator commas to TotalParams

The number of parameters can be quite large, and it would help the reading of the summary printout to have the TotalParams column & values at the bottom have thousand-separator-commas in them.

* Add thousand-separator commas to MultiLayerNetwork

Corresponding change to MultiLayerNetwork

Signed-off-by: Jxtps Jxtps <jxtps435@gmail.com>

* Update contributing and issue/PR templates (#7934)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix link to AdaDelta paper (#7942)

Fix link to AdaDelta paper hosted on matthewzeiler.com

Signed-off-by: Jxtps

* Fixes, and ignores for known/logged failing issues (#7943)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* SameDiff + DL4J/SameDiff: Multiple fixes (#28)

* #7919 HDF5 attribute buffer length fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #7909 Arbiter constructor exception ux improvements

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #7925 RNN output layer length checks

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #7939 Add listener for validating inputs are not incorrectly modified

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #7939 Integrate NonInplaceValidationListener into tests

* #7844 DL4J SameDiff fixes for variable minibatch size

* DL4J SameDiff fixes - ensure gradient for input placeholder is available

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Tweaks to ExternalErrorsFunction - use placeholders, make more robust

* Another fix

* More fixes

* More SameDiff/DL4J fixes

* Scope out scalar array creation in BaseScalarOp

* Remove debug code

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* [WIP] Final dev branch merge (#29)

* SameDiff: convertDataType and gradient check util improvements (#12)

* GradCheck util improvements

* StopGradient constructor + test

* SameDiff: Add datatype conversion

* Javadoc and add DataType.isNumerical()

* Small fix

* Fix SameDiff TF import test cases intermediate naming (workaround for bad default)

* TFGraphTestAllHelper: check intermediates in execution order

* Add missing debug listener

* [WIP] lstmBlock fix + other changes (#13)

- fixes lstmBlock issue
- changes NDArray method reshape(), permute(), transpose() by making them return instance instead of pointer
- CheckNumerics op
- fixes for ReduceBool IsInfOrNan & IsFinite

* Small test fix

* CheckNumerics op wrapper

* Compatibility of deserialization (#18)

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* SameDiff: add activation gradient checking support for debugging (#19)

* SameDiff gradient checker: first pass on activation gradient checks

* Fixes + tests for activation gradient checking

* Javadoc

* [WIP] Some nd4j data type corrections (#20)

* Adjust data type

* Set correct Data type.

* Size of proper data type.

* fix averaged cpu load (#22)

* [WIP] Multiple dataset iterators (#27)

* Splitting dataset into arbitrary number

* Fixes

* Multiple split of iterator

* Test

* Test

* Some fixes

* signature change

* one more tweak

Signed-off-by: raver119 <raver119@gmail.com>

* one more test for sequential use of DataSetIteratorSplitter

Signed-off-by: raver119 <raver119@gmail.com>

* Fixes

* Fixes

* one more test for Alexander

Signed-off-by: raver119 <raver119@gmail.com>

* Some fixes

* Some fixes

* one more test for Alexander

Signed-off-by: raver119 <raver119@gmail.com>

* minor test fix

Signed-off-by: raver119 <raver119@gmail.com>

* Some fixes

* Some fixes

* couple of assertions tweaked

Signed-off-by: raver119 <raver119@gmail.com>

* MDS splitter test :/

Signed-off-by: raver119 <raver119@gmail.com>

* Minor refactoring

* Multi dataset

* Some fixes

* More tests

* Small number of test fixes/improvements (failures on CI) (#31)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* [WIP] More CUDA stuff (#26)

* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* LRN BP CUDA

Signed-off-by: raver119 <raver119@gmail.com>

* less memory

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed bug with crop_and_resize op helper.

* get rid of unnecessary index-calculation dunction

Signed-off-by: Yurii <yurii@skymind.io>

* Fixed sort with nth_element cuda-based helper.

* Refactored nth_element.

* Refactored nth_element op and tests.

* Modified usage of dim array with sortTad routine.

* Refactored main routine of helper for non_max_image_suppression op.

* non_max_image_suppression op helper with cuda kernel implementation. Initial revision.

* fix vol2col cuda kernel

* meh

Signed-off-by: raver119 <raver119@gmail.com>

* topK concept

Signed-off-by: raver119 <raver119@gmail.com>

* unsorted topK with scanWitdh of 1

Signed-off-by: raver119 <raver119@gmail.com>

* correct vol2col tests

* sorted/unsorted topK

Signed-off-by: raver119 <raver119@gmail.com>

* implementation and fixing col2im/col2vol

* Corrected usage flags with input/output with reverse op.

* dup is const now

Signed-off-by: raver119 <raver119@gmail.com>

* percentile op

Signed-off-by: raver119 <raver119@gmail.com>

* group tests for mapool2d

Signed-off-by: Yurii <yurii@skymind.io>

* special test for george

Signed-off-by: raver119 <raver119@gmail.com>

* less threads for sortTad

Signed-off-by: raver119 <raver119@gmail.com>

* provide conv2d for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* remove auther in sort tad kernel code

Signed-off-by: Yurii <yurii@skymind.io>

* provide depthwise_conv2d for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* - max_pooling_with_argmax
- null check for special use

Signed-off-by: raver119 <raver119@gmail.com>

* dts cuda

Signed-off-by: raver119 <raver119@gmail.com>

* provide sconv2d for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* std cuda

Signed-off-by: raver119 <raver119@gmail.com>

* Refactored non_max_suppression op to conform TF implementation.

* Improved suppression helper.

* provide pooling3d for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* minor lstm rearrangements

Signed-off-by: raver119 <raver119@gmail.com>

* more of minor lstm rearrangements

Signed-off-by: raver119 <raver119@gmail.com>

* (bi)dynamic_rnn

Signed-off-by: raver119 <raver119@gmail.com>

* templates init order

Signed-off-by: raver119 <raver119@gmail.com>

* Refactored non_max_suppression op.

* Added cuda kernel for non_max_suppression.

* CPU sort by key/value

Signed-off-by: raver119 <raver119@gmail.com>

* CPU sort TAD by key/value

Signed-off-by: raver119 <raver119@gmail.com>

* CPU sort TAD by key/value tests

Signed-off-by: raver119 <raver119@gmail.com>

* Eliminate compiler error with cuda implementation.

* - repaired gradCheck in cuda
- provide conv2d_bp for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* missed signature

Signed-off-by: raver119 <raver119@gmail.com>

* provide depthwise_conv2d_bp for cuda

Signed-off-by: Yurii <yurii@skymind.io>

* Implementation of lup helper with cuda kernel. Initial commit.

* further work on backprops for convolutions

Signed-off-by: Yurii <yurii@skymind.io>

* CUDA linear sort by key/val

Signed-off-by: raver119 <raver119@gmail.com>

* CUDA tad sort by key/val

Signed-off-by: raver119 <raver119@gmail.com>

* start providing of backprop for pooling2d/3d

Signed-off-by: Yurii <yurii@skymind.io>

* Added atomicAdd for bool datatype.

* dynamic partition concept

Signed-off-by: raver119 <raver119@gmail.com>

* dynamic partition concept

Signed-off-by: raver119 <raver119@gmail.com>

* dynamic partition scalar CUDA

Signed-off-by: raver119 <raver119@gmail.com>

* important comment

Signed-off-by: raver119 <raver119@gmail.com>

* fix pooling2d/3d backprop helpers

Signed-off-by: Yurii <yurii@skymind.io>

* Added non-linear test with dynamic_partition.

* Improved test for dynamic_partition.

* dynamic_partition TAD concept

Signed-off-by: raver119 <raver119@gmail.com>

* - dynamic_partition TAD CUDA impl
- dynamic_partition TAD CPU fix

Signed-off-by: raver119 <raver119@gmail.com>

* - rewrite cpu code for usampling2d/3d
- write cuda code for usampling2d/3d

Signed-off-by: Yurii <yurii@skymind.io>

* dynamic_stitch CUDA vector case

Signed-off-by: raver119 <raver119@gmail.com>

* dynamic_stitch CUDA TAD case concept

Signed-off-by: raver119 <raver119@gmail.com>

* dynamic_stitch CUDA TAD case impl

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for dynamic_stitch 3D-4D cases.

* minor tests tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed type check for dynamic stitch.

* min/max bp

Signed-off-by: raver119 <raver119@gmail.com>

* rewrite code for upsampling2d/3d cpu

Signed-off-by: Yurii <yurii@skymind.io>

* reduce min/max/norm_max bp

Signed-off-by: raver119 <raver119@gmail.com>

* lup implementation. Additional enhancements.

* provide code for upsamling2d/3d backprop

Signed-off-by: Yurii <yurii@skymind.io>

* weightedCrossEntropyWithLogits

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed template math atomicMul for 64bit ints.

* Refactored dynamic_partition_bp op.

* inverseBroadcast fix

Signed-off-by: raver119 <raver119@gmail.com>

* DynamicPartitionBP test datatype fixed.

* - nd4j_atomicMul Windows fix
- cpu/NDArrayLambda.hpp excluded from CUDA

Signed-off-by: raver119 <raver119@gmail.com>
2019-06-27 18:37:04 +03:00
Alex Black cae4fc9760
Fixes, and ignores for known/logged failing issues (#7943)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-06-25 12:07:28 +10:00
jxtps e2db385682 Fix link to AdaDelta paper (#7942)
Fix link to AdaDelta paper hosted on matthewzeiler.com

Signed-off-by: Jxtps
2019-06-25 10:50:17 +10:00
Alex Black d3a9e5b41d
Update contributing and issue/PR templates (#7934)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-06-22 16:21:27 +10:00
jxtps 2a1ea24781 Add thousand-separator commas to TotalParams (#7915)
* Add thousand-separator commas to TotalParams

The number of parameters can be quite large, and it would help the reading of the summary printout to have the TotalParams column & values at the bottom have thousand-separator-commas in them.

* Add thousand-separator commas to MultiLayerNetwork

Corresponding change to MultiLayerNetwork

Signed-off-by: Jxtps Jxtps <jxtps435@gmail.com>
2019-06-19 13:22:07 +10:00
Alex Black 68ea5f3688
Dev branch merge: dev_20190606 (#7904)
* correct logsoftmax looss (#2)

* Small SameDiff listener fix (#4)

* Various fixes (#6)

* #7839 Fix for asXMatrix and tests

* #7866 EmbeddingSequenceLayer dtype fix + test

* #7856 SameDiff save/load stream methods

* #7859 RegressionEvaluation rank 4 fix + tests + axis configuration

* EvaluationBinary 3d/4d

* More evaluation 3d/4d tests

* #7847 Evaluation empty checks

* Small test ifx

* #7848 Fix median edge case

* Improve DL4J samediff layer tests

* [WIP] FastText wrapper implemented (#8)

* FastText implemented

* Some fixes

* Fix shapes for wordsNearest

* Validation of input vectors

* Fixes

* Fixed test

* Thread tagged

* Some tweaks

* setContextClassLoader for DeallocatorServiceThread

* Numpy format tests (#1)

* Various fixes (#11)

* #7852 SameDiff gather fix

* #7892 SameDiff placeholder to constant conversion

* #7890 validate input rank for MLN/CG init methods

* Fix broken permute shape calculation

* Permute and gather fixes

* Tests

* #7850 LogSumExp fix + test

* Handful of test fixes

* Empty arrays with non-scalar shapes (#10)

* minor rearrangements for lambdas

* empty tensors with non-scalar shapes

* numpy empty tensors with non-scalar shapes

* few more empty tweaks

* Small fixes

* conv3d signature update

* micro fix in batchnorm mkldnn

* Import fixes

* Fix

* MKL-DNN update

* Small fill fix

* fill with empty input + test

* Fixes

* Small error improvement

* Fix

* one special test

* couple of fixes for lstm

* Rewrite TFGraphMapper.getNDArrayFromTensor to be maintainable and less error prone

* Fixes

* FP16

* Unsigned

* BFloat16

* Fill op - empty tweaks

* - couple of fixes for empty arrays construction
- stack updated

* strided slice fix

* one transform test

* provide method for reducing shapeInfo in case of input array is empty

* Fixed reduceAlongDimensions to use empty input properly.

* couple of broadcast tests

* couple of tests broadcast tests + tweak to make them pass

* add check of non-empty to methods producing sub-arrays

* Fixed reshapeC with zeros in shape.

* complete empty check in reduce_... legacy ops

* Concat and cumsum/prod

* Tweak to empty shape inference on import

* add empty check to the rest of reduce legacy ops

* one more test

* correct typo in evalReduceShapeInfoEmpty

* Added tests for reduce_* ops to tests with zero shapes.

* few more tests for empty reductions

* Fixed strided_slice op with empty case and tests.

* one more empty reduction test

* Fixed strided_slice test.

* add empty check to NDArray::reshapei

* infOrMax

* empty min/max with infinity tests

* made unstack working correctly with empty arrays

* few IndexReduce tests + tweaks for empty shapes

* add test for empty concat

* few tests fixed

* Validation fix for reductions on empty shapes

* Reverse fix

* Reduction shape calc fixes

* SameDiff.generateOutputVariable: don't use shape function to determine number of outputs

* Range fix

* - NDArray constructor updated for scalars/empty arrays
- few tests fixed

* More fixes

* Empty creator fixes

* concat fix

* concat fix

* TF import tests: allow 'both all NaN' and 'both all inf' to pass

* Slice, zero fraction, and reshape fixes

* transpose, gather

* Zero fraction

* scalar cast fix

* Empty reduction axis support

* few more tests fixed

* Fixed input checks conforming with TF for concat op and tests.

* few tests fixed

* matmul scalar shape fix

* Fixed checkout for data type and scalarity with concat to allow non-empty scalars with vector concats.

* broadcast bool fix

* few more tests

* few more tests

* correct evalReduceShapeInfoEmpty

* argmax/argmin + tests

* one more empty edge case + one more test

* argmax/argmin/realdiv_bp tweaks

* empty reshape test + fix

* Helper fixes

* Small fixes

* Gather test fix

* Gather test fix

* Small fixes

* reduce scalar zero values

* scalar mean workaround

* Remove debug code

* along dim mean workaround

* one more test

* - equalsTo() tweak for empty arrays
- one more test

* broadcast tweaks
2019-06-15 21:34:34 +10:00
Alex Black 32e5cc1945
First round of runtime test improvements (#7875)
* Capsnet test runtime improvements

* Slow test speedups

* Next round of test speed improvements

* More test improvements

* Improve test speed

* Next round of test speedups

* Another round

* More test speedups

* Another round

* Another round of test speedups

* Another round of speedups...

* CuDNN test speedups + more tests extending BaseDL4JTest

* Minor fix + more BaseDL4JTest use in other modules
2019-06-13 20:40:40 +10:00
skymindops b5f0ec072f Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00