Commit Graph

1493 Commits (9aa56f27f15292f0d6845c789f655d3aa6778aea)

Author SHA1 Message Date
Yurii Shyrma 7a90a31cfb
Shyrma deconv3 (#69)
* - profiling cuda kernels for vol2col and im2col

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct addBias helper

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct mkl dilation formula and switch off mkl api for dilation deconvolutions

Signed-off-by: Yurii <iuriish@yahoo.com>
2019-11-21 21:17:30 +02:00
Samuel Audet ff73e6da3f ND4J: Fix OpenBLAS loading for nd4j-native (#64)
* ND4J: Fix OpenBLAS loading for nd4j-native and remove bundling of OpenMP

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>

* Bundle back libgomp.so.1 for Linux

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>

* Readd preload directories for ARM

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>

* Add back preloads for GCC on Windows

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>

* Add explicit preloadpaths for ARM and POWER to bundle correct library

Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2019-11-21 15:54:41 +03:00
raver119 064a56ccf1
Few fixes (#66)
* skip legacy transforms execution in case of empty input arrays

Signed-off-by: raver119 <raver119@gmail.com>

* - BroadcastBool ops now accept extraParams to make MatchCondition possible
- TrueBroadcastHelper now uses samediff::threads

Signed-off-by: raver119 <raver119@gmail.com>

* java side

Signed-off-by: raver119 <raver119@gmail.com>

* trigger jenkins

Signed-off-by: raver119 <raver119@gmail.com>

* update LessThanOrEqual opNum mapping

Signed-off-by: raver119 <raver119@gmail.com>

* update LessThanOrEqual opNum mapping

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-21 15:43:03 +03:00
raver119 83cb0d9329
[WIP] Create and small fix (#67)
* - create op
- skip exec for empty inputs for non_max_suppression
- EmptyHandling idea

Signed-off-by: raver119 <raver119@gmail.com>

* Create op and mapping for it

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-21 13:31:20 +03:00
shugeo dc0036f2c6
Shugeo image resize bicubic (#56)
* Added implementation files for image_resize and resize_bicubic ops.

* Image resize and image.resize_bicubic ops implementation. Initial revision.

* Finished with infrastructure development for image.resize_bilinear op and image_resizo op implementation.

* Refactored resize methods.

* Added processing for Mitchelcubic algorithm.

* Added check for input/output sizes.

* Added int and float types for crop_and_resize op.

* Refactored crop_and_resize output type check.

* Added helper for bicubic interpolation as TF v.1 does.

* Added TF v.1 bicubic helper for cuda platform.

* Added cached class for bicubic algorithm.

* Refactored cuda implementation for crop_and_resize helper to use proper output type.

* Added facilities for bicubic interpolation.

* Portion bicubic interpolation from TF.

* Added tests for resize_bilinear testing.

* Working implementation of bicubic interpolation and tests.

* Refactored routines with image_resize bicubic op helper.

* Refactored code with coding standards.

* Refactored cpu helpers for resize_bicubic op.

* Refactored bicubic helpers.

* Added bicubic resize facilities.

* Implementing cuda kernels for bicubic interpolation. Implementation step.

* Cuda implementation of resize_bicubic op helper.

* Refactor image.resize_bicubic op helpers.

* Refactored helpers for resize_bicubic. Added error checking with cuda implementation.

* Refactored cuda implementation of resize_bicubic op helper. The first working revision.

* Cuda arch implementation for resize_bicubic op helper. Full working single-threaded revision.

* Intermediate bicubic interpolation helper for cuda.

* Refactored cpu helper for resize_bicubic.

* Multithreaded cuda implementation for resize_bicubic.

* Fixed merge issues.

* Refactored nlp helpers.

* Replicated resize_bicubic for 3D also.

* Eliminated waste comments of unused code.

* Eliminated waste comments with unused code.

* Eliminated waste template definitions.

* Eliminated waste debug code.

* Eliminated waste comments.

* Fixed multithreading with helpers.

* Fixed test suites for float and double in float point input lists.

* Fixed usage of reshape with 3D/4D on resizes.

* Final fixes.

* Fixed resize_neighbor op problem.
2019-11-20 21:11:04 +02:00
shugeo 13e5c0a280
Shugeo release fix1 (#61)
* Added a pair of tests for failed ops.

* Fixed cpu helper for draw_bounding_boxes op.

* Refactored implementation of draw_bounding_boxes op to full conform with TF.

* Improved multithreading with draw_bounding_boxes op cuda helper.

* Eliminated log messages.

* Changed logging with draw_bounding_boxes op helper and tests.

* Resize_biliear with 3D input allowed.

* Refactored 3D input acception with resize_bilinear op.

* And another improvement.

* Refactored reshape of input/output for resize_bilinear.

* Improvements final.

* Finished with 3D replication for image.resize_bilinear/_nearest_neighbor.

* Added copyrights for TF code.

* Using new form of multithreading for cpu implementation.

* Fixed shape error.

* Added multithreaded with batches on crop_and_resize functor.

* Refactored multithreading with crop_and_resize and draw_bounding_boxes.
2019-11-20 13:37:48 +02:00
raver119 59e955cedc
- MKL-DNN version upgrade to 1.1.x (#62)
- MKL-DNN namespace changes to match DNNL rename

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-20 13:23:08 +03:00
raver119 7898f3c0cc
fix for is_increasing/non_decreasing ops for empty input case (#63)
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-20 11:12:15 +03:00
Max Pumperla d9e9733387
fix pydatavec for python 3... and python2 install problems (#8422) 2019-11-20 08:20:04 +01:00
raver119 3f38900c33
J9+ -> J8 ByteBuffer fix (#59)
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-20 07:43:17 +03:00
Fariz Rahman 630409cd53 Fixes Keras import issue (8373) (#54)
* test

* fix

* rem prn

* add test
2019-11-20 13:19:04 +11:00
Yurii Shyrma 66b84b38cf
Shyrma mmul (#58)
* - get rid of some copy procedures in mmulHelper ops

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on embedding cuda api for batched gemm (cublasGemmBatchedEx) in our mmulHelper class

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on cuda batched gamm api

Signed-off-by: Yurii <iuriish@yahoo.com>

* - write own cuda kernel performing batched gemm

Signed-off-by: Yurii <iuriish@yahoo.com>

* missing include in MmulHelper

Signed-off-by: raver119 <raver119@gmail.com>

* - forgot to keep in code previous correct kernels for mmulNxN, since it may happen that new onw will fail for some reason in future

Signed-off-by: Yurii <iuriish@yahoo.com>

* disable old tensordot

Signed-off-by: raver119 <raver119@gmail.com>

* - rewrite cuda kernels for usualGemm and usualGemv

Signed-off-by: Yurii <iuriish@yahoo.com>

* - profiling mmul helpers

Signed-off-by: Yurii <iuriish@yahoo.com>

* - prints to check shapes were added

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct type of output array Cin mmulNxN

Signed-off-by: Yurii <iuriish@yahoo.com>

* - take into account possible nans in C array

Signed-off-by: Yurii <iuriish@yahoo.com>

* slightly change numThreads message

Signed-off-by: raver119 <raver119@gmail.com>

* - make corrections in accordance to given notes in pr review

Signed-off-by: Yurii <iuriish@yahoo.com>
2019-11-19 15:39:36 +02:00
Alex Black da1944e8e1
SameDiff TF import (#49)
* Added implementation files for image_resize and resize_bicubic ops.

* Image resize and image.resize_bicubic ops implementation. Initial revision.

* Minor fix

* Some TF imports disabled.

* Finished with infrastructure development for image.resize_bilinear op and image_resizo op implementation.

* Refactored resize methods.

* Added processing for Mitchelcubic algorithm.

* adjust_contrast

* Small fix for TF import expected value loading when variable name starts with the test name

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Tests

* Tests added.

* Removed tf names absent in mapping.

* Some fixes.

* Small fixes

* Minor change

* Some failing tests.

* Disable failed test

* Ignore some tests

* Fix import class mapping

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix float property mapping (flatbuffers)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Override equality function for model 'dropout'

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fail tests

* Failed tests ignored temporarily.

* Minor fixes

* Small fix

* Conflict resolved

* Default implementations of tensorflowName and onnxName
2019-11-19 22:44:29 +11:00
raver119 ce2ef20f96 additional reverse signatures
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-19 13:54:07 +03:00
raver119 bbd59a3537
fake quant dtype validation fix (#60)
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-19 12:53:52 +03:00
Alex Black 8f96f71f2b
Mist gradient check (#57)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-19 00:12:59 +11:00
raver119 db7ca956c5
[WIP] Mish (#55)
* Mish activation function and its derivative

Signed-off-by: raver119 <raver119@gmail.com>

* signature fix

Signed-off-by: raver119 <raver119@gmail.com>

* mish as activation for dl4j

Signed-off-by: raver119 <raver119@gmail.com>

* javadoc

Signed-off-by: raver119 <raver119@gmail.com>

* minor optimization

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-18 13:21:26 +03:00
raver119 29990b1214
[WIP] Clang/macOS fixes (#8412)
* build fix for clang

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* [WIP] clang for jcpp (#53)

* clang as compiler for jcpp

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* we don't need macos profile

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-17 13:45:58 +03:00
raver119 c5cbdcd8f4
[WIP] clang for jcpp (#53)
* clang as compiler for jcpp

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* we don't need macos profile

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-17 09:45:30 +03:00
raver119@gmail.com 9101a0ee15 build fix for clang
Signed-off-by: raver119@gmail.com <raver119@gmail.com>
2019-11-16 22:18:50 +03:00
AlexDBlack da4bf0209b Merge
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-16 23:21:45 +11:00
Alex Black a856922fe9
#8409 Fix compgraph backprop issue with dual embedding layers from single input (#52)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-16 23:09:41 +11:00
Alex Black 09a827fb6d
Fixes and pre-release QA (#51)
* #8395 Keras import - support scaled identity weight init

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More Keras scaled weight init fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #8352 Deprecate duplicate SamplingDataSetIterator class

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Remove /O2 optimization for faster CUDA build

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Tweak regression test precision for CUDA

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix edge cases for buffer creation

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Update MKLDNN validation tests to new helper enable/disable settings

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Delete debugging class

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* MKLDNN test - add proper skip for CUDA backend

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Align WeightInitUtil with weight init classes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix for SameDiff test layers weight init when using IWeightInit classes

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-16 17:04:29 +11:00
raver119 1780dcc883
[WIP] Small fixes here and there (#50)
* one range test

Signed-off-by: raver119 <raver119@gmail.com>

* few Context convenience singatures

Signed-off-by: raver119 <raver119@gmail.com>

* one more range test

Signed-off-by: raver119 <raver119@gmail.com>

* "range" "fix"

Signed-off-by: raver119 <raver119@gmail.com>

* adjuct_contrast_v2 now allows scale factor to be provided via input_variable

Signed-off-by: raver119 <raver119@gmail.com>

* adjust_contrast now allows scale factor as variable too

Signed-off-by: raver119 <raver119@gmail.com>

* bitcast shape tests

Signed-off-by: raver119 <raver119@gmail.com>

* BitCast import dtype added

Signed-off-by: raver119 <raver119@gmail.com>

* few more BitCast signatures

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-15 17:04:29 +03:00
Serhii Shepel d7718c28fb
Drop unused profiles for artifacts distribution i8388 (#48)
* Drop unused profiles for artifacts distribution

* Drop skymindnexus-skil profile
2019-11-15 09:56:28 +02:00
Yurii Shyrma 62d8e0d409 - make agreement between our and mkl api dilation/padding formulas (#47)
Signed-off-by: Yurii <iuriish@yahoo.com>
2019-11-14 20:21:22 +03:00
raver119 cab90d3dc7 fix preload
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-14 20:00:24 +03:00
raver119 c5b912bddf
few changes for openblas and jcpp preloads (on macos) (#46)
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-14 19:50:24 +03:00
raver119 1eb3de90d7
[WIP] Platform helpers switches (#44)
* - platform helpers can be disabled on per-op basis now via Context::allowHelpers
- java has access to it as well

Signed-off-by: raver119 <raver119@gmail.com>

* global platform-helpers trigger

Signed-off-by: raver119 <raver119@gmail.com>

* few signatures renamed

Signed-off-by: raver119 <raver119@gmail.com>

* - few new env variables to follow
- maxThreads/masterThreads differentiation

Signed-off-by: raver119 <raver119@gmail.com>

* Javadoc update

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-14 14:35:02 +03:00
Alex Black 47d19908f4
Various fixes (#43)
* #8172 Enable DL4J MKLDNN batch norm backward pass

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #8382 INDArray.toString() rank 1 brackets / ambiguity fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #8308 Fix handful of broken links (inc. some in errors)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Unused dependencies, round 1

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Unused dependencies, round 2

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Unused dependencies, round 3

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Uniform distribution TF import fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-14 19:38:20 +11:00
raver119 6de00bf75f
[WIP] Weekly update of repo (#8390)
* [WIP] Fix compilation after nd4j changes (#37)

* Fix compilation.

* Some tests fixed

* Disable tests temporarily.

* Restored test

* Tests restored.

* Test restored.

* [WIP] perf tests (#40)

* special maxpool test

Signed-off-by: raver119 <raver119@gmail.com>

* special maxpool test

Signed-off-by: raver119 <raver119@gmail.com>

* Shyrma bnorm bp (#41)

Batchnorm backprop mkldnn

* Add SameDiff memory reuse memory manager (array cache) (#39)

* Attention op comments

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* ArrayCacheMemoryMgr - first pass

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Tweak array cache for use with SameDiff identity arrays

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* ArrayCacheMemoryMgr javadoc and properly get max memory

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* LRU cache policy + add tests

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Resize arrays internally if required for ArrayCacheMemoryMgr

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Test improvement

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small polish

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* SameDiff op runtime benchmarking listener (#42)

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* INLINE_LOOPS for windows

Signed-off-by: raver119 <raver119@gmail.com>

* [WIP] ThreadPool (#8)

This PR removes OpenMP use in 95% of cases
2019-11-13 17:15:18 +03:00
raver119 48df1acdfb
[WIP] ThreadPool (#8)
This PR removes OpenMP use in 95% of cases
2019-11-13 17:04:59 +03:00
raver119 f05c6ee139 INLINE_LOOPS for windows
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-12 15:12:31 +03:00
Alex Black 1d96bb9e6e
SameDiff op runtime benchmarking listener (#42)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-12 22:51:09 +11:00
Alex Black 18c01f5bdc
Add SameDiff memory reuse memory manager (array cache) (#39)
* Attention op comments

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* ArrayCacheMemoryMgr - first pass

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Tweak array cache for use with SameDiff identity arrays

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* ArrayCacheMemoryMgr javadoc and properly get max memory

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* LRU cache policy + add tests

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Resize arrays internally if required for ArrayCacheMemoryMgr

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Test improvement

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small polish

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-12 21:15:44 +11:00
Yurii Shyrma 0eda1e733e Shyrma bnorm bp (#41)
Batchnorm backprop mkldnn
2019-11-12 11:58:48 +03:00
Robert Altena c0f91e5c3c removing obsolete nd4j docs. (#8380)
Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-11-12 13:56:57 +11:00
raver119 cd961727bb
[WIP] perf tests (#40)
* special maxpool test

Signed-off-by: raver119 <raver119@gmail.com>

* special maxpool test

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-11 17:45:59 +03:00
Robert Altena 4cb281c01f Samediff import doc (#8301)
* first commit.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fix review items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* update reviewed items.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-11-11 15:34:12 +11:00
Alexander Stoyakin 6958f2ba24
[WIP] Fix compilation after nd4j changes (#37)
* Fix compilation.

* Some tests fixed

* Disable tests temporarily.

* Restored test

* Tests restored.

* Test restored.
2019-11-08 10:25:44 +02:00
AlexDBlack 0107fb10ab Merge remote-tracking branch 'konduit/master' 2019-11-08 18:11:45 +11:00
Alex Black 2f84ea666d
Uniform distribution op tweaks + 'specified output dtype' constructor (#38)
* Uniform distribution op tweaks + 'specified output dtype' constructor

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Validation tweak

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-08 18:08:25 +11:00
raver119 929c1dc5c7 - new NDArrayFactory scalar constructor
- minor tweak in randomuniform
- one more test

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-08 08:49:41 +03:00
raver119 51f3a1371d
[WIP] Random Uniform (#36)
* args

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* T args

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-07 17:09:47 +03:00
Alex Black 24980efde3
Fix LogSumExp along dimension (#35)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-07 23:36:15 +11:00
shugeo 679e42199a Shugeo strided slice bp fix2 (#33)
* Fixed crash and restored brocken functionality for strided slice.

* Added comments for strided_slice_bp main step.
2019-11-07 13:44:02 +03:00
Samuel Audet 73b5a508fc Update dependencies to just released JavaCPP and JavaCV 1.5.2
Signed-off-by: Samuel Audet <samuel.audet@gmail.com>
2019-11-07 17:57:34 +09:00
raver119 4276e63054 one more test
Signed-off-by: raver119 <raver119@gmail.com>
2019-11-07 08:49:27 +03:00
longzhendong 52c9918c6f Testing slice and concat (#8362) 2019-11-07 14:47:37 +11:00
shugeo 08853c7829
Shugeo random uniform int (#30)
* Corrected randomuniform declaration.

* Refactored uniform distribution for both cuda and cpu platforms.

* Refactored uniform distribution and tests.

* Fixed type usage with indices.

* Refactored uniform distribution implementation and tests to full conform with TF implementation.

* Refactored gamma function to use type util method.

* Copyright changes and fixes with ConstantHelper.

* Added error checking on allocate cuda device memory and operations.
2019-11-06 12:49:27 +02:00