Commit Graph

1014 Commits (a1fcc5f19f0f637e83252b00982b3f12b401f679)

Author SHA1 Message Date
Alex Black 4bcc1eee11
Merge pull request #8529 from KonduitAI/ab_jackson
nd4j-jackson: exclude java.xml.stream.XML*Factory from service loader
2019-12-13 22:13:47 +11:00
AlexDBlack 0ab39a2274 nd4j-jackson: exclude java.xml.stream.XML*Factory from service loader to avoid clashes with other non-shaded jackson etc on classpath
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-13 21:41:28 +11:00
Alex Black 228d06fbb0
Merge pull request #8526 from KonduitAI/master
Merge
2019-12-12 18:20:40 +11:00
Fariz Rahman 709295281e suppress err (#123) 2019-12-12 09:40:29 +03:00
Alex Black c0163f6e01
Merge pull request #8518 from KonduitAI/master
Update master
2019-12-10 15:28:21 +11:00
AlexDBlack 0df1b46c8c Merge 2019-12-10 15:08:50 +11:00
Alex Black 4920f22fff
Check for empty streams for NativeImageLoader + test (#121)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-10 12:11:05 +11:00
raver119 a5f5ac72b1
reduce bool changes (#118)
* reduce bool changes

Signed-off-by: raver119 <raver119@gmail.com>

* reduce bool tweaks

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-09 20:08:59 +03:00
Yurii Shyrma 425c747330 - permute threadsPerBlock and blocksPerGrid in signature of launching of cuda kernel for trueBroadcast op (#120)
Signed-off-by: Yurii <iuriish@yahoo.com>
2019-12-09 20:08:36 +03:00
Alex Black 0175ace4c3
Small tweaks (#119)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-09 23:08:00 +11:00
raver119 fc334ff47a Merge remote-tracking branch 'origin/master' 2019-12-09 11:17:31 +03:00
raver119 ee5d25caa9 cuda broadcast exec fix
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-09 11:17:16 +03:00
Alexander Stoyakin 927d591421 ResizeBicubic added (#117)
* ResizeBicubic added
Some fixes.

* Test fixed

* Narrowed argument type changed to boolean

* Clean up
2019-12-09 18:25:39 +11:00
raver119 cea68c18f1 cuda broadcast fix
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-09 09:27:50 +03:00
raver119 ae7933a428 cpu truebroadcast fix
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-09 08:01:12 +03:00
Alex Black b66154a9d4
Add ArraySavingListener for debugging (#114)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-09 14:16:11 +11:00
raver119 70e08c3a6c reshape validation fix
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-06 21:19:33 +03:00
raver119 b32dd1bf92
[WIP] resize_bicubic types (#116)
* resize_bicubic: allow more dtypes

Signed-off-by: raver119 <raver119@gmail.com>

* resize_bicubic: allow less dtypes

Signed-off-by: raver119 <raver119@gmail.com>

* Refactored resize_bicubic op to full conform with TF1.5 and tests.

* Corrected test to proper data type output.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Corrected double input test to float constant outputs.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Finished with correction of tests for bicubic interpolated resizes expected.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Fixed adjust_contrast ops to allow non-RGB inputs.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Refactored adjust_contrast_v2 to conform with TF one.

Signed-off-by: shugeo <sgazeos@gmail.com>

* AdjustContrast tests activated

* two typos fixed

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-06 18:58:37 +03:00
raver119 972fae60dc
Update master (#8511)
* cleaned up bert iterator tests (#110)

Signed-off-by: eraly <susan.eraly@gmail.com>

* Various pre-release fixes (#111)

* Various fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix default dtypes for MaxPoolWithArgmax

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small pre-release tweak (#112)

* Log UI address on launch as in previous Play-based UI

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Logging level tweak for UI

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* http not https

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* datavec python ensure host (#113)

* ensure host

* one more host ensure

* info->debug

* [WIP] reverse improvements (#115)

* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* reverse draft

Signed-off-by: raver119 <raver119@gmail.com>

* reverse kernel

Signed-off-by: raver119 <raver119@gmail.com>

* reverse kernel

Signed-off-by: raver119 <raver119@gmail.com>

* 2 micro fixes

Signed-off-by: raver119 <raver119@gmail.com>

* Shugeo resize fix5 (#102)

* Refactored resize images ops to use TF-like bool args as input.

* Refactored helpers for cpu implementation of resize_bilinear and resize_nearest_neighbor ops.

* Refactored cuda implementation for image.resize_bilinear and image.resize_nearest_neighbor ops helpers.

* Refactored nearest_neighbor resize op.

* Added a pair of tests for special case of resize_bilinear algorithm.

* Fixed issue with resize_bilinear op.

* Refactored cpu implementation for helpers with resize_nearest_neighbor op.

* Final fixed for resize ops to conform TF v.1.5

* Refactored cuda helpers for resize_neares_neighbor op.

* Fixed resize_bilinear to accept proper data.

* Fixed issue with non-float input for resize_bilinear op.

* Refactored cuda helper for resize_bilinear to proper process non-float inputs.

* Added tests for resize_bilinear to int inputs.

* Fixed ResizeBilinear wrapper

* Tests fixed

* Fixed float and bool constant to avoid overflow for some kind of compilers.

* Corrected float constants with float data type.

* Added f suffix for float constants.

* Corrected float constant to avoid overflow with initializing lists.

* Corrected float initializing list with float input.

* Corrected bool constant with initalizing list.

* Corrected float and bool values with initializing lists.

* Fixed wrong constant.

* Fixed issue with 1x1 input picture for resize.

* ResizeBilinear default values on import fix

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-06 11:10:44 +03:00
Philip Khor a6223d307b Minor edits to README for pydatavec and pydl4j (#8336)
* Restore badges for PyPi and Apache license; and edit links. Removed badge for build status as build status for Deeplearning4j overall is not meaningful here. Java-Python coffee image removed as we (probably) don't want to be pointing to the old repo.

Apache LICENSE file added for pydatavec as it was not previously included.

Signed-off-by: Philip Khor <philip.khor@skymind.my>

* move badges to top for consistency

Signed-off-by: Philip Khor <philip.khor@skymind.my>

* some typos

Signed-off-by: Philip Khor <philip.khor@skymind.my>

* Add gitter chat link to be consistent with jumpy README

Signed-off-by: Philip Khor <philip.khor@skymind.my>
2019-12-06 08:10:38 +01:00
Robert Altena e7730eded4 delete unused and refactor. (#8262)
Signed-off-by: Robert Altena <Rob@Ra-ai.com>
2019-12-05 22:25:41 -05:00
shugeo e09a785232 Shugeo resize fix5 (#102)
* Refactored resize images ops to use TF-like bool args as input.

* Refactored helpers for cpu implementation of resize_bilinear and resize_nearest_neighbor ops.

* Refactored cuda implementation for image.resize_bilinear and image.resize_nearest_neighbor ops helpers.

* Refactored nearest_neighbor resize op.

* Added a pair of tests for special case of resize_bilinear algorithm.

* Fixed issue with resize_bilinear op.

* Refactored cpu implementation for helpers with resize_nearest_neighbor op.

* Final fixed for resize ops to conform TF v.1.5

* Refactored cuda helpers for resize_neares_neighbor op.

* Fixed resize_bilinear to accept proper data.

* Fixed issue with non-float input for resize_bilinear op.

* Refactored cuda helper for resize_bilinear to proper process non-float inputs.

* Added tests for resize_bilinear to int inputs.

* Fixed ResizeBilinear wrapper

* Tests fixed

* Fixed float and bool constant to avoid overflow for some kind of compilers.

* Corrected float constants with float data type.

* Added f suffix for float constants.

* Corrected float constant to avoid overflow with initializing lists.

* Corrected float initializing list with float input.

* Corrected bool constant with initalizing list.

* Corrected float and bool values with initializing lists.

* Fixed wrong constant.

* Fixed issue with 1x1 input picture for resize.

* ResizeBilinear default values on import fix

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-05 22:05:33 +03:00
raver119 6a3c046ffd 2 micro fixes
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-05 20:44:11 +03:00
raver119 355c6b6096
[WIP] reverse improvements (#115)
* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* reverse draft

Signed-off-by: raver119 <raver119@gmail.com>

* reverse kernel

Signed-off-by: raver119 <raver119@gmail.com>

* reverse kernel

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-05 20:03:10 +03:00
Fariz Rahman 0e8a4f77bc datavec python ensure host (#113)
* ensure host

* one more host ensure

* info->debug
2019-12-05 23:27:32 +11:00
Samuel Audet e51e6ebfd2 Update CMake toolchains for more recent versions of Android NDK (#8502) 2019-12-05 12:46:01 +02:00
Alex Black ef4d3ffee8
Small pre-release tweak (#112)
* Log UI address on launch as in previous Play-based UI

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Logging level tweak for UI

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* http not https

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-05 20:59:46 +11:00
Maxime Michel ca20763619 Mention the new % unit for maxBytes and maxPhysicalBytes in Memory management documentation (#8435) (#8461)
Signed-off-by: Maxime Michel <maxime.michel@magnolia-cms.com>
2019-12-05 12:47:53 +09:00
Alex Black 2052ce7026
Various pre-release fixes (#111)
* Various fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix default dtypes for MaxPoolWithArgmax

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-05 14:20:03 +11:00
Susan Eraly 63ed202057 cleaned up bert iterator tests (#110)
Signed-off-by: eraly <susan.eraly@gmail.com>
2019-12-05 13:24:37 +11:00
Alex Black 9592072cef
Merge pull request #8457 from chentao106/Fix#8383
Fix#8383,getMostPopulatedClusters should sort by cluster's points siz…
2019-12-05 12:09:38 +11:00
Alex Black 3275fe35a3
Merge pull request #8495 from KonduitAI/master
Update master
2019-12-05 11:05:44 +11:00
Alex Black 91de96588c
BertIterator sentence pair support for supervised training (#108)
* bert iterator sentence pair handling

Signed-off-by: eraly <susan.eraly@gmail.com>

* bert iterator sentence pair handling -seg

Signed-off-by: eraly <susan.eraly@gmail.com>

* bert iterator sentence pair handling tests

Signed-off-by: eraly <susan.eraly@gmail.com>

* test with pairs long done

Signed-off-by: eraly <susan.eraly@gmail.com>

* more tests with bert iter sent pairs done

Signed-off-by: eraly <susan.eraly@gmail.com>

* fixed copyright, formatting

Signed-off-by: eraly <susan.eraly@gmail.com>

* bert iterator - added featurizer for sentence pair inference

Signed-off-by: eraly <susan.eraly@gmail.com>

* bert iterator - finished tests

Signed-off-by: eraly <susan.eraly@gmail.com>

* bert iterator - finished tests, polish

Signed-off-by: eraly <susan.eraly@gmail.com>

* collection labeled sentence provider

Signed-off-by: eraly <susan.eraly@gmail.com>

* lombok fix for pojo class

Signed-off-by: eraly <susan.eraly@gmail.com>

* java doc misc clean up

Signed-off-by: eraly <susan.eraly@gmail.com>

* Private access modifiers

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-12-04 23:35:38 +11:00
Alex Black 9cc8803b8d
DL4J + Keras import: Causal Conv1D support (#107)
* Keras causal conv1d support first steps

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add tests

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Causal conv mode

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Gradient check and fixes for causal conv1d

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Fix Conv1D import and testing

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small keras test fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* Don't allow setting causal convolution mode to conv2d/3d layers

Signed-off-by: Alex Black <blacka101@gmail.com>

* More robustly infer nIn for recurrent layers for ambiguous NCW and NWC cases

Signed-off-by: Alex Black <blacka101@gmail.com>

* Polish and cleanup

Signed-off-by: Alex Black <blacka101@gmail.com>
2019-12-04 22:52:06 +11:00
Alex Black 578a5abb68 DNNL/MKLDNN dilated causal conv1d + betainc (#103)
* - add padding calculation in same mode in causal conv1d op for right mkl paddings

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct causal condition in mkldnnUtils.cpp

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct some code which caused additional round errors is betainc op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - put float in place of template parameter in nan assign in betainc op

Signed-off-by: Yurii <iuriish@yahoo.com>
2019-12-04 14:50:17 +03:00
raver119 cb18d3d996
allow MKL-DNN on non-AVX machines (#104)
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-04 09:11:37 +03:00
Fariz Rahman 0d14032d26 TF Updates (#87)
* tf updates

* pom

* copyright

* graphrunner tests

* gpu test

* getSessionOptionsConfigProto

* dtype fix

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* cast graphs

* savemodel test fix

* testresource instead of local

* Logging level

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* gson dependency issue fix; fix GraphRunnerTest for no session options config case

Signed-off-by: Alex Black <blacka101@gmail.com>

* Final tweaks

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* few minor fixes

Signed-off-by: raver119 <raver119@gmail.com>

* one more fix

Signed-off-by: raver119 <raver119@gmail.com>

* Tweak configuration for GraphRunnerTest

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* nd4j align config

* tf warmup
2019-12-04 17:11:03 +11:00
shugeo 190575196c Refactored pad and mirror_pad ops to conform with TF. (#100) 2019-12-03 15:06:38 +03:00
raver119 d8339246d9 fix typo in test
Signed-off-by: raver119 <raver119@gmail.com>
2019-12-03 10:23:19 +03:00
Yurii Shyrma 1f5e15b541 Shyrma adjust (#98)
* - add possibility of passing scalar-array as input parameter for scale factor in adjust hue/contrast/saturation ops
- correct typo in function which calculates regularized incomplete beta integral

Signed-off-by: Yurii <iuriish@yahoo.com>

* - fix bug in betainc cuda kernel

Signed-off-by: Yurii <iuriish@yahoo.com>

* - start working on implementation of digamma function

Signed-off-by: Yurii <iuriish@yahoo.com>

* - further work on digamma function (cpu)

Signed-off-by: Yurii <iuriish@yahoo.com>

* - testing and fixing bugs in digamma op

Signed-off-by: Yurii <iuriish@yahoo.com>

* - make correction n cuda kernel for polyGamma

Signed-off-by: Yurii <iuriish@yahoo.com>

* - remove unnecessary stuff from betaInc cuda kernel

Signed-off-by: Yurii <iuriish@yahoo.com>

* - resolve conflicts in DeclarableOpsTests3.cpp after master branch has been merged

Signed-off-by: Yurii <iuriish@yahoo.com>

* - restore id number of Not opertion in legacy_ops.h

Signed-off-by: Yurii <iuriish@yahoo.com>

* - correct padding calculation in mkl dnn conv1d causal

Signed-off-by: Yurii <iuriish@yahoo.com>

* restore empty check in adjust_contrast_v2

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-03 09:40:45 +03:00
shugeo 1e9ff114aa Shugeo atomic tests (#97)
* Added atomic tests for atomicAdd, atomicSub and atomicDiv.

* Fixed atomicAdd for 16bit ints.

* Fixed atomicMul for 16 floats.

* Eliminated waste prints.

* Fixed problems with double type on matrix inverse helepers.

* Eliminated commented wrong code.

* Refactored atomicMul for 16bit types.

* few more minor tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed fake_quant_with_min_max_vars_per_channel args processing.
2019-12-02 21:40:54 +03:00
raver119 25b3cd9b80
[WIP] CUDA tests (#95)
* one more CI test

Signed-off-by: raver119 <raver119@gmail.com>

* export additional symbols

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* one more tweak for linux

Signed-off-by: raver119 <raver119@gmail.com>

* fix dtype in few tests

Signed-off-by: raver119 <raver119@gmail.com>

* missing sync and memset in couple of tests

Signed-off-by: raver119 <raver119@gmail.com>

* copy step for libnd4j cuda

Signed-off-by: raver119 <raver119@gmail.com>

* no-op on empty for adjust hue/contrast/saturation

Signed-off-by: raver119 <raver119@gmail.com>

* CUDA_VERBOSE Off

Signed-off-by: raver119 <raver119@gmail.com>

* BroadcastBool fix + few tests

Signed-off-by: raver119 <raver119@gmail.com>

* trigger jenkins

Signed-off-by: raver119 <raver119@gmail.com>

* trigger jenkins

Signed-off-by: raver119 <raver119@gmail.com>

* - ignore couple of warnings
- remove redundant compiler options

Signed-off-by: raver119 <raver119@gmail.com>
2019-12-02 21:37:21 +03:00
Alexander Stoyakin 5e152c0d9a TF import tests - adding missing operations (#65)
* Add and fix mappings.

* Intermediate

* Added and fixed some mappings

* Added op

* Missing constructors added.

* Added new mappings

* SDImage wrappers and minor tweaks.

* Added missing constructor

* Some corrections

* Cleanup

* Small fixes

* Ops wrappers

* Minor fixes.

* Max Pooling

* MaxPoolWithArgmax

* Some fixes

* Ignores for failures

* Some ops fixed.

* Some fixes

* Missing package added

* Some fixes

* Ignored tests fixed.

* Some fixes

* Merge master

* bitcast fix

Signed-off-by: raver119 <raver119@gmail.com>

* Bitcast fixed
2019-12-02 21:23:06 +11:00
Fariz Rahman 1adc25919c Python updates (#86)
* python updates

* fix cyclic deps

* konduit updates

* konduit updates

* fix list

* fixes

* sync pyvars test

* setuprun comments

* Version fix, other module test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* bug fix using advanced hacking skillzz
2019-12-02 19:20:23 +11:00
Alex Black 8123d9fa9b
SameDiff: Add Java-level assertion check/exception (#96)
Signed-off-by: Alex Black <blacka101@gmail.com>
2019-12-02 18:07:54 +11:00
raver119 4ada65b384
[WIP] MSVC-related tests fixes (#88)
* fix narrowing down cast

Signed-off-by: raver119 <raver119@gmail.com>

* trigger jenkins

Signed-off-by: raver119 <raver119@gmail.com>

* few more fixes for MSVC and Windows

Signed-off-by: raver119 <raver119@gmail.com>

* few more fixes for MSVC and Windows

Signed-off-by: raver119 <raver119@gmail.com>

* few more fixes for MSVC and Windows

Signed-off-by: raver119 <raver119@gmail.com>

* few more fixes for MSVC and Windows

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks
- tensormmul dtype validation

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks
- batched gemm dtype validation

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - few more tweaks

Signed-off-by: raver119 <raver119@gmail.com>
2019-11-30 16:02:07 +03:00
Alex Black 2be47082c9
#8470 TrainingConfig json fix for Evaluation instances (#93)
Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-30 20:08:30 +11:00
Alex Black 35ab4a72ba
TF import test resources loading precision fixes (#92)
* Fix precision issues when loading from CSV

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small tweak

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-30 18:58:37 +11:00
Alex Black 4fb9fa7748
Add ND4J namespaces (#83)
* Add NDValidation

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add bitwise namespace

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Math namespace op constructor fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Constructor fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add Math namespace

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Update NDBitwise

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add random namespaces

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Update

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* NN namespace

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2019-11-30 18:39:32 +11:00
shugeo dc66a52bc7 [WIP] Shugeo release fixes4 (#91)
* Fixed fake_quant_with_min_max_vars op.

* Refactored bitcast op.

* bad linspace removed

Signed-off-by: raver119 <raver119@gmail.com>

* Corrected tests for bitcast op.

* Eliminated debug prints.

* one fix

Signed-off-by: raver119 <raver119@gmail.com>

* one fix

Signed-off-by: raver119 <raver119@gmail.com>

* Added a pair of comments.
2019-11-29 16:05:08 +03:00