cavis/nd4s
Alexander Stoyakin 10d676e0b8 [WIP] nd4s - Scala operators for SameDiff (#113)
* Jar packaging for maven

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Typo fixed

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* minimal viable prototype for SD

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Tests corrected

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* few fixes for bfloat16 in java and cpp (#114)

Signed-off-by: raver119 <raver119@gmail.com>

* Nd4j refactoring (#112)

* refactoring

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* wip

* fix: make test public.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* make test public.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* fixes read refactoring.

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Enabled test

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test copied from nd4j

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* [WIP] bitwise ops (#115)

* - cyclic_shift_bits + test
- shift_bits + test

Signed-off-by: raver119 <raver119@gmail.com>

* OMP_IF replacement

Signed-off-by: raver119 <raver119@gmail.com>

* Thin wrapper added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Cleanup

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Shugeo cuda tests (#116)

* Added tests for get_seed/set_seed ops.

* Added missed tests for scatter_sub/mul/div ops.

* Added tests for hardsigmoid and hardsigmoid_bp.

* Added tests for hardtanh and hardtanh_bp ops.

* Added test for histogram op.

* Added tests for identity op.

* Refactored mergemaxindex op. Added tests for log1p,mergemaxindex, mod and mod_bp ops.

* Fixed tests for FloorDiv.

* Added test for rank op.

* Added tests for rationaltanh/rationaltanh_bp ops.

* Added tests for realdiv/realdiv_bp.

* Added tests for rectifiedtanh/_bp ops.

* Added tests for shapes_of op.

* Added tests for shapes_of op.

* Added tests for size op.

* Added tests for softplus/_bp ops.

* Added tests for softsign/_bp ops.

* Added tests for toggle_bits op. Fixed processing of OP_IMPL and so on defititions.

* Added test for truncatediv op.

* Added another test for truncatediv op.

* Added another test for histogram.

* Added tests for unstack_list op.

* Refactored to_int32/uint32/float16/float32/double/int64/uint64 ops and tests.

* Refactored mergemaxindex op helper for cuda platform and tests.

* Fixed cuda kernel for histogram op helper.

* Refactor skipgram to avoid early buffers shift.

* Fixed check up with non_max_suppression op cuda helper. Added cuda kernel implementation for skipgram op helpers.

* Added implementation of skipgram op helper for cuda platform. Working revision

* Fixed mergeMaxIndex kernel and move it to separate source file.

* Adding arithmetic

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Eliminated memory leaks and dropped waste prints with tests. (#117)

* Added tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* fix test

Signed-off-by: raver119 <raver119@gmail.com>

* no openmp for ClipByGlobalNorm

Signed-off-by: raver119 <raver119@gmail.com>

* Stubs for ops

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* [WIP] right shift ops (#118)

* right shift ops

Signed-off-by: raver119 <raver119@gmail.com>

* typo

Signed-off-by: raver119 <raver119@gmail.com>

* rotr test

Signed-off-by: raver119 <raver119@gmail.com>

* fix: IOException no longer thrown by read(). (#120)

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Small fix in TensorflowConversion class (#121)

Signed-off-by: Alex Black <blacka101@gmail.com>

* Shyrma concat2 (#119)

* - rewrite/improve concat

Signed-off-by: Yurii <yurii@skymind.io>

* - ged rid of unnecessary argument in concat kernel

Signed-off-by: Yurii <yurii@skymind.io>

* InferenceSession additional validation for shape calc (#122)

Signed-off-by: Alex Black <blacka101@gmail.com>

* [WIP] build fix (#124)

* AffinityManager changes

Signed-off-by: raver119 <raver119@gmail.com>

* build fixes

Signed-off-by: raver119 <raver119@gmail.com>

* OP/CONFIGURABLE_OP shapefn fix (#125)

Signed-off-by: raver119 <raver119@gmail.com>

* Some ops added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Nd4j refactoring (last one!) (#123)

* fix: IOException no longer thrown by read().

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* refactoring

* last refactorings

Signed-off-by: Robert Altena <Rob@Ra-ai.com>

* Advanced tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* [WIP] Java wrappers (#126)

* shift/rshift/rotl/rotr java/sd wrappers

Signed-off-by: raver119 <raver119@gmail.com>

* few additional wrappers

Signed-off-by: raver119 <raver119@gmail.com>

* minor naming tweak

Signed-off-by: raver119 <raver119@gmail.com>

* Test added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* one more build fix

Signed-off-by: raver119 <raver119@gmail.com>

* Jar packaging for maven

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Typo fixed

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* minimal viable prototype for SD

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Tests corrected

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Enabled test

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test copied from nd4j

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Thin wrapper added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Cleanup

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Adding arithmetic

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Added tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Stubs for ops

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Some ops added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Advanced tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Ops added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Small build fixes (#127)

* Small build fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Fix RL4J

Signed-off-by: Alex Black <blacka101@gmail.com>

* Test fixes

Signed-off-by: Alex Black <blacka101@gmail.com>

* Another fix

Signed-off-by: Alex Black <blacka101@gmail.com>

* parent module name fix

Signed-off-by: raver119 <raver119@gmail.com>

* [WIP] Roll rewritten (#128)

* Process correct input vector.

* Added tests for roll.

* Refactored roll to conform with TF. Eliminated memory leaks with Roll op tests.

* no thread_local for cpu

Signed-off-by: raver119 <raver119@gmail.com>

* Jar packaging for maven

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Typo fixed

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* minimal viable prototype for SD

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Tests corrected

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Enabled test

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test copied from nd4j

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Thin wrapper added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Cleanup

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Adding arithmetic

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Added tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Stubs for ops

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Some ops added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Advanced tests

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Ops added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Tests added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Boolen logic ops

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Test added

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>

* Shift operations

Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com>
2019-08-19 16:25:13 +03:00
..
project Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
src [WIP] nd4s - Scala operators for SameDiff (#113) 2019-08-19 16:25:13 +03:00
.gitignore Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.scalafmt.conf Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.travis.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
LICENSE Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
README.md Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
build.sbt Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
pom.xml [WIP] nd4s - Scala operators for SameDiff (#113) 2019-08-19 16:25:13 +03:00
sbt-pom.xml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00

README.md

ND4S: Scala bindings for ND4J

Join the chat at https://gitter.im/deeplearning4j/deeplearning4j

ND4S is open-source Scala bindings for ND4J. Released under an Apache 2.0 license.

Main Features

  • NDArray manipulation syntax sugar with safer type.
  • NDArray slicing syntax, similar with NumPy.

Installation

Install via Maven

ND4S is already included in official Maven repositories.

With IntelliJ, incorporation of ND4S is easy: just create a new Scala project, go to "Project Settings"/Libraries, add "From Maven...", and search for nd4s.

As an alternative, one may simply add the line below to build.sbt and re-build project.

val nd4jVersion = "1.0.0-alpha"

libraryDependencies += "org.nd4j" % "nd4j-native-platform" % nd4jVersion
libraryDependencies += "org.nd4j" %% "nd4s" % nd4jVersion

One may want to check our maven repository page and replace 1.0.0-alpha with the latest version.

No need for git-cloning & compiling!

Clone from the GitHub Repo

ND4S is actively developed. You can clone the repository, compile it, and reference it in your project.

Clone the repository:

$ git clone https://github.com/deeplearning4j/nd4s.git

Compile the project:

$ cd nd4s
$ sbt +publish-local

Try ND4S in REPL

The easiest way to play ND4S around is cloning this repository and run the following command.

$ cd nd4s
$ sbt test:console

It starts REPL with importing org.nd4s.Implicits._ and org.nd4j.linalg.factory.Nd4j automatically. It uses jblas backend at default.

scala> val arr = (1 to 9).asNDArray(3,3) 
arr: org.nd4j.linalg.api.ndarray.INDArray =
[[1.00,2.00,3.00]
 [4.00,5.00,6.00]
 [7.00,8.00,9.00]]

scala> val sub = arr(0->2,1->3)
sub: org.nd4j.linalg.api.ndarray.INDArray =
[[2.00,3.00]
 [5.00,6.00]]

CheatSheet(WIP)

ND4S syntax Equivalent NumPy syntax Result
Array(Array(1,2,3),Array(4,5,6)).toNDArray np.array(1, 2 , 3], [4, 5, 6) 1.0, 2.0, 3.0] [4.0, 5.0, 6.0
val arr = (1 to 9).asNDArray(3,3) arr = np.arange(1,10).reshape(3,3) 1.0, 2.0, 3.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0
arr(0,0) arr[0,0] 1.0
arr(0,->) arr[0,:] [1.0, 2.0, 3.0]
arr(--->) arr[...] 1.0, 2.0, 3.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0
arr(0 -> 3 by 2, ->) arr[0:3:2,:] 1.0, 2.0, 3.0] [7.0, 8.0, 9.0
arr(0 to 2 by 2, ->) arr[0:3:2,:] 1.0, 2.0, 3.0] [7.0, 8.0, 9.0
arr.filter(_ > 3) np.where(arr > 3, arr, 0) 0.0, 0.0, 0.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0
arr.map(_ % 3) 1.0, 2.0, 0.0] [1.0, 2.0, 0.0] ,[1.0, 2.0, 0.0
arr.filterBit(_ < 4) 1.0, 1.0, 1.0] [0.0, 0.0, 0.0] ,[0.0, 0.0, 0.0
arr + arr arr + arr 2.0, 4.0, 6.0] [8.0, 10.0, 12.0] ,[14.0, 16.0, 18.0
arr * arr arr * arr 1.0, 4.0, 9.0] [16.0, 25.0, 36.0] ,[49.0, 64.0, 81.0
arr dot arr np.dot(arr, arr) 30.0, 36.0, 42.0] [66.0, 81.0, 96.0] ,[102.0, 126.0, 150.0
arr.sumT np.sum(arr) 45.0 //returns Double value
val comp = Array(1 + i, 1 + 2 * i).toNDArray comp = np.array([1 + 1j, 1 + 2j]) [1.0 + 1.0i ,1.0 + 2.0i]
comp.sumT np.sum(comp) 2.0 + 3.0i //returns IComplexNumber value
for(row <- arr.rowP if row.get(0) > 1) yield row*2 8.00,10.00,12.00] [14.00,16.00,18.00
val tensor = (1 to 8).asNDArray(2,2,2) tensor = np.arange(1,9).reshape(2,2,2) [1.00,2.00] [3.00,4.00 5.00,6.00] [7.00,8.00]
for(slice <- tensor.sliceP if slice.get(0) > 1) yield slice*2 [10.00,12.00][14.00,16.00]
arr(0 -> 3 by 2, ->) = 0 0.00,0.00,0.00] [4.00,5.00,6.00] [0.00,0.00,0.00