10d676e0b8
* Jar packaging for maven Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Typo fixed Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * minimal viable prototype for SD Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Tests corrected Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * few fixes for bfloat16 in java and cpp (#114) Signed-off-by: raver119 <raver119@gmail.com> * Nd4j refactoring (#112) * refactoring Signed-off-by: Robert Altena <Rob@Ra-ai.com> * wip Signed-off-by: Robert Altena <Rob@Ra-ai.com> * wip Signed-off-by: Robert Altena <Rob@Ra-ai.com> * wip * fix: make test public. Signed-off-by: Robert Altena <Rob@Ra-ai.com> * make test public. Signed-off-by: Robert Altena <Rob@Ra-ai.com> * fixes read refactoring. Signed-off-by: Robert Altena <Rob@Ra-ai.com> * Enabled test Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test copied from nd4j Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * [WIP] bitwise ops (#115) * - cyclic_shift_bits + test - shift_bits + test Signed-off-by: raver119 <raver119@gmail.com> * OMP_IF replacement Signed-off-by: raver119 <raver119@gmail.com> * Thin wrapper added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Cleanup Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Shugeo cuda tests (#116) * Added tests for get_seed/set_seed ops. * Added missed tests for scatter_sub/mul/div ops. * Added tests for hardsigmoid and hardsigmoid_bp. * Added tests for hardtanh and hardtanh_bp ops. * Added test for histogram op. * Added tests for identity op. * Refactored mergemaxindex op. Added tests for log1p,mergemaxindex, mod and mod_bp ops. * Fixed tests for FloorDiv. * Added test for rank op. * Added tests for rationaltanh/rationaltanh_bp ops. * Added tests for realdiv/realdiv_bp. * Added tests for rectifiedtanh/_bp ops. * Added tests for shapes_of op. * Added tests for shapes_of op. * Added tests for size op. * Added tests for softplus/_bp ops. * Added tests for softsign/_bp ops. * Added tests for toggle_bits op. Fixed processing of OP_IMPL and so on defititions. * Added test for truncatediv op. * Added another test for truncatediv op. * Added another test for histogram. * Added tests for unstack_list op. * Refactored to_int32/uint32/float16/float32/double/int64/uint64 ops and tests. * Refactored mergemaxindex op helper for cuda platform and tests. * Fixed cuda kernel for histogram op helper. * Refactor skipgram to avoid early buffers shift. * Fixed check up with non_max_suppression op cuda helper. Added cuda kernel implementation for skipgram op helpers. * Added implementation of skipgram op helper for cuda platform. Working revision * Fixed mergeMaxIndex kernel and move it to separate source file. * Adding arithmetic Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Eliminated memory leaks and dropped waste prints with tests. (#117) * Added tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * fix test Signed-off-by: raver119 <raver119@gmail.com> * no openmp for ClipByGlobalNorm Signed-off-by: raver119 <raver119@gmail.com> * Stubs for ops Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * [WIP] right shift ops (#118) * right shift ops Signed-off-by: raver119 <raver119@gmail.com> * typo Signed-off-by: raver119 <raver119@gmail.com> * rotr test Signed-off-by: raver119 <raver119@gmail.com> * fix: IOException no longer thrown by read(). (#120) Signed-off-by: Robert Altena <Rob@Ra-ai.com> * Small fix in TensorflowConversion class (#121) Signed-off-by: Alex Black <blacka101@gmail.com> * Shyrma concat2 (#119) * - rewrite/improve concat Signed-off-by: Yurii <yurii@skymind.io> * - ged rid of unnecessary argument in concat kernel Signed-off-by: Yurii <yurii@skymind.io> * InferenceSession additional validation for shape calc (#122) Signed-off-by: Alex Black <blacka101@gmail.com> * [WIP] build fix (#124) * AffinityManager changes Signed-off-by: raver119 <raver119@gmail.com> * build fixes Signed-off-by: raver119 <raver119@gmail.com> * OP/CONFIGURABLE_OP shapefn fix (#125) Signed-off-by: raver119 <raver119@gmail.com> * Some ops added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Nd4j refactoring (last one!) (#123) * fix: IOException no longer thrown by read(). Signed-off-by: Robert Altena <Rob@Ra-ai.com> * refactoring * last refactorings Signed-off-by: Robert Altena <Rob@Ra-ai.com> * Advanced tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * [WIP] Java wrappers (#126) * shift/rshift/rotl/rotr java/sd wrappers Signed-off-by: raver119 <raver119@gmail.com> * few additional wrappers Signed-off-by: raver119 <raver119@gmail.com> * minor naming tweak Signed-off-by: raver119 <raver119@gmail.com> * Test added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * one more build fix Signed-off-by: raver119 <raver119@gmail.com> * Jar packaging for maven Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Typo fixed Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * minimal viable prototype for SD Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Tests corrected Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Enabled test Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test copied from nd4j Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Thin wrapper added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Cleanup Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Adding arithmetic Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Added tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Stubs for ops Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Some ops added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Advanced tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Ops added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Small build fixes (#127) * Small build fixes Signed-off-by: Alex Black <blacka101@gmail.com> * Fix RL4J Signed-off-by: Alex Black <blacka101@gmail.com> * Test fixes Signed-off-by: Alex Black <blacka101@gmail.com> * Another fix Signed-off-by: Alex Black <blacka101@gmail.com> * parent module name fix Signed-off-by: raver119 <raver119@gmail.com> * [WIP] Roll rewritten (#128) * Process correct input vector. * Added tests for roll. * Refactored roll to conform with TF. Eliminated memory leaks with Roll op tests. * no thread_local for cpu Signed-off-by: raver119 <raver119@gmail.com> * Jar packaging for maven Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Typo fixed Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * minimal viable prototype for SD Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Tests corrected Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Enabled test Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test copied from nd4j Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Thin wrapper added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Cleanup Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Adding arithmetic Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Added tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Stubs for ops Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Some ops added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Advanced tests Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Ops added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Tests added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Boolen logic ops Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Test added Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> * Shift operations Signed-off-by: Alexander Stoyakin <alexander.stoyakin@gmail.com> |
||
---|---|---|
.. | ||
project | ||
src | ||
.gitignore | ||
.scalafmt.conf | ||
.travis.yml | ||
LICENSE | ||
README.md | ||
build.sbt | ||
pom.xml | ||
sbt-pom.xml |
README.md
ND4S: Scala bindings for ND4J
ND4S is open-source Scala bindings for ND4J. Released under an Apache 2.0 license.
Main Features
- NDArray manipulation syntax sugar with safer type.
- NDArray slicing syntax, similar with NumPy.
Installation
Install via Maven
ND4S is already included in official Maven repositories.
With IntelliJ, incorporation of ND4S is easy: just create a new Scala project, go to "Project Settings"/Libraries, add "From Maven...", and search for nd4s.
As an alternative, one may simply add the line below to build.sbt
and re-build project.
val nd4jVersion = "1.0.0-alpha"
libraryDependencies += "org.nd4j" % "nd4j-native-platform" % nd4jVersion
libraryDependencies += "org.nd4j" %% "nd4s" % nd4jVersion
One may want to check our maven repository page and replace 1.0.0-alpha
with the latest version.
No need for git-cloning & compiling!
Clone from the GitHub Repo
ND4S is actively developed. You can clone the repository, compile it, and reference it in your project.
Clone the repository:
$ git clone https://github.com/deeplearning4j/nd4s.git
Compile the project:
$ cd nd4s
$ sbt +publish-local
Try ND4S in REPL
The easiest way to play ND4S around is cloning this repository and run the following command.
$ cd nd4s
$ sbt test:console
It starts REPL with importing org.nd4s.Implicits._
and org.nd4j.linalg.factory.Nd4j
automatically. It uses jblas backend at default.
scala> val arr = (1 to 9).asNDArray(3,3)
arr: org.nd4j.linalg.api.ndarray.INDArray =
[[1.00,2.00,3.00]
[4.00,5.00,6.00]
[7.00,8.00,9.00]]
scala> val sub = arr(0->2,1->3)
sub: org.nd4j.linalg.api.ndarray.INDArray =
[[2.00,3.00]
[5.00,6.00]]
CheatSheet(WIP)
ND4S syntax | Equivalent NumPy syntax | Result |
---|---|---|
Array(Array(1,2,3),Array(4,5,6)).toNDArray | np.array(1, 2 , 3], [4, 5, 6) | 1.0, 2.0, 3.0] [4.0, 5.0, 6.0 |
val arr = (1 to 9).asNDArray(3,3) | arr = np.arange(1,10).reshape(3,3) | 1.0, 2.0, 3.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0 |
arr(0,0) | arr[0,0] | 1.0 |
arr(0,->) | arr[0,:] | [1.0, 2.0, 3.0] |
arr(--->) | arr[...] | 1.0, 2.0, 3.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0 |
arr(0 -> 3 by 2, ->) | arr[0:3:2,:] | 1.0, 2.0, 3.0] [7.0, 8.0, 9.0 |
arr(0 to 2 by 2, ->) | arr[0:3:2,:] | 1.0, 2.0, 3.0] [7.0, 8.0, 9.0 |
arr.filter(_ > 3) | np.where(arr > 3, arr, 0) | 0.0, 0.0, 0.0] [4.0, 5.0, 6.0] ,[7.0, 8.0, 9.0 |
arr.map(_ % 3) | 1.0, 2.0, 0.0] [1.0, 2.0, 0.0] ,[1.0, 2.0, 0.0 | |
arr.filterBit(_ < 4) | 1.0, 1.0, 1.0] [0.0, 0.0, 0.0] ,[0.0, 0.0, 0.0 | |
arr + arr | arr + arr | 2.0, 4.0, 6.0] [8.0, 10.0, 12.0] ,[14.0, 16.0, 18.0 |
arr * arr | arr * arr | 1.0, 4.0, 9.0] [16.0, 25.0, 36.0] ,[49.0, 64.0, 81.0 |
arr dot arr | np.dot(arr, arr) | 30.0, 36.0, 42.0] [66.0, 81.0, 96.0] ,[102.0, 126.0, 150.0 |
arr.sumT | np.sum(arr) | 45.0 //returns Double value |
val comp = Array(1 + i, 1 + 2 * i).toNDArray | comp = np.array([1 + 1j, 1 + 2j]) | [1.0 + 1.0i ,1.0 + 2.0i] |
comp.sumT | np.sum(comp) | 2.0 + 3.0i //returns IComplexNumber value |
for(row <- arr.rowP if row.get(0) > 1) yield row*2 | 8.00,10.00,12.00] [14.00,16.00,18.00 | |
val tensor = (1 to 8).asNDArray(2,2,2) | tensor = np.arange(1,9).reshape(2,2,2) | [1.00,2.00] [3.00,4.00 5.00,6.00] [7.00,8.00] |
for(slice <- tensor.sliceP if slice.get(0) > 1) yield slice*2 | [10.00,12.00][14.00,16.00] | |
arr(0 -> 3 by 2, ->) = 0 | 0.00,0.00,0.00] [4.00,5.00,6.00] [0.00,0.00,0.00 |