Brutex
/
cavis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
cavis/nd4j
History
raver119 0613485654
compression ops (#436) 
* Added declarations for decode/encode_bitmap ops.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added implementation for bitmap encoding/decoding ops.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Added helpers for encode/decode bitmap ops.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Refactored encodingBitmap helper.

Signed-off-by: shugeo <sgazeos@gmail.com>

* threshold encode/decode skeleton

* helper skeleton

* minor import fix

* encoder shape fn & op impl

* thresholdEncode cpu impl

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* thresholdDecode cpu impl

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* Only cosmetical changes.

Signed-off-by: shugeo <sgazeos@gmail.com>

* placeholder

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* Added cuda implementation for bitmap decode helper.

Signed-off-by: shugeo <sgazeos@gmail.com>

* cuda thresholdEstimate

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* cuda thresholdDecode

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* next step

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - nano cmakelist update (get rid of Clion section)
- fixed forgotten throw in AtomicTests

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* thesholdEncode cuda impl

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* Added tests for bitmap encoding/decoding ops.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Fixed tests for encode/decode bitmaps.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Refactored decode/encode helpers.

Signed-off-by: shugeo <sgazeos@gmail.com>

* Fixed crashes with bitmap decode/encode helpers.

Signed-off-by: shugeo <sgazeos@gmail.com>

* bitmap encode/decode CPU

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* bitmap encode/decode CUDA

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* C API removed for threshold/bitmap encode

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* EncodeBitmap/DecodeBitmap Java side

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* EncodeThreshold/DecodeThreshold Java side

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* EncodeThreshold/DecodeThreshold Java side

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* few more tests for threshold encoding

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* minor test tweak

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* two special tests

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* encodeBitmap CPU fix

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* parallel_long/parallel_double proper spans fix

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* encodeThreshold CUDA fix

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* nano fix

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* grid tweaks

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* RTX adaptation for thresholdEncode

Signed-off-by: raver119 <raver119@gmail.com>

* don't allow threshold encoding for length < 2

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* get rid of NDArrayCompressor in EncodingHandler

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* one more minor update of EncodingHandler

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* one more minor tweak of EncodingHandler

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - matmul allows integer data types use
- EncodingHandler boundary default value
- few tests for integer matmul

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* minor fix of CUDA bitmap encode

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* boundary changed to integer everywhere

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* boundary changed to integer everywhere

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* re-enable CUDA deallocator

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* threshold encoder fix for systems without omp

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - encode_threshold now requires non-negative boundary
- minor tweak in EncodingHandler

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* restore parallelism in decode_bitmap

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* fall back to omp for encode_bitmap cpu

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* single time casts

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

* - additional test for encode_threshold
- sync buffers to device before calling for shape function

Signed-off-by: raver119@gmail.com <raver119@gmail.com>

Co-authored-by: shugeo <sgazeos@gmail.com>
 		2020-05-08 20:59:39 +03:00
..
ci Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
contrib Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
nd4j-backends compression ops (#436) 2020-05-08 20:59:39 +03:00
nd4j-common Fix deeplearning4j-util dependency + update remnants still using it (#428) 2020-05-04 15:54:03 +10:00
nd4j-common-tests Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-jdbc Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-parameter-server-parent Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-remote Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-serde Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-shade nd4j-jackson: exclude java.xml.stream.XML*Factory from service loader to avoid clashes with other non-shaded jackson etc on classpath 2019-12-13 21:41:28 +11:00
nd4j-tensorflow Refactor packages to fix split package issues (#411) 2020-04-29 11:19:26 +10:00
nd4j-uberjar Fix uberjar dependencies (#401) 2020-04-21 10:51:26 +10:00
.appveyor.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.codeclimate.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.gitignore Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
.travis.yml Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
LICENSE Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
README.md Various fixes (#43) 2019-11-14 19:38:20 +11:00
RaspberryPi.md Update links to eclipse repos (#252) 2019-09-10 19:09:46 +10:00
VERSION Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
buildAllversions.sh Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
buildmultiplescalaversions.sh Eclipse Migration Initial Commit 2019-06-06 15:21:15 +03:00
pom.xml Cleanup of multiple projects (#175) 2020-01-24 22:35:00 +03:00

README.md

ND4J: Scientific Computing on the JVM

Join the chat at https://gitter.im/deeplearning4j/deeplearning4j Maven Central Javadoc

ND4J is an Apache 2.0-licensed scientific computing library for the JVM. By contributing code to this repository, you agree to make your contribution available under an Apache 2.0 license.

It is meant to be used in production environments rather than as a research tool, which means routines are designed to run fast with minimum RAM requirements.

Please search for the latest version on search.maven.org.

Or use the versions displayed in: https://github.com/eclipse/deeplearning4j-examples/blob/master/pom.xml

Main Features

  • Versatile n-dimensional array object
  • Multiplatform functionality including GPUs
  • Linear algebra and signal processing functions

Specifics

  • Supports GPUs via with the CUDA backend nd4j-cuda-7.5 and Native via nd4j-native.
  • All of this is wrapped in a unifying interface.
  • The API mimics the semantics of Numpy, Matlab and scikit-learn.

Documentation

Documentation is available at deeplearning4j.org. Access the JavaDocs for more detail.

Installation

To install ND4J, there are a couple of approaches, and more information can be found on the DL4J website.

Install from Maven Central

  1. Search for nd4j in the Maven Central Repository to find the available nd4j jars.
  2. Include the appropriate dependency in your pom.xml.

Clone from the GitHub Repo

https://deeplearning4j.org/docs/latest/deeplearning4j-build-from-source

Contribute

  1. Check for open issues, or open a new issue to start a discussion around a feature idea or a bug.

  2. If you feel uncomfortable or uncertain about an issue or your changes, feel free to contact us on Gitter using the link above.

  3. Fork the repository on GitHub to start making your changes to the master branch (or branch off of it).

  4. Write a test, which shows that the bug was fixed or that the feature works as expected.

  5. Note the repository follows the Google Java style with two modifications: 120-char column wrap and 4-spaces indentation. You can format your code to this format by typing mvn formatter:format in the subproject you work on, by using the contrib/formatter.xml at the root of the repository to configure the Eclipse formatter, or by using the INtellij plugin.

  6. Send a pull request, and bug us on Gitter until it gets merged and published.