cavis/libnd4j/tests_cpu/layers_tests/VariableTests.cpp
raver119 3c4e959e21 [WIP] More of CUDA (#95)
* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* Implementation of hashcode cuda helper. Working edition.

* Fixed parallel test input arangements.

* Fixed tests for hashcode op.

* Fixed shape calculation for image:crop_and_resize op and test.

* NativeOps tests. Initial test suite.

* Added tests for indexReduce methods.

* Added test on execBroadcast with NDArray as dimensions.

* Added test on execBroadcastBool with NDArray as dimensions.

* Added tests on execPairwiseTransform and execPairwiseTransofrmBool.

* Added tests for execReduce with scalar results.

* Added reduce tests for non-empty dims array.

* Added tests for reduce3.

* Added tests for execScalar.

* Added tests for execSummaryStats.

* - provide cpu/cuda code for batch_to_space
- testing it

Signed-off-by: Yurii <yurii@skymind.io>

* - remove old test for batch_to_space (had wrong format and numbers were not checked)

Signed-off-by: Yurii <yurii@skymind.io>

* Fixed complilation errors with test.

* Added test for execTransformFloat.

* Added test for execTransformSame.

* Added test for execTransformBool.

* Added test for execTransformStrict.

* Added tests for execScalar/execScalarBool with TADs.

* Added test for flatten.

* - provide cpu/cuda code for space_to_Batch operaion

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for concat.

* comment unnecessary stuff in s_t_b

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for specialConcat.

* Added tests for memcpy/set routines.

* Fixed pullRow cuda test.

* Added pullRow test.

* Added average test.

* - correct typo in NDArray::applyPairwiseTransform(nd4j::pairwise::BoolOps op...)

Signed-off-by: Yurii <yurii@skymind.io>

* - debugging and fixing cuda tests in JavaInteropTests file

Signed-off-by: Yurii <yurii@skymind.io>

* - correct some tests

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for shuffle.

* Fixed ops declarations.

* Restored omp and added shuffle test.

* Added convertTypes test.

* Added tests for execRandom. Eliminated usage of RandomBuffer with NativeOps.

* Added sort tests.

* Added tests for execCustomOp.

* - further debuging and fixing tests terminated with crash

Signed-off-by: Yurii <yurii@skymind.io>

* Added tests for calculateOutputShapes.

* Addded Benchmarks test.

* Commented benchmark tests.

* change assertion

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for apply_sgd op. Added cpu helper for that op.

* Implement cuda helper for aplly_sgd op. Fixed tests for NativeOps.

* Added test for assign broadcastable.

* Added tests for assign_bp op.

* Added tests for axpy op.

* - assign/execScalar/execTransformAny signature change
- minor test fix

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed axpy op.

* meh

Signed-off-by: raver119 <raver119@gmail.com>

* - fix tests for nativeOps::concat

Signed-off-by: Yurii <yurii@skymind.io>

* sequential transform/scalar

Signed-off-by: raver119 <raver119@gmail.com>

* allow nested parallelism

Signed-off-by: raver119 <raver119@gmail.com>

* assign_bp leak fix

Signed-off-by: raver119 <raver119@gmail.com>

* block setRNG fix

Signed-off-by: raver119 <raver119@gmail.com>

* enable parallelism by default

Signed-off-by: raver119 <raver119@gmail.com>

* enable nested parallelism by default

Signed-off-by: raver119 <raver119@gmail.com>

* Added cuda implementation for row_count helper.

* Added implementation for tnse gains op helper.

* - take into account possible situations when input arrays are empty in reduce_ cuda stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Implemented tsne/edge_forces op cuda-based helper. Parallelized cpu-based helper for edge_forces.

* Added kernel for tsne/symmetrized op heleper.

* Implementation of tsne/symmetrized op cuda helper. Working edition.

* Eliminated waste printfs.

* Added test for broadcastgradientargs op.

* host-only fallback for empty reduce float

Signed-off-by: raver119 <raver119@gmail.com>

* - some tests fixes

Signed-off-by: Yurii <yurii@skymind.io>

* - correct the rest of reduce_ stuff

Signed-off-by: Yurii <yurii@skymind.io>

* - further correction of reduce_ stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for Cbow op. Also added cuda implementation for cbow helpers.

* - improve code of stack operation for scalar case

Signed-off-by: Yurii <yurii@skymind.io>

* - provide cuda kernel for gatherND operation

Signed-off-by: Yurii <yurii@skymind.io>

* Implementation of cbow helpers with cuda kernels.

* minor tests tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* minor tests tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - further correction of cuda stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Implementatation of cbow op helper with cuda kernels. Working edition.

* Skip random testing for cudablas case.

* lstmBlockCell context fix

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for ELU and ELU_BP ops.

* Added tests for eq_scalar, gt_scalar, gte_scalar and lte_scalar ops.

* Added tests for neq_scalar.

* Added test for noop.

* - further work on clipbynorm_bp

Signed-off-by: Yurii <yurii@skymind.io>

* - get rid of concat op call, use instead direct concat helper call

Signed-off-by: Yurii <yurii@skymind.io>

* lstmBlockCell context fix

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for lrelu and lrelu_bp.

* Added tests for selu and selu_bp.

* Fixed lrelu derivative helpers.

* - some corrections in lstm

Signed-off-by: Yurii <yurii@skymind.io>

* operator * result shape fix

Signed-off-by: raver119 <raver119@gmail.com>

* - correct typo in lstmCell

Signed-off-by: Yurii <yurii@skymind.io>

* few tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* CUDA inverse broadcast bool fix

Signed-off-by: raver119 <raver119@gmail.com>

* disable MMAP test for CUDA

Signed-off-by: raver119 <raver119@gmail.com>

* BooleanOp syncToDevice

Signed-off-by: raver119 <raver119@gmail.com>

* meh

Signed-off-by: raver119 <raver119@gmail.com>

* additional data types for im2col/col2im

Signed-off-by: raver119 <raver119@gmail.com>

* Added test for firas_sparse op.

* one more RandomBuffer test excluded

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for flatten op.

* Added test for Floor op.

* bunch of tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* mmulDot tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Implemented floordiv_bp op and tests.

* Fixed scalar case with cuda implementation for bds.

* - work on cuda kernel for clip_by_norm backprop op is completed

Signed-off-by: Yurii <yurii@skymind.io>

* Eliminate cbow crach.

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Eliminated abortion with batched nlp test.

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed shared flag initializing.

* disabled bunch of cpu workspaces tests

Signed-off-by: raver119 <raver119@gmail.com>

* scalar operators fix: missing registerSpecialUse call

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed logdet for cuda and tests.

* - correct clipBynorm_bp

Signed-off-by: Yurii <yurii@skymind.io>

* Fixed crop_and_resize shape datatype.

* - correct some mmul tests

Signed-off-by: Yurii <yurii@skymind.io>
2019-08-05 11:27:05 +10:00

227 lines
6.3 KiB
C++

/*******************************************************************************
* Copyright (c) 2015-2018 Skymind, Inc.
*
* This program and the accompanying materials are made available under the
* terms of the Apache License, Version 2.0 which is available at
* https://www.apache.org/licenses/LICENSE-2.0.
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*
* SPDX-License-Identifier: Apache-2.0
******************************************************************************/
//
// @author raver119@gmail.com
//
#ifndef LIBND4J_VARIABLETESTS_H
#define LIBND4J_VARIABLETESTS_H
#include "testlayers.h"
#include <NDArray.h>
#include <graph/Variable.h>
#include <flatbuffers/flatbuffers.h>
using namespace nd4j;
using namespace nd4j::graph;
class VariableTests : public testing::Test {
public:
};
TEST_F(VariableTests, TestClone_1) {
auto array1 = NDArrayFactory::create_<float>('c', {5, 5});
array1->assign(1.0);
auto var1 = new Variable(array1, "alpha");
var1->setId(119);
auto var2 = var1->clone();
ASSERT_FALSE(var1->getNDArray() == var2->getNDArray());
auto array2 = var2->getNDArray();
ASSERT_TRUE(array1->equalsTo(array2));
ASSERT_EQ(var1->id(), var2->id());
ASSERT_EQ(*var1->getName(), *var2->getName());
delete var1;
std::string str("alpha");
ASSERT_EQ(*var2->getName(), str);
array2->assign(2.0);
ASSERT_NEAR(2.0, array2->meanNumber().e<float>(0), 1e-5);
delete var2;
}
TEST_F(VariableTests, Test_FlatVariableDataType_1) {
flatbuffers::FlatBufferBuilder builder(1024);
auto original = NDArrayFactory::create<float>('c', {5, 10});
original.linspace(1);
auto vec = original.asByteVector();
auto fShape = builder.CreateVector(original.getShapeInfoAsFlatVector());
auto fBuffer = builder.CreateVector(vec);
auto fVid = CreateIntPair(builder, 1, 12);
auto fArray = CreateFlatArray(builder, fShape, fBuffer, nd4j::graph::DataType::DataType_FLOAT);
auto flatVar = CreateFlatVariable(builder, fVid, 0, nd4j::graph::DataType::DataType_FLOAT, 0, fArray);
builder.Finish(flatVar);
auto ptr = builder.GetBufferPointer();
auto restoredVar = GetFlatVariable(ptr);
auto rv = new Variable(restoredVar);
ASSERT_EQ(1, rv->id());
ASSERT_EQ(12, rv->index());
auto restoredArray = rv->getNDArray();
ASSERT_TRUE(original.isSameShape(restoredArray));
ASSERT_TRUE(original.equalsTo(restoredArray));
delete rv;
}
TEST_F(VariableTests, Test_FlatVariableDataType_2) {
flatbuffers::FlatBufferBuilder builder(1024);
auto original = NDArrayFactory::create<double>('c', {5, 10});
original.linspace(1);
auto vec = original.asByteVector();
auto fShape = builder.CreateVector(original.getShapeInfoAsFlatVector());
auto fBuffer = builder.CreateVector(vec);
auto fVid = CreateIntPair(builder, 1, 12);
auto fArray = CreateFlatArray(builder, fShape, fBuffer, nd4j::graph::DataType::DataType_DOUBLE);
auto flatVar = CreateFlatVariable(builder, fVid, 0, nd4j::graph::DataType::DataType_DOUBLE, 0, fArray);
builder.Finish(flatVar);
auto ptr = builder.GetBufferPointer();
auto restoredVar = GetFlatVariable(ptr);
auto rv = new Variable(restoredVar);
ASSERT_EQ(1, rv->id());
ASSERT_EQ(12, rv->index());
auto restoredArray = rv->getNDArray();
ASSERT_TRUE(original.isSameShape(restoredArray));
ASSERT_TRUE(original.equalsTo(restoredArray));
delete rv;
}
TEST_F(VariableTests, Test_FlatVariableDataType_3) {
flatbuffers::FlatBufferBuilder builder(1024);
auto original = NDArrayFactory::create<double>('c', {5, 10});
auto floating = NDArrayFactory::create<float>('c', {5, 10});
original.linspace(1);
floating.linspace(1);
auto vec = original.asByteVector();
auto fShape = builder.CreateVector(original.getShapeInfoAsFlatVector());
auto fBuffer = builder.CreateVector(vec);
auto fVid = CreateIntPair(builder, 1, 12);
auto fArray = CreateFlatArray(builder, fShape, fBuffer, nd4j::graph::DataType::DataType_DOUBLE);
auto flatVar = CreateFlatVariable(builder, fVid, 0, nd4j::graph::DataType::DataType_DOUBLE, 0, fArray);
builder.Finish(flatVar);
auto ptr = builder.GetBufferPointer();
auto restoredVar = GetFlatVariable(ptr);
auto rv = new Variable(restoredVar);
ASSERT_EQ(1, rv->id());
ASSERT_EQ(12, rv->index());
auto restoredArray = rv->getNDArray();
auto conv = restoredArray->asT<float>();
ASSERT_TRUE(floating.isSameShape(restoredArray));
ASSERT_TRUE(floating.equalsTo(conv));
delete rv;
delete conv;
}
/*
TEST_F(VariableTests, Test_FlatVariableDataType_4) {
flatbuffers::FlatBufferBuilder builder(1024);
auto original = NDArrayFactory::create<float>('c', {5, 10});
std::vector<Nd4jLong> exp({5, 10});
auto vec = original.asByteVector();
auto fShape = builder.CreateVector(original.getShapeAsFlatVector());
auto fVid = CreateIntPair(builder, 37, 12);
auto flatVar = CreateFlatVariable(builder, fVid, 0, nd4j::graph::DataType::DataType_FLOAT, fShape, 0, 0, VarType_PLACEHOLDER);
builder.Finish(flatVar);
auto ptr = builder.GetBufferPointer();
auto restoredVar = GetFlatVariable(ptr);
auto rv = new Variable(restoredVar);
ASSERT_EQ(37, rv->id());
ASSERT_EQ(12, rv->index());
//auto restoredArray = rv->getNDArray();
ASSERT_EQ(PLACEHOLDER, rv->variableType());
ASSERT_EQ(exp, rv->shape());
//ASSERT_TRUE(original.isSameShape(restoredArray));
//ASSERT_TRUE(original.equalsTo(restoredArray));
delete rv;
}
*/
TEST_F(VariableTests, Test_Dtype_Conversion_1) {
auto x = NDArrayFactory::create_<float>('c', {2, 3}, {1, 2, 3, 4, 5, 6});
Variable v(x, "alpha", 12, 3);
auto vd = v.template asT<double>();
auto vf = vd->template asT<float>();
ASSERT_EQ(*v.getName(), *vf->getName());
ASSERT_EQ(v.id(), vf->id());
ASSERT_EQ(v.index(), vf->index());
auto xf = vf->getNDArray();
ASSERT_TRUE(x->isSameShape(xf));
ASSERT_TRUE(x->equalsTo(xf));
delete vd;
delete vf;
}
#endif //LIBND4J_VARIABLETESTS_H