cavis/libnd4j/tests_cpu/layers_tests/ContextTests.cpp
raver119 3c4e959e21 [WIP] More of CUDA (#95)
* initial commit

Signed-off-by: raver119 <raver119@gmail.com>

* Implementation of hashcode cuda helper. Working edition.

* Fixed parallel test input arangements.

* Fixed tests for hashcode op.

* Fixed shape calculation for image:crop_and_resize op and test.

* NativeOps tests. Initial test suite.

* Added tests for indexReduce methods.

* Added test on execBroadcast with NDArray as dimensions.

* Added test on execBroadcastBool with NDArray as dimensions.

* Added tests on execPairwiseTransform and execPairwiseTransofrmBool.

* Added tests for execReduce with scalar results.

* Added reduce tests for non-empty dims array.

* Added tests for reduce3.

* Added tests for execScalar.

* Added tests for execSummaryStats.

* - provide cpu/cuda code for batch_to_space
- testing it

Signed-off-by: Yurii <yurii@skymind.io>

* - remove old test for batch_to_space (had wrong format and numbers were not checked)

Signed-off-by: Yurii <yurii@skymind.io>

* Fixed complilation errors with test.

* Added test for execTransformFloat.

* Added test for execTransformSame.

* Added test for execTransformBool.

* Added test for execTransformStrict.

* Added tests for execScalar/execScalarBool with TADs.

* Added test for flatten.

* - provide cpu/cuda code for space_to_Batch operaion

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for concat.

* comment unnecessary stuff in s_t_b

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for specialConcat.

* Added tests for memcpy/set routines.

* Fixed pullRow cuda test.

* Added pullRow test.

* Added average test.

* - correct typo in NDArray::applyPairwiseTransform(nd4j::pairwise::BoolOps op...)

Signed-off-by: Yurii <yurii@skymind.io>

* - debugging and fixing cuda tests in JavaInteropTests file

Signed-off-by: Yurii <yurii@skymind.io>

* - correct some tests

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for shuffle.

* Fixed ops declarations.

* Restored omp and added shuffle test.

* Added convertTypes test.

* Added tests for execRandom. Eliminated usage of RandomBuffer with NativeOps.

* Added sort tests.

* Added tests for execCustomOp.

* - further debuging and fixing tests terminated with crash

Signed-off-by: Yurii <yurii@skymind.io>

* Added tests for calculateOutputShapes.

* Addded Benchmarks test.

* Commented benchmark tests.

* change assertion

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for apply_sgd op. Added cpu helper for that op.

* Implement cuda helper for aplly_sgd op. Fixed tests for NativeOps.

* Added test for assign broadcastable.

* Added tests for assign_bp op.

* Added tests for axpy op.

* - assign/execScalar/execTransformAny signature change
- minor test fix

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed axpy op.

* meh

Signed-off-by: raver119 <raver119@gmail.com>

* - fix tests for nativeOps::concat

Signed-off-by: Yurii <yurii@skymind.io>

* sequential transform/scalar

Signed-off-by: raver119 <raver119@gmail.com>

* allow nested parallelism

Signed-off-by: raver119 <raver119@gmail.com>

* assign_bp leak fix

Signed-off-by: raver119 <raver119@gmail.com>

* block setRNG fix

Signed-off-by: raver119 <raver119@gmail.com>

* enable parallelism by default

Signed-off-by: raver119 <raver119@gmail.com>

* enable nested parallelism by default

Signed-off-by: raver119 <raver119@gmail.com>

* Added cuda implementation for row_count helper.

* Added implementation for tnse gains op helper.

* - take into account possible situations when input arrays are empty in reduce_ cuda stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Implemented tsne/edge_forces op cuda-based helper. Parallelized cpu-based helper for edge_forces.

* Added kernel for tsne/symmetrized op heleper.

* Implementation of tsne/symmetrized op cuda helper. Working edition.

* Eliminated waste printfs.

* Added test for broadcastgradientargs op.

* host-only fallback for empty reduce float

Signed-off-by: raver119 <raver119@gmail.com>

* - some tests fixes

Signed-off-by: Yurii <yurii@skymind.io>

* - correct the rest of reduce_ stuff

Signed-off-by: Yurii <yurii@skymind.io>

* - further correction of reduce_ stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Added test for Cbow op. Also added cuda implementation for cbow helpers.

* - improve code of stack operation for scalar case

Signed-off-by: Yurii <yurii@skymind.io>

* - provide cuda kernel for gatherND operation

Signed-off-by: Yurii <yurii@skymind.io>

* Implementation of cbow helpers with cuda kernels.

* minor tests tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* minor tests tweaks

Signed-off-by: raver119 <raver119@gmail.com>

* - further correction of cuda stuff

Signed-off-by: Yurii <yurii@skymind.io>

* Implementatation of cbow op helper with cuda kernels. Working edition.

* Skip random testing for cudablas case.

* lstmBlockCell context fix

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for ELU and ELU_BP ops.

* Added tests for eq_scalar, gt_scalar, gte_scalar and lte_scalar ops.

* Added tests for neq_scalar.

* Added test for noop.

* - further work on clipbynorm_bp

Signed-off-by: Yurii <yurii@skymind.io>

* - get rid of concat op call, use instead direct concat helper call

Signed-off-by: Yurii <yurii@skymind.io>

* lstmBlockCell context fix

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for lrelu and lrelu_bp.

* Added tests for selu and selu_bp.

* Fixed lrelu derivative helpers.

* - some corrections in lstm

Signed-off-by: Yurii <yurii@skymind.io>

* operator * result shape fix

Signed-off-by: raver119 <raver119@gmail.com>

* - correct typo in lstmCell

Signed-off-by: Yurii <yurii@skymind.io>

* few tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* CUDA inverse broadcast bool fix

Signed-off-by: raver119 <raver119@gmail.com>

* disable MMAP test for CUDA

Signed-off-by: raver119 <raver119@gmail.com>

* BooleanOp syncToDevice

Signed-off-by: raver119 <raver119@gmail.com>

* meh

Signed-off-by: raver119 <raver119@gmail.com>

* additional data types for im2col/col2im

Signed-off-by: raver119 <raver119@gmail.com>

* Added test for firas_sparse op.

* one more RandomBuffer test excluded

Signed-off-by: raver119 <raver119@gmail.com>

* Added tests for flatten op.

* Added test for Floor op.

* bunch of tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* mmulDot tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Implemented floordiv_bp op and tests.

* Fixed scalar case with cuda implementation for bds.

* - work on cuda kernel for clip_by_norm backprop op is completed

Signed-off-by: Yurii <yurii@skymind.io>

* Eliminate cbow crach.

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Eliminated abortion with batched nlp test.

* more tests fixed

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed shared flag initializing.

* disabled bunch of cpu workspaces tests

Signed-off-by: raver119 <raver119@gmail.com>

* scalar operators fix: missing registerSpecialUse call

Signed-off-by: raver119 <raver119@gmail.com>

* Fixed logdet for cuda and tests.

* - correct clipBynorm_bp

Signed-off-by: Yurii <yurii@skymind.io>

* Fixed crop_and_resize shape datatype.

* - correct some mmul tests

Signed-off-by: Yurii <yurii@skymind.io>
2019-08-05 11:27:05 +10:00

356 lines
9.9 KiB
C++

/*******************************************************************************
* Copyright (c) 2015-2018 Skymind, Inc.
*
* This program and the accompanying materials are made available under the
* terms of the Apache License, Version 2.0 which is available at
* https://www.apache.org/licenses/LICENSE-2.0.
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*
* SPDX-License-Identifier: Apache-2.0
******************************************************************************/
//
// Created by raver119 on 30.10.2017.
//
#include "testlayers.h"
#include <ops/declarable/CustomOperations.h>
using namespace nd4j;
using namespace nd4j::ops;
using namespace nd4j::graph;
class ContextTests : public testing::Test {
public:
};
TEST_F(ContextTests, Basic_Test_1) {
VariableSpace variableSpace;
auto _20 = NDArrayFactory::create_<float>('c', {2, 2});
auto _21 = NDArrayFactory::create_<float>('c', {2, 2});
_20->assign(1.0f);
_21->assign(2.0f);
variableSpace.putVariable(2, 0, _20);
variableSpace.putVariable(2, 1, _21);
Context block(1, &variableSpace);
block.pickInput(2, 0);
block.pickInput(2, 1);
ASSERT_EQ(2, block.inputs()->size());
ASSERT_EQ(2, block.width());
ASSERT_TRUE(variableSpace.hasVariable(2, 0));
ASSERT_TRUE(variableSpace.hasVariable(2, 1));
ASSERT_NEAR(1.0f, block.variable(0)->getNDArray()->meanNumber().e<float>(0), 1e-5);
ASSERT_NEAR(2.0f, block.variable(1)->getNDArray()->meanNumber().e<float>(0), 1e-5);
}
TEST_F(ContextTests, Basic_Test_2) {
VariableSpace variableSpace;
auto _20 = NDArrayFactory::create_<float>('c', {2, 2});
auto _21 = NDArrayFactory::create_<float>('c', {2, 2});
_20->assign(1.0f);
_21->assign(2.0f);
variableSpace.putVariable(-1, _20);
variableSpace.putVariable(-2, _21);
Context block(1, &variableSpace);
block.pickInput(-1);
block.pickInput(-2);
ASSERT_EQ(2, block.inputs()->size());
ASSERT_EQ(2, block.width());
ASSERT_TRUE(variableSpace.hasVariable(-1));
ASSERT_TRUE(variableSpace.hasVariable(-2));
ASSERT_NEAR(1.0f, block.variable(0)->getNDArray()->meanNumber().e<float>(0), 1e-5);
ASSERT_NEAR(2.0f, block.variable(1)->getNDArray()->meanNumber().e<float>(0), 1e-5);
}
TEST_F(ContextTests, Basic_Test_3) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto _20 = NDArrayFactory::create_<float>('c', {2, 2});
ctx.pushNDArrayToVariableSpace(1, 1, _20);
ASSERT_TRUE(variableSpace.hasVariable(1, 1));
}
TEST_F(ContextTests, Basic_Test_4) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto _20 = NDArrayFactory::create_<float>('c', {2, 2});
_20->linspace(1);
auto _21 = NDArrayFactory::create_<float>('c', {2, 2});
_21->linspace(10);
ctx.pushNDArrayToVariableSpace(1, 1, _20);
ASSERT_TRUE(variableSpace.hasVariable(1, 1));
ctx.pushNDArrayToVariableSpace(1, 1, _21);
auto vA = ctx.variable(1, 1);
ASSERT_TRUE(vA->getNDArray()->equalsTo(_21));
}
TEST_F(ContextTests, Basic_Test_5) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto _20 = NDArrayFactory::create_<float>('c', {2, 2});
_20->linspace(1);
auto exp = _20->dup();
ctx.pushNDArrayToVariableSpace(1, 1, _20);
ASSERT_TRUE(variableSpace.hasVariable(1, 1));
ctx.pushNDArrayToVariableSpace(1, 1, _20);
auto vA = ctx.variable(1, 1);
ASSERT_TRUE(vA->getNDArray() == _20);
ASSERT_TRUE(vA->getNDArray()->equalsTo(exp));
delete exp;
}
TEST_F(ContextTests, Basic_Test_6) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto v0 = ctx.ensureVariable();
auto v1 = ctx.ensureVariable(1);
ASSERT_TRUE(variableSpace.hasVariable(1, 0));
ASSERT_TRUE(variableSpace.hasVariable(1, 1));
auto var0 = variableSpace.getVariable(1, 0);
auto var1 = variableSpace.getVariable(1, 1);
ASSERT_TRUE(v0 == var0);
ASSERT_TRUE(v1 == var1);
}
TEST_F(ContextTests, Basic_Test_7) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto v0 = ctx.ensureVariable();
auto v1 = ctx.ensureVariable(1);
ASSERT_TRUE(variableSpace.hasVariable(1, 0));
ASSERT_TRUE(variableSpace.hasVariable(1, 1));
auto var0 = variableSpace.getVariable(1, 0);
auto var1 = variableSpace.getVariable(1, 1);
ASSERT_TRUE(v0 == var0);
ASSERT_TRUE(v1 == var1);
auto _10 = NDArrayFactory::create_<float>('c', {2, 2});
_10->linspace(1);
auto _11 = NDArrayFactory::create_<float>('c', {2, 2});
_11->linspace(10);
ctx.pushNDArrayToVariableSpace(1, 0, _10);
ctx.pushNDArrayToVariableSpace(1, 1, _11);
auto z0 = variableSpace.getVariable(1, 0);
auto z1 = variableSpace.getVariable(1, 1);
ASSERT_TRUE(v0 == z0);
ASSERT_TRUE(v1 == z1);
}
TEST_F(ContextTests, Basic_Test_8) {
VariableSpace variableSpace;
Context ctx(1, &variableSpace);
auto _10 = NDArrayFactory::create_<float>('c', {2, 2});
_10->linspace(1);
auto _11 = NDArrayFactory::create_<float>('c', {2, 2});
_11->linspace(10);
ctx.pushNDArrayToVariableSpace(1, 0, _10);
ctx.pushNDArrayToVariableSpace(1, 1, _11);
auto z0 = variableSpace.getVariable(1, 0);
auto z1 = variableSpace.getVariable(1, 1);
auto v0 = ctx.ensureVariable();
auto v1 = ctx.ensureVariable(1);
ASSERT_TRUE(v0 == z0);
ASSERT_TRUE(v1 == z1);
}
TEST_F(ContextTests, Basic_Test_9) {
VariableSpace variableSpace;
auto in = NDArrayFactory::create<float>('c', {5, 5});
Context ctx(1, &variableSpace, true);
ctx.pushNDArrayToVariableSpace(1, 1, &in, false);
}
TEST_F(ContextTests, Basic_Test_10) {
VariableSpace variableSpace;
Context ctx(119, &variableSpace);
}
TEST_F(ContextTests, Prototype_Test_1) {
ContextPrototype prototype(nullptr, 119, true);
prototype.pickInput(12, 3);
prototype.pickInput(12, 4);
prototype.getTArguments()->push_back(2.0);
prototype.getTArguments()->push_back(-2.0);
prototype.getIArguments()->push_back(17);
prototype.getIArguments()->push_back(119);
Context ctx(&prototype, nullptr);
ASSERT_EQ(ctx.nodeId(), prototype.nodeId());
ASSERT_EQ(ctx.isInplace(), prototype.isInplace());
ASSERT_EQ(2, ctx.inputs()->size());
ASSERT_EQ(2, ctx.getTArguments()->size());
ASSERT_EQ(2, ctx.getIArguments()->size());
ASSERT_EQ(2.0, ctx.getTArguments()->at(0));
ASSERT_EQ(-2.0, ctx.getTArguments()->at(1));
ASSERT_EQ(17, ctx.getIArguments()->at(0));
ASSERT_EQ(119, ctx.getIArguments()->at(1));
}
TEST_F(ContextTests, Prototype_Test_2) {
ContextPrototype prototype(nullptr, 119, false);
prototype.setOpNum(179);
Context ctx(&prototype, nullptr);
ASSERT_EQ(ctx.isInplace(), prototype.isInplace());
ASSERT_EQ(ctx.opNum(), prototype.opNum());
ASSERT_EQ(0, ctx.inputs()->size());
ASSERT_EQ(0, ctx.getTArguments()->size());
ASSERT_EQ(0, ctx.getIArguments()->size());
}
TEST_F(ContextTests, test_short_context_1) {
auto array0 = NDArrayFactory::create<float>('c', {3, 2}, {1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
auto array1 = NDArrayFactory::create<float>('c', {3, 2}, {-1.f, -2.f, -3.f, -4.f, -5.f, -6.f});
Context ctx(1);
ctx.setInputArray(0, array0.buffer(), array0.shapeInfo(), array0.specialBuffer(), array0.specialShapeInfo());
ctx.setInputArray(1, array1.buffer(), array1.shapeInfo(), array1.specialBuffer(), array1.specialShapeInfo());
ASSERT_EQ(2, ctx.width());
auto input0 = ctx.array(0);
ASSERT_TRUE(input0 != nullptr);
auto input1 = ctx.array(1);
ASSERT_TRUE(input1 != nullptr);
ASSERT_TRUE(input0->buffer() == array0.buffer());
ASSERT_TRUE(input0->shapeInfo() == array0.shapeInfo());
ASSERT_TRUE(input0->specialBuffer() == array0.specialBuffer());
ASSERT_TRUE(input0->specialShapeInfo() == array0.specialShapeInfo());
ASSERT_TRUE(input1->buffer() == array1.buffer());
ASSERT_TRUE(input1->shapeInfo() == array1.shapeInfo());
ASSERT_TRUE(input1->specialBuffer() == array1.specialBuffer());
ASSERT_TRUE(input1->specialShapeInfo() == array1.specialShapeInfo());
}
TEST_F(ContextTests, test_short_context_2) {
auto array0 = NDArrayFactory::create<float>('c', {3, 2}, {1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
auto array1 = NDArrayFactory::create<float>('c', {3, 2}, {1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
auto z = NDArrayFactory::create<float>('c', {3, 2});
auto exp = NDArrayFactory::create<float>('c', {3, 2}, {2.f, 4.f, 6.f, 8.f, 10.f, 12.f});
Context ctx(1);
ctx.setInputArray(0, array0.buffer(), array0.shapeInfo(), array0.specialBuffer(), array0.specialShapeInfo());
ctx.setInputArray(1, array1.buffer(), array1.shapeInfo(), array1.specialBuffer(), array1.specialShapeInfo());
ctx.setOutputArray(0, z.buffer(), z.shapeInfo(), z.specialBuffer(), z.specialShapeInfo());
ASSERT_EQ(2, ctx.width());
nd4j::ops::add op;
op.execute(&ctx);
ASSERT_EQ(exp, z);
}
TEST_F(ContextTests, test_short_context_3) {
auto array0 = NDArrayFactory::create<float>('c', {3, 2}, {1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
auto array1 = NDArrayFactory::create<float>('c', {3, 2}, {1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
auto exp = NDArrayFactory::create<float>('c', {3, 2}, {2.f, 4.f, 6.f, 8.f, 10.f, 12.f});
Context ctx(1);
ctx.setInputArray(0, array0.buffer(), array0.shapeInfo(), array0.specialBuffer(), array0.specialShapeInfo());
ctx.setInputArray(1, array1.buffer(), array1.shapeInfo(), array1.specialBuffer(), array1.specialShapeInfo());
ASSERT_EQ(2, ctx.width());
nd4j::ops::add op;
op.execute(&ctx);
ASSERT_EQ(1, ctx.fastpath_out().size());
auto z = ctx.fastpath_out()[0];
ASSERT_EQ(exp, *z);
}