5.1 KiB

Raw Blame History

BRUTEX CAVIS Build Requirements

You can build CAVIS on either x86_64 linux or x86_64 windows (sometimes referred to as amd64). Support for native GPU (CUDA) or CPU (AVX2/ AVX512) support can be enabled.

Linux on x86_64 is the primary target platform. CAVIS is written in Java, C, C++ and Kotlin, therefore you will need a Java JDK and C/C++ compiler.

Introduction 2.
Some paragraph
1. Sub paragraph
Another paragraph

Build Environment Prerequisites

CAVIS uses the Gradle Build Tool and CMake to execute the build. A Gradle Wrapper is provided, that will pull the right version of gradle.

Linux x86_64 Prerequisites

Linux Kernel 5.10 or higher

Required build tools:

Tools	Version	Remarks
Gradle	> 7.2
CMake	> 2.24.10
GNU Make	3.82

Compiler	Version	Remarks
GCC	7.3.1 - 10.2.1	C and C++ compiler
nvcc	> 11.4	only in case you want to compile for CUDA support
javac	JDK 11	openjdk-11

Amazon Linux 2

Preparation shell script

sudo  amazon-linux-extras install java-openjdk11 && \
      sudo yum install --assumeyes git make gcc gcc-c++ openssl-devel java-11-openjdk-devel && \
      wget https://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2.tar.gz && \
      tar -xf cmake-3.24.2.tar.gz && cd cmake-3.24.2/ && \
      ./bootstrap && make && sudo make install && cd ..

In case you have less than 8GB RAM, a swap file should be created

sudo dd if=/dev/zero of=/swapfile count=8k bs=1M && sudo chmod 600 /swapfile && \
sudo mkswap /swapfile && sudo swapon /swapfile

You can make the swap change permanent (surviving a reboot) by putting it into /etc/fstab file.

echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

In order to resolve Brutex Network IP6 endpoints, you need to add an additional nameserver

 echo 'nameserver 8.8.8.8' | sudo tee -a /etc/resolv.conf

Some paragraph

The first paragraph text

Sub paragraph

This is a sub paragraph, formatted in heading 3 style

Troubleshooting

Gradle process just crashes

This is likely an out-of-memory problem, check

dmesg

for

oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=java,pid=16477,uid=1000
[ 6291.694315] Out of memory: Killed process 16477 (java) total-vm:11373580kB, anon-rss:1373168kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:11220kB oom_score_adj:0

and increase i.e. swapfile size (by adding another one).

sudo apt-get -q update ret=$? while [ $ret -ne 0 ]; do echo Waiting for instance to be ready ... sleep 20s sudo apt-get -q update ret=$? done

echo Instance is ready.

#Create swap echo Adding swap file sudo fallocate -l 16G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile echo Swap created.

#echo "deb http://deb.debian.org/debian/ buster main" | sudo tee /etc/apt/sources.list #echo "deb http://deb.debian.org/debian/ buster main contrib non-free" | sudo tee -a /etc/apt/sources.list

sudo apt-get -q --yes install build-essential default-jdk git ninja-build gnupg software-properties-common snapd make g++-multilib g++ g++-8 gcc gcc-8 libopenblas-base libopenblas-dev ret=$? while [ $ret -ne 0 ]; do echo Waiting for instance to be ready ... sleep 20s sudo apt-get -q --yes install build-essential default-jdk git ninja-build gnupg software-properties-common snapd make g++-multilib g++ g++-8 gcc gcc-8 libopenblas-base libopenblas-dev ret=$? done #sudo snap install core #sudo snap install cmake --classic

wget https://github.com/Kitware/CMake/releases/download/v3.20.4/cmake-3.20.4-linux-x86_64.sh chmod a+x cmake-3.20.4-linux-x86_64.sh sudo sh cmake-3.20.4-linux-x86_64.sh --skip-license

echo "supersede domain-name-servers 172.31.0.2, 8.8.8.8" | sudo tee -a /etc/dhcp/dhclient.conf echo "nameserver 8.8.8.8" | sudo tee -a /etc/resolv.conf

Buildparameter:

-P<xxx>
CAVIS_AVX_EXTENSION = {avx2 | avx512}, default is avx2

Zeppelin Spark dependencies

To add the dependency to the language models, use the following format in the Dependencies section of the of the Spark Interpreter configuration (Interpreters -> Spark -> Edit -> Dependencies):

groupId:artifactId:packaging:classifier:version

In your case it should work with

edu.stanford.nlp:stanford-corenlp:jar:models:3.8.0

Native cpu code under linux needs libc6-dev /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found

5.1 KiB Raw Blame History