Installing IBM Visual Insights 1.2 with RedHat v7 OS on Power AC922 (Summery)

Cahyati Supriyati Sangaji (My Note)

In this article I will share my experience about the setup and installation process of IBM Visual Insight 1.2 with Redhat OS on Power Server AC922. For the installation process with Ubuntu OS, you can see in this article “Installing IBM Visual Insights 1.2 with Ubuntu OS on Power AC922 (Summery)”.

Note for you this installation process has been completed, you have finished installing Redhat OS on Power Server and Redhat subscription.

Step-by step if need to resize directory after OS installation process :

lsblk
df -h
tar -czvf /root/home.tgz -C /home .
tar -tvf /root/home.tgz
ll
umount /dev/mapper/rhel-home
lvremove /dev/mapper/rhel-home
lvcreate -L 1TB -n home rhel (/home size)
mkfs.xfs /dev/rhel/home
mount /dev/mapper/rhel-home
lvextend -r -l +100%FREE /dev/mapper/rhel-root
df -h
sudo reboot

Setup Server Power AC922

1. Enable common, optional, and extra repo channels.

sudo subscription-manager repos --enable=rhel-7-for-power-9-optional-rpms 
sudo subscription-manager repos --enable=rhel-7-for-power-9-extras-rpms 
sudo subscription-manager repos --enable=rhel-7-for-power-9-rpms

2. Install packages needed for the installation.

sudo yum -y install wget nano bzip2

3. Enable the Fedora Project Extra Packages for Enterprise Linux (EPEL) repository:

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm 
sudo rpm -ihv epel-release-latest-7.noarch.rpm

4. Load the latest kernel :

sudo yum install kernel-devel 
sudo yum update kernel kernel-devel kernel-tools kernel-tools-libs kernel-bootwrapper 
reboot

5. or do a full update:

sudo yum install kernel-devel 
sudo yum update 
sudo reboot

Redhat operating system and repository setup is done.

NVIDIA Components: IBM POWER9 specific udev rules

1. Copy the /lib/udev/rules.d/40-redhat.rules file to the directory for user overridden rules:

sudo cp /lib/udev/rules.d/40-redhat.rules /etc/udev/rules.d/

2. Edit the /etc/udev/rules.d/40-redhat.rules file:

sudo nano /etc/udev/rules.d/40-redhat.rules

3. Comment out the entire “Memory hotadd request” section and save the change:

# Memory hotadd request #SUBSYSTEM!="memory", ACTION!="add", GOTO="memory_hotplug_end" #PROGRAM="/bin/uname -p", RESULT=="s390*", GOTO="memory_hotplug_end" #ENV{.state}="online" #PROGRAM="/bin/systemd-detect-virt", RESULT=="none", ENV{.state}="online_movable" #ATTR{state}=="offline", ATTR{state}="$env{.state}" #LABEL="memory_hotplug_end"

4. Optionally, delete the first line of the file, since the file was copied to a directory where it cannot be overwritten:

# do not edit this file, it will be overwritten on update

5. Restart the system for the changes to take effect:

sudo reboot

Installing the GPU driver

Install the driver by following these steps:

Note: These instructions are intended for installation on a single Red Hat instance. If the GPU driver must be installed on many Red Hat instances, follow the instructions in this article: NVIDIA and Red Hat: Simplifying NVIDIA GPU Driver Deployment on Red Hat Enterprise Linux.

1. Download the NVIDIA GPU driver:

• Go to NVIDIA Driver Download.
• Select Product Type: Tesla.
• Select Product Series: P-Series (for Tesla P100) or V-Series (for Tesla V100).
• Select Product: Tesla P100 or Tesla V100.
• Select Operating System, click Show all Operating Systems, then choose the appropriate value: — Linux POWER LE RHEL 7 for Power®
• Select CUDA Toolkit: 10.2.
• Click SEARCH to go to the download link.
• Click Download to download the driver.

Important: An rpm file should be downloaded. If a different type of file is downloaded, verify that you chose the correct options and try again.

2. Install CUDA and the GPU driver.

Note: For AC922 systems: OS and system firmware updates are required before you install the latest GPU driver.

wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda-repo-rhel7-10-2-local-10.2.89-440.33.01-1.0-1.ppc64le.rpm
sudo rpm -i cuda-repo-rhel7-10-2-local-10.2.89-440.33.01-1.0-1.ppc64le.rpm
sudo yum clean all
sudo yum -y install nvidia-driver-latest-dkms cuda
sudo yum -y install cuda-drivers

5. Set nvidia-persistenced to start at boot :

sudo systemctl enable nvidia-persistenced

6. Restart to activate the driver.

After that, verify that the CUDA drivers are installed by running the /usr/bin/nvidia-smi application or run the following command:

nvidia-smi

IBM Visual Insights stand-alone installation prerequisites

Next install docker and nvidia-docker2.

Follow these steps to install Docker on RHEL. For full details, refer to https://github.com/NVIDIA/nvidiadocker#rhel-docker.

1. Install docker

sudo yum install docker

Note :

docker-1.13.1–108.git4ef4b30.el7 has a known issue with the Nvidia GPUs. The docker-1.13.1–104.git4ef4b30.el7 version can explicitly be installed, or newer versions of RHEL Docker work as well. Ensure that docker-1.13.1–108.git4ef4b30.el7 is NOT installed.

2. Reboot the system.

3. Add the package repositories:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) 
curl -s -L https://nvidia.github.io/docker/$distribution/docker.repo | sudo tee /etc/ yum.repos.d/docker.repo 
sudo yum install -y nvidia-container-runtime-hook 
sudo systemctl restart docker

4. Install nvidia-docker2

Set the repository and update. I am using Redhat, so I follow the Redhat settings:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-container-runtime/$distribution/nvidia-container-runtime.repo | \
  
sudo tee /etc/yum.repos.d/nvidia-container-runtime.repo

Install nvidia-docker 2.0 according to the guideline.

# Install the nvidia runtime hook
sudo yum clean expire-cache
sudo yum install -y nvidia-container-runtime-hook

# Adjust SELINUX Permissions
sudo chcon -t container_file_t  /dev/nvidia*

Verify the setup.

docker run --rm nvidia/cuda-ppc64le nvidia-smi
Note: for Maximo Visual Inspection
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | sudo tee /etc/ yum.repos.d/nvidia-docker.repo 
sudo yum install -y nvidia-container-runtime-hook
sudo chcon -t container_file_t  /dev/nvidia*
sudo docker run --rm -e NVIDIA_VISIBLE_DEVICES=all nvidia/cuda-ppc64le:10.2-cudnn7-devel nvidia-smi(For CUDA 10.2)

The previous step is the last step for setup the AC922 Server, before the IBM Visual Insights 1.2 installation process.

Installing Visual Insights stand-alone

To install IBM Visual Insights stand-alone, complete the following steps:

1. Download the product tar file from the IBM Passport Advantage website. If you have IBM Id, you can download from the Software Access Catalog website.
2. (Optional) Verify the downloaded tar file by following these instructions:

a. Download these files:

visual-insights-ppc-1.2.0.0-ppa.sig 
visual-insights-1.2.0.0-key.pub 
visual-insights-ocsp-1.2.0.0-key.pub 
visual-insight-ocspchain-1.2-key.pub

b. To verify the tar file by using the CISO code signing service, run the following command and ensure that the output is Verified OK:

openssl dgst -sha256 -verify PowerAI_Vision_1.1.5.0_public_key.pub \
-signature visual-insights-ppc-1.2.0.0-ppa.sig visual-insights-ppc-1.2.0.0-ppa.tar

c. To validate the tar file with the signing certificate authority directly, run the following command and ensure that the output includes Response verify OK:

openssl ocsp -no_nonce -issuer visual-insight-ocspchain-1.2-key.pub \
-cert visual-insights-ocsp-1.2.0.0-key.pub -VAfile visual-insight-ocspchain-1.2-key.pub \
-text -url http://ocsp.digicert.com -respout

3. Unzip and untar the product tar file, and run the installation command for the platform you are installing on.

sudo yum install ./.rpm

The install files are extracted to visual-insights-arch-1.2.0.0-ppa.

4. Load the IBM Visual Insights images from the directory that contains the extracted tar file. The user running the script must have Docker privileges:

sudo /opt/ibm/vision/bin/load_images.sh -f ./file_name.tar

Note: The installation process can take some time to complete.

5. Open ports for the firewall to access IBM Visual Insights by running this script:

sudo /opt/ibm/vision/sbin/firewall.sh

6. After the installation is complete, you can start IBM Visual Insights by running this script:

sudo /opt/ibm/vision/bin/vision-start.sh

A user named admin is created with a password of passw0rd.

Note: The startup script will modify ownership and permissions on /opt/ibm/vision/volume so that the containers can run under a non-root ID and access the data. You must read and accept the license agreement that is displayed before you can use IBM Visual Insights.

It can take several minutes to start IBM Visual Insights. To check the status of the startup process, run this script:

sudo /opt/ibm/vision/bin/helm.sh status vision

In the output from the helm.sh status vision script, you can verify which IBM Visual Insights components are available by locating the Deployment section and identifying that the AVAILABLE column has a value of 1 for each component.

After the application startup has completed and the user interface is available, it can be accessed at https://<hostname>/visual-insights/, where hostname is the system on which you installed IBM Visual Insights.

The last, Install any available fix packs.

Installation Issue:

Can not start service because “coredns”.

Solution :

systemctl stop docker
iptable --flush
iptables -tnat --flush
systemctl start docker

References:

IBM PowerAI Vision 1.1.5 Guide

IBM Visual Insights 1.2 Guide

CUDA Driver Installation Guide

Installing on RHEL 7 (Docker & Nvidia-docker2)

Using nvidia-docker 2.0 with RHEL 7

Installation Issue