Weight quantization

Up till now, I was relentlessly pursuing the best accuracy I can get with my neural network. The size of my Core ML app is of course increasing.

While going through the presentation slides of What’s New in Core ML, Part 1, Apple talked about the size of weight being a factor in the overall Core ML app size.

This got me thinking to try post-training quantization on my fully trained models. Training with low precision never worked for me in the past, especially while doing backprogagations.

Core ML Tools 2.0 introduced new utilities to reduce a fully trained model to a lower precision model.


Utilities to compress Neural Network Models. Only available in coremltools 2.0b1 and onwards

On my sandbox app, I managed to reduce the model size from 34Mb to 9.2Mb. This was not the only benefit. The runtime appears to be 2X faster on iPhone XS Max.

In my apps, this is where I feel quantized neural network would make a difference in the inference phase, though accuracy will be dependent on the environment in which the neural network is used.

On a side note, there is also a ‘post_training_quantize’ flag in the TensorFlow Lite conversion tool.

My AI insfrastructure at home

At home, I use machine learning /AI for mainly research purposes as I have the freedom to experiment new ideas or learn new networks without any external constraint and requirement.

The only requirement I sort of imposed to myself is that AI infrastructure must be keep it as simple as possible.

First, because I am its sole user hence deployment and packaging are not an issue. Second, I want to spend most of my time creatively. I am an engineer and a landscape photographer. Creativity is the reason I like fiddling with machine learning.

A « prepare, compute and store » workflow

My AI infrastructure is based on a « prepare, compute and store » workflow. The prepare stage involves preparing my dataset and Jupyter notebooks on my MacBook and controlled under GIT. Sometimes I also use my iPhone to prepare the dataset with iOS shortcut app and store the dataset using the JSON format. The compute phase is the most time consuming phase and I use either TensorFlow or Keras under docker. It is unwise to store the gigabytes under the docker image, hence I use a NAS for storage.

There are cloud based solutions such Domino, Google collaborate, etc which could certainly improve and speed up my infrastructure. However, at home, I am not targeting for a production platform.

What is Epoch, Batch Size and Iterations?

One Epoch is when an ENTIRE dataset is passed forward and backward through the neural network only ONCE.

Since, one epoch is too big to feed at once it is divided into several smaller batches of size batch_size.

Iterations is the number of batches needed to complete one epoch.

For example, a dataset of 1000 samples can be divided into batches of 200 and it will take 5 iterations to complete 1 epoch.

What is input_data in TFLearn?

tflearn.layers.core.input_data (shape=None, placeholder=None, dtype=tf.float32, data_preprocessing=None, data_augmentation=None, name=’InputData’)

In TFLearn, the input_data is the input layer to the neural network. It is used to specify how the input looks like, before adding any of the usual layer in the sequential model.

For example, in the MNIST data set where a 784 array represents 28×28 images with a shape (None, 28, 28, 1] with one color channel. Usually None could be interpreted as the batch_size.

How to install yosys on Fedora 26

Unlike my other posts in the past decade where I referred to install anything from a rpmbuild, today I just want to get the install done and move on.

Prequisites for the yosys installation:
$ sudo dnf install clang tcl-devel readline-devel libffi-devel mercurial iverilog

Yosys build:
$ git clone https://github.com/cliffordwolf/yosys.git
$ cd yosys/
$ make
$ make test
$ sudo make install

Build the Z3 SMT solver
$ git clone https://github.com/Z3Prover/z3
$ cd z3/
$ python scripts/mk_make.py
$ cd build/
$ make
$ sudo make install

Quartus 17.0 Prime install on Fedora 26

After installling Quartus on a x86_64 fedora 26, I fell upon the lib12.so.0 error.

$ quartus

quartus: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directory

The simple solution is

$ sudo dnf install libpng12-devel.i686

Unfortunately I was not unable to program the bitstream on the fpga, thus I started debugging the jtag connection:

$ dmesg | tail
$ lsusb | grep Altera

$ gedit /etc/udev/rules.d/51-usbblaster.rules &

Paste the following on the file:

# For Altera USB-Blaster permissions.
SUBSYSTEM=="usb", ATTR{idVendor}=="09fb", ATTR{idProduct}=="6001", MODE="0666"

$ sudo mkdir /etc/jtagd
$ sudo cp /home/chitlesh/intelFPGA_lite/17.0/quartus/linux64/pgm_parts.txt /etc/jtagd/jtagd.pgm_parts.txt
$ jtagd --foreground --debug

JTAG daemon started
Using config file /etc/jtagd/jtagd.conf
Remote JTAG permitted when password set
No USB device change detection because libudev.so.0 not found
Can't bind to TCP port 1309 - exiting

It appears I was missing libudev.so.0

$ sudo dnf install systemd-devel.i686
$ sudo killall -9 jtagd
$ jtagconfig

1) USB-Blaster variant [2-1.2]
Unable to lock chain - Insufficient port permissions

$ sudo udevadm control --reload
$ sudo killall -9 jtagd
$ sudo /home/chitlesh/intelFPGA_lite/17.0/quartus/bin/jtagconfig

1) USB-Blaster [2-1.2]
020F30DD 10CL025(Y|Z)/EP3C25/EP4CE22

Furthermore, since I imported the nios software projects from windows, I had to delete the .metadata directory to avoid strange incompatibility errors while compiling nios software projects.


uvm_dpi 32-bit requirements on Fedora 26

This morning I was trying to compile uvm’ s dpi plug on Fedora 26 and ended with the following error:

Model Technology ModelSim – Intel FPGA Edition vlog 10.5c Compiler 2017.01 Jan 23 2017
vlog “+incdir+uvm-1.2/src” uvm-1.2/src/dpi/uvm_dpi.cc
** Warning: ** Warning: (vlog-7032) The 32-bit glibc RPM does not appear to be installed on this machine. Calls to gcc may fail.

— Compiling DPI/PLI C++ file uvm-1.2/src/dpi/uvm_dpi.cc
** Error: (vlog-70) Compilation of the C/C++ src files failed with the error messages given below.

In file included from /usr/include/features.h:434:0,
from /usr/include/bits/libc-header-start.h:33,
from /usr/include/stdlib.h:25,
from uvm-1.2/src/dpi/uvm_dpi.cc:33:
/usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
compilation terminated.

The solution is simply:

$ sudo dnf install glibc-devel.i686

Modelsim Intel-Altera 10.5c on Fedora26

This blog post is about how to run Modelsim Intel-Altera 10.5c on Fedora 26 (4.12.13-300.fc26.x86_64) after its installation. Modelsim Intel-Altera 10.5c comes with intelFPGA_pro 17.0.

First install the 32-bit libraries.

$ sudo dnf install glibc.i686 zlib.i686 bzip2-libs.i686 libXft.i686 libXext.i686 ncurses-compat-libs.i686

Download the freetype of el7 from here and copy the so files to modelsim’s internal directory.

$ rpm2cpio freetype-2.4.11-9.el7.i686.rpm | cpio -idmv
$ cp usr/lib/libfreetype.so.6* intelFPGA_pro/17.0/modelsim_ase/lib/

Add the following lines to the .bashrc :

$ export PATH=$PATH:~/intelFPGA_pro/17.0/modelsim_ase/bin
$ alias vsim="LD_PRELOAD=\"/home/$USER/intelFPGA_pro/17.0/modelsim_ase/lib/libfreetype.so.6\" vsim"

If vsim is launched at this time, you will get the following error:

Error: cannot find “/home/chitlesh/intelFPGA_pro/17.0/modelsim_ase/bin/../linux_rh60/vsim”

This error occurs because the intelFPGA’s vsim script has not considered linux kernels 4.*.

[chitlesh@localhost ~]$ uname -r

Add this following line on line 210 of bin/vsim.

4.[1-9][0-9]*) vco="linux" ;;

Below line 50 dir=`dirname "$arg0"`, add

export LD_LIBRARY_PATH=${dir}/lib

If DPI-C is not required, rename all the gcc directories into _gcc and use +UVM_NO_DPI.

That’s it.


If you find this blog post useful, please do leave a comment below to let me know if you have successfully run Altera Modelsim on Fedora.