Installing Pytorch and Transformers on IBM Power 9 architecture
To install pytorch and related components on the power 9 architecture follow these steps.
The first step is to setup conda by following conda docs.
  • Specifically, I downloaded Anaconda installer from here (scroll at the bottom of the page). I am using the the 64-Bit (Power8 and Power9) Installer (276 MB), and at the time of writing has the following download link.
  • You might need to restart your terminal after the above step.
1
source activate
Copied!
After setting up conda, follow the documentation related to IBM cloud AI to install python modules. But make sure you are using the latest channel. Alternatively, copy the below commands.

Add a new config channel.

1
conda config --prepend channels \
2
https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/linux-ppc64le/
Copied!
Note that I am using the early release version rather than 1.7 stable version. I am doing so because of the sentencepiece module needed for the transformers (huggingface).

Create a new conda environment and activate it

1
conda create --name early_access python=3.6
2
conda activate early_access
Copied!

Install pytorch and ipython.

1
conda install pytorch
2
conda install ipython
Copied!
List of all packages available for this channel are here.
Installing Huggingface transformers.
  • Install rust compiler
  • Restart terminal
    • source activate
  • Download Huggingface source and follow the below instructions.
1
git clone https://github.com/huggingface/transformers
2
cd transformers
3
pip install .
Copied!

Check if everything is working.

1
ipython
2
>> import torch
3
>> import transformers
Copied!
Copy link
Contents