# Instructions

*Written by Cheng-Wei Lee* (clee2 \[at] mines \[dot] edu)

***

The very first step is to log into [Kestrel](https://www.nrel.gov/hpc/). Since there is no external login nodes available yet, you need to use login the SSH gateway first via

```
ssh -AY [user name]@hpcsh.nrel.gov
```

{% hint style="info" %}
\[user name] is your user account 

-AY is for X11 forwarding and is optional
{% endhint %}

Once on the ssh gateway, you can log into Kestrel by

```
ssh [user name]@kl2.hpc.nrel.gov
```

{% hint style="info" %}
there are multiple login nodes for Kestrel, e.g. kl1 and kl2
{% endhint %}

Once on the login nodes of Kestrel, we can load the following modules

```
source /nopt/nrel/apps/env.sh
module purge
module load anaconda3/2022.05
module load PrgEnv-intel/8.5.0
module swap cray-mpich cray-mpich-abi
module unload cray-libsci
module load intel-oneapi-compilers/2023.2.0
module load intel-oneapi-mpi/2021.10.0-intel 
module load intel-oneapi-mkl/2023.2.0-intel
```

{% hint style="info" %}
We only need anaconda3 to set up pylada.  The rest are the libraries used to compile VASP and thus needed to run VASP

We need to use the specific mpi, which can be different from the default one
{% endhint %}

Once anaconda is loaded, we can now use conda to create a python virtual environment

```
conda create -n [name] python=3.9
```

{% hint style="info" %}
\[name] is the user defined name for the virtual environment

python 3.9 has been working fine for pylada and pymatgen
{% endhint %}

Once the virtual environment is set up, we can activate it via

```
conda activate [name]
```

{% hint style="info" %}
This requires the module of anaconda3 being loaded
{% endhint %}

With the environment set up, we also need git to install pylada

```
module load git
```

Now with everything set up, we can follow the instruction on the [pylada's github website](https://github.com/pylada/pylada-light)

(We use pip to install pylada)

```
pip install git+https://github.com/pylada/pylada-light
```

{% hint style="info" %}
One common error message is likely due to lack of compiler 
{% endhint %}

Once the pylada is installed, we need to create a file name "ipython\_config.py" with the path of

```
~/.ipython/profile_default/ipython_config.py 
```

And it needs to have the following content

```
c = get_config()
c.InteractiveShellApp.extensions = [ "pylada.ipython" ]
```

In order to submit jobs using pylada, we also need the following file at home directory

```
~/.pylada
```

with the following content

```

vasp_has_nlep = False

################## QDEL definition ################

mpirun_exe = "srun --mpi=pmi2 -n {n} {program}"   # Kestrel requires --mpi=pmi2 for vasp

qdel_exe = "scancel"

qsub_exe = "sbatch"

################### QSTAT definition ##############
def ipython_qstat(self, arg):

  """ Prints jobs of current user. """
  from subprocess import Popen, PIPE
  from IPython.utils.text import SList
  # get user jobs ids
  jobs   = Popen(['squeue', '-u', '[user name]','--format','%.18i %.9P %j %.8u %.2t %.10M %.6D %R'], stdout=PIPE, encoding='utf-8').communicate()[0].split('\n')

  names  = [lines.strip().split()[2] for lines in jobs[1:-1]]
                                                  
  mpps   = [lines.strip().split()[0] for lines in jobs[1:-1]]
                                                  
  states = [lines.strip().split()[4] for lines in jobs[1:-1]]
                                                  
  ids    = [lines.strip().split()[0] for lines in jobs[1:-1]]

  return SList([ "{0:>10} {1:>4} {2:>3} -- {3}".format(id, mpp, state, name)   \
                 for id, mpp, state, name in zip(ids, mpps, states, names)])

##  return SList([ "{0}".format(name)   \
##                 for id, mpp, state, name in zip(ids, mpps, states, names)])	

##################### PBSSCRIPT #####################

pbs_string =  '''#!/bin/bash -x
#SBATCH --account={account}   
#SBATCH --nodes={nnodes}
#SBATCH --ntasks-per-node={ppn}
#SBATCH --export=ALL
#SBATCH --time={walltime}
#SBATCH --job-name={name}
###SBATCH --mem=176G.  # uncomment to requrest specific memory (usually not needed)
#SBATCH --partition={queue}
####SBATCH --qos=high # uncommented if high qos is needed (usually not needed)
###SBATCH -o out
###SBATCH -e err

# Go to the directoy from which our job was launched
cd {directory}

# Make sure the library are loaded properly for vasp
source /nopt/nrel/apps/env.sh
module purge
module load anaconda3/2022.05
module load PrgEnv-intel/8.5.0
module swap cray-mpich cray-mpich-abi
module unload cray-libsci
module load intel-oneapi-compilers/2023.2.0
module load intel-oneapi-mkl/2023.2.0
module load intel-oneapi-mpi/2021.10.0-intel 

source activate [path to virtual environment] 
export OMP_NUM_THREADS=1 #turns off multithreading, needed for VASP

{header}
python {scriptcommand}
{footer}
'''

```

{% hint style="info" %}
There are two places requiring modification

1. \[user name] is your account
2. \[path to virtual environment] is the path to your python environment.  you can check the python by <mark style="background-color:purple;">conda info -e</mark>
   {% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://3d-materials-lab.gitbook.io/3dmaterialslab-tutorials/installing-pylada-on-hpc/instructions.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.