Dr. Yun Wang

Dr. Yun Wang

Bioinformatician.

hpc commands

2 minute read

Published:

useful hpc commands

OTP

Get the SECRET by scanning it with you preferred App (Google Authenticator; iPhone Passwords),

or with QR code Generator (https://apps.apple.com/app/id1601025694) to extract the secret under the link of the form: totp/user%20TOTP53846126?secret=SECRET&period=30&digits=6&issuer=MPCDF&image=https%3A//selfservice.mpcdf.mpg.de/images/MPCDFLogo.png”

Extract the secret from the link with:

QRCODE="totp/user%20TOTP53846126?secret=<SECRET>&period=30&digits=6&issuer=MPCDF&image=https%3A//selfservice.mpcdf.mpg.de/images/MPCDFLogo.png"

SECRET=$(echo ${QRCODE} | awk -F'[=&]' '{print $2}')

Add the function encode_otp and decode_otp to your ~/.zshrc.

encode_otp () {

  USAGE="usage: encode_otp SOME_SECRET"

  if [ -z ${1} ] ; then printf "${USAGE}" ; return 1 ; fi

  echo -n "OTP name: "
  read OTP_NAME
  echo ""
  echo -n "Encryption password: "
  read -s  PASS


  echo ${1} | openssl enc -aes-256-cbc -md sha512 -pbkdf2 -iter 1000000 -salt -out ~/.otp_${OTP_NAME}.ae -k ${PASS}

}

decode_otp () {

  USAGE="usage: decode_otp OTP_NAME"

  if [ -z ${1} ] ; then printf "${USAGE}" ; return 1 ; fi
  if [ -z ${2} ] ;
    then
      echo -n "Encryption password: "
      read -s  PASS
      echo ""
  fi

  SECRET=$(openssl enc -d -aes-256-cbc -md sha512 -pbkdf2 -iter 1000000 -salt -in ~/.otp_${1}.ae -k ${PASS} 2> /dev/null )
  if [ $? = 0 ] ;
   then
     echo -n "OTP: "
     oathtool -b --totp "${SECRET}" | pbcopy
     pbpaste
  else
    echo "failed to collect otp .."
    return 1
  fi

}

Use encode_otp ${SECRET} to encode the secret.

Now every time you need the OTP code just type: decode_otp <OTP_NAME>. The OTP will be directly on your clipboard.

For windows, change oathtool -b --totp "${SECRET}" to python oathtool.py "${SECRET}", see oathtool.py, remove pbcopy and pbpaste

build a tunnel

ssh -L 2222:raven.mpcdf.mpg.de:22 wangy@gate2.mpcdf.mpg.de In another terminal, run scp -P 2222 wangy@localhost:/home/file_to_copy ./

get computing node and run R

srun -t 1440 --mem 30G --cpus-per-task 10 --pty  bash
singularity exec -B /nexus:/nexus /nexus/posix0/MAGE-flaski/service/images/posit-latest.sif   /bin/bash
export R_LIBS_USER="/nexus/posix0/MAGE-flaski/service/posit/home/wangy/R/x86_64-pc-linux-gnu-library/4.4"
/opt/R/4.4.0/bin/R

limit submitted jobs

for file in $( ls *.fa ); do
    rm -rf ${logs}search.${file%.fa}*.out
    while [  $(squeue -u wangy | wc -l) -gt $RUNNING_jobs ];
        do echo "sleeping"; sleep 300
    done

    if [[ ! -f ${outDB_dir}${file%.fa}.1e-10.m8 ]] ; then
        ids=${ids}:$(sbatch --partition $SLURMPARTITION --parsable << SOF
#!/bin/bash
#SBATCH --cpus-per-task=8
#SBATCH -o ${logs}search.${file%.fa}.%j.out
#SBATCH --job-name="mmeqs search"
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --time=30:00:00
#SBATCH --mem=100000
if [[ "$SERVER" == "RAVEN" ]] ; then module load singularity ; fi 
${SING} << SIN
#!/bin/bash
${HOMESOURCE}
##-- threads
    mmseqs search --threads 8 ${query_dir}${file%.fa} ${database_dir}${database}  ${outDB_dir}${file%.fa}.1e-10.m8 ${tmp}tmp_${file%.fa} -e 1E-10 --max-seqs 50000
SIN
SOF
)
    fi
done

tmux

put this piece of code in bash_profile

if [ -z "$TMUX" ]; then
    (tmux ls | grep -vq attached && tmux at) || tmux
fi

You May Also Enjoy