HPC Instructions

You must request access to the cluster prior to usage. Please send an email to help@stat.ncsu.edu requesting access to the cluster. This cluster is only available to users in the Department of Statistics.

Prerequisites

Off campus users must be connected to the campus VPN service before attempting the following instructions.

File Transfer

You need to transfer your files to and from the cluster. This is done using SFTP protocol.

  • {from mac terminal} $ scp [program] unityid@hpc.stat.ncsu.edu:
  • {from pc} start winscp and connect to hpc.stat.ncsu.edu with your Unity ID and Campus Password
    • Drag and Drop files to the HPC server

Login to Server

You need to login to the server. This is done using SSH protocol.

  • {from mac terminal} $ ssh unityid@hpc.stat.ncsu.edu
  • {from pc} start putty and connect to hpc.stat.ncsu.edu with your Unity ID and Campus Password
  • Submit a Program

    There are 2 ways to submit a program to run on the cluster.

    1. bwsubmit : will submit a job that will allow you to use 1 processor core and up to 8GB of RAM. The run time limit is 1 day (24 hours) for these jobs. This is meant for people who need to run many quick scripts. If your code runs for more than 1 day or you need more than 8GB of RAM, you should to use parallel programming and run with bwsubmit_multi. Because the limit is 1 day, you can use 50 cores at a time. There is a separate machine and queue for 1 processor jobs, so 128 total cores are available all users combined. After 128 cores are in use, jobs will queue up. Job run times may vary significantly because hyper-threading is enabled on this machine to utilize it completely.
    2. bwsubmit_multi : will schedule a job using 4, 8, or 16 processor cores each and 8GB of RAM for each processor core. Request multiple processors to get more memory allocation, even if you do not use the cores. Your job will also be limited to the number of processors you request. If you try to use more than you have requested, the performance of your program will suffer greatly. The BLAS/LAPACK libraries will automatically use the number of cores you request.

     

    Important Notes: If your program uses more than the allocated amount of RAM, or your job runs for more than 1 day with bwsubmit, 10 days with bwsubmit_multi, the scheduler will kill your job. You can have up to 32 cores worth of jobs running at a time (16 max per job). If the cluster fills up completely, it will limit everyone to 16 cores running at a time to try to serve more people. It automatically adjusts every hour. Do not submit jobs under other users names to bypass the per user core limit. This is a violation of NC State policies and both accounts will be heavily restricted or removed from the cluster.

  • bwsubmit [program] [filename] OR bwsubmit_multi [processor_cores] [program] [filename]
  • EXAMPLES:
    • bwsubmit sas program.sas OR bwsubmit_multi 4 sas program.sas
    • bwsubmit r sim.r OR bwsubmit_multi 8 r sim.r
    • bwsubmit exec fortran.exe OR bwsubmit_multi 16 exec fortran.exe
    • bwsubmit matlab prog.m OR bwsubmit_multi 4 matlab prog.m
    • WARNING: System is case sensitive.
      Meaning if your program is named Program.Sas it must be submitted as

      bwsubmit sas Program.Sas

    Check for Installed R-Packages

  • R-PkgCheck {package_name1 package_name2}
  • R-PkgCheck akima MASS

    Useful Commands

  • bwcheck – Check the status of the jobs/queues.
  • bwkill {jobnum} – To stop/kill a submitted job.
  • bwload – To see the cluster load/status.
  • quota – To see your current quota usage.
  • nano {filename} – Allow you to make easy modifications to your files.
  • LogOff and Wait

  • exit — LogOff Server
  • You’ll receive an email when your program has finished running. Once complete transfer your results from the cluster. Repeat steps for additional simulations. Please start small to understand this environment.