FAQ: eJet
- What is eJet?
- How fast is it?
- Why do I want to use it?
- How do I get started?
- How do I access eJet?
- How do I use eJet?
- What file systems are available?
- Does ejet have cron servers?
- Where are the vis nodes?
- I need more memory for my job than is available on a compute node (or what is a serial node)?
- How do I compile my code?
- How do I submit jobs?
- How do I ask for help?
A: eJet is an HPC cluster at GSD. It uses Intel processors and Myricom interconnect to increase performance over iJet. We have seen 100% speed increases for RUC20, WRF, and MM5 over runs on iJet.
Performance of the system isn't improved by CPU performance alone. The storage system will provide 50 TB, over 3 times more than is currently available. More importantly, the speed at which processes can read and write to the storage will increase by an order of magnitude.
Usage of eJet should feel the same as iJet. It is necessary to recompile your codes, but the compilers and tools are the same. There are a few exceptions to this, please read the FAQ for more details.
System specifications:
Compute nodes:
296 Dual EM64T 3.2 GHz (64- bit Intel Xeon processors)
2 GB RAM
Myrinet PCIX-D NIC. (latency is 7 microsec, bandwidth is 460 MB/s bidirectional).Serial nodes:
4 Dual EM64T 3.2 GHz
16 GB RAM
Myrinet PCIX-D NIC.Vis nodes:
2 Dual EM64T 3.2 GHz
8 GB RAM
Myrinet PCIX-D NICStorage:
Capacity: 50 TB.
Performance: Approx. 1.2 GB/s aggregate
Single process performance over 100 MB./sA: The performance of eJet will depend on your code. However in many instances, the performance of iJet is about twice that of eJet. Please see the following links for graphs showing the performance of eJet as compared to other systems.
WRFV2: http://box.mmm.ucar.edu/wrf/bench/wrf-perf.bmp
MM5: http://www.mmm.ucar.edu/mm5/mpp/helpdesk/20040304a.htmlA: iJet is still going to be around, so if iJet works fine for you, then keep using it. However, if your code is IO bound, doesn't scale well, or you would need to run larger jobs, eJet is for you. Also, if your code requires more memory that iJet has, or you need single processor nodes that have lots of memory (up to 16 GB) this is also a good reason to use eJet.
A: Get an account on eJet. If you have an account on iJet, you should already have an account on eJet. Since the home file system is different for eJet, please email hpcshelp.fsl@noaa.gov and have them ensure your account is setup properly on eJet before you begin.
You will need to request a change to your project allocation so that you may use the eJet system
A: All authorized users of an approved project will need to complete the User Application.
You can access ejet by ssh -l username ejet.fsl.noaa.gov after your application has been approved.
A: Usage of eJet is very similar to iJet. The instructions below assume some familiarity with iJet. Please see the other FAQ titles regarding Building and Compiling code.
Q: What file systems are available?
A: The eJet home file system is /ahome. You can access your iJet home file system as /ihome. Since eJet and iJet are different architectures, we recommend you only do this to transfer source file systems between the two systems.
Q: Does ejet have cron servers?
A: Yes! They are called ecron0 and ecron1. You may install crontabs on either of these nodes. Unless you processes only run for a few seconds, please use the cron servers to schedule processes to the cluster via qsub.
A: There will be visualization nodes for eJet. They are named evis0 and evis1. If you require certain software that typically are available on the vis servers, please request they be installed.
Q: I need more memory for my job than is available on a compute node (or what is a serial node)?
A: Our nodes have been configured with 16 GB of ram for those pre-/post- processing applications that need lots of memory and cannot be parallelized. You may submit single processor jobs to the serial nodes. The parallel environment name is called eserial.
- A process on the serial nodes may only use 1 cpu.
It is helpful to request how much memory you are going to use for the process. Please add the SGE option -l mem_size=(memsize)M is the amount of ram in MB. If you are not sure how to figure out home much memory your application will use, ask for help.
A: Currently only the Intel compilers are installed. We are working to provide the 64-bit Portland Group compilers.
Compiling codes on eJet with the Intel compiler is the same as iJet.
The C/C++ compiler is named icc.
The Fortran 77/90/95 compiler is named ifort.When you are compiling MPI applications, please use the wrapper scripts, mpicc, mpif77, and mpif90.
Good optimizations settings for the Intel compiler are "-O3 -axP".
If you are familiar with the PGI compilers but want to use the new system, please ask for help on using the Intel compilers. We will let you know when the PGI compilers are ready.
A: Job submission works the same on eJet as iJet. The output of sgestat is a bit different, but the options to qsub are the same.
Please use the parallel environment 'ecomp' to access ejet.
A: Please email your questions to hpcshelp.fsl@noaa.gov. If your question pertains to a current or previous job, please provide the job number, the location of the submission script, and the location of the output files.
