Nodes
Contents
Information on Installed Nodes
There are 4 "generations" of nodes installed on the cluster. The oldest nodes are AMD dual core, the medium-old nodes are Intel Core 2 quad cores, and the latest nodes are Intel Xeon quad cores and Intel Core i7 quad cores with HyperThreading. Our last acquisition is the AMD Opteron Supernode. The specifications of each node are given in detail below.
All nodes use Fedora 18 as their operating system.
AMD Opteron supernode
Opteron supernodes have 4 CPUs, 12 real cores each, that is 48 physical cores in total. The machines are to be used for highly parallel computations with huge MPI traffic. THey are connected with Infiniband interconnect. Please ask the admins if you would like to use it. Each supernode has a small disk array mounted in /data. If the user of the supernode uses only one node, the user should use that filesystem for I/O.
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| node0 | 4x AMD Opteron 6234 @ 2.4 GHz | 128 GB | 100 GB |
| node1 | 4x AMD Opteron 6234 @ 2.4 GHz | 128 GB | 100 GB |
Intel Core i7 Nodes
Core i7s nodes are quad cores with HyperThreading. This means that they have 4 cores, but present 8 cores to the operating system. Therefore you will see these nodes having 8 CPUs when looking at them in Ganglia. Since they actually only have 4 cores, you will not get a linear increase in speed once the CPU is running more than 4 threads. However, since there still is a perceptible increase in speed, it is beneficial to use all 8 threads rather than just limiting yourself to 4.
We currently have 31 of these nodes in the cluster. They spread three generations of corei7: The gen3 queue is Nehalem (11 nodes), gen4 queue is Sandy Bridge (3 nodes), and gen5 is Haswell (17 nodes).
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| node2 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node3 | Core i7 860 @ 2.80 GHz | 16 GB | 90 GB |
| node4 | Core i7 860 @ 2.80 GHz | 12 GB | 230 GB |
| node5 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node6 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node7 | Core i7 860 @ 2.80 GHz | 8 GB | 90 GB |
| node8 | Core i7 860 @ 2.80 GHz | 12 GB | 90 GB |
| node9 | Core i7 2600 @ 3.40 GHz | 32 GB | 184 GB |
| node10 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node11 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node12 | Core i7 4770 @ 3.40 GHz | 16 GB | 230 GB |
| node13 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node14 | Core i7 860 @ 2.80 GHz | 8 GB | 230 GB |
| node15 | Core i7 860 @ 2.80 GHz | 8 GB | 80 GB |
| node19 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node21 | Core i7 2600 @ 3.40 GHz | 32 GB | 184 GB |
| node26 | Core i7 4770 @ 3.40 GHz | 16 GB | 650 GB |
| node30 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node31 | Core i7 2600 @ 3.40 GHz | 32 GB | 184 GB |
| node35 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node36 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node37 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node38 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node39 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node40 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node41 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node42 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node43 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node44 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node45 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
| node46 | Core i7 4770 @ 3.40 GHz | 32 GB | 80 GB |
Intel Xeon Nodes
There are workstations re-purposed as compute nodes. They are comparable to the Core i7s, but with more cores.
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| node32 | 2x Xeon X5680 @ 3.33GHz | 48 GB | 50 GB |
| node33 | 2x Xeon X5660 @ 2.80GHz | 48 GB | 50 GB |
Intel Xeon Virtual Nodes
There are five virtual nodes running inside Windows workstations. Each of them has a quad core Intel Xeon of SandyBridge architecture, identical to the Core i7s, but without HyperThreading. This machines are connected to the rest of the cluster nodes by a virtual private network (VPN), which is ~1.5 order of magnitude slower than the Gigabit interconnect used by non-virtual nodes. Therefore these machines should not be used for MPI jobs. Virtual nodes are best suited for sequential programs, OpenMP jobs, and code development.
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| vnode1 | Xeon E31220 @ 3.10 GHz | 8 GB | 50 GB |
| vnode2 | Xeon E31220 @ 3.10 GHz | 8 GB | 50 GB |
| vnode3 | Xeon E31220 @ 3.10 GHz | 8 GB | 50 GB |
| vnode4 | Xeon E31220 @ 3.10 GHz | 8 GB | 50 GB |
| vnode5 | Xeon E31220 @ 3.10 GHz | 8 GB | 50 GB |
Intel Core 2 Quad Nodes
These are slightly older machines. They have 4 physical cores and no HyperThreading, so when you look at them in Ganglia you will see that they have 4 CPUs. They are still reasonably fast.
We currently have 5 of these nodes in the cluster. In addition, node18 has a similar architecture, but has 2 CPUs (8 cores total).
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| node18 | 2x 4-core Xeon L5420 @ 2.50 GHz | 8 GB | 80 GB |
| node25 | Core 2 Quad Q8200 @ 2.33 GHz | 8 GB | 649 GB |
| node27 | Core 2 Quad Q8200 @ 2.33 GHz | 8 GB | 649 GB |
| node28 | Core 2 Quad Q8200 @ 2.33 GHz | 8 GB | 10 GB |
| node29 | Core 2 Quad Q8200 @ 2.33 GHz | 8 GB | 90 GB |
| node34 | Core 2 Quad Q6600 @ 2.40 GHz | 8 GB | 50 GB |
AMD Dual Core Nodes
These are the "ancient" machines. They have two physical cores, and also no HyperThreading. If speed isn't that important you should use these nodes. They're also good at adding some speed to your MPI MCNP runs when using the balance flag. Compiling and other disk I/O tasks are also suited well to these nodes, since all nodes have the same delay when writing to the file servers.
We currently have 6 of these nodes in the cluster.
| Node Name | Node CPU | RAM | Swap |
|---|---|---|---|
| node16 | Athlon 64 X2 5200+ | 8 GB | 184 GB |
| node17 | Athlon 64 X2 5200+ | 8 GB | 184 GB |
| node20 | Athlon 64 X2 5200+ | 8 GB | 184 GB |
| node22 | Athlon 64 X2 5200+ | 8 GB | 10 GB |
| node23 | Athlon 64 X2 5200+ | 8 GB | 10 GB |
| node24 | Athlon 64 X2 5200+ | 8 GB | 10 GB |
