Helix Systems > New Users > User Guides
 

Disk Storage on Helix Systems

[/home/...] [/scratch/...] [/scratch/export] [/stripe/...] [/afs/...] [/data/...]


 

Home Directories (/home/...)

A home directory is established for each new user when his or her account is created, as a subdirectory of /home, with an initial disk quota of 200 megabytes. Home directories are shared between the SGI systems. That is, the same files and directories appear in your home directory on helix, nimbus and quasar. Files in home directories may be considered permanent in the sense that they remain on the system as long as the user wishes. Home directories are backed up to tape on a daily basis. Users who are over quota at the time of login will receive a warning message at that time.

Charge

Disk storage in home directories is charged at the rate $0.0033/megabyte/day.

Technical notes

  • The directories in /home are actually symbolic links pointing to user filesystems named /u1, /u2, /u3, etc. Because the actual physical location of your home directory is subject to change, you should always refer to it by the /home path in programs and scripts.
  • Home directories actually reside on disks attached to specialized file servers. They are made accessible to the other systems via the Network File System (NFS).

A user may simply need more permanent disk storage in his or her home directory. Users who require higher quotas should use the ptr command at the command prompt to make their request.

Temporary Storage (/scratch/...)

Instead of /tmp or /usr/tmp, the /scratch directory should be used for the temporary storage of relatively large amounts of data. We recommend that a user create a directory in /scratch and place files there. The /scratch directory is shared across all of the SGI systems in the same way as /home. While /tmp and /usr/tmp are the traditional temporary directories on UNIX systems, users are discouraged from making use of them on the NIH Helix Systems since they are often used by system utilities, compilers, and application programs. Files in /tmp and /usr/tmp are subject to removal at any time.

Files in /scratch which have not been accessed for 14 days are automatically deleted. The usefulness of /scratch depends on users' cooperation. It is inappropriate to regularly 'touch' files in /scratch as a means for extending the 14 day time limit. Users who require more permanent disk space should request it. See table below.

Files on /scratch are not backed up. Disk storage on /scratch is not charged.

Extra Storage

Storage

/home/...

/scratch/...

/scratch/export

/stripe/...

/afs/...

/data/...

Purpose

home directory

temporary files

temporary files with NFS access

high performance I/O

AFS home and group directories

large files

Shared by

all SGI systems

all SGI systems

all SGI systems

not shared

all SGI systems

all SGI systems and Cluster

Quotas

yes

no

no

yes

yes (ALW)

yes

Creation

with Helix account

created by user

created by user

by request via ptr command

with ALW account

with Biowulf account

Backups

daily

none

none

daily

yes (ALW)

weekly (RAID disks)

Time Limits

none

deleted at 14 days

deleted at 14 days

none

none

none

High Performance

no

no

no

yes

no

no

Charged

yes

no

no

no (yes later)

yes (ALW)

no (yes later)

Network Disk Storage (/scratch/export)

/scratch/export, a subdirectory of /scratch, has the same characteristics as /scratch, but in addition is exported as part of the Network File System (NFS). This space is accessible from workstations which can act as NFS clients. Data in this directory should not be considered secure since it is accessible from the network. The /scratch/export filesystem is shared between all SGI of Helix Systems. To mount /scratch/export using NFS, use the following mount command:

mount helixscratch.nih.gov:/scratch/export /mnt

where /mnt is a directory present on your workstation.

High Performance Disk Storage (/stripe/...)

The /stripe file systems on nimbus and quasar have been configured to give maximum input-output performance for large files. Users who have applications which do intensive file I/O will benefit from placing files on /stripe.

The /stripe file system is managed more like home directories than temporary directories. On request, users are given a directory on /stripe with the same name as their home directory. For instance, user 'quux' with a home directory of /home/quux would also have a striped directory, /stripe/quux. For convenience, users may want to create a symbolic link in their home directory which points to the striped directory, e.g., /home/quux/stripe -> /stripe/quux. This would be accomplished with the command:

ln -s /stripe/quux /home/quux/stripe

Directories on /stripe are assigned quotas, are backed up on a daily basis, and may be considered permanent. However, files which have not been accessed for 6 months are subject to automatic compression by the system. The /stripe filesystem is not shared between systems.

**Disk storage on the /stripe file system is not currently charged, but will be at a to-be-announced date in the future.**

Users who need striped disk storage should contact the Helix systems staff via the ptr command (simply type ptr at the command prompt and follow the instructions).

Andrew File System (/afs/...)

AFS is a network distributed filesystem with its root at /afs, and is accessible from all SGI systems. Access to the AFS filespace is controlled by authentication to various AFS administrative "cells". Helix Systems users with NIH ALW (Advanced Laboratory Workstation) accounts may authenticate themselves by using the klog command at the command prompt. AFS/ALW filespace is administered by the ALW Project.

Storage Shared with the Biowulf Linux Cluster (/data/...)

In addition to its SGI systems, the Helix Systems also include a Linux cluster named Biowulf. All of these systems, SGI and the cluster, also have access to additional permanent storage on 4 Network Appliance filers, which are specialized high-performance NFS fileservers using RAID-4 disk arrays. This storage is accessible as the /data directory and is used for very large datasets. Filesystems accessible through /data are the only ones shared across SGI systems and the Linux cluster.

Using Disk Space Efficiently

Use the gzip program to reduce the amount of disk space used by infrequently accessed files. Depending on the file, this can significantly reduce space requirements. For example, the disk space required for the text file below was reduced by more than 50%:

helix% ls -l

-rw------- 1 quux 925594 Apr 20 12:48 TDATA

helix% gzip TDATA

helix% ls -l

-rw------- 1 quux 334198 Apr 20 12:50 TDATA.Z

Note that compressed files are renamed by the gzip command to have an extension of .z. The gunzip command is used to return a compressed file to an uncompressed form. The zcat command is used to produce an uncompressed byte stream from a file without uncompressing it, for example:

zcat huge-postscript-file.Z | lpr

See the gzip man page for more information.

 

 

Helix Systems, CIT, NIH
last update: October 16, 2003