Accessing Data on hola

Accessing the "alfred" Account

The data is accessible through the "alfred" account on hola.cosmo.fas.nyu.edu, in the same manner as the 2D spectroscopic reductions at Princeton. Essentially, you should send me your SSH Version 1 protocol public key (with a passphrase, please) and I will add it to the authorized keys on the alfred account. If you know what I mean by that, fine and dandy. If not, please follow the steps below.

Because of incompatibilities between different implementations of ssh2 (OpenSSH versus Secure Shell SSH), we will only be supporting ssh1 access to hola. So the first thing to do is to determine how to access ssh1 on your system, by typing:
```
ssh -V
```
to see the ssh version. If the version is 1.xxx, then you already use ssh1. If the version is 2.xxx, you can access ssh1 by typing ssh1 or ssh -1. Similar rules will apply to other ssh commands, in particular scp. Whichever method worked for ssh (putting "1" at the end of the command or giving the argument "-1") will also work for these other commands. In more detail, if "ssh -V" gives you a message like:
```
ssh: SSH Secure Shell 2.4.0 (non-commercial version) on i686-pc-linux-gnu
```
you can access ssh1 using the command ssh1. You will typically find this on RedHat Linux 6.x machines. If you see a message like:
```
OpenSSH_2.5.2p2, SSH protocols 1.5/2.0, OpenSSL 0x0090600f
```
you can access ssh1 using the command ssh -1. You will typically find this on RedHat Linux 7.x machines. If you see a message like:
```
SSH Version 1.2.27g [i686-unknown-linux], protocol version 1.5.
Standard version.  Does not use RSAREF.
```
you access ssh1 using just the command ssh. Fermi machines all have ssh1 by default.
Now, run the command:
```
ssh-keygen -t rsa1
```
First, it will ask you where you want to store your key. The default is fine. Next, it will ask for a passphrase, which you should give (and remember). Then it will create a world-readable file in your .ssh directory called "identity.pub". This is a public key; it will not compromise your account in any way to send it on email or to plaster it on billboards or anything else. So ...
Email me the contents of .ssh/identity.pub. If you can include it in your email without having your mail program linewrap, that is better. If not, I can deal. I will add this to the "authorized_keys" file on the alfred account. This will make it possible for you to log into hola using your passphrase. I will email you back when this is done (minutes if I am online).
Finally, test the account by logging into alfred:
```
ssh1 alfred@hola.cosmo.fas.nyu.edu
```
(Using "ssh" or "ssh -1" as determined by the first step). You will be able to browse the LSS samples in the /global/data/sdss/lss/sample# directories on the machine. Now you will be ready to download the data you need.

Downloading the Data

Once you have access to the machine, you can retrieve the data from "/global/data/sdss/lss/sample#" using scp or rsync. Usually, there is no reason to use anything but the latest sample (the highest number). The details of the file structure within each sample should be contained in the README or READIT files. "rsync" provides an excellent way to keep your data up to date. For example, if you are at your local machine, you can use the command:

rsync -arv --rsh="ssh1" alfred@hola.cosmo.fas.nyu.edu:/global/data/sdss/lss/sample8/ .

The "ssh1" should be replaced with the appropriate command for the machine you are using (as determined in the first step above). This command will download the whole "sample8" tree into the directory you are in. "rsync" is smart, in the sense that if you run the command again, it will only update the directory (if we made changes to the distribution). So one might run the rsync periodically using cron.

Many people will only be interested in the large-scale structure samples -- ie. ra, dec, redshift, and weights. This information is all compactified into several different samples, and are all in the "lss" subdirectory of the main "sample#" directory. So to just get the large-scale structure samples, do:

rsync -arv --rsh="ssh1" --include "lss" --include "lss/**" --exclude "*" alfred@hola.cosmo.fas.nyu.edu:/global/data/sdss/lss/sample8/ sample8

Again, repeating the command will just update your files (much quicker than replacing everything with scp!).

Snail Mail: Michael Blanton; Email: mb144@nyu.edu