Playing with iSCSI on Solaris.
I've been hearing bits and pieces about iSCSI for a while now, but not much of how to use it. So I decided to grab my two toy Solaris machines and give it a shot.
It still seems to be somewhat underdocumented; although there's a docs.sun.com page for iSCSI administration, it mostly centres around the initiator (read: client), not so much from the target (read: server). The iscsitadm man page doesn't even exist on my machines, although I notice that it's present in the OpenSolaris source tree, so it's at least in the pipe for the future.
To set up a very simple shared "LUN", I used my PC running OpenSolaris as the target/server, and my SPARC running Solaris 10 06/06 as the initiator/client.
On my PC, the first thing I did was create a scratch-area to hold the backing store. Easy enough, with ZFS:
PC# zfs create zp01/iscsi-play
The next part is to create the "LUN". This can be a raw disk, but instead I went with a plain file as a backing store, which seems to be the default. Also unstated is that the target is created as a "disk" device... there are other options, including "tape", and "raw".
PC# iscsitadm create target -b /zp01/iscsi-play/test-volume -z 10g test-volume
And then you can see the following:
PC# ls -l
total 551255
-rw------- 1 root sys 10737418240 Feb 22 16:47 test-volume
If you think it created such a big file awfully fast, you're right. For the moment, test-volume is a
sparse file. Keep that in mind the first time you write to the LUN/file. It'll be mind-bogglingly slow as the OS fills out the file.
I can see the target I've created with the following:
PC# iscsitadm list target -v
Target: test-volume
iSCSI Name: iqn.1986-03.com.sun:02:241dafd0-6d3b-ef2f-e6c9-8b62f63ece19.test-volume
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0100000c7694924000002a0045de016a
VID: SUN
PID: SOLARIS
Type: disk
Size: 10G
Backing store: /zp01/iscsi-play/test-volume
Status: offline
Note the "Status: offline" bit? We can fix that next. Also, I need to make note of that long "iSCSI Name" string... I'll need that later.
First, I want some kind of authentication. So I set a CHAP username and password with the following commands:
PC# iscsitadm modify admin -H shumira
PC# iscsitadm modify admin -C
Enter secret:<gimmesomedisk>
Re-enter secret:<gimmesomedisk>
So now I have a CHAP username of 'shumira' and a password/secret of 'gimmesomedisk'.
Next, I needed to create a Target Port Group Tag (TPGT). I do so with the following:
PC# iscsitadm create tpgt 1
PC# iscsitadm modify tpgt -i <my PC's IP address> 1
This creates a TPGT and associates my IP address with it. I'm not sure if it was strictly necessary in my case -- although if I had multiple interfaces, and I can see a machine using iSCSI having a dedicated high-speed NIC to the purpose -- I'd probably want to.
Now, since I'm not using iSNS or some "dynamic" discovery method, I enable static discovery...
PC# iscsitadm modify admin -S enable
Now, for the client!
On my initiator box, I need to set up discovery and create a static configuration so it can find the target. I begin with the following, using iscsiadm:
PC# iscsiadm add discovery-address <my PC's IP>
That lets iSCSI know which machine to discover LUNs on. Next, I need to provide the SPARC with the pre-set CHAP authentication credentials:
SPARC# iscsiadm modify initiator-node --authentication CHAP
SPARC# iscsiadm modify initiator-node --CHAP-name shumira
SPARC# iscsiadm modify initiator-node --CHAP-secret
Enter secret:<gimmesomedisk>
Re-enter secret:<gimmesomedisk>
Now I have to add a static configuration. Remember the "iSCSI Name" from way up above? I need that now.
SPARC# iscsiadm add static-config iqn.1986-03.com.sun:02:241dafd0-6d3b-ef2f-e6c9-8b62f63ece19.test-volume,<my PC's IP address>,1
The ",1" at the end is the TPG tag I created above. So the string is basically <iSCSI Name>,<IP>,<TPGT>
Now, I have to enable static discovery on the client's side, as well:
SPARC# iscsiadm modify discovery --static enable
And that should be that. Now I used devfsadm to discover the new "disk", and format to check that it's there.
SPARC# devfsadm -i iscsi
SPARC# format
Searching for disks...done
c1t0100000C7694924000002A0045DE016Ad0: configured with capacity of 10.00GB
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <QUANTUM FIREBALLP AS60.0 cyl 28739 alt 2 hd 16 sec 255>
/pci@1f,0/pci@1,1/ide@3/dad@0,0
1. c1t0100000C7694924000002A0045DE016Ad0 <SUN-SOLARIS-1 cyl 32766 alt 2 hd 4 sec 160>
/scsi_vhci/ssd@g0100000c7694924000002a0045de016a
Woo-hah. There it is. From there, you can label and slice it just like you would any other disk in Solaris. Keep in mind that since the backing store is a sparse file, you can expect some godawful performance at first. At one point I'd thought that newfs had hung.
And there you have it. An iSCSI-shared LUN. Surprisingly easy, actually. Keep in mind that this is a very basic setup, and I am definitely no expert. I would be surprised if I *haven't* done something wrong, which might make it insecure or inefficient. But it was enough to get the idea, which I've written down here, mostly for myself.
How do I undo all this?
Well, first, if I've newfs'd/mounted the filesystem, unmount it. Then I get rid of the static mapping on the SPARC side with the following:
SPARC# iscsiadm list static-config
Static Configuration Target: iqn.1986-03.com.sun:02:241dafd0-6d3b-ef2f-e6c9-8b62f63ece19.test-volume,<my PC's IP address>:3260,1
SPARC# iscsiadm remove static-config iqn.1986-03.com.sun:02:241dafd0-6d3b-ef2f-e6c9-8b62f63ece19.test-volume,<my PC's IP address>:3260,1
Update the iSCSI driver, and then clean up lost device nodes under /dev/dsk and /dev/rdsk...
SPARC# devfsadm -i iscsi
SPARC# devfsadm -C
On the PC side, I clean up with the following commands, where 0 is the LUN and 1 is the TPGT...
PC# iscsitadm delete target -u 0 test-volume
PC# iscsitadm delete tpgt -A 1
You'll notice that when you delete the target, the backing store files are deleted as well! Don't expect to easily recover from doing this.
One last thing... it happened silently, so someone might not notice this, but when you start mucking about with iscsitadm, it automatically starts the "iscsitgt" service under SMF. You can show that it's running with the following:
# svcs -a | grep iscsi
disabled Feb_07 svc:/network/iscsi_initiator:default
online 15:06:09 svc:/system/iscsitgt:default
If you're just mucking about with a toy, like I was, you might want to turn it back off when you're done. I don't know if it presents any kind of security risk -- I suspect not -- but there's not much reason to leave it going. So:
PC# svcadm disable iscsitgt
... and you're done. Or maybe just starting? :)
Posted at
05:28PM Feb 22, 2007
by Brandon Hume in Solaris |