I’ve always been an EMC Celerra guy since I cut my teeth on it so many years ago,  and it’s support of de-duplication (single instancing) left a lot to be desired by me – mainly that it could not work across filesystems.  When I first started investigating Isilon I had high hopes for de-dupe across the entire array because it din’t have separate file systems.. New announcements today bring that idea to light finally. It is certainly not a surprise that OneFS will support de-dupe,  but the fact that it allows it across the entire IFS is a huge benefit.   Data written to the cluster will be written full size,  but in post processing objects and files will be de-duped using an 8k block size. EMC is suggesting you’ll see a 30% reduction in storage consumption, but your mileage will vary.  This is great news for the Isilon TCO – it’s already low overhead in rawRead More →

By:
On:
In: Storage

I was browsing the ECN Symmetrix Forum when I ran across this post where someone was asking for a summary of capacity allocated to each storage group – something you can easily get with ControlCenter or ProSphere but not with the SymCLI.  Don’t get me wrong, SymCLI cave give you the information just not in a nice easy command.  So I duh out a script to share.     You just run this script and specify the SID and watch it come back with a list of ever device allocated sorted by storage group. [code]LUN, SG, Pool, Size, Allocated, Written, Compressed Size, FAs 1424,TestSG1,SATA_R6,275.0,226,257.8,226.9,”FA-9E:1,FA-7E:1,FA-10E:1,FA-8E:1″ 0C77,TestSG1,SATA_R6,300.0,237,283.4,237.4,”FA-6E:1,FA-3E:1,FA-4E:1,FA-5E:1″ 0A4D,TestSG2,FC_R5,150.0,98,99.0,98.5,”FA-6E:1,FA-3E:1,FA-4E:1,FA-5E:1″ 0C60,TestSG2,FC_R5,100.0,6,50.8,6.1,”FA-6E:1,FA-3E:1,FA-4E:1,FA-5E:1″ 0A02,TestSG99,FC_R5,50.0,5,12.0,5.7,”FA-6E:1,FA-3E:1,FA-4E:1,FA-5E:1″ [/code] This can take a little while to run – but it’s easy for reporting allocations.Read More →

By:
On:
In: Storage

One of the most annoying managements tasks of a Network Attached Storage is managing permissions for Windows files.  This is even further complicated using systems which allow for work Windows and Unix shares – Such as an EMC Isilon. If you haven’t picked it up by now,  i’m a command line guy with a strong unix background – so I don’t like using the windows GUI to control SMB permissions.  What I needed was really simple,  just to clone permissions (both unix and windows) from one file to another.  You can use setfacp/getfacl, but it’s kludgy and  deprecated in favor of ls/chmod.  So I came up with a simple perl script which can be ran on Isilon to clone permissions easily and quickly. As you can see in the example below permissions get cloned from <source> to <target>. [box]isilon-1# ls -led source -rwxrwx–x + 1 root wheel 1580 Mar 20 13:14 source OWNER: user:root GROUP: group:wheel CONTROL:dacl_auto_inherited,sacl_auto_inherited 0: user:DOMAINCincyStorageallow inherited file_gen_read,file_gen_write,file_gen_execute,std_write_dac,delete_child,inherited_aceRead More →

By:
On:
In: Storage

We recently turned on FAST on one of our EMC VMAX arrays and I was curious as to how much data was moving.  I didn’t have an easy way to see what percentage of data changes from day to day,  let alone hour to hour.  Getting the current data is actually really easy using symtier -sid <sid> list -vp,  but I wanted something to view historical information.  So I whipped up a quick perl script which will parse the output of symtier and recode it in a CSV format for importing into Excel. You can download it here: It’s really simple,  just run the script and pass it a sid and the path/name of a file to record to.. and it does the rest.  Like so: [box]mark@shell# ./record.pl 1234 ./tier_usage.csv[/box] A few second later you’ll see the information in the file you specify.  Now for the historical part you’ll need to setup a crontab entry which runsRead More →

By:
On:
In: Storage

Someone on the Symmetrix Support Forums on the EMC Community Network was asking for a way to report on Tier Usage by Storage Group. That’s pretty simple and easy to see with the following command: [box]symfast -sid xxxx list -demand –association[/box] However that is raw output and even if output is XML it’s just not that easy to parse into Excel.  So I decide to write a quick perl script to parse the output and turn it into a CSV.   It’s fairly simple,  but you do need to edit the script to specify the name of all your tiers. That is done on line 15. [box]my @tiers = (‘R5_SSD’, ‘R5_FC’, ‘R6_SATA’); [/box] Each tier just needs to be added as an element to the array.  Then you just run the perl script passing the SID of the array in question and you’ll get a CSV showing the Storage Group, Demand, Usage, and Growth like this: [box]SG,Read More →

By:
On:
In: Storage

This afternoon I was checking out my thin pool statistics when I noticed one of the pools had a significantly higher allocation usage than I was expecting.  I started digging into my documentation (I track every device created in a postgres database for tracking and trending) to see if a whole lot of devices we created over the weekend – and none were.  I started looking for an easy way to find bound luns that are not in a storage group – nothing!  I started to write my own shell script and the next thing you know I ended up with an inventory Perl script which tracks every meta device including lun id, thin pool, storage group, total thin allocation in Gb, and the FA ports they are bound to.   It’s simple enough to get with SMC but that is a lot of clicking.  So here it is! [box] ./inventory.pl lun_id,size,thin_pool,gb_alloc,%_alloc,gb_written,%_written,sg,FAs 11AA,200, 450gb_R5,97,49,95,48,StorageGroup,”FA-7F:0,FA-10F:0,FA-9F:0,FA-8F:0″ 2C2D,75, 2tb_R6,0,0,0,0,,””[/box] TheRead More →

By:
On:
In: Storage

After seeing my Perl scripts for migrating from EMC Celerra to Isilon a few people have asked me how I moved my actually data from the source Celerra to my target Isilon. The answer is pretty simple: EMCOPY. EMCOPY is basically a fast version of robycopy used for moving data while maintain ACLs and permissions.. It’s CIFS/SMB centric and very easy to use. You run it from a windows machine with access to both shares and it copies data. I wrote a quick script to generate my EMCOPY commands based on my previous ones. Wanted to share that with everyone.     [box]Now it’s time for a short note and small disclaimer. This script works for me in my environment. Do your homework and make this work for you. While I have tried to make this robust and fully functional, it’s important to note “stuff happens”, this is still provides “as is” with no warranty whatsoever. UseRead More →

By:
On:
In: Storage

I’ve been working on a massive migration from an EMC Celerra to our new shiny Isilon. We’ve got a ton of file systems, CIFS, and NFS shares which I don’t want to re-create manually. “Work Smarter, Not Harder” is what Scrooge McDuck always said, right? So I created a few simple Perl scripts to export information from the Celerra Control Station and created a shell script to run on Isilon to migrate the directories and shares. I’ve only tested with with 6.5 – not sure of the change in syntax in 7.x. This doesn’t actually move the data or the permissions, as that is a whole different project. It simply parses a “nas_fs” query to make directories, create a hard quota to mimic sizes, and then created the NFS and SMB share. It’s a set of three scripts that can be run as needed. [box]Now it’s time for a short note and small disclaimer. This script worksRead More →

By:
On:
In: Storage

As a storage administrator one of most boring tasks I have to do is making and allocating new storages to hosts.  To me it’s the equivalent of creating a user.  Easy to do, but very boring.  If you know me at all, which you probably don’t, you know I hate boring.  For a lot of people it’s just a few right clicks in SMC/Unisphere and you’ve got yourself a new LUN.   For me it’s a quick SSH into a server with SYMCLI, a few edits to files, a few commands, and bam. I’d rather spend my time working on future planning, new architecture, or almost anything else. I finally had enough free time to write some Perl code which parses a simple text file to create new Meta devices on an EMC VMAX array. Actually creating a new Meta device is a fairly simple thing: First you create the thin devices, then you combine them all togetherRead More →

By:
On:
In: Storage