NFS Migration (Celerra/VNX to ISILON) – the troubleshooting part

Recently I had a post on the NFS migrations from Celerra/VNX systems to Isilon array. This post is an extension to that, describing how to tackle the common errors/issues during the migration activities. I had to try different things in troubleshooting the issues, but sometimes, we may not be in a position to spend a lot of time in troubleshooting. Hence I thought of putting this post to listing some of the issues, I faced – you may come across.

Continue reading

First of all, the most common issue is with the checkpoint creation on the source (Celerra/VNX).

If you are getting a message similar to ‘msg SnapSure file system creation failed’ during any copy session, the command is failing to create a snapshot of the source filesystem. It could be due to many reasons including space shortage. You may try creating a manual snapshot on the source FS by logging into the source system. You will have to troubleshoot this issue if the manual checkpoint creation fails. free up some space if space is the issue. You may try the below command for a manual checkpoint creation on the source system.

#fs_ckpt Test_FS -name Test_FS-ckpt01 -Create

Next, Permission issues… I know I don’t have to mention it here. Mostly, from the error message itself it will be clear what you have to check. Ensure you are using the correct credentials for the NDMP user in the migration command. This user should have sufficient rights on the source system. You must have sufficient privileges on the target system you are logged on. If not the super user, the logged in user should be capable of creating and modifying the directories and their contents.

Another one could be related to the snapshot creations on the target – Isilon array. Snapshot creation may fail due to many issues as said above including space issue. In the below example, the snapshot creation failed as there is an existing snap with the same name.

ISILON568# isi_vol_copy_vnx Cel097-dm2:/Test_FS/NFS01 /ifs/data/Test_NFS01 -sa ndmp:NDMPpass -incr
Snapshot with conflicting name ‘isi_vol_copy.011.1.snap’ found. Remove/Rename the conflicting snapshot to continue with further migration runs.
snapshot already exists
ISILON568-1#

We can either delete or rename the existing snap to resolve this issue. Here it was safe to delete the existing snap, and so I did the same.

ISILON568-1# isi snapshot snapshots list| grep isi_vol_copy.011
134 isi_vol_copy.011.0.snap /ifs/.ifsvar/modules/isi_vol_copy/011/persistent_store
136 isi_vol_copy.011.1.snap /ifs/.ifsvar/modules/isi_vol_copy/011/persistent_store
ISILON568-1#

ISILON568-1# isi snapshot snapshots delete –snapshot=134
Are you sure? (yes/[no]): y
ISILON568-1#

Other issues could be anything related to network connectivity. The source and target systems should be able to communicate for a successful migration. Similarly, an ongoing migration session may fail due to network disruptions also.

I will update this post, if I find more similar issues during the migration sessions. Feel free to have your suggestions/questions in the comments section below.

Posted in EMC, ISILON, VNX | Tagged , , , | 3 Comments

Dell EMC Elect 2017 – Don’t be late, nominate now

Dell had their TechCenter Rockstar program running and EMC had the EMC Elect program to appreciate the social contributions. From this year, following the merger, Dell EMC have combined these programs to form the all new Dell EMC Elect program.

Wanna be a founding member of the Dell EMC Elect program? If you believe your (or your peer’s) contributions deserve to be recognised, nominate now.

Continue reading

Dell Rockstar and EMC Elect were two of the leading community recognition programs in the industry.  VIP treatment at major Dell EMC events (even though I’ve never been to EMC World before), blogger briefing sessions, goodies etc… are some of the attractions of the Elect program. With the new edition, the expectations are really high.

Here’s the link for you to nominate self or your peers. Your contributions will be validated by an expert panel formed by EMC Elect founders and members.

Time is running out. Nominations are open until 17th March only.

Posted in EMC, Recognition program | Tagged , , | Leave a comment

VMware vExpert 2017 announced

VMware, a few hours back announced the batch of 2017 vExperts. I am honoured to be a member of this group, fourth year in a row for me.

This year, there are 2017 members in the group. I would like to congratulate all my fellow members..!

I am sure that there’s no need for an introduction about the program for anyone. But, if you want to check few of the program benefits or FAQs you may find it here.

You may find the full list of 2017 members and more details here in the announcement.

Posted in Announcements, Recognition program, VMware | Tagged , , | Leave a comment

NFS migration from Celerra/VNX to ISILON

I was working on a few NFS migrations from our Celerra/VNX systems to the ISILON systems. I thought of putting a post with the steps and gotchas in a Celerra-ISILON migration. It took so long, and now it is the time for that post.

Our first step should be to identify the hosts accessing the export and to understand the type of applications running on the export. We should work with server/application teams for this and for finding the suitable window for a possible migration window. The size of the exports, network parameters, and the nature of applications being used – we should take many things into consideration as we do the pre-work of the migration. ISILON by default, do not support 32-bit exports running on the NFS exports and this need a special bit setting to be enabled. So, this is one thing we should be ready for as we plan the migration. We will discuss how this is done as we discuss the actual migration steps.

Once we are done with all the pre-works, we will start the migration steps. To reduce the downtime window during the actual data migration, we will run a pre-copy of the data to the target ISILON array.

Continue reading

Create the NFS export directory and a quota on the target ISILON array. While setting the quota limits, we should consider the source FS size. Here I am assuming a 100 GB source FS and setting 90GB advisory threshold on the target directory.

mkdir Test_NFS01

isi quota quotas create /ifs/data/Test_NFS01 directory –advisory-threshold 90GB –hard-threshold 100GB –container yes

You can try listing the quota and it should be showing 0GB as the current utilisation. Now initiate a full copy of the data from the source (Celerra/VNX) export to the newly created directory.

isi_vol_copy_vnx Cel097-dm2:/Test_FS/NFS01 /ifs/data/Test_NFS01 -sa NDMPuser:NDMPpass -full

The isi_vol_copy_vnx utility creates a snapshot of the source Filesystem and initiates data transfer from it. The NDMP credentials on the array need to be used in the command in the place of NDMPuser and NDMPpass. Once done, we have one full copy of the data on the target array. You can now list the quota again and you should be seeing the change in utilisation now.

isi quota quotas list | grep NFS01

Now we are all set for the migration window. (Note: In case if the number of days between full-copy and the actual cutover is high – maybe due to a change in schedule etc… – you may run another incremental copy with incr switch a few hours prior to the cut-over. This will ensure the delta will be less and will help in completing the migration faster.)

The application team will start the migration proceedings by stopping the application instances running on the export. The server team will unmount the exports from their end after all the applications are stopped. We can un-export the source export and this will ensure there are no new writes happening to the export being migrated.

server_export server_x  -unexport -perm /Test_FS/NFS01

We will initiate a final incremental copy of the export to migrate the changes from the source export.

isi_vol_copy_vnx Cel097-dm2:/Test_FS/NFS01 /ifs/data/Test_NFS01 -sa NDMPuser:NDMPpass -incr

After the successful completion of the data transfer, we are good to create a new export on the target directory. We will add the export permissions as in the source export. We will create an NFS alias for the export for our convenience. NFSZone in the below commands needs to be replaced with NFS Zone on your ISILON array.

isi nfs exports create /ifs/data/Test_NFS01 –zone NFSZone –root-clients host1, host2 –read-write-clients host1, host2 –read-only-clients host5 –description Test_migration -f

isi nfs alias create /Test_NFS01 /ifs/data/Test_NFS01 –zone NFSZone

Now the 32 bit part. As discussed already, if your application is 32 bit, you have to enable 32-bit settings on the new NFS export.

isi nfs exports modify EXPID –zone=NFSZone –return-32bit-file-ids=yes

Replace the EXPID with the ID of the target export. You can now verify the bit settings by viewing the export.

isi nfs exports view EXPID –zone=NFSZone | grep -i return

We are now done with the migration. The server team can now mount the new export on the servers and application can be run on the new export.

Next… ?

You may read our post on some of the common issues in migration and their troubleshooting steps.

Posted in EMC, ISILON, VNX | Tagged , , , | 4 Comments

Time for a change – a new beginning


I am very pleased to announce that I have joined the Dell EMC’s Converged division today.

The converged division is one of the areas of toughest competition these days and it is very important for each vendor to deliver their best to remain in the competition. Dell EMC is the leader in converged systems sector with 43% share of the total market revenue as per the Q2 results from IDC. It will be very interesting to work with the leaders and will be equally challenging in the effort for continuous improvement.

In my previous roles, I was working closely with EMC as a partner and later as a customer.

The company has gone through the biggest transformation following the merger with Dell which was completed a few months ago. The merger announcement took the entire industry by surprise and remained a hot topic for many months. But all those concerns had no place as we have seen one of the smoothest mergers in IT. The company recently at the Dell EMC World, announced their new delivery model with Dell servers taking the compute part of their HCI systems. This will further accelerate the growth and will offer cost-efficiency for the customers.

Looking forward to the new challenges and opportunities with the new role. I would request all your prayers and support for me for my new assignment with Dell EMC Converged.

Posted in Announcements, EMC, General | Tagged , , | 2 Comments

WordPress optimization – here’s how I improved my website performance


C’mon… You mean a 1001st blog post on this topic ??

Yes, I know there are so many resources available online on WordPress optimization. But still, I just thought of putting this post – some steps I took to improve the performance stats of this site. Maybe it helps someone and of course a quick reference guide for me for a future.

I have been working to improve the response time of my website. (That said, being a storage guy I would have taken the ‘response time’ factor in consideration earlier). I have spent some time selecting the right plugins and getting it configured correctly for the things to work. I was having a caching plugin installed from some time. Still, my GTmetrix scores were poor. That clearly said I was having something configured incorrectly (or not configured). This is how I managed to get things right.

First of all and the most important is the caching plugin. I found W3 Total Cache (W3TC) being much advised everywhere and the active install count (1M+) will underline that. I was having W3TC installed but was not properly configured. I believe this blog post will be a good guide for configuring W3TC.

Continue reading

check_sideEven after having settings in as advised for W3TC, GTmetrix was not happy enough with my website. Because I am not using CDN ? I decided to go for that next. As many suggested and as its free (yes, free except a very few pro features) I have chosen CloudFlare. This  YouTube video will walk you through the steps to configure W3TC with CloudFlare. One important thing to remember here is to keep Minify option disabled in W3TC if you are going to configure CloudFlare. CloudFlare will do the minifying task for you.

Once you have configured W3TC with CloudFlare, you can see many options in GTmetrix in green including minification, gzip compression, browser/page cache etc…

Note : If you are editing any of the WordPress configuration files, just be sure to have a backup in advance. I know you are smart.. 🙂

Next big thing for me to do was to deal with the images. Images were not optimized and were not scaled. I installed an image optimization plugin, and that did the optimization tasks for me. But for ‘serve scaled images’ warning, I had to manually adjust the size of the images. A huge task, especially when you have too many images listed in the warning. Not sure if there are plugins available to do this task.

There were many inactive plugins, too many old post revisions etc.. which needed cleanup. I have removed the plugins which were not in use and used a DB cleaner plugin for cleanup of the Database. I have removed the post revisions, etc… with the DB cleaner plugin.

Very few steps it look like, but I had to do a lot of experiments on these and I could manage to get a decent performance score. I still have a few more things to work on and I may be updating this post later. Based on the type of the content in your website, you may have to adjust few more things in addition for better page load speed. You may refer the missing points from GTmetrix report and of course the internet is there to assist you further.

Posted in Blogging, General, WordPress | Tagged , , , , , | Leave a comment

Dell EMC World 2016 – through the product announcements


EMC declared 2016 as the Year-of-Flash and they had a few interesting products added in their All-Flash portfolio this year. ISILON AFA, VMAX 250F contributing more to this line-up and a few other announcements were the highlights of the Last week at Dell EMC World.

I thought of putting all the major product announcements from the event together in a single place here in this post.

dellemcworld2016

Image courtesy : flickr.com/dellphotos

Dell Servers & HCI : One of the key announcement at Dell EMC World was their move to go with the Dell Servers in their Hype-Converged solutions. VxRack and VxRail systems are gonna be in the market soon with Dell Servers in them. This new approach will help customers with cost-efficiency and better support.

Continue reading

ISILON AFA (Nitro) : The All-Flash ISILON array is one of the biggest attractions from the Dell EMC World 2016. I have written a detailed post here about the product. The ISILON AFA (previously known as the Project Nitro) is a solution for the performance intensive NAS workloads. AFA chassis can be added to the existing SAS and NL nodes as a performance tier also.

XtremIO File : XtremIO All-Flash Array will have NAS functions added by late 2017. Dell EMC World had a preview of the functions which is still in the development phase. The FluidFS Scale-Out NAS filesystem will make XtremIO a true Unified AFA with common protocols including NFS, SMB, HDFS, NDMP  etc… with the block storage. You can read more about the product on this Register post here

EMC ECSEMC ECS announced a few feature enhancements including some from a management perspective with the new release ECS 3.0. ECS has a new denser hardware introduced which increases capacity density by 64%. Dell EMC now have a software-only solution also of the ECS which can be installed on certified Dell servers.

ECS will be soon available as a dedicated cloud – ECS-as-a-Service from Dell EMC. Customers can provision their ECS from the cloud, hosted by Virtustream. Integration with existing storage systems is another attraction from ECS now. ECS can be added as an additional storage tier with other storage products including VNX, VMAX etc…

VMAX 250F : A week before Dell EMC World, Dell EMC added this new array to their VMAX AFA solutions along with 450 and 850 systems. New VMAX 250F, an upper-midrange model with enterprise capabilities can offer more than 1 Million IOPS. The basic configuration of this system can be as small as 1 V-Brick with 11 TB of storage. You can find more specificaitons, comparision etc… from EMC Store here

Some of picks from Data Protection is Software-Only version of Data Domain with Dell servers support. You can read more from the announcement here.

I believe that sums up the major announcements from the event. If you feel I missed something important, please feel free to let me know. Comments are always welcome.

Posted in All-Flash, Announcements, EMC, Events | Tagged , , , | Leave a comment

ISILON Nitro – All-Flash ISILON array from Dell EMC


At EMC World 2016, EMC announced the “Project Nitro” – a high performance NAS storage array. And today at Dell EMC World, Dell EMC announced the GA (pre-order from today and GA by 2017) of the All-Flash ISILON array ISILON Nitro. The first All-Flash NAS array to add to their All-Flash portfolio.

EMC announced 2016 as the ‘Year of Flash’ we have seen new All-Flash arrays and enhancements to the existing arrays from them. DSSD, VMAX All-Flash Array etc.. and now ISILON Nitro. This array is targeting performance intensive NAS workloads including Media and broadcasting, Electronic Design Automation (EDA), Life Science, Analytics etc…

dell-emc-isilon-nitro

 

As per studies the flash storage will be dominating the primary storage by 2020. The reduction in $/GB for Flash and the performance requirements for the modern data center are accelerating this growth. We are seeing AFA model in NAS storage also now and ISILON Nitro is the most recent in this category.

Continue reading

Hardware : ISILON Nitro comes with a new hardware architecture from the elder ones in the family. Nitro chassis will have 4 nodes and 60 SSD modules (1.9/3.8/15.4 TB), that is approx 1PB out of 4U rack units. Each node will have 15 SSDs associated (4×15= 60 per chassis)and 2 I/O ports and 2 cluster ports. Each node will have 16-core CPU, 256GB RAM, and 2x 40GbE front-end IO ports. For cluster connectivity, customers can have Infiniband or 40GbE connectivity. For adding existing node types (S/X/NL etc…) customers can use IB connectivity and 40GbE interface for adding additional Nitro chassis to the cluster.

With Nitro, the minimum cluster size is a single chassis, i.e, 4 nodes. Each cluster can grow up to 100+ chassis in a single namespace.

Let’s have some comparison :

ISILON Nitro can have up to 6 times more IOPS than the S-Series per rack unit, 11 times more throughput than X-series per rack unit. HD series is the high-density array available today from the ISILON family with 472TB capacity. With ISILON Nitro, the capacity will be almost 2x compared HD series.

ISILON Nitro supports all ISILON protocols and enterprise features including data protection, data management, data security and compliance capabilities. Nitro chassis can be used as a Flash tier with an existing ISILON array also. ISILON array has the ability to automatically tier the data. With the new Flash tier, the array can have efficient tiering of the data in an existing ISILON array.

Why All-Flash NAS ?

I didn’t even think of NAS arrays in All-Flash configuration an year back. But some of the modern data processing requirements are very much performance intensive and this needs the file storage to be capable. HD video processing and broadcasting,  High-throughput DNA sequencing (Next-Gen Sequencing), IoT, Analytics etc.. are some of the examples for performance intensive NAS workloads.

In a common man’s example, our televisions are now capable of displaying high quality, high resolution pictures and the technology is getting improved day-by-day. We will look for quality video while watching a soccer game, a stage show etc… The digital broadcasting technologies are in transformation for meeting this requirement and the underlying storage arrays also should be capable of processing, storing and retrieving the data in the back-end for making this to happen. That is where All-Flash arrays will play a significant role.

With Nitro, ISILON has taken a step in advance as we will be seeing this area getting even wider in the coming years and there will be lot of developments happening here.

Posted in All-Flash, Announcements, EMC, Events, ISILON | Tagged , , , , , | 2 Comments

Vote now for the Top vBlogs of 2016


As many of you might have already known, this is the time for voting for the top vBlogs for the year of 2016. An yearly program by Eric Siebert of vsphere-land to identify the industry’s top bloggers in various categories. This year’s voting is in partnership with VMTurbo .

You can now vote for the best blogs in various categories including virtualization, storage, scripting, VDI, independant blogger, new blogger etc…

It takes a huge effort to maintain a blog site and keep it updated. Top vBlogs voting is a real encouragement for the bloggers to continue investing time and effort in publishing more value added contents.

Top vBlog 2016

Continue reading

Some of the inactive (blogs which are not being updated) entries from the whole list of vLaunchPad blogs have been removed from the list out for voting. So this time we have around 319 blogs (from 400+ in last year’s list) which were active in 2015.

Now, here is the link for voting for your favorite blogs for the year. To value all the efforts behind this successful program, let’s

keep-calm-and-vote-for-the-best-16

Here you can read more about this year’s program. And here is the results from the last year’s voting.

Posted in General, Vitulization, VMware | Tagged , , | Leave a comment

EMC UNITY -“SIMPLICITY IS THE ULTIMATE SOPHISTICATION”

Today at EMC World, EMC announced the EMC Unity unified storage solution. The al-new Unified storage solution, available in All-Flash and Hybrid variants. As the subject says, Unity is Simple in many aspects from purchase to service. New controllers, no more Java-based UI, new management over the cloud, vVols and container support…

let’s discuss some of these in detail.

EMC Unity

Continue reading

When we say the system is a true unified array, the Unity is having the same controller managing the functions of SAN and NAS. Having different controllers for each function was a drawback with the VNX and Celerra systems. This reduces the efforts and complexity in cabling the system, from an implementation perspective. This also reduces the rack space needed for the system to 2U from 12 or 10 Units.

The image below will explain you the models available. The base system will be Unity 300 and the elder ones in the family are Unity 400, 500 and 600. All-Flash systems have the similar models with an F in front.

EMC Unity Models

Let us see how Unity is simplified in various areas.

Purchase and Install : The implementation partners and the EMC employees are well aware of the sizing tools available for designing a new system. With EMC Unity, the implementation engineer can upload a configuration taken from an existing VNX/VNXe or CX array and get the new Unity system configuration. That is designing and ordering a new system, simplified.

For the installations, the racking and cabling are much more simple with Unity as we have fewer modules to rack and stack and very less number of cables to be laid. It takes less than 15 minutes to setup a new system with the configuration wizard.

If the customer wants to have a similar type of configuration (in terms of drive and pool layout etc…), it is made easier with Unity. Customers can clone the configuration to multiple systems with Unisphere Central.

Management and Monitoring : The interesting part as far as the customers are concerned. Java or Flash based management is a real pain for the administrators, yes, HTML is the future. EMC Unity offers HTML 5 based management UI, supported by all the major browsers available. HTML 5 is faster and secure compared to its predecessors. The new Unisphere UI offers customizable cool dashboards, easy navigation from a Unified management perspective, and very good look and feel.

Pool and Thin provisioning – Unity offers a Unified storage pool which can serve space for both SAN and NAS functionalities. The NAS Filesystem is capable of automatically expanding and shrinking. This enables improved space efficiency. The space released by an FS shrink will be sent back to the pool and can be used for SAN provisioning. The FS shrink will be reflected at the target FS if the FS is being replicated to a different system.

CloudIQ – CloudIQ is the cloud-based monitoring solution available with Unity. You can monitor your system from anywhere with CloudIQ. It helps in pro-active alerting and support. CloudIQ also enables better availability and performance with the monitoring and planning. EMC is now providing an early-access for the CloudIQ solution.

Unity introduces new Unified CLI and better integration with Rest API for management functions. The simple HTTP commands make easier integration. Customers can prepare their own scripts based on the APIs and they can access the APIs from CLI, Browsers, and Apps also.

Service : Unity supports embedded ESRS with the help of embedded docker container. This allows the support team to connect to the system and the system to send alerts to EMC. The UDoctor software has been improved to have a detailed log collection from the system. The new UI has a better system view which shows a detailed status and health information of the components.

Flash Optimized architecture : The Unity architecture is optimized for the best performance. The new write-caching with the Unity systems is a key for performance improvements. The new writes will be received by the full system cache and will be served to the disks.

Unity Write Caching

Pricing : The Unity storage systems comes with all-softwares and the starting price is $10K for hybrid arrays and $18K for All-Flash models. You can find more details about the products, compare the products, and place the order for Unity system form the EMC Store. For more details and specifications please check the datasheet here.

Posted in All-Flash, Announcements, EMC, Events | Tagged , , , , | 2 Comments