How to kill a bumblebee with a meteor

so missing contacts still plague us here. So I have been forced to pull from tape back from a month ago so there is a static set of contacts to refer to until this happens again. So from the tape I pulled a full virtual machine backup and dropped it into VirtualCenter with VMware Convertor. From there I left it on an internal network, don't want two of the same exchange server up at the same time! I powered it on and attempted to exmerge the users mailbox for the contacts. I immediately came to the issue of needing a DC for exchange to authenticate against. So I cloned our existing DC and dropped it into this private vlan. I then had to seize FSMO roles, point AD and DNS to local host and re-add the exchange server to the "new" domain. Now I can get onto the server but exchange doesn't like letting you point to an existing database - instead I am trying to create a new one and copy over the old data and have the DB rebuild from the logs. After doing this a couple of times the database will not mount with the error being "The Database Files in this store are inconsistent" Running eseutil /p against DB now.

Thin Prov / Dedupe

So I am in preparation to snapmirror our critical volumes to our DR filer. Part of this preparation includes thin provisioning and deduplication to minimize the data traveling across our pipe. To thin provision on our NetApp FAS2020 I first need to remove the guarantee on both the volume and the space reservation on the LUN. If you attempt to remove the guarantee on the volume to none you will get a message complaining that the fractional reserve needs to be set to 100%. This is to protect the LUN from writing past the extent of the volume. To avoid allocating 100% more space to the volume first set the LUN space reservation to none. Now you can set the fractional reserve to 100% without adding more space to the volume (the FR will NOT come into play once you set the vol guarantee to none). After setting the FR I then set the vol guarantee to none and viola! the volume is thin provisioned. Now to dedupe!

Last friday I had the unpleasant experience of rack-mounting a 170lbs IBM server aptly named Atlas. I am actively seeking a lighter replacement since the controller in it has since died and the rails are bending from the weight.

Update! SME on non-working Exchange server has since been abandoned. NetApp essentially threw the towel in and I was already rebuilding the mail server. We were having performance issues with that mail server anyway and it had been P2V'd and probably not done the right way. I have reinstalled the mail server from scratch and am in the process of migrating mailboxes over. SME and Snapdrive installed properly. So it was a flukey port issue with the other mail server. It would have been nice to drag a solution out of all the work but sometimes the only solution is to scrap it and start over from a clean slate.

Inaccessible VM

Thanks to kjb007 over at VMware forums for this - disconnecting and removing ESX from VC inventory cleared the inaccessible VM.

SME still not working Netapp contacted me today and we worked a little on it. The service is running but the port is not listening. TCP Port Sharing service is not working or something...

Storage VMotion

So I successfully blew away my LUN that I had attempted to shrink. It has successfully locked both LUNs so attempting to remove them I get a "Resource is in use" error. Clearly my virtual machine is sitting on both due to the incomplete migration.

I successfully backed up Exchange Store last night. Only difference was I unchecked AOFO and the job ran successfully. Also as a note VCB will not run on physical RDM's so I can't back up either of our exchange information stores that way.

Pulling Drives from a Virtual Exchange Server

I did this once before and it didn't work out well but I am a little more confident now that everything will work accordingly. Since we moved our Exchange DB and Logs to RDM's we have the empty drives left behind consuming unreclaimed space. As long as you don't delete the virtual disk this is safe. Powered down the Exchange server and removed the three hard disks from the Edit Settings option on the Virtual Machine. Reboot and in my case....Success!

Now time to figure out why Backup Exec 11d is failing consistently to backup exchange database and flush logs. This happened roughly around the time I migrated the exchange store and logs to the RDM LUNs. The logs are still growing and threatening to fill up drive. As usual Backup Exec explains the error in their usual descriptive method: "Other Errors" I click the link for additional information to be greeted by a "No Exact Matches Found for: V-79-32772-8992. I look up on the web page and see the Symantec logo "Confidence in a connected world." Sure feels that way! Guess I am on my own here.

Storage VMotion and Shrinking LUNS

Ah yes! Tuesday. I have resized a LUN and made it 100 GB from 250 GB to make room to migrate it. First attempt LUNs (initial and target) went offline due to lack of space. Yes - I know it is unsupported to do this but hey I'm a gamblin man. So LUNs back online after resize and aiming to complete migration ends in "The virtual disk is either corrupted or not a supported format" uh oh. Time to pull from a snapshot!

Rightfax connector has been fixed by pointing the gateway to our front-end exchange server - seems odd as months ago the same issue was fixed by pointing it to the server it was set to.

SME still not working on our troublesome exchange box. Netapp not available for comment. At this point I will create a brand new exchange server and point to our RDM's where the DB and logs are. Should be interesting...

Vanishing Outlook Contacts

Randomly disappearing Outlook contacts in a very important persons mailbox were giving me trouble the last couple of days. After days of research and troubleshooting, access control, bulk logging, and pressure from above to find a fix it turns out he was deleting the folders that he kept the distribution lists with those contacts in them. So when you went to Recover Deleted Items the contacts did not show but the folders they resided in did. Awesome. Just as a lesson look between the keyboard and the chair for issues like this. These problems never get old no matter how often they happen.

RFax Connector Issues

Following my work last friday migrating Exchange DB and logs to RDM for Snapmanager Rightfax is unable to send notifications or faxes to Rightfax users once routed or forwarded. The problem began when I moved the Exchange Databases to the RDM's. The RFax Connector just continuously scans the RFaxGate folder for H*.TMP files but never seems to find them. Not my day...

SME Logs

So if you ever forget the password for the account you installed Snap Manager for Exchange under just check SnapMgrService.InstallLog under Program Files\NetApp\SnapManager For Exchange\ sitting in plain text. That can't be too secure as the Snap account needs admin rights to server and admin rights to Information Stores for the Exchange Servers.

SnapManager for Exchange 5.0

Installed SnapManager for Exchange 5.0 last Saturday. Some caveats I was not aware of is it uses Windows Task Scheduler which has been causing issues. It runs fine for the day but every day after requires the password be re-entered for the task to run. On Windows 2003 server box windows task scheduler should have advanced button to browse to the logs (it doesn't and the logs do not even exist in C:\Windows) and there is no notification option - maybe I am missing something.

Multipathing set up and is working! In a FC environment this was pretty straightforward.

Another big issue we ran into was installing SME 5.0 on both virtual Exchange 2003 servers. On one it installed seamlessly but on our second exchange server (P2V'd) the discovery phase of SME failed. It prompts "Please specify and Exchange Server you want to manage." I select the localhost and click Add and... "Snapmanager initialization failed. Error Code:hr=0x80004005 The specified server is either not running or the server name is incorrect." Opened case with Netapp - their support as of last 3 months has been lackluster but worth a shot. Stumped level 1 so they escalated the issue to level 2 and still no solution. We've done countless troubleshooting steps from repairing .NET, to reinstalling SME 5.0, Snapdrive 6.0 and 6.1, editing the default port number (808) that the webservice uses, editing permissions and others...and no dice.