When I was told that you could only grow (not shrink) the storage volume for a running Windows VM in XenCenter, I took that as a challenge. Guess what, there is a way to shrink it! Here is how:

1. Use XenCenter to add a new disk to the existing VM, making it the size you want to srink the current server to.
• Select the VM from the list on the right, click Storage.
• Click “Add…” and create the new volume the size you want it.
2. Log into the VM, (you can use the Console tab in XenCenter) and start ‘Disk Management’ and Initialize the new disk.
• You can find it in Start->Administrative Tools->Computer Management.
• Click on the new disk. Mine showed up as “Disk 1, Unknown… Not Initialized”.
• If you click on the words “Not Initialized” it will be selected.
• Next select Action->All Tasks->Initialize Disk. Select MBR and OK.
3. Format a new partition on your new volume (you might be able to skip this step, did not try).
• Right click on the black “Unallocated” partition.
• Select “New Simple Volume”.
• Click Next twice, the size will be defaulted to the full disk, and specify a new drive letter.
• In my example, I use E:\.
• Format as NTFS using Quick Format, and click Finish.
• Once you see formatting finish, and the new partition turns blue and is marked “Healthy” (Primary Partition). Proceed to the next step.
4. In the VM, download and run XenConvert.
• I used version 2.4.1.
• It requires Microsoft .NET v4.0, so you may need to download that from Microsoft and install it before running XenConvert.
5. Start XenConvert and select From: Volume and To: Volume.
• Set the Source Volume to your boot drive (C:).
• Set the Destination Volume to your new drive (E:).
• Say “Yes” to the warning about losing free space.
• Click Convert, and accept the warning about erasing data on your Destination volume (E:)
• Go have a coffee, or something, and come back later.
6. When XenConvert is finished, use “Disk Management” again in the VM to activate the new partition.
• Right Click the new partition and select “Mark Partition As Active”.
7. Shut down the VM.
• I did this from inside the VM using Start->Shut down.
8. Now re-order the drives on the VM so the new drive is in position 0.
• In XenCenter, select the VM, and select the Storage tab.
• Detach the original drive by selecting it and clicking the “Detach” button.
• Select the new drive, and click Properties, and set the Position to 0.
9. Now you can boot the VM and voila! It’s now srunken to the size you wanted.
• If it all works the way you want it you can go delete the original drive in XenCenter to reclaim the space.

Running a production application in the cloud can be great because it’s possible to add and remove servers from a cluster dynamically using a provisioning API. These automatic additions and removals can be triggered by system utilization levels that you measure, such as concurrent network connections, memory utilization, or CPU utilization. When you need more capacity, you can add more servers, and when they are not needed anymore, you simply turn them back off. You only pay for the time those servers were running, so it’s more economic than having a large number of servers deployed all the time.

Most simple web clusters rely on a single database sever that all the application servers connect to. This way, all of the application servers have concurrent access to the same data. This can be problematic in the elastic use case when workloads increase, and more servers are added to the cluster. If the work is bottle-necked on storing or accessing data in the database server, adding additional application servers will not help. It will actually make the problem worse.

I spoke on a panel at Zendcon yesterday, which was covered in an Infoworld article where my remarks were published. The article says:

Panelists also debated use of SQL and database connectivity in clouds. SQL as a design pattern for storage “is not ideal for cloud applications,” said Adrian Otto, senior technical strategist for Rackspace Cloud. Afterward, he described SQL issues as “typically the No. 1 bottleneck” to elasticity in the cloud. With elasticity, applications use more or fewer application servers based on demand. Otto recommended that developers who want elasticity should have a decentralized data model that scales horizontally. “SQL itself isn’t the problem. The problem is row-oriented data in an application,” which causes performance bottlenecks, said Otto.

The author Paul Krill did a good job here of accurately reporting my position on this subject. Data stored in databases are arranged in tables of rows and columns. A new piece of data adds a new row. Each row has multiple columns that separate fields of a single record of data in the table. The truth is that most web applications work very well with this data design pattern. Those should continue to use SQL databases with row oriented data. However, there are some applications where data may be arranged differently to make reading the data more efficient.

If you have a big table of data, and you want to pull out just a little bit of it using a query, the database server must determine the location of that data in the table by consulting the table’s index, and return the desired portion that matches the constraints given in the query. This makes the reading of data relatively expensive from a computational perspective. If data were instead arranged in lots of columns instead, it could be retrieved more efficiently, and the data could be more easily distributed over a larger number of servers yielding the horizontal scalability that cloud applications want. This works very well in cases where the number of reads are very high, but the data is not updated very frequently in proportion to the reads.

Let’s use a blog application as an example. Blog posts are written once, and maybe updated a few times, possibly once each time a comment is submitted. However, on a busy web site, a blog post may be read millions of times. If the posts were stored in a column oriented storage system like Cassandra, they could be quickly and easily retrieved using the id number of the blog post. The listing of recent blog posts can also be arranged in a column so that the front page of the blog site with the listing of the articles can be generated. Using this approach requires that the data be properly arranged as it’s stored, putting the computational burden on the (infrequent) write rather than on the (frequent) read.

Using a distributed system to store data in columns allows the data to be evenly distributed over an arbitrary number of servers, eliminating the central data bottleneck. Adding more servers in the correct proportion of application servers and storage servers can result in true horizontal scalability, meaning that the capacity increases as a direct proportion of how many servers are in the cluster.

Why doesn’t everyone do this already? For some good reasons:

1. The concept of running applications in clouds is still relatively new. The related technology is still maturing.
2. Existing software tends to use SQL already. If you want to use an existing CMS platform, chances are it will require a central SQL database.
3. Most heavy-read workloads can be scaled well using data caching techniques. If applications don’t write data very often, it may not be necessary to scale beyond a single database server.
4. You must anticipate exactly how the application will use the data, and arrange it just right.
5. It may be harder to analyze the data. Once your data is arranged in a column store, you may not be able to query it in arbitrary ways. You may only be able to pull it out using it’s id numbers, or by systematically scanning all of it to find the parts you want.
6. Distributed data storage (aka: NoSQL) systems like Cassandra, Hbase, Redis, etc. are complicated, and there is a considerable learning curve associated with setting them up and maintaining them. In some cases these systems are not as good in terms of data durability or data consistency as the prevailing SQL database systems. These tradeoffs can be difficult to navigate.
7. Today’s software developers are very familiar with SQL as a data storage and access paradigm. They can very quickly develop software that relies on the ACID qualities of a SQL database.

If you have an application that you want to deploy into a cloud, and you want it to be very elastic, you should think about the subject of how you arrange your data. If you use a centralized data design, you will probably have scalability bottlenecks when you add lots of servers. You should aim to decentralize the data in a way that you can easily add more servers to horizontally scale the environment, and not stumble on the limits of the database server. This is particularly important in situations where you need the application to write a lot of data, and a cache is not a suitable solution for you.

Over time, the reasons why not to use column oriented data will begin to shrink, and better tools and services will make it easier to do. Until then, I suggest that you carefully consider if you need maximum elasticity. If not, then it’s perfectly appropriate to keep using the same centralized row-oriented data paradigm. Use a cache like memcached in cases where you have heavy reads, and when it’s acceptable to show slightly outdated information to readers. The truth is that traditional solutions work really well for most web applications. However, if you have one of the more unique situations where you need true horizontal scalability, take a good look at a different arrangement for your data, and the systems and tools to make that possible for you in the cloud.

I was doing some work integrating with a payment gateway in a PHP application, and decided it would be a good idea to validate credit card numbers using a Luhn Algorithm formula prior to forwarding them to the payment gateway for processing. I looked for existing PHP ones, and found a few.

The more I found the less I liked any of them. Some of them actually had bugs or typos and did not work at all, and most of them would incorrectly validate a credit card number that was all zeros.

I wrote my own that I’m pretty happy with. It’s a good deal more efficient that most that I found. It does not repeat the same math on the same figures like some of them out there do.

The function contains 7 lines of code. Can you make this function better without making it harder to read and understand? Please let me know.