Terry Wilcox

The Reality of Virtualization

I just read a short article from the June 2006 issue of Application Development Trends called, "The Reality of Virtualization."

A few of the points the article made that I liked where that cost reduction may be the reason that companies start using virtualization technologies, but the agility benifits currently aren't widely considered.

The article also talked briefly about how current virtualization players are positioning themselves to compete with each other. They claimed that Xen is what motivated Microsoft to take virtualization seriously because Microsoft sees Xen as a "greater threat" than VMware. An interesting thought considering the fact that Microsoft Research actually funded some of the initial Xen work and that Microsoft announced they'll support Xen running on Microsoft Server products.

RubyCLR

According to here Microsoft has just hired John Lam. John Lam is the developer behind RubyCLR.

This move makes me very happy. I really enjoy the Ruby programming language and perhaps this will lead to increased support for Ruby in Microsoft products. This could lead to IIS being easier to work with for Rails that alone would be great. This could also lead to quicker development of RubyCLR which would benifit the Windows community and the Ruby developer community.

I haven't played with RubyCLR much yet, but since I happen to enjoy programming in Ruby quite a bit I will have to look into it. I imagine that RubyCLR would make it easy to create a GUI with Visual Basic and the performance critical sections in C# or C++ with Ruby holding the whole thing together.

Matz At BYU

Yesterday BYU had the honor of hosting Matz (Yukihiro Matsumoto, chief designer of the Ruby programming language). The presentation was interesting and funny which shows that Matz was very well perpared and has an excellent sense of humor considering that he had to give the presentation in English, but Japanese is his native language.

After the presentation I thought a little about the future of computer science. As I understand it right now PhD's in math are expected to be fluent in foreign languages because quite a bit of interesting work in math is done by people who don't speak English natively. In 10 or 20 years will the same be expected of computer scientists? To stay up to date in this field will we have to learn to communicate in Japanese or German or Korean or any other language? Matz is nice enough to learn English and speak to us in our language, but what about other Japanese computer scientists who don't speak English so well? I think they probably have interesting things to say and I wouldn't be surprised if in the future computer science PhD's here in the States are going to have to be comfortable giving presentations in other languages.

Microsoft Opens up Virtualization Format

Microsoft has opened up its virtualization format. This means that third party companies can use the format "without fear of infringing on Microsoft patents."

I spent a few minutes thinking about what Microsoft is going to get out of this. They get a little bit of free publicity inside of the virtualization community. This will encourage third party developers to create products that build on Microsoft's virtualization format. Perhaps it will also encourage researchers working on filesystems for virtual machines to pay attention to Microsoft's virtualization offerings.

Read more about this here:
http://news.com.com/Microsoft+opens+up+access+to+virtualization+format/2100-1007_3-6126384.html

Pitfalls Of Server Virtualization

I found an article here about some of the "Pitfalls of Server Virtualization." Some of the things highlighted include: understanding license restrictions, understanding what your servers can realistically do, and getting hard drives to work quicker.

I'd say in my personal experience the hard drive issue does seem to be crucial. When I try to run virtual machines on my laptop it seems like the disk is the big bottleneck. I guess this also goes into the category of understanding what my computer can realistically do.

Softricity

I found a video that talks about a software company called Softricity. Softricity is turning software into an on demand, centrally managed service. Applications are downloaded on demand and with virtualization they run on the machine without causing conflicts with other applications. Users are given permission to use the software they need and after that they can use the software without right away.

No DLL trouble. Each application is an island and each application is available right when it is needed. This is real neat stuff and Microsoft is looking to buy this company so we might see this integrated into Windows.

The video is here.

Timing Issues Revisited - Mean vs. Median

Well, I think I may have used the wrong method to decide how a benchmark performed. On this last series of tests I was using the median of five iterations on the integer suite of the cpu2000 benchmark. I was using the median mainly because that is what SPEC uses.

It seems though that I may have made a bad decision. With sixteen virtual machines I noticed that there was occationally large differences between different run times. For example,
Iteration #1 172
Iteration #2 49.6
Iteration #3 180
Iteration #4 178
Iteration #5 179

SPEC reported the median which was 178 for the above benchmark, but really the number I was expecting is closer to the mean which is 151.72. One reason I decided on using the median in the first place is occationally SPEC will report some wierd and completely impossible number for one of the runs. I.e.
Iteration #1 .0000326
Iteration #2 153
Iteration #3 157
Iteration #4 160
Iteration #5 159

The "ESX Server Performance Paper and Resource Management for CPU-Intensive Workloads" paper used two methods of measuring performance: an average runtime and a system throughput which is the number of iterations completed per hour. They would have had to let the benchmark run much longer than an hour to college the throughput data.

Virtualization Paper - ESX Server Performance and Resource Management for CPU-Intensive Workloads

Next weeks paper is ESX Server Performance and Resource Management for
CPU-Intensive Workloads. This is a pretty interesting paper with some
work which overlaps with things we have done. We can use this to get
ideas on how to build on some of their tests and how to setup some of
our own tests.

http://www.vmware.com/pdf/ESX2_CPU_Performance.pdf

Arsenic and Old Lace

My fiancée and I went to a preview performance of “Arsenic and Old Lace” last night and we went away happy. The play is a product of WWII America and even though the genre is theatre of the grotesque most people would probably just call it a comedy. Basically through exaggeration and good acting the play makes mass murders look funny.

This is a very good production and very funny. I highly recommend seeing this play.

Anah’s Grade: A
My Grade: A-

Changes to the Testing Enviornment

I upgraded the version of ESX in our testing enviornment from 2.1 to 2.5.2 today. This fixes a few known bugs with vmkusage when used on a host machine with hyperthreading enabled.

I would also like to recommand that we try to stop using using the Administrator account in the VirtualCenter to manage virtual machines. I believe all of us on the project have our own user accounts.

Also, I have recently learned how to play with the configuration files for virtual machines. In so doing I have set it up so the virtual machines I run my benchmarks on use the host machine's TSC instead of using a virtualized TSC. I believe upgrading versions to 2.5.2 and using the host machine's TSC may eliminate most of the strange timing issues we've noticed. This may allow us to get accurate benchmark results even when the host machine is overloaded.

Virtualization Reading Group

Well, finally after several minutes of randomally trying permutations of my name and common passwords I was able to log back into this thing. Maybe if I posted more I wouldn't have to go through this every time. The virtualization reading group is now meeting weekly at Friday at 10:00. This is a reminder email to any interested person that the virtualization reading group will be meeting at 10:00 this Friday. The paper under discussion will be Intel Virtualization Technology from the May issue of Computer. A link to the paper is http://cache-www.intel.com/cd/00/00/22/19/221961_221961.pdf The future reading schedule may be found on the lab calander at http://www.eclab.byu.edu/calendar/ Please send me any ideas about future papers you'd like to discuss in the group.

Virtualization Reading Group Meeting

The Virtualization Reading Group will be meeting this Friday at 10:30. The paper we'll be discussing this week will be Memory Resource Management in VMware ESX Server (http://www.usenix.org/events/osdi02/tech/waldspurger/waldspurger_html/esx-mem-html.html). Everyone is invited. Please come prepared. If anyone has any papers they think would be good for the group then please send me an email.

First Meeting

I'd like to remind everyone parties that the virtualization reading group will be meeting at 10:30 this Friday in the lab. The first paper can be found at http://www.cambridge.intel-research.net/~rneugeba/pub/2004-safehw-oasis.pdf.

Benchmark Results To Date

All of the following results have to do with running different benchmarks on virtual servers running on VMware ESX. We are trying to characterize the behavior of virtual machines. The first set of tests that we ran was to run the freebench benchmark on the virtual machines and increase the number of virtual machines running at the same time. Here are our results. The machines performance is being reported relative to some base machine that was picked by the freebench crew. Notice that we are showing the summation of the relative performances. Also notice how similiar this is to a graph that Harsh made several months ago when testing network performance. The increase from 1-4 is because the host machine has four cpu's. There is a continued increase in performance until around 8 machines. From 9 - 20 the summation of the relative performance stays flat suggesting that performance had peaked. After 20 machines, however, it appeared like performance started to pick up again. In order to tell if performance was actually increasing or not we compared the relative performance times and wall clock times. Based on the wall clock time for one run and the relative performances collected from the last test we tried to estimate what wall clock time would be other values. We then measured to wall clock time for those values and compared it with our estimates. Here is the graph. Notice that up until 20 the relative performance was a good measure for wall clock time, but after 20 it is not a good measure. Also notice that the actual wall clock time seems to be linear which makes sense if the performance has peaked. To make sure this behavior isn't limited to the freebench benchmark we then ran the SPECInt benchmark. We recieved similiar results after 20. Now that we have confirmed that this behavior is present in more than one set of benchmarks we have a couple different directions we can go. We can continue to run this on other sets of benchmarks (like network tests) to see if the trend holds for them as well. We could also work to better understand how these relative performance numbers are generated so we can better guess why they are off. Here are spreadsheat files with our results. Download file