Justin Carmony » Servers

VirtualBox Networking Setup

Justin Carmony — Mon, 22 Mar 2010 06:32:43 +0000

Every time I go to set up VirtualBox and several VMs I always forget how I setup the networking for them. Once again, I went through the pain of trying to figure it all out, so I might as well document it for future use.

The Problem

When creating virtual machines, especially virtual linux servers, I want them to be able to do the following things:

Connect to the Internet
Connect over SSH from my Host Machine to the Guest Machine
Have the servers connect to each other on a private internal network

Out of the box, a VM is configured for #1, however it cannot do #2 and #3. There are some solutions that do some archaic port forwarding by issuing special commands from the Terminal. This is just painful, and if you just want one port to work, that’s manageable, but what if you want ssh, ftp, myql, http, https, etc? Any on multiple VMs? What a nightmare! What is painful is when googling for a solution, almost every “answer” is this crazy port forwarding method. There is a much cleaner, better way!

Solution – Host to Guest Connectivity

Go into your VM’s settings and go to “Network”, it will list options for 4 different adapters. Instead of hacking one adapter to do all the work, just setup more adapters for your configuration. For adapter #2, set it to Host-only. This enables an IP range for communicating from the Host to the Guest and vice versa. Start up your VM, and if you are using something like Ubuntu, edit your /etc/network/interfaces file so it looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

### ADD THESE LINES
auto eth1
iface eth1 inet dhcp

After adding these lines, run the command:

sudo /etc/init.d/networking restart

Now VirtualBox is configured to host this traffic over the 192.168.56.x IP range (I’m not sure if/where you can change this). So do an ifconfig on your guest machine, and it should show you the IP it has been assigned. If you want, you can change your /etc/network/interfaces so it can be statically assigned. You should now be able to SSH into your box on it’s IP:

ssh -l justin 192.168.56.101

Solution – Guest to Guest Communication

Now, what if I have a Cassandra server, a Web Server, and a MySQL server each on their own VMs. What you do is create a third adapter set to “Internal Network”. Then in your /etc/network/interfaces file you set eth2 to a static IP of your choosing. I typically use the 10.0.0.x range. So each server is respectively 10.0.0.10, 10.0.0.20, and 10.0.0.30. However, over the 10.0.0.x range there is no DHCP, no Gateway, no nothing, just direct communications over these IPs.

Conclusion

So please, for the love of everything good, don’t resort to crazy port forwarding, complicated XML file editing, solutions for VirtualBox. Just use each type of interface configuration for what you need. Thank you!

The Cloud to Bare Metal

Justin Carmony — Mon, 22 Mar 2010 06:02:22 +0000

A Little Background

It was the summer of 2008 when I attended the LT Pact conference that I first drank the “Cloud Kool-Aid.” As you can see from my notes, I was pretty excited about this whole concept. Excited enough, one client of mine signed up for Layered Tech’s grid layer (which is what they were demoing at the conference).

One interesting thing I heard at the LT Pact conference was one of their keynotes was as IT Researcher for Gartner. He outlined what phases the current industry would go through with “cloud computing”:

First, several early adopters would have great success implementing the technology.
Second, mainstream hype would “over-hype” cloud computing.
Due to the hype, many people would start “using the cloud” as a silver bullet, thinking the cloud would be perfect for anything.
A “cloud crash” would occur where disillusioned IT managers realized the cloud had its drawbacks, and wasn’t the perfect solution for any problem. More over-hyped people will start to move away from the cloud.
After the dust had settled and the hype faded away, the actual cloud technology would find a common place in today’s IT.

Currently I think different people and groups are at different stages of the last three listed above. Some are still raving and hyping the cloud for everything. Some are bitter about their disillusionment. Many are now looking beyond the hype at the real concepts behind “cloud computing” and finding “the services are wildly different. While many parts of Web hosting are pretty standard, the definition of ‘cloud computing’ varies widely.” (InfoWorld)

After a few hours, the fog of hype starts to lift and it becomes apparent that the clouds are pretty much shared servers just as the Greek gods are filled with the same flaws as earthbound humans. Yes, these services let you pull more CPU cycles from thin air whenever demand appears, but they can’t solve the deepest problems that make it hard for applications to scale gracefully. Many of the real challenges lie at the architectural level, and simply pouring more server cycles on the fire won’t solve fundamental mistakes in design.

In a rebuttal blog post, Don MacAskill defending accusations that EC2 was over 50% slower than stated.

… let me explain what I think is happening: Amazon’s done a poor job at setting user expectations around how much compute power an instance has. And, to be fair, this really isn’t their fault – both AMD and Intel have been having a hard time conveying that very concept for a few years now…

Bottom line? EC2 is right on the money. Ted’s 2.0GHz Pentium 4 performed the benchmark almost exactly as fast as the Small (aka 1.7GHz old Xeon) instance. My 866MHz Pentium 3 was significantly slower, and my modern Opteron was significantly faster.

So what about that guy with the Ruby benchmark? Can you see what I missed, now? See, he’s using a Core 2 Duo. The Core line of processors has completely revolutionized Intel’s performance envelope, and thus, the Core processors preform much better for each clock cycle than the older Pentium line of CPUs. This is akin to AMD, which long ago gave up the GHz race, instead choosing to focus on raw performance (or, more accurately, performance per watt).

My Experiences

Virtualized Servers can be awesome. They have some great advantages, while also having clear disadvantages (if you look beyond the hype). While I know I’ve quoted a lot about Amazon EC2, I have only experimented with it. However, I have a good deal of experience with 3Tera’s AppLogic, and have recently moved a website to Rackspace Cloud.

Learning Curve

While Rackspace Cloud was more straight forward, learning 3Tera’s AppLogic is a large task. I spent several months learning a lot (and tearing my hair out some days) and then another year of learning to deal with it’s quirks. It is pretty cool once you get it down, but in all honesty its features didn’t line up with what the company using it needed. Every now and then we spin up a custom server for a website or something else. However, AppLogic is built more for people who are doing a large volume of similar instances. So while it works, it probably isn’t the best fit for us.

Also, with AppLogic’s components, they are engineered for very very specific tasks. Doing things outside of their scope *is* possible, however there was a vast amount of tinkering, reading, blood, sweat, and tears trying to get it to work. There are still things, like with the way AppLogic handle network routing, that are just unique at best.

Bare Metal to The Cloud

Each instance I’ve gone to deploy on the Cloud, either Rackspace Cloud, Amazon EC2, or AppLogic, the project itself had started on a dedicated server that we then moved to a virtualized server. Since when virtualized servers started to pop up, even in the earlier days of simple VPS solutions, you would be moving from dedicated bare metal to a virtual server. The inherit problem is that virtual servers, especially on shared hardware, have *less* resources available to them, especially CPU power. On Bare Metal Dedicated Servers, I rarely run into CPU issues. When the CPU power is being shared with several other “virtual servers”, you suddenly only have a quarter of what was available.

Then you have your neighbors. While moving Clipish to the Rackspace Cloud the little 1GB RAM server felt peppy. I was confident in our choice. Clipish isn’t complicated, and it’s web services are pretty easy going. The only intense part is when you clear the image cache. Then when a web server request comes in and the image isn’t there, it pulls the image from the Database, performs the ImageMagick operations to resize, watermark, create a thumbnail, etc on demand. During the early morning, it isn’t a problem at all. However, one night I “cleared the cache” during peak times, and within minutes the server had locked up.

Turns out on our old server, the web services could easily burst to all 8 cores for a few minutes while regenerating hundreds of image caches. Even the Rackspace server could burst during off-peak hours and repopulate. However, during peak times, when the dozen or so “neighbors” we’re using their allotted CPU, and our server couldn’t “burst” like before. So what did we have to do? We wrote a script that would loop through our image library one at a time and regenerate it. This works, but after the move and we were experiencing these problems, we thought “did we make the right move to go to the cloud?”

Cloud to Bare Metal

I think the approach with the “cloud” needs to be this. You have your baby project, so throw it up on the cloud. Let it cost $15-$20 bucks a month. Take advantage of the daily backup snapshots, etc, and just worry about development. Let it grow and at some point you’re going to hit a fork in the road:

Bare Metal Benefits outweigh Cloud Benefits – Bare Metal is typically cheaper than Cloud power. If you are not spinning up and down servers on a daily biases, but is a more fixed size, then quickly scaling isn’t an issue. Cost can and should be a huge priority for hosting your application. If your application needs more higher CPU, Memory, or Bandwidth that doesn’t fit the Cloud model then going Bare Metal is a good idea.
Cloud Benefits outweigh Bare Metal Benefits – If your application, however, can fit on the cloud and the budget at the same time, then it can be a beneficial to stay on the cloud. Just remember you can’t expect a Cloud server to outperform a Bare Metal server of the same specs. But you can take advantage of things like spinning up new servers in minutes, take quick backups of virtual machines, etc.

Conclusion

I guess my biggest advice when it comes to deciding on bare metal vs cloud is to understand the pros & cons. Remember that while you can burst to more CPU, you shouldn’t rely on that always being the case. The Cloud does afford some cool flexibility, but it isn’t perfect, and it can be a lot more expensive than going bare metal. Also, if you’re on bare metal, and moving to the cloud, the cost of raw power is a great deal more expensive on the Cloud.

XAMPP for Mac – My Frustrations & Solutions

Justin Carmony — Sat, 14 Feb 2009 23:15:34 +0000

These last few days I’ve been trying to get XAMPP to work on my MacBook Pro these last couple of days and it has been frustrating! It seemed I kept running into more and more problems. For those who don’t know, XAMPP is tool for developers to run a LAMP stack (Linux, Apache, MySQL, PHP) on your machine locally. I’ve been running it on my Windows machines for years, and I have been running my production servers on Linux.

Installing just XAMPP and getting it to run with the Control Panel was easy. All my headaches started when I tried to get my vhosts installed and working. The problem I ran into getting XAMPP to run on my MacBook Pro were the following:

XAMPP for Mac is different from XAMPP for Windows
Max OS X comes with a Version of Apache Installed
Mac OS X has additional permissions that are different from Linux

XAMPP for Mac != XAMPP for Windows

My first mistake was that I assumed that XAMPP for Mac is the same as XAMPP for Windows. After digging into it, the file structure and config files are pretty different. The php.ini file is different, and all the binary files are in /xampp/xamppfiles/bin/. On windows the binary files are in their own folders. It took me awhile to figure out where everything was at. This led me into my next problem:

Mac OS X comes with a Version of Apache Installed

When I was trying to debug why stuff wasn’t working, I opened up the terminal and started running commands. There is just one problem, OS X already has a version of Apache installed. So when I cd’d into the bin directory, I would execute this command:

apachectrl -t -D DUMP_VHOSTS

There was just one problem, it was executing the apachectl that was installed by Mac OS X, not the one for XAMPP. So it all my debugging with the config files, etc. wasn’t working. I was going insane. Finally I figured out what was going on and I had to execute this command instead:

./apachectl -t -D DUMP_VHOSTS

It started behaving like I expected! This lead my to my third hang up.

Mac OS X has additional permissions that are different from Linux

This threw me for a loop to. I don’t understand exactly why this was happening, but I kept gettinga Permission Denied when trying to view the vhost I was trying to setup. The way I used to do it on Windows is I have a partition for all of my development files. I would have a folder at the root of that drive called SVN_Repositories that holds all the SVN Repositories I use for work. So on my MacBook Pro at / I did the same, I had my SVN_Repositories folder. So my vhost would point to a directory like with a name something like this: “/SVN_Repositories/ExampleRepo/trunk/httpdocs/”

So my linux side clicked in and thought “Hrm, it looks like there is something wrong with the permissions. Let me change the folder permissions to 777 just to test.” So I did that and it still didn’t work. This it made me think it still was a vhost configuration issue. So I spent hours testing all sorts of stuff and just getting more frustrating. Then I read somewhere that XAMPP can have problems with OS X’s unique permissions on addition to the FreeBSD stuff it does. It said to host the vhost content in either the Applications folder or Users folder. So I moved my SVN_Repositories folder to my Users folder so it was like this: “/Users/username/SVN_Repositories/ExampleRepo/trunk/httpdocs/”

It finally worked! So the lessons I learned were:

Put your vhost content in the Users folder.
If you try things from the terminal, make sure you do ./apachectl so you execute the correct binary file.
Most of your config files are in the /Applications/xampp/etc/ folder.

As always, hopefully this helps someone. Feel free to leave any questions or comments.

LT Pact ’08 – Overview of Experience

Justin Carmony — Fri, 27 Jun 2008 22:52:45 +0000

I just left the last session of the LT Pact ’08 conference and it has been fun. I’ve learned a lot and there were a lot of common themes among the conference. I’ll briefly touch on the themes and over the next week or so I’ll be writing more in-depth on each one. Besides, I need to enjoy Las Vegas while I’m here and not just blog.

Virtualization & Cloud Computing

Virtualization is the future of server hosting. It won’t be optimal for all scenarios of course, but there are so many advantages to virtualization that they can’t be passed up:

Backups & Snapshots
Allocate hardware resources appropriately
Fast deployment
Easier management of data centers
Scalability

The abstraction of an Application’s OS and the hardware offers so many different options and empowers engineers to successfully utilize their hardware to it’s fullest power. There is a very large emphasis on viewing utilization on a per application level and not per server. This allows for many servers to be deployed with only one “role” for the organization, properly spec’d for their function.

Cloud computing is the next step of Virtualization. Instead of just properly allocating resources to virtual servers, cloud computing allows for organizations to “tap” into a Data Center’s cloud and utilize their CPU, Memory, and Disk Space, pay for exactly what they use, and then walk away. This allows for very agile development. Not only scale as you go, but some times in a blue moon you need access to extremely large amounts of CPU and Disk Space, use it, and then move on. It isn’t practical to buy all of that horsepower to only be used once. They gave a story of the New York Times and how they used the Amazon web services to perform an incredible task that just needed to be done once. To be honest, it was incredible and paradigm shattering.

The Forrester Researcher that spoke warned of avoiding the “hype” of cloud computing and keeping things in a realistic expectations. Hype creates unreal expectations and customers find themselves in a state of “disillusionment” once reality hits them. In short, cloud computing is the future, but keep it in a realistic context.

Sun & MySQL

The team is managing to turn eyes. While sticking true to open source roots and looking at open source as a friend, the Sun & MySQL partnership is starting to look very promising. I’m actually going to download and try out OpenSolaris. From what it sounds like, a dedicated OpenSolaris machine could be the perfect high performance database machine with MySQL. Only time will tell, and we’ll see if that trent catches on.

Blogging: The Software, The Community, and The Business of It

Wow, did they talk a lot about blogging. While most of it I already had a hunch about, there were some serious perspectives that I saw, especially from Matt Mullenweg, that were very insightful. Matt explained that blogging is so much more than just a publishing platform, that “[it] is about people talking to people.” The other two panelists were very insightful, and it was interesting many of them had different views on almost any topic. how

Personal blogging should be, as they said, personal and can be just about anything. To keep it interesting personally, it should be just whatever your mind is on. Now this doesn’t mean to blog every single thought (that for some twitter people… sheesh), but to blog what you feel is important. It should be something you do for yourself first and your readers benefit is a byproduct of that desire to write. Whether for no readers or millions, your blogs should be motivated personally.

Business blogs are more important to stay on topic and put a personal face on your blog. Being relevant will help your customers know what your company is about, and will help in SEO. There were a lot of pointers and I’ll put them down in the upcoming weeks.

My only complaint was that throughout the entire session members of the audience repeatedly raised their hands, but not to ask questions, but just add “comments” which were basically saying what the panelists in different words. It was frustrating because it constantly happened, and we were only able to make it through about 1/3 to 1/2 of the questions. I didn’t sit down so I could hear the audience just agree with anything, but to here what the panelists had to say.

Final Thoughts

I didn’t realize how influential Layered Tech has been in the server hosting world, and how large of clients they have, like WordPress.com. They are pushing the boundries with 3Tera and other companies. I hope that I will be able to attend next year and if so, I look forward to it.

LT PACT 2008 – Day 1

Justin Carmony — Fri, 27 Jun 2008 06:31:52 +0000

Whew, today was a good day. I took a lot of notes and over the next few days I’ll be blogging on the different topics. Hopefully you’ll enjoy. Related posts:Getting ready for LTPact 2008 Why I love ecto To Blog or Not To Blog… Related posts:

How to tell what Linux OS you’re running

Justin Carmony — Tue, 24 Jun 2008 22:01:23 +0000

I deal with many different servers and it is hard to keep track of which is running which Distro of linux. Here is the quick way to tell:

uname -a

uname –help

You can’t believe how hard it was to find that out.

No related posts.

Getting ready for LTPact 2008

Justin Carmony — Tue, 24 Jun 2008 15:35:01 +0000

Well, I’m doing my laundry and getting ready to leave tomorrow for LTPact 2008. I’m looking forward to hearing about new technologies in server hosting. They also have coming several speakers on blogging, including Matt Mullengweg, founder and creator of WordPress. Layered Technologies has been a fantastic sponsor for CEVO.