Solution, open up the terminal and run the command:

sudo chown <username> /dev/bpf*

After a reboot, these permissions get reset, so you need to do it after each reset (and hence why I need this every now and then but can never remember the command).

Related posts:

1. XAMPP for Mac – My Frustrations & Solutions
2. Ubuntu Desktop Terminal – Su
3. MySQL Stored Procedure, NAME_CONST, and Character Sets

Today some memories came from when I first started with web development. So before I forget anymore of the details, I thought I would share my unique way I became a web developer. I never went to school to learn Computer Science or Information Systems. I’ve had a total of 2 programming classes, which were my in sophomore and junior years of High School, one of which we lovingly nicknamed Warcraft 101, because we would spend the entire time getting out butts kicked in Warcraft 2 by my classmate Daren (We all got A’s, we had just finished the entire coursework in 2 and a half weeks).

The reason I share this is perhaps others who hope to become a web developer can learn some of the valuable lessons I picked up along the way. That, and that my close programming friends and colleagues can get a kick out of my humble roots.

Heritage

How I ended up in Technology to begin with is probably because part of it is in my blood. My Father, Kevin Carmony, has been the owner of several technology businesses. So I was exposed frequently to computers and other technologies my whole life. I remember some time when I was about five or six, in the late 1980s, playing against my younger brother on a multiplayer ASCII LAN game at the old Streamlined Information Systems office. It was wandering around a 2D maze hunting these 8-bit monsters, and each other, and it was amazing fun.

I also had access to a lot of expensive software that were hand-me-downs. One of them I remember is Photoshop 3 and 4, spending hours trying to design websites with it. To give you an idea how old that is, the current version is CS5 (aka version 12). The first program I really used to make a website was (and get ready to gasp) Microsoft FrontPage in 1997. There were also a couple of website books laying around that I would read, or at least try to.

On the other side of the equation was my mother, while herself wasn’t extremely technical, encouraged and “sponsored” my learning when I was young. By sponsoring I mean she many times bought computers, digital piano keyboards, DSL (we were some of the very first people to have “high” speed internet in our town), and paid for many other expenses. I remember a several hundred dollar long distance bill because I would call the “Provo” dial-up connection instead of the broken “Ogden” phone number. I even once ordered a “temporary” AOL dial-up account, with her credit card and without permission, because our ISP was down, and I needed to check my website. I forgot to cancel it afterward, and it racked up a few months of fees before my mother realized what had happened. She never yelled or mad over these “expenses”, that I realize now as an adult, were not cheap. She just told me in the future what to do to avoid causing them again.

She was also very patient with a son whose grades weren’t the best, and who would rather work on a website than do his homework. I know if she would have came down hard on me, my GPA would probably be higher (its wasn’t bad, just not great), but I wouldn’t have learned what I did, nor make the connections that eventually fast tracked my career. I owe her a great deal (I love you mom!)

Humble Beginnings

I first learned about websites, and how I could make them, while sitting in my 7th Grade Band Class. My fellow clarinet player Kenny Cottrell explained to me in between songs about HTML, Notepad, and how I could learn to make my own website. So I set off to learn HTML. In my excitement, I found an online book on HTML, and printed the entire 400 page book on my mother’s home printer. Single page, of course, because I couldn’t figure out how to easily do double page, and it was on her expensive laser printer (this was 1997) with expensive cartridges.

My very first website was hosted on one of my father’s web servers, and using FrontPage, I made a website about a game called The Realm, one of the very first graphical Massive Multiplayer Online games (MMO, think World of Warcraft). It was bad, really bad. I can’t find any pictures or old files from it, but you can take my word for it. When spending weekends at my Grandma’s house with my Dad, I would spend hours designing and writing websites, or at least try to. They were all bad, but I learned a lot by trying over and over again. I would design a site, look a a professional site, and try to see why mine stunk while the professional ones were so much nicer. I remember looking at Amazon‘s rounded corner tabs for hours trying to get mine to look just as nice.

I then got my first domain, RPGLegacy.com, in 1998 and started a website with game reviews and walkthroughs for PlayStation RPG games. I remember writing reviews and information for games like Final Fantasy VII, Suikoden I & II, and even more obscure (and terrible) titles like The Granstream Saga. I started to get perhaps 100 visitors a month, and I thought that was great. I even got emails from people asking me for help. One subject in particular was in Breath of Fire 3 dozens had emailed me about getting stuck in a castle. I myself had gotten stuck in the same place for hours, and posted an in-depth solution for finding Honey the Robot in the castle. For being a 15 year old kid in junior high, it was a lot of fun, and I learned a lot. I had moved to Dreamweaver for making the website, and the designs weren’t half bad. However, I don’t have a screenshot of how the old site used to look.

Counter-Hack

I was 16 and I in high school. My classmate Daren (the same who whooped me at Warcraft 2) introduced me to a game called Counter-Strike. It was a First Person Shooter, and it was a lot of fun. However, at the same time, there were a lot of these “cheats” and “hacks” programs that were coming out. My friends and I would be accused of “cheating” and “hacking” when in reality we weren’t. Also, some of the claims on how we were cheating were rather absurd. So I started to investigate the truth and fiction behind these programs. I learned a lot, and thought I would share that knowledge, so I started Counter-Hack (http://www.counter-hack.net) in the summer of 2001.

Little did I know how extremely popular this website would become. Within a week or two I had a website wtih decent content up using Dreamweaver. This was 100% HTML based, no CSS, trying to use Dreamweaver’s Template system. What I would have given for some PHP or even WordPress. While with RPGLegacy, I had maybe a max of 5,000 visitors over it’s entire lifespan, within a month I had 30,000 visitors to Counter-Hack. Soon I was getting thousands of visitors per day. About the same time I met Anthony Ouwehand (nicknamed H3X), who had ran another popular website about video game hacks. He graciously helped me with his PHP/MySQL skills. He designed, and developed, the Counter-Hack website that was launched in 2002 and ran until 2008 when everyone involved with the project just were too busy, and the site had ran it’s course and purpose. Two years of that time the project ran with the rest of the volunteers while I served an LDS Mission in Torreon Mexico.

During the years with Counter-Hack, a few highlights were interviews with Wired for news stories, working with Valve Software to help recover the HL2 Leak, and an interview and article with Rolling Stone Magazine. During it’s height, Counter-Hack was covering dozens of games with hundreds of thousands of visitors a month. During it’s later years, Counter-Hack implemented a Wiki system for much of it’s content, something that was pretty new at the time. All and all, it was a great experience with dozens of volunteers and great memories. For a hobby during High School and the year after graduating, I couldn’t have asked for a better experience. I learned a lot before even starting college.

Developing Out of Necessity for CEVO

The reason why for the background story is for two reasons: I had started practicing web design and development from a very early time, and I gained a unique background and knowledge with my work with Counter-Hack. So in March 2005, being home only a few weeks from Mexico and still with a heavy mexican accent, I was approached to work with a company called CEVO. It was an online video game league that was emerging, and their first game was Counter-Strike. They needed someone to help consult with preventing cheating and “hacking” in their matches, and I was a perfect fit. So I joined as a consultant, and quickly became the Operations Director within a few months. CEVO also had brought on Eric Ping to be the CTO, and the company started to grow.

One of the challenges CEVO faced was it was a completely distributed team. Charlie Plitt, the owner, lived in Baltimore, MD, Eric lived in Ephrata, WS, and I lived in Ogden, UT. We had staff and volunteers that literally lived in all 50 states except Hawaii and Alaska, as well as some across Canada. Our customers also lived all over North America, and we relied on the Website, Email, and VoIP to run the entire company. It was insane and awesome at the same time. This put a ton of demand on Eric, our only developer for the website, to get new features implemented quickly. During the first two years, we had re-designed the website a total of 6 times before finally settling down on the current design, which is being redesigned now as well. Eric couldn’t handle it all, and since we didn’t have the funds to hire another developer, I thought since I knew something about web design and programming, I could help out.

Eric took me under his wing and basically mentored me along as I started to take on project after project. I had become familiar with PHP and MySQL when I was tinkering with Counter-Hack’s code, but now I was really learning. I was also learning extremely quickly because I had no other option. It wasn’t like working at another company where I had a team of Senior Developers that could bail me out. Eric worked such late hours that normally if I was awake, he was asleep, and we meet for a few hours in the afternoon and evenings. But if I had a bug, or a problem, there was only one person who could solve the problems: myself. Working on a team of two developers for a start-up company is extremely demanding, and I was constantly looking for ways to work more efficiently. It was the only way to meet CEVO’s growing demands, by learning how to be a quicker and better developer. So while I had a day-job of doing Tech Support for a local company, I spent every other free moment working for CEVO. As CEVO grew, we ran into scaling problems and performance problems, things a normal “Jr. Developer” wouldn’t have hands-on experience dealing with.

Eventually, I had learned so much working for CEVO, that I was able to quit my day-job doing tech support, and did contract work on the side. I helped launch some e-commerce websites, some basic business websites, and spent the rest of the time working on CEVO. I had started to go back to school for my Business Administration degree at WSU, and I was pretty much busy non-stop. But it was a lot of fun, and allowed me a lot of freedom not having a normal 9 to 5 job.

Ambient Partners, LLC

In 2006, I had the most amazing thing happen to me: I met my wife. We had dated for almost a year and we decided to get married. So by the end of 2006, I had a serious realization: I needed a steady income. While contract work paid really well, I could go a month or two without a check from clients. When living at home with relatively no expenses with the exception of my car, I could get away with this. But health insurance, rent, groceries, etc. I couldn’t live that way. So I decided to find a full-time programming job. Fortunately, the company I had worked tech support at had split with their California office and changed names. They were a company primarily doing software for DVD Rental Kiosks, and needed a Web Developer to do work on their web technologies. So by January 1st, 2007, I became a full-time employee for Ambient Partners. Our development team consisted of myself, a Senior Developer, and the CTO. As I look back at what the three of us accomplished as a development team, it amazes me.

When I joined, our main client had about 100 kiosks in the field. After doing the company website, I was given a very big project: RBO, Rent-Buy-Online. We wanted to provide clients with “white-label” solution to allow people to reserve and buy DVDs on a website, and then go pick them up at a Kiosk. There were two parts to this solution. The website itself, and the web services to supply the website with data. I had never written web services before, so I had a lot to learn very quickly. Also, it was to be written completely in ASP .NET, a framework, and C#, a language, I had never worked with before. So I started to learn how to build this project in .NET. It was very difficult on multiple fronts, but on in particular that I had underestimated was going from a loosely-typed language (PHP) to a strongly-typed language (C# .NET). So I tried several different methods, started a few different projects, and after a three months I got an email from my boss. It was short and sweet: “Justin, money is tight and we can’t afford to keep you on staff if you’re unable to make real progress on RBO. Either we need to see some real progress very soon, or staffing changes will be made.”

I learned very quickly that spinning your wheels trying to do something perfect, but never getting done, is an excellent way to stay unemployeed. I immediately shifted gears after the thought of explaining to my new wife that I had lost my job. Fortunately, after the fear of doing something wrong was overcome by my fear of not finishing anything, I completed RBO to version 1 in record time. I had made some ugly coding decision that we refractored out later, or scratched completely, but it was a working prototype. My boss was happy, and I was happy and still employed. While the front website has been redone for Blockbuster Express, it is powered by the same web services I built in 2007.

After building RBO, I was tasked with building a reporting & support suite that would manage millions of transactions. This second project went much smoother, and I put a lot of effort into it. I knew the people making the decision whether or not to buy our multi-million dollar software suite would be personally using this piece of our solution, so I wanted it to make them really smile. I was told after the demo of our new software suite, the executives all mentioned they were looking forward to using their “executive reporting tool.” Several months later, after successfully building some pretty slick software with the rest of the team (of which other things were even cooler then what I made, like Chris’s auto-updater system), Ambient Partners was purchased by NRC.

Dating DNA

By 2008 when the NCR deal was underway, I had a choice. I could stay, take a very nice raise, and work for a very large corporation. But, deep down inside, I had a bad feeling about working for NCR. As a small team, we were very effective and there was almost zero political or bureaucratic non-sense in Ambient. As for NCR, I would go from being a developer in a company of three developers to a company of hundreds, if not thousands, of developers. I was also tired of working in .NET and Web Services, as I was moving away from what I loved to do: building cool websites with cool technology.

My father at the same time was working on his new business: Dating DNA. They had a web developer in San Diego, but he was expensive because living in San Diego is expensive. So Kevin asked me if I would be interested in working for Dating DNA. We tested the waters by having myself build a Web-based iPhone App for Dating DNA (the App SDK hadn’t been released yet.) It worked really well, so I turned down the offer to become an NCR employee, and returned to my roots as a developer: working from home working with code I love.

Through 2008 and 2009 I worked as a full-time developer for Dating DNA. I took over all responsibilities for all their technology. In those two years, we did a lot. I’ll have to write a new blog post to completely cover everything we did that was awesome, but here were a few highlights: Built a real-time score generation system that could calculate hundreds of scores per second. Built the iPhone’s first Dating App, and to this day is a top ranking App in the charts and highest rated dating app. We scaled from 3,000 users to hundreds of thousands of users. We built a handful of new iPhone Apps, the main one being Clipish. We built custom chat rooms using Ajax and Comet, and a bunch of other stuff.

Alienware & CEVO

All this time, I was still doing work in the evenings with CEVO. In 2009, we were approached with the opportunity to do something we hadn’t done before in CEVO. Dell’s brand Alienware wanted us to make them a website like CEVO’s, only completely branded for Alienware. We built, from the ground up, and custom solution for Dell and Alienware, and Alienware Arena was born. This was a great project to work on, and we were able to get it done on an extremely tight timeline and a strict budget. I did 100% of all the graphical design, following Alienware’s look and feel, and I’m very proud with the result. It was built by myself, Eric Ping, and our new talented developer Mike Stevens. While I can’t say how many members Alienware Arena now has, it is a lot. Building such a successful website for a large company like Dell doesn’t come around very often. Each year Dell has us add more and more features to it, and it has been a great project for CEVO.

Utah Open Source

One thing that happened in November 2008 that I would consider one of the crucial events that “fast-tracked” my education as a web developer was being introduced to the local Open Source groups in Utah. I spoke at the Utah PHP Usergroup and was introduced to the Utah Open Source Foundation. Through these groups, I met dozens, and eventually hundreds, of talented, passionate people. While through CEVO I fast-tracked my web design and PHP development, through the Open Source groups I broadened my knowledge of so many more technologies. I learned about nginx, nagios, redis, memcached, php apc, linux server administration, and Git just to name a few off the top of my head. What is great is not only did I learn about these things, but I met people who know a lot about them. So when I ran into problems, I already knew a solution that could work, and knew people I could ask questions too. That, and I’ve made a lot of great friends through the different meetings, lunches, and conferences. I’ve picked up some contract work through my connections with these groups, and overall they have been extremely beneficial and great.

Chief Technology Officer

All of these different things played a part in me having the job I have today, CTO of Dating DNA. I’ve written already about my new responsibilities as our new CTO, and what it means for the company. These few short months as CTO we’ve made a lot of changes to handle even more scaling (especially with holiday surges and such). We’re in the process of improving our already fast score generation system, and moving our user photos to a more scalable solution in the near future. We’re evaluating our usability and such for our website, and seeing if a redesign on certain areas would be beneficial. There is a lot of work to be done, but I truly enjoy it.

Advice & Lessons Learned

After reading and thinking about the different things I’ve experienced and gone through to get me to this point, I’ve had a few thoughts that I think can help anyone in our field, and other fields of work:

• Surround yourself by people who help each other learn – Looking back, all of the people who have really helped me along with my career and education (not just schooling) have been people who help everyone learn. They each others things, and then learn from others. They harbor a culture of continual learning, and being in the tech industry which is always changing, this is critical.
• Always be learning something new, always – Trust me, there is always something to learn in this industry. Even if you’re learning something that isn’t directly involved, you never know what it might lead to. My work with Counter-Hack lead me to CEVO, which in turn lead to an accelerated web development “course” of “holy crap, we need to get this done and working or we crash and burn.” Even to this day, a lot of the things I learned about how hacks work I apply in other areas of computer science.
• Don’t let formal education be your only source of knowledge – Those that know me know I can be a little “harsh” when talking about formal educational institutions. Especially my frustrations with certain types of developers that are produced from these institutions. But the bottom line is this: they can be a great source of knowledge, and you can learn a lot from them. However, if you don’t learn additional information outside of the classroom, you are going to be sorely disappointed at how much you know when you graduate. Fundamentals are crucial, but practical application is just as important.
• Apply and build something important to you – There isn’t a better teacher than experience, and getting is as soon as possible, and as frequent as possible, will help a lot. I’ve spent more years unprofessional doing my job than professionally (at least for another year or two). Working on something meaningful to you, not just going through the motions of tutorials, really teach someone what it is like to do this kind of work.
• Networking and getting to know people is crucial – There is such an important emphasis on skill in the technical world, that knowing people and their actual abilities is vital. Also being know for your set of knowledge is important. That way when you want to learn something new, you know who to seek out. Before becoming Dating DNA’s CTO, and the rumor went out I was considering a new job, I had a lot of people contact me to see if I was interested in certain positions. There is no down side to being a “social” developer. Just because this isn’t “Marketing” doesn’t mean social networking isn’t important. I personally don’t like the term “networking” since it makes it seem like a chore. Make lots of friends in the programming groups and circles in your area and community, and it will be beneficial.
• Love what you do – If you don’t love what you do, then there is a good chance you will not go far in this industry. It doesn’t mean this is the only thing you do. Other hobbies and activities are important. But if you dread going to work, and do your work, every day, then it’s time to find something else.

This went a lot later in the evening than I thought, so I hope my thoughts are coherent, and if nothing else, entertaining. Its been a great deal of fun since those first days with Photoshop 3 trying to design something that didn’t look terrible. I look forward to the next few decades to see where web technology takes us.

Related posts:

1. So You Want To Be A Web Developer?
2. Web Development 10-Years Ago & Now
3. Being a Productive Developer

This is where Redis comes in. I hate doing any extensive logging to disk on a production server, however sometimes you need to do some logging to track down a bug. With Redis, it instead would be logged to memory and periodically written to file. So it was very, very quick, and did not slow down anything on production.

So to log where how many times our class was being called, we used PHP’s debug_backtrace() function. This would return an array with each row holding information about the stack trace. Using this, we could identify the different parts of our program that was using our class and this SQL statement. I used Rediska as a PHP library to communicate with Redis.

So, inside of our class right before the SQL statement, I included this code:

// Get Instance of Rediska for the Debug Connection
$redis = new Rediska_Manager::get('debug');

$stacktrace = debug_backtrace();

// Loop through the stacktrace
foreach($stacktrace as $k => $v)
{
    // Unset args and object variables, since they will throw off our hash calculation.
    unset($stacktrace[$k]['args']);
    unset($stacktrace[$k]['object']);
}

// Start an output buffer to capture the var_dump;
ob_start();
var_dump($stacktrace);
$print = ob_get_contents();
ob_end_clean();

// Hash what it returns, since it will the same each time for each stacktrace
$hash = md5($print);

// Increment the count
$redis->increment('debug.photoAlbum.count:'.$hash);
// Set the Content of the Stack Trace so it can be read later.
$redis->set('debug.photoAlbum.content:'.$hash, $print);

Few important lines to note. We unset ‘args’ and ‘object’ because it will unique for each function call, we just want to know where the function is being called. We then set two values to redis, one to hold the count, and another to hold the content.

Now, to read the data, I just made a PHP script:

// Get Instance of Rediska for the Debug Connection
$redis = new Rediska_Manager::get('debug');
// Get all the keys matching the patern for the counts.
$keys = $redis->getKeysByPattern('debug.photoAlbum.count*');

$sorted = array();
$strings = array();

$total = 0;

// loop through the keys to strip out the hash
foreach($keys as $key)
{
    // Get the hash part
    $parts = explode(':', $key);
    $hash = $parts[1];

    $count = $redis->get('debug.photoAlbum.count:'.$hash);

    $total += $count;

    $string = "Count (".$hash."): ".$count."\n"
            .$redis->get('debug.photoAlbum.content:'.$hash)."\n"
            ."\n\n\n";

    $sorted[$hash] = $count;
    $strings[$hash] = $string;
}

// Reverse Sort by Numberic numbers
arsort($sorted, SORT_NUMERIC);

echo "Grand Total: ".$total."\n\n";

foreach($sorted as $hash => $count)
{
    echo $strings[$hash];
}

A few notes, first off, the KEYS function for Redis has some limitations. Mainly, it should be used sparingly, and only for admin pruposes. The documentation explains it pretty well.

Second, this script will sort by count and print the count along with the content of the stack trace.

This helped me identify a situation where it was calling the class several times instead of just once. The bottom line is, after figuring out the problem spot, we were able to fix it, saving us several thousand expensive queries every hour. I highly recommend this technique for debugging issues on production servers. Its working well for me.

Related posts:

1. Creating Chatroom / Walls with Redis & PHP
2. “Nerfing” a PHP Object
3. MS SQL 2005 (T-SQL) Row Count for Each Table

Also, I’ve never had a very strong need to learn C. Almost all of my work is web based, and so traditional LAMP skills were more than enough. However, things are changing in the industry, and there are a few reasons why I want to polish up on my C and re-learn it.

First off, there are a lot of new, cool technologies coming out for web development. The whole NoSQL concept and the different solutions for it are being written in C based languages. Even after all these years, many are being written in C. Now, I doubt I would be willing to write my own NoSQL solution when there are great ones out there, but I would love to contribute bug fixes, or be able to read and understand on a lower level how they work.

Second, I’ve been cautious or nervous compiling my own binaries on Linux. Almost always, if a new PHP release came out, I would want to wait for my distribution to release and update. Sometimes, that can be months, and even years. By learning C and learning more how it works, I hope to be able to compile my own binaries when needed, and not feel so hopeless if something breaks.

Third, PHP is written in C. There have been times where I’ve wondered exactly how a specific function or worked, or have found a bug I would love to submit a fix for. Its amazing how large the PHP community is, and yet how few contribute to help. I would love to become a code contributer, and learning C is a prerequisite definitely. Also, with web development and PHP, sometimes certain tasks could be performed much quicker when compiled, such as the score generation system for Dating DNA. It would be great if I could generate millions of compatibility scores at a C level, instead of just in PHP.

Fourth, is iPhone development is not going anywhere but up. We pay the bills and more with our iPhone sales for Dating DNA and Clipish, and having the option to do some iPhone Development when needed would be great. Granted, iPhone Development is done in Obj-C 2.0, but when I first tried iPhone development, almost every single one of my problems were from a fundamental lack of understanding on how Obj-C 2.0 worked. A better of understanding of C would help me greatly in this.

Firth, it would just be good to learn. Even if I don’t use it a lot, learning the programming language that is used by so many of my tools would be great. So, along side my posts on Redis, PHP, and web service development, you might be seeing on beginner information on Learning C.

Related posts:

1. iPhone SDK – Learning the Basics by Removing the Touch
2. WordPress & OOP
3. Project Management & Entrepreneurship

So for Dating DNA & Clipish I put together the “Basic Server Administration Document” which detailed everything a person would need to know to perform basic sys admin tasks. It was nicked named the “Oh Crap” document. Fortunately, while I was gone, there were no problems, and the document was not needed.

For my other main project, CEVO, that was not the case. CEVO has 4 times the servers that Dating DNA has, and is a bit more complex. But there are two other programmers who also monitor the servers, so I wasn’t worried. Then, while I was gone, half the physical servers for a Virtual Server “Grid” went down, which was my pet project. The CTO for CEVO, Eric, had to try to figure out what was wrong and co-ordinate with the Data Center to figure what happened and how to fix everything. Since we host our Databases on this “Grid”, if it goes down, everything practically goes down. At the time, I was in Glacier Bay, watching Glaciers crumble and fall into the ocean, with zero internet connectivity.

So thank you Eric for putting up with everything while I was gone. It was all fixed by the time I came back.

But if there is anyone wondering if an Alaskan Cruise is worth it, it definitely is. My father-in-law took us, which was very nice of him. I’ll probably post some pictures in the near future of the trip.

Related posts:

1. Quick Update
2. Speaking: Utah Open Source Conference 2009

A Little Background

It was the summer of 2008 when I attended the LT Pact conference that I first drank the “Cloud Kool-Aid.” As you can see from my notes, I was pretty excited about this whole concept. Excited enough, one client of mine signed up for Layered Tech’s grid layer (which is what they were demoing at the conference).

One interesting thing I heard at the LT Pact conference was one of their keynotes was as IT Researcher for Gartner. He outlined what phases the current industry would go through with “cloud computing”:

• First, several early adopters would have great success implementing the technology.
• Second, mainstream hype would “over-hype” cloud computing.
• Due to the hype, many people would start “using the cloud” as a silver bullet, thinking the cloud would be perfect for anything.
• A “cloud crash” would occur where disillusioned IT managers realized the cloud had its drawbacks, and wasn’t the perfect solution for any problem. More over-hyped people will start to move away from the cloud.
• After the dust had settled and the hype faded away, the actual cloud technology would find a common place in today’s IT.

Currently I think different people and groups are at different stages of the last three listed above. Some are still raving and hyping the cloud for everything. Some are bitter about their disillusionment. Many are now looking beyond the hype at the real concepts behind “cloud computing” and finding “the services are wildly different. While many parts of Web hosting are pretty standard, the definition of ‘cloud computing’ varies widely.” (InfoWorld)

After a few hours, the fog of hype starts to lift and it becomes apparent that the clouds are pretty much shared servers just as the Greek gods are filled with the same flaws as earthbound humans. Yes, these services let you pull more CPU cycles from thin air whenever demand appears, but they can’t solve the deepest problems that make it hard for applications to scale gracefully. Many of the real challenges lie at the architectural level, and simply pouring more server cycles on the fire won’t solve fundamental mistakes in design.

In a rebuttal blog post, Don MacAskill defending accusations that EC2 was over 50% slower than stated.

… let me explain what I think is happening: Amazon’s done a poor job at setting user expectations around how much compute power an instance has. And, to be fair, this really isn’t their fault – both AMD and Intel have been having a hard time conveying that very concept for a few years now…

Bottom line? EC2 is right on the money. Ted’s 2.0GHz Pentium 4 performed the benchmark almost exactly as fast as the Small (aka 1.7GHz old Xeon) instance. My 866MHz Pentium 3 was significantly slower, and my modern Opteron was significantly faster.

So what about that guy with the Ruby benchmark? Can you see what I missed, now? See, he’s using a Core 2 Duo. The Core line of processors has completely revolutionized Intel’s performance envelope, and thus, the Core processors preform much better for each clock cycle than the older Pentium line of CPUs. This is akin to AMD, which long ago gave up the GHz race, instead choosing to focus on raw performance (or, more accurately, performance per watt).

My Experiences

Virtualized Servers can be awesome. They have some great advantages, while also having clear disadvantages (if you look beyond the hype). While I know I’ve quoted a lot about Amazon EC2, I have only experimented with it. However, I have a good deal of experience with 3Tera’s AppLogic, and have recently moved a website to Rackspace Cloud.

Learning Curve

While Rackspace Cloud was more straight forward, learning 3Tera’s AppLogic is a large task. I spent several months learning a lot (and tearing my hair out some days) and then another year of learning to deal with it’s quirks. It is pretty cool once you get it down, but in all honesty its features didn’t line up with what the company using it needed. Every now and then we spin up a custom server for a website or something else. However, AppLogic is built more for people who are doing a large volume of similar instances. So while it works, it probably isn’t the best fit for us.

Also, with AppLogic’s components, they are engineered for very very specific tasks. Doing things outside of their scope *is* possible, however there was a vast amount of tinkering, reading, blood, sweat, and tears trying to get it to work. There are still things, like with the way AppLogic handle network routing, that are just unique at best.

Bare Metal to The Cloud

Each instance I’ve gone to deploy on the Cloud, either Rackspace Cloud, Amazon EC2, or AppLogic, the project itself had started on a dedicated server that we then moved to a virtualized server. Since when virtualized servers started to pop up, even in the earlier days of simple VPS solutions, you would be moving from dedicated bare metal to a virtual server. The inherit problem is that virtual servers, especially on shared hardware, have *less* resources available to them, especially CPU power. On Bare Metal Dedicated Servers, I rarely run into CPU issues. When the CPU power is being shared with several other “virtual servers”, you suddenly only have a quarter of what was available.

Then you have your neighbors. While moving Clipish to the Rackspace Cloud the little 1GB RAM server felt peppy. I was confident in our choice. Clipish isn’t complicated, and it’s web services are pretty easy going. The only intense part is when you clear the image cache. Then when a web server request comes in and the image isn’t there, it pulls the image from the Database, performs the ImageMagick operations to resize, watermark, create a thumbnail, etc on demand. During the early morning, it isn’t a problem at all. However, one night I “cleared the cache” during peak times, and within minutes the server had locked up.

Turns out on our old server, the web services could easily burst to all 8 cores for a few minutes while regenerating hundreds of image caches. Even the Rackspace server could burst during off-peak hours and repopulate. However, during peak times, when the dozen or so “neighbors” we’re using their allotted CPU, and our server couldn’t “burst” like before. So what did we have to do? We wrote a script that would loop through our image library one at a time and regenerate it. This works, but after the move and we were experiencing these problems, we thought “did we make the right move to go to the cloud?”

Cloud to Bare Metal

I think the approach with the “cloud” needs to be this. You have your baby project, so throw it up on the cloud. Let it cost $15-$20 bucks a month. Take advantage of the daily backup snapshots, etc, and just worry about development. Let it grow and at some point you’re going to hit a fork in the road:

• Bare Metal Benefits outweigh Cloud Benefits – Bare Metal is typically cheaper than Cloud power. If you are not spinning up and down servers on a daily biases, but is a more fixed size, then quickly scaling isn’t an issue. Cost can and should be a huge priority for hosting your application. If your application needs more higher CPU, Memory, or Bandwidth that doesn’t fit the Cloud model then going Bare Metal is a good idea.
• Cloud Benefits outweigh Bare Metal Benefits – If your application, however, can fit on the cloud and the budget at the same time, then it can be a beneficial to stay on the cloud. Just remember you can’t expect a Cloud server to outperform a Bare Metal server of the same specs. But you can take advantage of things like spinning up new servers in minutes, take quick backups of virtual machines, etc.

Conclusion

I guess my biggest advice when it comes to deciding on bare metal vs cloud is to understand the pros & cons. Remember that while you can burst to more CPU, you shouldn’t rely on that always being the case. The Cloud does afford some cool flexibility, but it isn’t perfect, and it can be a lot more expensive than going bare metal. Also, if you’re on bare metal, and moving to the cloud, the cost of raw power is a great deal more expensive on the Cloud.

Related posts:

1. LT Pact ’08 – Overview of Experience
2. Aptana Studio – PHP IDE Alternative to PDT, Zend Studio

I’ve gone back and looked at this year of 2009, and the things I’ve blogged about, and I thought I’d give a little recap.

My CSS Conversion

At the beginning of the year I realized I had 100% percent to using CSS. I’m still a little in awe that being such a table junkie I finally did it. There were a few key things that I did to help me learn all the difficult things with CSS. I hope I can help some poor web developer out there understand their CSS a little better.

A Year of PHP IDEs

I really invested some time in trying out and experimenting with several PHP Integrated Development Environments (IDEs). I had been using Zend Studio for several years, however more and more I found myself frustrated with it’s quirks. However, I haven’t had time yet to blog about the new IDE I am using, NetBeans. I’ve made the switch from Zend Studio to NetBeans, and while I’m not completely sold, I’m still giving it some time. Maybe over christmas break I’ll blog some about it.

Interesting Challenges w/ MySQL

While I haven’t blogged about most of them, I have had some interesting challenges w/ MySQL over the last year. One was converting 40 Million rows from a MyISAM engine to an InnoDB Engine. I’ve been having some serious heart-to-heart conversations with MySQL (metaphorically of course) on how I’m going to scale it to millions and billions of rows. I think I’ve mapped out in my head how we’re going to handle it for Dating DNA. That will be one challenge I’ll be addressing in 2010.

New Blog Design

This year I got around to re-designing my blog using the Carrington Theme as a base. I kinda like it, but it still could use a little more polish. One day I’ll find time to give it that extra polish.

Memcached

This year I really got into the theory on how to design an application with Memcached. I’ve decreased the load on several websites by using it, and wrote an in-depth article on the theory of caching, and how to implement it effectively. I’ve been amazed at how well it works, and that I hadn’t used it before.

A Year of Presentations

I’ve given two presentations this year, my Memcached presentation at UPHPU, and my “Real Life Scaling” presentation at the Utah Open Source Conference. I got great feedback from both presentations, and I look forward to what I’ll present on this upcoming year. I’m thinking about putting together a presentation called “Being a Web Ninja with jQuery,” and showing of the dozen of awesome applications I’ve built using jQuery.

Utah Open Source Conference 2009

This year I helped organize and throw together the Utah Open Source Conference. I was made the Sponsorship Manager, and it was interesting trying to talk to companies in a recession about donating to a non-profit conference. I haven’t had a chance to blog about different techniques on how to professional ask for money for something like this, but hopefully in 2010 I’ll be able to share some insights.

Things I Didn’t Blog About

There were a lot of things I didn’t get around to blogging about this year. I’ll quickly touch on them here, and hopefully in 2010 I’ll be able to blog about them more.

Web Services – I’ve done a LOT of web service work. Web services provide a unique challenge, since many times its hard to debug them. At my old job, one of the biggest challenges is we really couldn’t debug our web services until all of the other pieces of the puzzle were done. Then we spent long hours with trial and error, debugging via dumping logs. I’ve learned a lot of tricks with PHP and web services. Hopefully I’ll be blog more about those soon.

Performance Tuning LAMP – I’ve had to do a lot of tuning to keep the Dating DNA website afloat. Because of the awesomeness of our iPhone apps, we’ve had a lot of load on our web servers. There are a lot of things you can do to get the LAMP stack to run well.

Alienware Arena – Here was a fun little project I was able to crank out for probably the largest client I’ve done work for: Dell. We made them a website for their Alienware brand called “Alienware Arena.”

There were a lot of challenges to this website, mainly timeline and budget. But we were able to walk away happy with our work, and the Dell / Alienware team were really happy. It was interesting going through and trying to copy / mimic the look and feel of the Alienware website. I’m really proud of what we made.

Never Ending iPhone App Store Saga – Anyone who knows me has heard of the bloody wars that my current employer has had with Apple, their iPhone, and the App Store. The ridiculous approval process, the extremely long wait periods, and Apple’s OCD when it comes to their phone. I won’t say much anything else on the topic, but boy it has been frustrating.

What to Expect with 2010

Boy, 2010 will be interesting. What to expect? I’ll be doing some pretty sick programming in jQuery, as I’ll be adding integrating chat to the Dating DNA website. What else? A lot more when it comes to integrating websites into other social network platforms.

From a business platform, 2010 will be a lot of work spent on taking some great ideas and trying to monetize them. While working on awesome stuff is great, its important to pay the bills with them. I will have an entire year to sign up Sponsors for the 2010 UTOS Conference instead of 2 short months.

Hopefully I’ll get in a better habit of documenting my discoveries and solutions here in my blog. Sharing information is the best way to help everyone out.

I hope everyone has a happy holiday and good luck in the new year! 2010, here we come!

Related posts:

1. Speaking: Utah Open Source Conference 2009
2. So You Want To Be A Web Developer?
3. Presentation: Real Life Scaling

I have learned the hard way over the years that it doesn’t matter how clever you think you are or how much everyone loves you on your team. The fact is, there are always potential circumstances that can come to a head, resulting in the ax coming down on your valuable head.

Sometimes these doomsday series of events are in your control. Sometimes it’s like being blindsided by a bus.

…

We might as well start with the obvious. If your job consistently isn’t getting done, then you will eventually be toast. All it takes is a few missed deadlines and your manager will have no choice because “you know what” rolls downhill. A manager can only absorb so many blows from unhappy end users or their own boss.

Why wouldn’t you see this one coming? Well, you might be thinking the missed deadlines are not your fault. Your excuses may include “the design was bad” or “the deadlines are not realistic” or “they are making me code in Java and I am a .NET expert.”

Guess what? Excuses don’t matter. Results matter.

While I hope I never get fired from a job, hopefully these tips can help any developer out.

Related posts:

1. Great PHP Article: Leadership in Software Development
2. PHP Article: 10 Principles of the PHP Masters
3. Article: 5 Things Your Clients Should Know

The Problem

Many times while working with PHP and bigger frameworks, you’ll have classes that extend classes that extand classes. We have an ORM system that interfaces our Database with PHP classes. This makes accessing and updating information easy for us. So lets say we have a class called “Post” that stores a blog post. Our ORM class just takes a few lines of code to hook up our “Post” class to the “posts” database tables. We just extend our base DataObject, which contains all sorts of references to other class isntances. Long story short, this PHP code is awesome, but there is one issue.

Lets say I have an AJAX call that gets some post data, so I want to pass my “Post” class through a JSON parser to send back a JSON version of the PHP class. The problem was that the JSON parser was picking up on the extended classes’s private members and such, so it was spitting back a whole lot of stuff we didn’t want it to that belonged to the ORM classes.

The Solution

So what do we do? We “nerf” the class. Nerfing is a term used by gamers about video games. It means “[to make] a change to a game that reduces the desirability or effectiveness of a particular game element. The term is also used as a verb for the act of making such a change. The term is used as a reference to the NERF brand of toys which are soft and less likely to cause serious injury.” (Wikipedia)

So we make a “Nerfed” version of the class, with no functions, or extended protected members, just the data. We do this by using two parts of PHP:

• stdClass PHP Class
• PHP’s Reflection Class

Just attach this function to your class and you’ll be able to “nerf” it.

<?php

class ExampleClass extends ComplexClass
{

  // .. Some PHP Class code ..
  public function GetNerf()
  {
    $nerf = new stdClass();

    // Save the values of the fields
    $dbFields = $this->dbFields;

    foreach ($dbFields as $n => $v)
    {
      $nerf->$n = $v;
    }
    //var_dump($nerf);

    $me_ref = new ReflectionClass(get_class($this));
    $me_properties = $me_ref->getProperties();
    foreach($me_properties as $m)
    {
      if($m->isPublic())
      {
        $name = $m->getName();
        $nerf->$name = $this->$name;
      }
    }

    return $nerf;
  }

}

?>

Hopefully that makes sense, but the Reflection capabilities of PHP are pretty simple and powerful. Hopefully over the next few weeks I’ll be posting some more tricks with PHP Reflection.

Related posts:

1. PHP Singletons, Sub-Classing, and HAS-A Relationships
2. Aptana Studio – PHP IDE Alternative to PDT, Zend Studio
3. List of 50 PHP Tools

Doh! I meant to schedule this for Thursday, so oops! Already off my schedule, but oh well.

These last few weeks my productivity seems to have been fluctuating, and it has been irritating. So I’ve decided over the next few weeks to keep track of how I work, see what works well and what doesn’t, and figure out how I can be more productive.

Just as I would suggest any company with scaling/growth problems, I’m going to do the same with me: metrics. You can’t really know where and what to fix if you can’t tell how they are performing. So I’m going to start to keep track of my time in a lot more detail. I think this will help me in so many fronts, and I’ve known for a long time I need to do this. I’ve signed up for a demo account with Harvest, a time tracking software. Why? They seem to have a great interface, both Windows and Mac apps, and iPhone web and native app. So it will be easy to keep track of what I’m doing no matter what or where I am.

Over these next few weeks I’ll write how things are going, see where I can approve, and how that goes.

Related posts:

1. So You Want To Be A Web Developer?
2. Two-Way Sync between Google Calendar, Windows, OS X, and iPhone
3. Adeona – Open Source LoJack for Laptops