I’m excited to say I will be attending PHP Tek 2011 in one week. This will be the first “PHP” conference I’ve attended, and hope I will learn a lot while also having some fun. It’ll be nice to meet some people who I’ve only talked to online.

I’ve always wanted to attend a PHP conference for quite some time now, but it seemed for several years either it wasn’t in the budget, or I was just too busy. But I kept hearing great things about Tek, and it’s organizers, so when the Call for Papers opened up for Tek 11 this year, I submitted. Unfortunately I wasn’t able to make it on the presenter list, so I immediately registered and booked my flights so I wouldn’t find an excuse later on to not go.

For those in the Utah PHP Users Group who are not able to come out with me, and other friends of mine, let me know if there is anything interesting in the schedule, and I’ll see if I can attend and take some notes for you.

Alright, quick fix for a problem I always forget. When running Wireshark on OS X, when I go to select an interface to capture on, I get an error telling me there are no available interfaces to capture on. This is because Wireshark is running as a user that doesn’t have ownership on these interfaces.

Solution, open up the terminal and run the command:

sudo chown <username> /dev/bpf*

After a reboot, these permissions get reset, so you need to do it after each reset (and hence why I need this every now and then but can never remember the command).

I lived in Mexico for two years when I was serving my LDS Mission. There I discovered my favorite food of all time: tacos al pastor. Also known as “Shepherds Tacos.” They were awesome. I’ve found similar tacos here in Ogden at the Sonora Grill. However, there is one problem: the food health codes in the USA prohibit the tradition methods of cooking tacos al pastor. Here is a traditional method of cooking:

The key is cooking it on the rotisserie spit. It allows for a crispy outside and juicy inside. But as listed on Grub Street Chicago, there is a serious downside to this method:

If the restaurant isn’t busy, your stomach will be in a tizzy: that spit meat can go bad pretty quickly, and if there’s not a lot of turnover at the taqueria, you may turn over (this has happened to us before and it’s not pretty).

It takes at least 24 hours to prepare and marinate the meat, some people taking 48 or even 72 hours to prepare the meat. That means many people will prepare a very large amount of meat at a time to help save on the work. When the meat is cooking on the spit, the outside cooks much faster than the inside. So you start cutting and serving from the outside, letting the inside cook more. But if the restaurant is running slow, the inside can go bad before it is cooked.

The Food Safety guidelines state that you can’t start serving the meat until the entire piece of meat is finished cooking. So when Sonora Grill tried to do Tacos al Pastor, they would have to cook the entire piece of meat on the spit, and either keep it up at temperature, or cool it down and re-heat it later. This would cause the meat to dry out, and just wasn’t very good. So they changed their recipe and method of cooking. So they still taste great, but it isn’t the same.

Now, I won’t be preparing meat for hundreds of people, just for myself and friends. So I can plan, prepare, and cook the meat in a safe manner using the spit. So I thought “alright, I can do this myself, and have my favorite tacos.”

So I started researching some recipes. I found several recipes, and I’ve posted them at the end of this post (incase they go down, since the interent isn’t forever).

The second problem I face is finding a vertical rotisserie. After googling around I found out it is called a vertical broiler, and are also used in making traditional gyros. But the decent ones run about $1,000 bucks. I’m not ready to drop that kind of money on a spur of the moment feeling. So, here is my plan:

First, try to get the marinade down. I’ve been told this is the hardest part of making these tacos. Then, I can cook the meat in the oven and finish it on a pan. So the crispy/juicy factor won’t be quite right, but it will be pretty decent.

Second, there are some smaller smaller vertical rotisserie cookers for about $60-70 bucks. I figure I can try out one of these and see if it can work for making smaller batches.

Then, finally, if I can regularly make good tacos al pastor that are just lacking the spit, and if I am still committed, I’ll buy a vertical broiler. Then during the warm summer months I’ll be cooking my favorite tacos regularly!

Wish me luck!

Read More

I just read an article about “Node.JS and the JavaScript Age.” It was from a very “enthusiastic” point of view. My guess is they had tinkered around with Node.JS and using with client-side JavaScript rebuilding their Dashboard. You can do some pretty cool stuff with it, and it has a lot of potential. It is very easy to install and get going, much like Redis. Here ia an excerpt from the article:

This decision was driven by a realization: the LAMP stack is dead. In the two decades since its birth, there have been fundamental shifts in the web’s make-up of content, protocols, servers, and clients. Together, these mark three ages of the web:

I. 1991-1999: The HTML Age.

The HTML Age was about documents, true to Tim Berners-Lee’s original vision of a “big, virtual documentation system in the sky.” The web was dominated by static, hand-coded files, which web clients crudely formatted (with defaults that offend even the mildest of typographiles). Static documents were served to static clients.

II. 2000-2009: The LAMP Age.

The LAMP Age was about databases. Rather than documents, the dominant web stacks were LAMP or LAMP-like. Whether CGI, PHP, Ruby on Rails, or Django, the dominant pattern was populating an HTML template with database values. Content was dynamic server-side, but still static client-side.

III. 2010-??: The Javascript Age.

The Javascript age is about event streams. Modern web pages are not pages, they are event-driven applications through which information moves. The core content vessel of the web — the document object model — still exists, but not as HTML markup. The DOM is an in-memory, efficiently-encoded data structure generated by Javascript.

LAMP architectures are dead because few web applications want to ship full payloads of markup to the client in response to a small event; they want to update just a fragment of the DOM, using Javascript. AJAX achieved this, but when your server-side LAMP templates are 10% HTML and 90% Javascript, it’s clear that you’re doing it wrong.

Claiming LAMP is Dead is Reaching

It goes on to explain how LAMP like architectures dead. I would think that is a very overzealous point of view, and to perhaps put it in some perspective:

During each of these “Ages” new technology was developed to solve problems. HTML was created to create inter-linking information. As the demand grew and became more difficult to manage, LAMP like tools like PHP, Django, Ruby on Rails, and others were developed to help make delivering this information easier. It wasn’t so much replacing the previous technology, but adding on to it. Sometimes you will replace older technology with newer ones, but we’re still using HTML & CSS, and augment it when needed.

But LAMP-like systems are deployed in production all over the place, powering some of the most visited websites on the net. While there are some people who are using Node.JS, like Yammer, but Node.JS isn’t serving nearly the volume of content like PHP, Ruby, Python, or event .NET.

What is Node.JS?

Basically, it is a lightweight framework wrapped around Google’s V8 JavaScript Engine. V8 is the JavaScript engine that powers Google Chrome, their open source Web Browser. Another part of Node.JS is that it is event based, which can make is very quick and efficient.

I’ve been tinkering with Node.JS for a couple of weeks, and it is very interesting and has a lot of potential. It’s unique scoping with JavaScript and event-based methodology makes certain things easy to accomplish that are more difficult in other traditional tools. There are some pretty cool libraries for Node.JS like Socket.IO, a library to support multiple transports using feature detection. So if your browser supports WebSocket or Adobe Flash Socket, it will use that, or AJAX long polling and multipart streaming, or even the Forever Iframe technique. So your application will use which ever type of communication is most efficient.

A Little Bit of a Reality Check

But lets be honest for a moment, Node.JS was created in 2010. It is still very, very young. In the video on the front page of the Node.JS is a video of it’s creator presenting to a PHP Users Group about Node, and even cautions using it in production. It still has a long way to go to maturing as a tool. Granted, being based off the V8 Engine brings a lot of maturity to the project.

As for replacing technologies like PHP, Ruby, and Django, I’m extremely skeptical that they are “dead.” Instead, I see Node.JS augmenting our existing technologies. Like how I’ve implemented Redis as a data store where MySQL was poorly suited, I see Node.JS functioning as a tool for real-time communication for the future of websites.

I hope to write more about Node.JS, and look forward to using it to solve new and unique problems. But I won’t be rewriting all my websites in Node.JS, it would be a nightmare. So like with NoSQL, and Cloud Computing, yes it is a new tool, but it won’t radically remove everything else up to this point. Like with Redis, I didn’t get rid of MySQL, I just use Redis when it is a better choice.

I mentioned on my twitter account that my very first project in PHP is used in production today, and I had a few people ask me what it was. So I thought I would show some examples of it in action. While I had done other things in PHP, this was the first time I had sat down and built someone by myself instead of just modifying someone else’s code. I built it for the very first time back in 2005, and since then have had to update it a few times. But it is still used regularly to host CEVO’s tournaments.

It basically is a system of managing a tournament bracket. Here is an example:

Example Tournament Brackets

It basically is a PHP script that pulls from a database with key-value pairs. In a template file, in which you would create the bracket’s HTML, you would insert tags with an id. Then, depending on if you are viewing or editing it, it with either insert the value, or a input box used for changing the value.

There was a basic admin listing of the brackets that are active, as well as access to archived brackets:

Admin Panel

You could add a new Bracket and select a template to use. When you select a template, it will show a preview of the template below.

Create a Bracket

Once you create a bracket, you can edit it. It isn’t pretty, but it gets the job done:

Editing a Bracket

Here is an example of the finished product that I just created. If you want a better example, here is the recent League of Legends tournament bracket. While it isn’t pretty, it works. Some of the biggest problems with it is the editing and managing of the templates. You can only use some sort of WYSIWYG editor to be able to make sense of them, since they are a giant table. What that ends up leading to is template files with 2,000+ lines. It still amazes me this chunk of PHP code has been used over 1,000 times for different tournaments around the world.

One day we are going to rewrite the system to allow the end user to make their own templates, and more interaction & connectivity between the bracket and CTE (CEVO Tournament Engine). But until, my first real PHP project can continue to chug along just fine.

Marco Tabini, co-founder of php|architect and Blue Palabora, posed an interesting topic: Is PHP running out of itches to scratch?

I think it’s fair to say that the pace at which PHP core is being developed has slowed down considerably over the past couple of years, while the development of many projects based on it, like programming and application frameworks, has sped up and continues to grow at a fast pace.

He goes on to point out some of the reasons for the slow down of development for the PHP Core. But at the end, he states it ultimately boils down to find a better way to bring features build downstream back upstream.

The risk facing us, as I see it, is not that Drupal, or WordPress, or whoever may decide to fork PHP or abandon it altogether. Rather, the problem is that there is no real way for these projects to provide upstream positive feedback to PHP core.

As I pointed out that the meeting that Cal references (I was the instigator of the discussion), core developers can’t improve PHP if they don’t know what needs improving, and downstream developers are forced to resort to needlessly duplicate functionality that they could instead feed into and pull out of core. This, in turn, would enable the latter to focus on what really makes their projects unique and make the whole PHP ecosystem better in the process.

So I thought I would do my part by first my blogging about a few “itches” I’d love to see scratched. My first itch would be some form of Object Notation for PHP.

Read More

I read an opinion article that put into words exactly my opinion on the trends in the public and private sector. Now, I have family and in-laws who work for the government, as well as relatives who have worked a long time and retired. I believe the government has it’s place to do thing in our country.

However, the trend is that more and more people are working for the public sector, and this article puts it into perspective.

More Americans work for the government than work in construction, farming, fishing, forestry, manufacturing, mining and utilities combined. We have moved decisively from a nation of makers to a nation of takers. Nearly half of the $2.2 trillion cost of state and local governments is the $1 trillion-a-year tab for pay and benefits of state and local employees. Is it any wonder that so many states and cities cannot pay their bills?

In a very simplistic view, the government gets it’s funding from taxes, which come from the citizens and private companies. But if the private sector continues to shrink, and the public sector grow, how can we sustain that?

The other troubling factor is that it seems this trend isn’t going to change anytime soon.

Don’t expect a reversal of this trend anytime soon. Surveys of college graduates are finding that more and more of our top minds want to work for the government. Why? Because in recent years only government agencies have been hiring, and because the offer of near lifetime security is highly valued in these times of economic turbulence. When 23-year-olds aren’t willing to take career risks, we have a real problem on our hands. Sadly, we could end up with a generation of Americans who want to work at the Department of Motor Vehicles.

While I’m not an expert in economics and employment, I do believe that if these trends continue, eventually it will end up with much more drastic consequences. Look at the current situation with possible furloughs for non-essential government workers. Unlike with a company which can downsize workforce if needed, the government will end up going much longer until it has to tell hundreds of thousands of people, if not more, that they cannot come to work until budgets are fixed.

What is even more ridiculous is my brother offered to go on voluntary furlough weeks ago, to help those who couldn’t afford to not work for a week or two. Apparently several employees all offered. But they were told no one would be furloughed unless everyone was required to be furloughed, even if others volunteered.

So I recommend to anyone to read this article, and hopefully to understand the concept that the public sector cannot outgrow all other sectors without becoming a burden to it’s citizens and businesses.

Article: The Wall Street Journal – We’ve Become a Nation of Takers, Not Makers

This last weekend I had a fun opportunity of restoring roughly 912 Million Rows to a database. 902 Million belonged to one single table (902,966,645 rows, to be exact). To give an idea of growth, the last time I blogged about this database we had about 40 million rows. This giant table, the “Scores” table, is has a very small schema: two ints, a tiny int, and a DECIMAL(5,2).

Problem

Our current backup system uses mysqldump. It dumps a 25 GB sql dump file, which compresses to about 2.5GB using gzip. The last time we needed to restore a backup it was only about 9GB, and it took several hours.

This time, I created the database, and from the mysql prompt I issued the following command:

\. /path/to/backup/database_dump.sql

It would run great until it got to about 10% of the way through the scores table. However, it would start to slow down. Because the rows were so small in our scores table, each INSERT statement had about 45,000-50,000 records. So each line had roughly 1MB of data.

At first it would insert a set of 50,000 in half a second or so. However, after a few million records, it would slow down to three second, and got to about 10 seconds per INSERT statement. This was a huge problem, given that I had roughly 18,000 INSERT statements, and at 10 seconds per INSERT, it would take 50 hours to restore. Our website was down during this restore, since it was our primary database. So being down for over two days was not an option.

While trying to diagnose the problem I noticed something. While using the MySQL command “show processlist” the thread for the Database Restore would be in the sleep state for 9-10 seconds, and then the query would execute in under 0.2 seconds. So it wasn’t a problem with MySQL storing the data, but a problem with reading the data from such a large database dump file.

So I tried from the server’s command line “mysql -u user_name -p database_name < /path/to/backup/database_dump.sql" with the same result. The longer into the file I got, the longer it was taking for MySQL to read the query.

Solution

So, after some thinking late at night at 3 AM, I came up with an idea. Why not split up the database sql dump into multiple files. So I used the linux “split” command like this:

cd /path/to/backup/
mkdir splits
split -n 200 database_backup.sql splits/sql_

This produced several dozen files in order, and it took about 10 minutes. The -n option told split to split each file up into 200 lines each. So the files were then named sql_aa, sql_ab, sql_ac all the way to sql_fg. Then, I did the following command using cat to pipe the files to mysql:

cd splits
cat sql_* | mysql -u root -p database_name

The only problem with this method is you don’t see a status report for each query executed, it just runs until you hit an error, displaying the error. If no errors occur, it will just return you to the prompt. So to monitor the progress I would execute a “show processlist;” command on mysql to see how far we were.

4 1/2 hours later, the entire database was restored. A few things to note, I didn’t try just using cat on the original file to see if it would read the file differently than the was mysql was trying. But the important thing is I got the database restored in a relatively timely manner.

Hopefully, in the very near future, we will have moved to a new score system that doesn’t have almost a billion rows in it.

I have a couple of scripts that use the linux “mail” command to send results of cron jobs, backups, and such. The problem is I don’t want to setup more overhead than I need, nor do I want to setup non-secure services. So after asking the guys on the #uphpu channel, they recommended postfix. So for a Ubuntu 10.04 server, just run the following commands:


aptitude install postfix #when prompted, select Internet Site
aptitude install mailutils


Then bingo, the mail command should work. You can test it by doing:


mail justin@example.com


It will ask for any Cc addresses, and a subject. Then, type your message, and when you are done hit “Ctrl-D” for done.

Today as I was talking with a friend about Git and when or not we should switch a repository from SVN to Git, I thought up a pretty good analogy. So I thought I would share this to those who are looking to learn Git, and hopefully put them in the right mindset.

The Goal

Ultimately, when people talk about Source Control, they have the same concepts in mind. I want to be able to easily update my source control, collaborate with others, and have a history of changes that I can review and use to make other changes. So for our Analogy, these basic concepts, we’re going to call Getting from point A to point B.

SVN, The Car

With SVN (aka Subversion), I like to compare it to a car. You can hop in, drive around, and get from point A to point B with relative ease. You do need to learn how to drive a car, but after a little while, you get the hang of it. Eventually, you can get pretty good.

You get from point A to point B by combining things like accelerating forward and turning left and right. Maybe you perform even more complicated maneuvers like accelerating in reverse and turning the opposite direction to back up. Some people can even tackle crazier terrain and go off roads with their cars. All and all, it is a very versatile and well suited for the job. It isn’t very difficult to learn, and gets the job done.

Git, The Helicopter

Git, like SVN, can get someone from point A to point B. However, they approach the task with some fundamental differences. I like comparing it with going from driving a car to flying a helicopter. The helicopter is very versatile as well. It doesn’t require roads to travel along, and doesn’t get caught in traffic like cars can. They don’t have to deal with red lights or stop signs. To the normal driving looking up at them, it seems that they just “go.”

This is because you don’t just drive forwards and backwards, turning left and right. With a helicopter, you also go up and down. This adds an additional layer of complexity. You gain more mobility, but is crucial you understand the fundamental differences. For example, unlike a car when you accelerate and you go forwards, with a helicopter you punch the accelerator and you go up, not forwards. For anyone who has ever played a video game with realistic controls for a helicopter, it is hard. People have recorded tutorials teaching how to fly helicopters in games, and show the fundamental differences on how you approach flying. Even when flying planes in these games is similar to driving a car, many concepts are backwards when flying a helicopter.

Trying to Drive a Helicopter

Read More