Senator Lee,

My name is Justin Carmony, and I am the C.T.O. for Dating DNA, LLC, a company that makes iPhone Apps. This is my first time writing a senator, but I feel it is important to convey my thoughts on SOPA and PIPA.

We have struggled with people downloading our Apps from 3rd Party Websites and not paying for them. We are able to monitor and track these “cracked” apps when they connect to our services, and these “cracked apps” number in the hundreds of thousands over time. That is potentially millions of dollars in lost revenue.

However, I do NOT support SOPA, PIPA, or bills like it that give “blacklist” or “takedown” authority without due process.

Even though we struggle with piracy, these bills will do very little in preventing unauthorized distribution of copyrighted materials. However, they are extremely dangerous in terms of abuse against legitimate businesses. Even though the cost of piracy can be great, we already have tools and laws to help us minimize piracy. We work hard at providing a great service, and we’ve even successfully turned “pirates” into “legal, legitimate customers.” SOPA & PIPA will do very little in helping us stop real piracy, but will also expose us to “takedowns claims” without any oversight or due process.

There is already testimonies on legal records of Media Companies such as Warner Music and Universal Music group admitting to using DMCA to take down content they had no copyright ownership to. Their motives instead were to silence groups being critical of them. (see “A Business Person’s SOPA Primer” — http://avoidingagoatrodeo.com/2011/12/26/a-businesspersons-sopa-primer/)

Please, as my elected representative, pull your support of SOPA. Our small business would not survive a “takedown” if a competitor used SOPA with false claims. We depend on the internet to keep us in business. I hope you can see, and understand, that due process and legal oversight should still apply to the internet.

Thank you,

Justin Carmony
Utah Citizen

Related posts:

1. Comcast Limit to 250GB a month for Residential
2. Google Apps Dropping IE 6
3. Yahoo vs. Rhapsody – Why Yahoo Lost

PHP

Its been a fun year with the Utah PHP Usergroup. I look forward to it each month, and if you’re in Utah, you definitely should come. We’ve had a lot of interesting talks, such as running your app in different clouds, several MySQL Forks and NoSQL stuff.

I also had a great time going to my first PHP specific conference. While I’ve attended and spoken at other conferences, Tek 11 was a lot of fun. Met a lot of smart and fun people.

As for work, I do almost exclusively PHP work. We power the Dating DNA website and APIs using PHP. All of the Clipish Apps’ APIs are powered by PHP. Alienware Arena, The GeForce StarCraft II Pro/Am Tournament, Thermaltake eSports, and a few other gaming tournaments sites, all powered by PHP.

I’m hoping to blog more about PHP this year. While I use it all over the place, it seems I write more about all the other pieces that interface with and support PHP.

New Technologies

There have been a handful of new technologies gaining traction. We’ve been using Redis in production for quite awhile now, and finding more and more uses for it. I’ve played around quite a bite with node.js, and I see some potential for augmenting PHP with it. MongoDB, CouchDB, and a few other data stores have shown a lot of potential in situations that play to their strength. I’m still a major fan of MySQL, but it’s nice to good options for things that MySQL isn’t as strong at.

New Goals

I’ve been working out, eating well, and such for almost 8 weeks now as well as my wife. The net effect is I’ve lost 16 pounds, and my wife has lost quite a bit too. This was done over the holidays even. We still have some ways to go to get to our goals, but we’re hoping to be successful in 2012 and hit our target goals. We’re even starting to enjoy lifting, cardio, and eating right.

I’m also setting the goal to blog once a week this year. I’ve tried daily, and that is just too much, but once a week is manageable. Hopefully it’ll be relating the technologies I use, to help share the cool stuff we get to use in-house.

Upcoming in 2012

We’re launching a new product in 2012 for Dating DNA, hopefully within a month or so. I’m also hoping to go to a few conferences this year, and perhaps speak at a few of them. Also, on the fun side of things, we’ll be going on a cruise in April, and perhaps a trip to China in September.

I hope everyone had an excellent 2011, and and amazing 2012!

Related posts:

1. My 2009 Technology Recap
2. Attending PHP Tek 2011
3. Goal for 2011: Learn C

Well, this year for Christmas, Daniel wanted to help raise up $250 for… a water buffalo. Yup, thats right, a Water Buffalo.

What in the world for? To donate for a family in a rural. Why? From the Heifer International website:

A water buffalo can lead a hungry family out of poverty and give them a chance for a bright future filled with hope and free from hunger.

In poor Filipino villages, water buffalo from Heifer provide draft power for planting rice and potatoes, milk for protein and manure for fertilizer and fuel. A farmer can plant four times more rice with a buffalo than by hand.

Water buffalo haul heavy loads to the market, where the sale of extra produce brings in vital income for clothing, medicine and school. By renting their buffalo to neighbors, Heifer partner families can earn money for home improvements. And one day, those same neighbors might receive a water buffalo of their own as recipients pass on the gifts of animals and training.

Protein-rich milk, strength to till soil, manure to enrich the land … so many benefits. And, in turn, water buffalo are happy just to graze on coarse grasses and other plants not suitable for harvesting.

So Daniel wants to raise the funds for one!

I need your help.

Well actually I need your help to help me to give someone a Water Buffalo.

Think about it for a second. You can say you gave someone a Water Buffalo for Christmas. You know Water Buffaloes right?

Like this one:

While we were in Cambodia I noticed how much of the work is still done manually. Also in the few “Real China Adventures” we have had I have been amazed at what life is like outside the cities. Don’t get me wrong the people seem happy and content. But I am sure there are places they are not. And I think we should help them.

So I have started a Heifer International Team, Daniel’s Shoe Factory, and I would like you to help me raise $250 to give someone in South East Asia a water buffalo.

Want more information: Click Here

How to help me: Click Here and join or donate.

I am hoping to raise $250 by Jan 1st 2012.

So step up. Unless you yourself are in need of a water buffalo, then no worries.

So lets help Daniel buy a Water Buffalo for a family in need for this Christmas!

Related posts:

1. Bad PR, The Internet, and Censorship – Just Take It
2. Merry Christmas & Happy Holidays

Ultimately, I’m approaching it with a great deal of hesitation. Here is why:

When you start up a Video Game, much like watching a Movie, you will see some credits. In Modern Warfare 2, you’ll see Activision’s name, which is the Publisher, and Infinity Ward, which is the Game Development Studio. While the publisher can be involved in making (or ruining) a game, ultimately its up to the studio to produce a high quality game.

However, the Infinity Ward who made Modern Warfare 2 is not the same Infinity Ward who made Modern Warfare 3. Activision fired the lead game designer and a co-founder of Infinity Ward, and about half the staff resigned soon after that. They are now in a lawsuit against Activision who tried to withhold 60% of the staff’s bonuses.

As a Developer myself, there are two things I think are critical when working on a product, especially a video game: Design Direction and Talented Developers. From what I’ve read on the several news stories about staff leaving Infinity Ward, the majority were Leads and Seniors developers, animators, etc. So I would say the bulk of Infinity Ward’s talent left. Why does this matter?

Call of Duty, Call of Duty 2, Call of Duty 4: Modern Warfare, and Call of Duty: Modern Warfare 2 were are great games. Each added on top of the other, trying new things and bringing noticeable improvements to each installment. Also, while Infinity Ward grew, many of the core team members were the same throughout the series. So you had a great team, with a lot of talent, and experience working together to make great games.

Meanwhile, other Call of Duty games by other studios like Treyarch, released games like Call of Duty: World at War and Call of Duty: Black Ops. While these games weren’t bad games, they weren’t ground breaking either. They felt like minor updates instead of fresh, new material.

So now we have an Infinity Ward with half of their staff brand new, and trying to release a game on a very tight deadline. So I wouldn’t be surprised if there are technical issues at release, especially for the PC. From all the previews I’ve ready about Modern Warfare 3, it sounds like it really is just an updated version of MW2 with new maps and a new game play mode.

So while the videos and images of Call of Duty: Modern Warfare 3 look impressive, and the marketing behind it is great, I’m not holding my breath. But this will be the first time I won’t be purchasing Call of Duty: Modern Warfare 3 for PC, my personal choice for playing FPS games. I’ll likely pickup a copy of it for XBox 360 so I can play with friends, but here is my prediction:

Critics will give it high ratings (9+), actual players will give it okay ratings (7+), but Activision and Infinity Ward will have made their biggest mistake. They will have moved from being innovators to just producers. While they might sell millions of copies of this game, and they might pump out a few more titles, they open themselves to being displaced as the leader in the future. I’m hoping that future games will be able to bring in a breath of fresh air while Activision milks their franchise into the ground.

Who knows, maybe I’ll be proven wrong and Modern Warfare 3 will be great. I just have my doubts, especially when you have an exodus of senior talent from a company. I’ve seen that happen before, and rarely does the company produce the same quality as before.

Related posts:

1. Ghostbusters – The Video Game
2. I’m Boycotting Electronic Arts (EA) Games
3. EA – Lack of Quality & Out of Touch

Sonora Grill is one of the best Mexican restaurants in Utah. Not only is it great food, it is very unique. The best way I explain to those who have never been is that it is “gourmet mexican food.” Most people I tell this to seem rather confused by this, and think it is some kind of oxymoron. When they think of Mexican restaurants, they think of those we have around here.

Utah, and especially Ogden, has many, many Mexican restaurants. Here is a map of the google results for “mexican restaurant ogden”:

Most of these restaurants are what I would call Americanized Mexican food. It can be good, but it isn’t very authentic. I lived in Mexico for two years, eating with lots of different families and at many restaurants. So I know authentic mexican food when I have it. Also, there are several restaurants locally that cater to the Latino community. They are very authentic, typically in-expensive, and pretty tasty. But many times their menu is only in Spanish, and most of my friends don’t care for the really authentic food.

Then, there is the Sonora Grill. Their approach is to take the authentic ingredients, flavors, and dishes of Mexico and do amazing things with them. They have a professionally & classically trained chef who approaches mexican food like any other fine dining genre. They are serious about using the freshest ingredients and going above-and-beyond when it comes to preparing finely crafted dishes. They do all of this with a passion for the food, which really shows.

So when you go to the Sonora Grill, you get very flavorful food that each flavor is carefully balanced and blended in. It is like taking Mexican food to the next level. They will import a lot of their ingredients like spices, chiles, and cheese from Mexico so they can have the best flavors. They have some of the basics, like more generic Burritos & Quesadillas, but their best dishes are their house specialties. So while it might be tempting to go with the “safe burrito,” try one of their specialties. Here are my personal favorites:

Sweet Pork Tacos – Also know as Tacos al Pastor in Mexico, these tacos are made of pork that has been marinated for a very long time, traditionally a full day. These were my favorite tacos in Mexico. While Sonora Grill can’t cook them 100% authentic (I’ve written about Tacos al Pastor on exactly why), these tacos come as close as you can get here in the States.

Chile Rellenos – They have a few menu items with Chile Rellenos, and I will tell you these are the best I’ve ever found. A Chile Relleno is a poblano pepper that is stuffed with mexican cheeses and sometimes other things. They are then battered and fried for a nice crisp on the outside. I love them, and recommend everyone try them at least once.

Market Fresh Sea Food Special – I rarely get sea food unless I’m at a restaurant on the coast, but Sonora Grill does it right. It seems every week they have a new special with an in season fish that they can get and serve fresh. I’ve had some amazing dishes that were excellent that were specials. Also, they typically cost the same as a normal entre, instead of being twice as much like other restaurants. So for $15 – $18 it is a great deal.

Chicken Chalupas – Forget what you know at Taco Bell, this appetizer has three Chalupas which have a thick corn masa base and piled on top with shredded chicken, tomatillo salsa, onions, cilantro and sour cream. They are really yummy.

I haven’t tried all the dishes here yet, but some others I’ve heard good things about are the Alambres (aka Fajitas) and Ceviche (specially cooked sea food). I also saw they added Pozolé, which is a great mexican soup, so I need to try that as well.

Ogden’s Best Kept Secret

Now, I think this is Ogden’s best kept secret: Sonora Grill’s Regional Dinners. Four to five times a year, Courtney, the chef, will select a region of Mexico and prepare a five course meal (which technically would be 7-8 courses since many times the plates will have multiple things on them). It is $32 per person, and afterwards you feel like you should have paid way more than that. This is where their passion for Mexican cuisine really shows.

The chef will plan the dishes weeks in advance, importing in special ingredients for just this dinner. Then, the day of the Dinner (and sometimes even days before that), the chef will spend hours and hours preparing the food. It is an opportunity for the staff to say “If the restrictions on serving a busy restaurant weren’t an issue, and we had all the time and ingredients we could ever dream of, these are the dishes we’ll make.” I’ve been to two of these dinners now, and it was the absolute best mexican food I’ve ever had.

Tonight’s dinner was for the region of Oaxaca, which is know for their Mole (pronounced “mo lay”), which is basically a sauce. It is labor intensive to make Mole, and they made not one, but five different types for tonight’s dishes. Chef Courtney spent about seven hours, since 5 AM in the morning, preparing all the different Moles. Each one was carefully and artfully prepared. I’ve never had such well-prepared Mole before, it was extremely impressive and down right yummy.

I took a few pictures from my iPhone during the dinner. I didn’t get all the plates, but here were some of them. Now, I really enjoy Mexican food, but I couldn’t remember all the details of each dish, so I can’t remember all the ingredients. But I do know they all tasted great.

The appetizers, including a very delicious mole tamale & salad

A Chile Relleno stuffed with beef, pork, and black beans on a yummy tomato sauce.

Two different types of Mole with fresh corn tortillas.

A yummy fried platanos (bananas) desert.

The other dinner we attended was the Mexico City region, which had amazing Beef Tongue Tacos that tasted like Barbacoa Tacos, and Duck in an Ash mole. Both dinners were really good (though if I had to pick a favorite, I’d say the Mexico City dinner).

They are done with the dinners for this year, but will start them back up in the spring of 2012. They post the details on their website and Facebook.

So thank you, Steve, Courtney, and the rest of the gang at Sonora Grill. You can really tell that you all strive to make some serious mexican food that is great. I look forward to the Regional Dinners next year!

– Justin Carmony

The Sonora Grill Website

Related posts:

1. Tacos al Pastor – The Quest Begins
2. Great Eats: Grideli’s In Ogden

This morning started like most other sunday mornings. I was running around getting ready for church, taking care of a few things, and then I stepped outside to our neighborhood. The local Scouting program (which I had just been put in charge of) had done a simple fund-raiser: people could get for a small donation a flag placed in their yard on holidays by the scouts. Independence Day, Veterans Day, Labor Day, etc.

So today, when I stepped outside, I saw those flags in the yards of all our neighbors. It wasn’t one of the planned days to go out and setup the flags. Without my direction, the young men of our neighborhood thought today the flags should be placed out. So they dropped whatever they were doing before church, and went out and got the flags put in place. These few young men, who were the same age as I was when the towers fell and so many lost their lives, took their time to give a small remembrance for that day. It set the tone for me as I saw all those flags, and made me so appreciative of their service. It was a small reminder as well that we do have great young men and women growing up in our nation. They might drive me nuts sometimes, but I am grateful for their work this morning.

Related posts:

1. XKCD & 11th Grade
2. The LDS General Conference

No related posts.

The Problem

With Dating DNA, our goal was to display a compatibility score with every other user. This score is generated by taking two sets of answers to our 20 page survey, and running it through our algorithm. While it is super convenient for our users, this poses a problem. We wanted not only for people to be able to visit a profile and see a score, which is easy to generate a score on demand. We wanted our users to be able to browse other profiles sorted by their score with them. This requires us to pre-generate and store these scores, and then later query them.

So Ultimately, in theory, our “scores” data scaled at the following rate, with X as the number of registered users:

X^2 – X

That is at an exponential rate, which is practically impossible to scale at. The very first version of Dating DNA (before I took over the project) had about 1,500 users. The scores were stored in a single table. Every night, a “cron job” would run and get a list of every user, and loop through every possible iteration and re-generate each score. At 1,500 users that was 2,248,500 records. That is a lot for just 1,500 users. With our current user count, we would roughly have 359,999,400,000 score records. Thats 359 Billion records if you don’t want to count the commas.

This old system of daily cron jobs broke at about 2,000 users. We would have problems with the cron job taking over 48 hours to complete, and would end up with 3 scripts running at the say time. One for today, one for yesterday, and one for the day before that.

Smart Logic & Threading

We solved our first problem by using some common sense and smart logic. I won’t detail the lengthy measures we go through, but we can basically boil down our entire user base to an estimated top 5000 matches for any given user. If we have a heterosexual man named Joe, he doesn’t care about the hundreds of thousands of other heterosexual men who he scores a 2 or 3 with, but the heterosexual women he scores above a 6 with. So we don’t store the score for Frank, Jimmy, and Alan with Joe, but Sally, Rachael, and Tiffany.

The second part we solved was pre-generating scores for a user. After a User has reached a point in the survey where we have all the information we need to generate scores, and they are just filling in some miscellaneous, we put them in a queue. We then have a server process than is continuously running checking this queue, and spinning off multiple generation “threads” that crunch the data and store the score. We’ve spent a lot of time perfecting this system. Currently we typically can generate any given user’s matches in roughly 5 to 20 seconds, depending on how busy our website is.

Storing The Score in MySQL

The problem we now faced was the write through put of MySQL. Even through sharding and partitioning, we wanted to have a goal of sustaining 1,000 registrations per minute in a scalable and high performance manner. Which comes down to about 83,000 records per second that are either being inserted or updated. We then needed to be able to retrieve large volumes of scores just as fast.

I believe we could have bent MySQL to our will and got it to work, but it would be at a high cost of server power, and that cost wouldn’t scale well with our revenue stream. After we moved from the MySQL storage of the scores, I ran a query to see how many scores we were indeed storing. The final total was 950,363,992. Just 50 Million shy of one billion. It took 1 hour 49 min 38.27 sec to calculate that count. It is evidence that even though MySQL wasn’t the best choice for storing this data, it did it pretty well considering this single table was holding 90 times more data than any other table.

Picking Another Solution

In 2009 we started to throw around ideas for a new scoring system. I cannot stress enough when talking to others about “NoSQL” solutions the best solution for any given job is based on your data’s characteristics. User registration data needs to be treated differently than activity logging and basic stats. It might be okay to lose a few minutes of activity logging (depending on the app), but you definitely don’t want to be losing user accounts.

With Dating DNA’s scores, we had one great advantage. The data could be somewhat volatile, because we can always re-generate a set of matches for any given users. Of course, we didn’t want to lose all of it, because having to regenerate everyone’s scores is a major pain and extremely resource intensive. But if we lost a few minutes, anything lost could easily be regenerated. So when we started research for a solution, we were willing to sacrifice some persistance for performance. We wouldn’t be doing the same for our user registration data.

At first, I was contemplating building a completely in-house project to handle the data storage and retrieval of the scores. It would be a lot of work, and decided against. So I then thought about hacking together a custom solution with memcached. The idea would be a user’s set of matches would be stores in a variable in memcached. So the website and generation scripts would interface with memcached, and a server process would write inactive sets of scores (people who weren’t logged in) to a file on the disk. When they logged it and scores were being pulled and stored for that user, it would load the data from disk into memcached.

While the general concept was sound, the actual execution would be difficult. Memcached only supported strings for values, and we would still need some sort of database to manage which users had the data in memory vs disk, and the server process (probably just a php, python, or node.js script running continuously) would have to be running constantly, and if that broke things could get messy.

It boiled down too many points of failure and complexity. But it was a step in the right direction, so we kept looking for a better solution.

Redis, the Advanced Key-Store

I was talking with Joseph Scott, an employee of Automattic as a Bug Exorcist (not joking, his real title), and he mentioned I should look into Redis. He gave me a brief overview of what it was, and I shuffled that info back in my brain. I can’t remember how much longer it was before I checked out and compiled a copy of Redis, but I quickly discovered it could be a viable storage system for our scores.

So I spun up a virtual machine, installed Redis, and started to pound away at it. One of the things I wanted to test was the new feature (at the time) of Virtual Memory for redis. What this allowed was for Redis to make it’s own Virtual Memory on the server and store the lest recently used data to disk. When a Redis object was retrieved and it was in the VM, it would swap it back into memory, and swap older data to disk. This was just like the idea I had before with using an archaic system with memcached, but much more elegant.

The second thing was Redis’s support for multiple data types. So instead of having a json encoded string that held the scores and user ids for another user, we could have a hashtable or even a sorted set. It was a much more elegant solution than what we were thinking of before.

Some Limitations to Redis

However, there were a few limitations that we faced when implementing Redis. Redis works flawlessly with smaller sets of data. But the larger your data set, the more careful and aware you need to be about a few things that will kill your redis instance.

First off, with memcached, if you set a memory limit, it is a hard limit. I’ve never seen memcached use more memory than what you allow it to use. Redis, on the other hand, has soft memory limits. This is because of the way the Background Saves work so you can have persistant data. When a Background Save is issued, Redis will fork itself, and have one thread save a snapshot to the disk, and the other thread will continue to operate. In order to do this, Redis will exceed the standard memory limits, and your memory usage will go up much quicker. Once the background save is complete, it will close the forked backup process and sync the memory back to one data set. (I’m not a computer science guy, nor do I know a lot of lower level programming, so I might be describing this not 100% accurate, but this is how I envision it in my head).

Now, if you are not using the Virtual Memory, this isn’t that bad. However, when using Virtual Memory, the Background Saves take a great deal longer (from seconds to almost a minute or so), which isn’t too bad, but there is a catch. You will not be able to swap to the VM Disk until after the BG Save. This means that all Redis objects swaped to memory will stay in memory until after the BG Save is complete. This is because, like the memory from the fork, the Virtual Memory file is being used for the BG Save instead of the process handling requests.

So the one limitation we’ve encountered is we cannot run scripts that “query” large amounts of data from Redis. For example, it would be very simple to get a listing of users using the KEYS command, and then loop through the values using a HLEN to read the length. This will cause you to swap from and to the virtual memory a great deal. If a Background Save is occuring, Redis will not swap to disk until the BG Save is complete. This means if you have 10 GB of data in Virtual Memory, and you have a 1 GB instance of Redis, you will suddenly be reading gigs of data into Redis’s memory. If you are on a 2GB machine, you can easily use up all the memory on the server and then start using the System’s Swap.

Once you start using the Operating System’s Virtual Memory, it is game over. Your Redis instance’s performance will tank, and your Background Save might not finish, and you will need to restart redis.

There are some ways to give Redis a “Hard” limit on memory, but we opted to configure our servers in a way that doesn’t require this. If Redis hits the memory limit, it can start throwing write operation errors, which we didn’t want.

How We Configure Redis

After much trial and error and testing internally, we believe we found a sweet spot. We deploy a redis server, and spin up three redis instances on a different port each. Each is configured with 4 GB of Virtual Memory using 4096 size pages, and only 256 MB of “memory” using the vm-max-memory setting. While you would think this would mean a hard limit, it is a soft limit, and more of a goal “we’ll try to only use 256 MB of memory to store the data, if we’ll exceed it if needed.” Given our patterns of usage, Redis’s actually usage fluctuates (based on ps aux’s reporting) between 640 MB to 1100 MB of RAM, depending on if a background save is being executed or not. Redis is configured to perform background saves every 10 minutes, which take about a minute to perform.

So between the other admin services running on the box, and the three redis instances, we use just over 3GB RAM:

The amount of CPU required is extremely low, and almost 100% from writing the background saves to the disk:

So what do we get in return? We estimate each instance with 256 MB of data can hold roughly 2,000 active users. So with a single server we can support 6,000 users online at any given moment. We can store the scores for roughly 360,000 users on a single 4 GB box, which is about 1.8 Billion scores. Then, if we need more, we just provision another box, and our system will start assigning users to the instances on that machine.

Because of our ability to re-generate the scores, we decided to only convert the users who had logged in the past three months to the new system. If a user who hadn’t logged in since then logged in again, it would in the background assign them to a redis instance and rebuild their matches for them.

Using Redis with PHP

I recommend currently to use the PHP Redis client predis. I’ve used others like Rediska, but I prefer the straight forward approach of predis.

A high level view of how we use Redis with our PHP based website is we have a class called RedisManager than manages pretty much all the connections to Redis. It supports lazy connections (which is important to us, since we don’t want to have to connect to every instance of Redis we have), and I hope to open source it some day soon.

One key performance trick we’ve noticed is to use Pipelining to the redis instance. We don’t use this so much on the website, but our score generation “threads.” Writing thousands of scores one by one eats up a lot of network overhead versus sending them in batches (we send in batched of 500 or 1000, depending on the situation). Using pipelining is extremely fast for us, and I highly recommend it for any large batch of commands.

Using Redis Elsewhere in Dating DNA

Now, it might seem that we’ve put a lot of thought and effort into using Redis, and I want to make sure it was understood that Redis itself wasn’t difficult to use, but the volume of data were were dealing with. On Dating DNA, we also use Redis to power out in-app chat system (which I’ve written about previously), and it works great and is currently only using 146.70 MB of RAM, and serves thousands of requests per second.

The Future

I still have a lot of great ideas for Redis and Dating DNA, both with the score system, and outside of it. I plan on writing several reporting tools for Redis and hope to share them on github. I am currently working on the code and scripts for automatic deployment for Redis servers for Dating DNA, so we can scale easily with the push of a button. I’m excited for the work that is being done on Redis, and highly recommend it to anyone.

If there are details you would like to know more about, leave a comment and I’ll try to answer them. If you see me at tek11, feel free to ask me about this, and I can show you in detail how it works (internet permitting).

Related posts:

1. MySQL & PHP – SQL_CALC_FOUND_ROWS – An easy way to get the total number of rows regardless of LIMIT
2. MySQL, 40 Million Rows, MyISAM to InnoDB, 45 Minutes
3. Restoring Large MySQL Dump – 900 Million Rows

I’ve always wanted to attend a PHP conference for quite some time now, but it seemed for several years either it wasn’t in the budget, or I was just too busy. But I kept hearing great things about Tek, and it’s organizers, so when the Call for Papers opened up for Tek 11 this year, I submitted. Unfortunately I wasn’t able to make it on the presenter list, so I immediately registered and booked my flights so I wouldn’t find an excuse later on to not go.

For those in the Utah PHP Users Group who are not able to come out with me, and other friends of mine, let me know if there is anything interesting in the schedule, and I’ll see if I can attend and take some notes for you.

Related posts:

1. Goal for 2011: Learn C
2. Speaking: Utah Open Source Conference 2009
3. UTOSC – Day One

The key is cooking it on the rotisserie spit. It allows for a crispy outside and juicy inside. But as listed on Grub Street Chicago, there is a serious downside to this method:

If the restaurant isn’t busy, your stomach will be in a tizzy: that spit meat can go bad pretty quickly, and if there’s not a lot of turnover at the taqueria, you may turn over (this has happened to us before and it’s not pretty).

It takes at least 24 hours to prepare and marinate the meat, some people taking 48 or even 72 hours to prepare the meat. That means many people will prepare a very large amount of meat at a time to help save on the work. When the meat is cooking on the spit, the outside cooks much faster than the inside. So you start cutting and serving from the outside, letting the inside cook more. But if the restaurant is running slow, the inside can go bad before it is cooked.

The Food Safety guidelines state that you can’t start serving the meat until the entire piece of meat is finished cooking. So when Sonora Grill tried to do Tacos al Pastor, they would have to cook the entire piece of meat on the spit, and either keep it up at temperature, or cool it down and re-heat it later. This would cause the meat to dry out, and just wasn’t very good. So they changed their recipe and method of cooking. So they still taste great, but it isn’t the same.

Now, I won’t be preparing meat for hundreds of people, just for myself and friends. So I can plan, prepare, and cook the meat in a safe manner using the spit. So I thought “alright, I can do this myself, and have my favorite tacos.”

So I started researching some recipes. I found several recipes, and I’ve posted them at the end of this post (incase they go down, since the interent isn’t forever).

The second problem I face is finding a vertical rotisserie. After googling around I found out it is called a vertical broiler, and are also used in making traditional gyros. But the decent ones run about $1,000 bucks. I’m not ready to drop that kind of money on a spur of the moment feeling. So, here is my plan:

First, try to get the marinade down. I’ve been told this is the hardest part of making these tacos. Then, I can cook the meat in the oven and finish it on a pan. So the crispy/juicy factor won’t be quite right, but it will be pretty decent.

Second, there are some smaller smaller vertical rotisserie cookers for about $60-70 bucks. I figure I can try out one of these and see if it can work for making smaller batches.

Then, finally, if I can regularly make good tacos al pastor that are just lacking the spit, and if I am still committed, I’ll buy a vertical broiler. Then during the warm summer months I’ll be cooking my favorite tacos regularly!

Wish me luck!

References

Recipe #1

Link: http://www.fromaway.com/cooking/tacos-al-pastor

So, how can you create this dish at home, without building your own vertical rotisserie, stacking raw marinated pork into a big pile, or hiring your own swarthy, mustachioed taquero to make your tacos? You can’t. But, you can come awfully close. Here’s how to prepare tacos al pastor at home.

Tacos al Pastor

Ingredients:

2 pound boneless pork loin
2 Guajillo chiles
1 Ancho chile
1 teaspoon cinnamon
1 teaspoon oregano
1 teaspoon crushed whole black pepper
1 teaspoon ground sea salt
2 finely chopped white onions
6 cloves of garlic, crushed
2 teaspoons of achiote paste
1 cup of pineapple juice
1 white onion, diced
1 handful fresh cilantro, chopped
Corn tortillas
Lime wedges
Method:

Slice pork as thinly as possible, diagonally against the grain, and set aside. Rehydrate dried chilies by covering in hot water. Let soak until soft, drain water, and remove tops, seeds, and membranes from each chili. Combine with cinnamon, oregano, black pepper, sea salt, garlic, achiote paste, and pineapple juice in food processor, and pulse until thoroughly combined. Transfer pork to gallon freezer bag, and pour marinade on top. Mix to make sure all pieces are coated, and marinate, refrigerated, for 24 hours. Preheat oven to 350 degrees, and remove meat from bag. Arrange into “roll” or “loaf” shape on a baking sheet, cover with additional marinade, and cook for about an hour. The pork will still be pink, but don’t worry; finish it until cooked to your liking in a pan on the stovetop. Place a few slices of pork on a corn tortilla, and top with diced onions, cilantro, and a healthy squeeze of lime.

Recipe #2

Link: http://www.batista.org/pastor.html

Introduction:

Tacos al Pastor is perhaps the best taco you could ever try. The problem is that good tacos are hard to find the United States since the recipe is such a secret.
In Mexico city there are taquerias dedicated mostly or even exclusively to tacos al Pastor. Tacos al Pastor are made from pork meat that has been marinated in a secret recipe and then cooked in a rotisserie with pineapple on top. The tacos should be very small (almost like 2 bites size) and garnished with cilantro (coriander), chopped onion and the pineapple. Add your favorite hot sauce.

This space was empty for a long time and I apologize for it. The recipe I started with was not bad, but it was not as good as I wanted it to be. I sent this recipe to Mexico City to be reviewed but nobody would talk much about it. Then in my last visit I wanted to have someone review it but I encountered the same problem.

The fact is that everyone seems to have a different list of ingredients for the marinade and nobody wants to share it. It is after all their business to keep their recipe different and better from others as well as secret. Restaurants are sold and bought just for the value of their recipe. You get the idea? Restaurant owners have emailed me an said that there is no way the would disclose their recipe. It is a big business.
Further down in the page you will find a recipe for you to try, but note that is not the real thing. The good news is that we can now sell you the real stuff in bottles, read below.

Below is the recipe I started with, if you experiment with it and improve on it, please tell me about it. Other ingredients mentioned to me by people are:

Achiote and red vinegar. Achiote can provide some color (red)
Orange juice. Adds to the flavor.
Coca-cola (Coke). Another recipe we make is Carne con Coca, which is pork meat cooked in Coke. Someday I’ll write it for you. It makes things taste very good and ads some sweetnes.

This recipe serves 4.

Ingredients:

10 chiles Pasilla
10 chiles Guajillo
1/2 garlic bulb
1/4 litter White Vinegar
1/4 tsp. Cumin
5 cloves
salt
pineapple (fresh or canned)
2 lbs thin pork meat
1 onion
Fresh cilantro (coriander)
1 Beer
2 Limes
Pre-cooking:

Cut the pork meat in thin stakes or slices if necessary. Normally each stake would rest on top of each other while marinating and cooking.
The following is the recipe for the marinade, this is a lot so you won’t have to make it very often.

Take the seeds out of the chiles, cut them in little pieces and mash them together with the garlic, cloves, and cumin, avoid touching the chiles and vinegar with your bare hands if possible. The vinegar and chiles can “cook” your hands, trust me. A food processor would help here.
Boil the ingredients from the above step in the vinegar until it makes some sort of a heavy paste. Making sure that it won’t burn, so mix it often.
Once fully cooked drink the beer while you let the marinade cool down.

Apply the paste to the meat putting one steak on top of the other. At a real taqueria they would form a top that eventually goes into the rotisserie. Since we do not have the rotisserie you simply pile the meat together and store in the fridge for at least 6 hours or overnight.

Cooking:

In a taqueria they would roast the “top” of meat with pineapple on top of it so that the juice gives the meat some of its flavor. The meat would be rotated constantly as the cook cuts very small slices of meat and pineapple to be served in a taco.
Chop the cilantro and onion.
Without a rotisserie, our only choice is to cut the pork in small bits so that it can be eaten easily in a taco.
Use a frying pan and cook the meat with small pieces of pineapple making sure the pork is fully cooked and just about to burn in some cases (well roasted).
Cut the limes in quarters.
Serve the tacos with chopped cilantro, onion, and the limes.
Tips:

I definitely need some (tips) this time. Or perhaps you’d like to send me a rotisserie.
Squeeze a lit bit of lime juice in the taco before you eat it and add habanero salsa.
If you only have large tortillas, cut them using a small plate of the size you want and a knife.
For microwave, add 15 seconds per tortilla and heat them between paper napkins.
Now for a totally different, radical recipe we have this, Courtesy of Jayson Mcpeak:

My name is Jayson Mcpeak, I am a certified chef (c.c.) through the ACF. I would like to share with you my recipe for Tacos Al Pastor.

5 lb boneless pork shoulder
1 lbs chorizo sausage (remove from packaging into a mixing bowl, must come to room temp)
1 medium size whole pineapple skin removed (canned may be used, fresh is preffered)
1/4 cup dry menudo seasoning found in most ethnic stores
3 tsp kosher salt
1 lime juiced

You will need a spit or some type of rotating cooking device for a more authintic recipe. If using an upright spit: Thinly slice the boneless pork shoulder roughly 1/4 inch thick, lightly brush on lime juice to the meat, starting with the largest slice on the bottom while working your way up, mix the dry ingredients togeter in a shallow pan, lightly dust the bottom piece first, then spear the slice onto the spit. Now generously hand smear the chorizo to the top of the slice, besure to reserve enough to finish the job. Repeat all steps until done. Now top the fully loaded spit with the pineapple, begin cooking. Besure to collect all drippings and rebaste using these very flavorfull tidbits. Depending on your heat source cooking time may vary. However you may slice the outer portions as soon as they reach a proper temp of 165* For a horizontal spit: follow all prep methods above, except for the meat layout and pineapple setup. Starting with the smallest slice getting bigger in the middle and smaller at the end. Make sure to use a foil drip catch under the meat. Crushed pineapple may be used here alternating between slices, use lightly.

No related posts.