This in the “In-Depth” part of my presentation on “Streamling Web Development with SVN & Trac.” I can’t possibly cover all the aspects of this topic. I like to post a “in-depth” article right after my presentation with links, tutorials, and all my resources used for the presentation. This allows for me to spend more time on questions & answers. I’d appreciate any feedback on the presentation, and if you have any questions feel free to ask. This article is an extension to the presentation, so I suggest listening to the presentation first if it seems like there are any pieces missing.

Presentation

Here is the Scribd version of my Presentation. Since it didn’t like some of the things I used in the presentation, I uploaded a pdf version as well.

UPHPU - Streamlined Web Development

Overview

As a web developer, I like to spend the most time possible on adding improvements, making changes, and ensuring quality. What I don’t like spending time on is repetitive tasks trying to manage the project. Over the last few years, I’ve been trying to focus on “automating” the processes of web development, letting programs, scripts, and othe technologies keep me focused on real development. I’ve pooled together all the information I’ve found on the subject, and hopefully I can help anyone searching for more information.

The Development Cycle

There is one constant in any type of software development: change. Every project is going to have new features, bug fixes, feature changes, etc. Our goal is to help create, verify, and deploy these changes as quickly as possible, while ensuring quality. There are five basic steps, in my opinion, to this cycle:

• Code Changes
• Test / QA Changes
• Stage Changes
• Deploy Changes (Push Changes Live)
• Feedback

Code Changes

This is where developers get down and get their hands dirty. To be honest, this is my favorite part of the process, as I would think most developers would agree. However, if we only focus on this aspect, our products will suffer from bugs and errors.

I use SVN to manage all of my code and my changes. “Subversion (SVN) … is used to maintain current and historical versions of files such as source code, web pages, and documentation.” (Wikipedia) Basically, it is a great tool to manage your source code, whether it is PHP, HTML, CSS, JS, Images, etc. If you are unfamiliar with SVN and source control concepts, I highly recommend visiting the online SVN Book. The chapters on Fundimental Concepts and Basic Usage are great primers for SVN. To give a tutorial on SVN is outside the scope of this article, so I would suggest reading & experimenting with SVN before trying to continue this article.

Important steps to working with SVN is that you must be able to run your website code locally. Some beginners (myself included when I started), use SVN like a glorified FTP, using it to upload their files to the server, and “commit” hundreds of tweaks and changes in a short period. Soon you will have thousands of revisions that aren’t meaningful. Running your code locally, by either installing your enviroment or using a local server (physical or virtual), you can make a single “change” that you’ve tested & ensured before “commiting” it to your SVN repository. This way you review your SVN repository history, each revision is meaningful. With good commit comments, you will see “Added extra field for middle name on registration” in the history, and all code changes needed for that one “feature change” will be in a single revision entry.

There are many tutorials for running your code locally. If you want to install Apache, MySQL, and PHP on your machine, I suggest using XAMPP. It works with Windows, Linux, Mac OSX, and others. I have a video tutorial for installing XAMPP on windows.

If you want to investigate using Virtual Machines for a local test environment, I would suggest checking out Sun’s VirtualBox. It is Open Source (GPL) and I like to use it when I don’t want to pay for the expensive VMWare solutions. There is a great article on using a Virtual Ubuntu Server as a local development enviroment. This tutorial is based on using VirtualBox to virtualize Ubuntu within Ubuntu. Cal Evans takes the same approach, but you can read how he tried doing it using a windows environment.

Testing / QA Changes

Once you’ve coded and tested your changes locally, you commit your changes to the SVN repository. Using a post-commit hook with SVN, we update

General Setup

Here is a diagram showing the basic setup for streamlining a project with LAMP, SVN, and Trac.

Related posts:

1. Speaking: Utah PHP Usergroup - Streamlined Web Development Next Thursday, November 20th, at 7 PM Mountain Time (9PM...
2. Web Development 10-Years Ago & Now This week's blogging challenge was our computer software now and...
3. Zend Studio vs PHP Development Tools I've been using Zend Studio (ZS) for the last...

Related posts brought to you by Yet Another Related Posts Plugin.

I’m very pleased to announce the topic for November’s UPHPU Meeting : Streamlined Web Development using a VCS (Version Control System) and Trac.

Starting the meeting off will be Code Greene’s Kevin Carter. Kevin, who has used Git in a production environment for several months now, will be presenting a primer on how to get started with Git. Additionally, he’ll cover the advantages of distributed source code control and possibly some topics on using git in an svn eniroment. His blog can be found at http://dexterthedragon.com.

Following Kevin will be are main speaker, Justin Carmony. His presentation will focus on the unique challenges with developing, deploying, & managing Website development. Justin Carmony has put together his trips, tricks, and suggestions for streamlining the development cycle. With a focus on automation, using SVN and Trac allows developers to focus on development rather than redundant tasks. Whether a small project with a single developer, or a large project with a big team, SVN and Trac in a LAMP environment can help any developer. Time permitting he will show some of his own live examples of using SVN & Trac. If you have a specific topics or questions you would like addressed in his presentation, please feel free to let him know on his blog.

Justin Carmony has worked in web development professionally for the last four years. With emphasis on PHP, .NET, and Web Services, he has worked on projects ranging from simple websites to complex communications between thousands of remote systems. He currently is working as an independent contractor & private consultant. If you have any questions, comments, or curiosities you can contact him (justin AT justincarmony DOT com) or read about his latest endeavors at his blog (www.justincarmony.com/blog).

We will have streaming as usual and slides will be posted early.

If you’re in the Salt Lake City area, and would like to attend, it will be at the following address: 12393 Gateway Park Place Suite 600, Draper, Utah.

If you’re not able to make it, but would still like to listen in & participate via IRC, check out the UPHPU’s live meeting page. If you have any specifics on what you would like to hear, or have any questions that you would like me to answer in my presentation, leave a comment below.

Related posts:

1. Streamlined Web Development: In-Depth Note: This is a work in progress. I will finish...
2. Zend Studio vs PHP Development Tools I've been using Zend Studio (ZS) for the last...
3. Web Development 10-Years Ago & Now This week's blogging challenge was our computer software now and...

Related posts brought to you by Yet Another Related Posts Plugin.

Related posts:

1. It’s Time for Campaignin’ - Elections Video I couldn't help resist posting this video I found over...
2. PHP Video Tutorial - Getting Started - Installing XAMPP I've decided to give this a try and make...

Related posts brought to you by Yet Another Related Posts Plugin.

This solution will return 1 row if it exists, and 0 rows if it doesn't. Here is a PHP example using this:

Related posts:

1. MySQL & PHP - SQL_CALC_FOUND_ROWS - An easy way to get the total number of rows regardless of LIMIT Here is a little trick I found awhile back. I...
2. MS SQL 2005 (T-SQL) Row Count for Each Table At work I needed a solution to give me a...
3. ASP .NET, LINQ, GridViews, and GUID Errors I found a simple problem today. Here is the deal,...

Related posts brought to you by Yet Another Related Posts Plugin.

If EA wants to punish its consumers so heavily for posting on their forums what might be inappropriate or out of line, that's fine, just ban their forum account. But if you ban their entire online gaming connections, that is going too far. This is my last straw with EA, and until I see significant changes in their attitude towards consumers, they will not receive a single penny from me. My money as a consumer is an award for good companies, and I don't believe in awarding bad companies.

Update: it appears EA has retracted it's statement to clarify that bans are handled individually, not globally. However, until I see better quality from EA, I'm not buying their games, period.

Related posts:

1. EA - Lack of Quality & Out of Touch An article posted on Slashdot just set me off today....
2. Starting Book Reviews Alright, to help me continue the learning process in my...

Related posts brought to you by Yet Another Related Posts Plugin.

The lawsuit even states, "Although the FREESPIRE mark was recently assigned as part of an asset purchase, [Robertson] expressly reserved all rights in and to all claims against Carmony arising from the freespire.com domain name and website." Now, why would Robertson want to carve out that one thing to hang on to? He's no longer involved with Freespire, so why does he care? Is he desperate for money? With Robertson's companies struggling and laying employees off, does anyone really believe this is the most important thing Robertson should be concerning himself with? His motives are pretty transparent--to suppress freedom of expression that is critical of Robertson.

This is a nuisance lawsuit, intended to silence my freedom of speech. I will defend my right to free speech by continuing to point freespire.com to my blog and keep reporting on Robertson's disturbing actions.

Enough is enough Michael. You once stated that the "The Record Labels Want My Minivan" playing the victim of a personal lawsuit. Seems rather hypocritical now that you're suing my father. What about my stocks in Linspire? I was hoping to use them to help pay for school or a down payment on my house. But of course my house, car, or personal money doesn't matter to you Michael, since you drove Linspire into the ground and sold it's assets off in a single year. I'm glad to see you valued the shareholders to much. Seems its only a crime when its "your minivan" at risk.

If you want to read up more on Michael Robertson & the Linspire/Freespire issue, you can checkout my father's blog.

Related posts:

1. Funny iPhone Comic I haven't written in a few days, and I've been...
2. About Hi, I'm Justin Carmony. I am a professional web designer...

Related posts brought to you by Yet Another Related Posts Plugin.

Because most PHP applications rely heavily on the ability to read and store information in a database, many times designing how you interact with that database is critical. The reason being is if you design something, but don't foresee a requirement or necessity, it can be extremely time consuming to go back and change. These "oversights" can be costly in time, money, and resources.

I've thought of some of the biggest oversights I've had when working with PHP and MySQL and put them in a list. This is my personal list, and I'm sure some people can think of some other oversights that belong on the list as well. This list is just for PHP & MySQL, not PHP and any database. I know many people like using software like Doctrine to allow switching between different database types. That is beyond the scope of this article.

Many of the examples in this article are very simple and are designed for showing techniques of better programming, not for actual use. Straight copy and paste most likely will not create sound, robust solutions. Search the web for a solid MySQL PHP Class to handle these issues for you, or you can make your own.

Now that I've thrown out my disclaimers and thoughts, lets venture on to by list:

Oversight #1 - No Data Access Layer

I first heard the term Data Access Layer was when I was reading about ASP .NET. Basically a Data Access Layer (DAL) is a layer of code that you access data through. Sounds simple right? Many new php developers don't have any type of DAL. Here is an example of not having a Data Access Layer:

Now, that wasn't so bad, was it? Lots of developers just use the mysql commands. I have several projects I worked on my early days with PHP that fall victim to this oversight. One in particular now has grown out of hand so bad that we've decided to start from scratch for a whole new version. Why? Lets say you have 3000+ php files, and your boss says "Hrm, we're seeing some problems with performance. Can you display at the bottom of each page the # of queries you use on that page?" If you coded your entire project like the example above, you would be totally screwed. You would have to find each and every mysql_query() and add some counter at the end. It would be a managing Nightmare. So how cold you solve this problem?

First of all, you've satisfied the boss's wish of keeping a counter for the total # of queries. The trick is to have all queries go through the function "Query($sql)." This will ensure the counter stays current. You can also add code to catch errors, display error messages, log queries, etc. Having a class to wrap around your database connection, queries, etc. can make your life easier, and more simple.

Oversight #2 - Design for Only One Database Connection

I have a project that I've taken over right now that is suffering from being programmed into handling only one database connection. "But why would you ever need to connect to more than one database?" Well, I like to have a database for important information: invoices, users, payments, etc. Stuff that is updated every day & information that the end user manipulates. Then I like to have another database for logging statistical information: user log-ins, page hits, visitor counts. The problem I've ran into lately is I want to have hourly backups for my important information, but only daily backups for simple statistical information. It would be much easier if they were separated in different databases, but many times they are in the same. I don't want to take the extra time and space to backup the statistical information.

Many people who run into this problem use a Singleton class to handle their Data Access Layer. A singleton class is a class that can only have one instance at a time in execution. This can be suer easy and quick to use in your code, but may lock you into a situation where your code only handles a single database connection:

Now your boss comes in and asks you to connect to another database to get a list of customers. You think "uh oh, I can't really do that with my design, now can I?" The solution I like to use is having Database Connection classes, and then a Database Connection Manager. Imagine I have the MySqlDAL class from the previous example in oversight #1. I would create singleton class called MySqlManager. It's job would be to handle the creation & management of all it's classes. Here is an example:

Now you have a MySQL Connection Manager class that can handle the creation, management, and disconnection of your SQL connections. Your boss will be happy to hear you can connect to another database with easy!

Oversight #3 - No Developer Logging

There are many times where I wish I could what was going on behind the scenes. It is very easy to add Query Logging to your Data Access Layer if you choose. I won't give an example how to do so, but there are some several very important parts to it. First, you need to be able to enable & disable query logging. Second, you should log whether it was successful or not. Also, if you play around with PHP's back tracing abilities you can trace where each query was made. I'll make a blog post about this another day.

Oversight #4 - Queries Written In Procedural Processes

I absolutely *loathe* when I get into a situation where queries are being written all over the place. If there is one constant in any software development, it is that things will always change. When queries are being placed in procedural code, it makes management of database changes a total hassle. "But what about find and replace?" That can help you find places where you need to change, but you still can miss other places in your code, which can lead to bugs, headaches, and self-combustion. Here is an example:

Not seeing the big issue here? Lets say you copy and paste that line of code '$links = $dal->FetchArray("SELECT * FROM user_links ORDER BY id");' on twenty different pages. Then your boss comes and says "Hey, you know how the user_links table has a title? Well every where we display the title, I'd like to have it show which # of link it is, so example: Link #1 - My Blog, Link #2 - My Facebook., Oh, and can we order them by date_added? Thanks!"

Have fun trying to find every place that query is executed. Don't be surprised when only 80% of the queries get changed, and the other 20% become bug tickets with your boss wondering why you didn't find them before. This is an example of writing queries all over the place when you need them, instead of organizing them and abstracting them by Classes and Functions. Here is an example of abstracting the query:

Now your boss will give you a raise when you can make changes this quickly. While this example may seem simple, there are many times where I've had projects that queries have required major changes and re-writes. Bugs quickly service if queries that are the same, or similar, are copied and re-copied all over the place. This leads to a giant mess very quickly. So remove queries from the procedural flow of things by abstracting them in classes and functions.

Oversight #5 - No Separation of Reads & Writes

This oversight has stung me for the first time just a few weeks ago. A project of mine had reached critical mass with its size, and out grew its beefy MySQL dedicated server. I started investigating doing a Master-Slave replication system, but every article said I needed to route SELECT statements to the slave servers, and just about every else to the Master (like Insert, Update, Delete, etc). This project didn't have any Data Access Layer at all, and 2+ years of development on it. There is no way I could go back and re-write all the queries on budget and timeline, especially without causing bugs. especially if Insert/Update/Delete commands on accidently got routed to a slave server.

My suggestion is funneling all queries into two functions, Select() and Execute(). Then, when you create a MySqlDAL instance, it would actually have a $this->master_connection; and $this->slave_connection; It wouldn't connection on creation, but on the first instance a query was executed. This is done to release stress on the Master server, the whole purpose of Master-Slave replication setups. Here is a mock class that would handle this: