Planet Kodiak

Show A Twitter Status Update On A Page Using Javascript

Cory Fischer — Sun, 22 Aug 2010 07:09:13 +0000

So you want to show your Twitter status on your website? Don’t want to deal with server-side scripts? Luckily, Twitter gives you an excellent API to call that makes this pretty easy.

Because we live in crazy times, lets start at the end. The Twitter call will be made after all your code. It returns a JSON object to your code. Don’t worry if you don’t know JSON, it’s a pretty simple object and no real world knowledge of JSON is necessary to get your code up and running. The API is as follows (note: change the “planetkodiak” URL to your own user ID).

The important parts here are the user name and the count. The username above is planetkodiak.json. If your twitter username is “bob” then use bob.json. The count is important in that it will return your most recent posts. In this example we’re only returning the most recent.

Remember, this should come after the rest of your code as it’s expecting a certain function.

Now that we’ve started at the end, lets continue to the beginning (then we’ll finish in the middle because I can’t make it TOO easy for you). In the you can add a javascript with inline code or an external file; your choice. For the example below I’ll give you the code for the inline script.

function twitterCallback2(twitters) {
	var statusHTML = [];
	for (var i=0; i<twitters.length; i++){
		var username = twitters[i].user.screen_name;
		var status = twitters[i].text.replace(/((https?|s?ftp|ssh)\:\/\/[^"\s\<\>]*[^.,;'">\:\s\<\>\)\]\!])/g, function(url) {
			return '+url+'">'+url+'';
		}).replace(/\B@([_a-z0-9]+)/ig, function(reply) {
			return  reply.charAt(0)+'+reply.substring(1)+'">'+reply.substring(1)+'';
		});
		statusHTML.push(''+status+'');
	}
	document.getElementById('twitterStatus').innerHTML = statusHTML.join('');
}

The name of this function is important, as the JSON is expecting it. It is also ready for multiple statuses if you choose to change the amount of twitter statuses from one to many.

The idea of this function is to build up a variable with the Twitter content and replace the inner HTML of an element with the ID of “twitterStatus” with the contents of that variable. If you’ll notice the only thing we’re putting in the HTML is this:

statusHTML.push(''+status+'');

The only variable we’re using is the status itself. The status is set a few lines above with this code:

var status = twitters[i].text;

It also includes some replacement text for URL’s. The “twitters” variable is the JSON variable that is sent to this function. It is a multi-dimensional array with a number of key values; “text” is the one with the value of the Twitter status itself, therefore the code is: twitters[i].text.

If you wanted to include your Twitter ID, you would do:

var id = twitters[i].id;

If you wanted to include your Twitter username, you would do:

var username = twitters[i].user.screen_name;

Here is a full list of key/value pairs you can use from one of my recent Twitter updates:

	{
	"in_reply_to_status_id":null,
	"contributors":null,
	"geo":null,
	"retweet_count":null,
	"truncated":false,
	"created_at":"Fri Aug 13 07:29:29 +0000 2010",
	"coordinates":null,
	"source":"web","place":null,
	"retweeted":false,
	"favorited":false,
	"user":
		{
			"contributors_enabled":false,
			"description":"Cory Fischer - Software developer at PlanetKodiak.com",
			"lang":"en","geo_enabled":false,
			"profile_use_background_image":true,
			"time_zone":"Alaska",
			"created_at":"Sat Jan 16 02:33:15 +0000 2010",
			"location":"Washington State",
			"notifications":null,
			"profile_background_color":"9ae4e8",
			"verified":false,
			"profile_background_image_url":"http://s.twimg.com/a/1281662294/images/themes/theme1/bg.png",
			"follow_request_sent":null,
			"following":null,
			"profile_text_color":"000000",
			"profile_background_tile":false,
			"profile_link_color":"0000ff",
			"followers_count":5,
			"protected":false,
			"profile_image_url":"http://a0.twimg.com/profile_images/864422532/logo_300x300_normal.png",
			"listed_count":0,
			"friends_count":7,
			"profile_sidebar_fill_color":"e0ff92",
			"name":"Planet Kodiak",
			"statuses_count":11,
			"screen_name":"planetkodiak",
			"id":105344133,
			"show_all_inline_media":false,
			"utc_offset":-32400,
			"favourites_count":0,
			"profile_sidebar_border_color":"87bc44",
			"url":"http://www.planetkodiak.com"
		},
	"in_reply_to_user_id":null,
	"id":21045178144,
	"in_reply_to_screen_name":null,
	"text":"Ironically, the W3 (the site that heads up the web accessibility initiative) has a web accessibility issue http://bit.ly/cK7Ail"
	}

Now, once you’ve got your javascript function configured the way you want, you need to create an HTML element with the ID of “twitterStatus”.

The order of events is to have your twitterCallback2() function in your header, have your twitterStatus element in your body, then link to the Twitter API last. In it’s simplest form, your code should look like this:



<script type="text/javascript">
function twitterCallback2(twitters) {
	var statusHTML = [];
	for (var i=0; i<twitters.length; i++){
		var username = twitters[i].user.screen_name;
		var status = twitters[i].text.replace(/((https?|s?ftp|ssh)\:\/\/[^"\s\<\>]*[^.,;'">\:\s\<\>\)\]\!])/g, function(url) {
			return '+url+'">'+url+'';
		}).replace(/\B@([_a-z0-9]+)/ig, function(reply) {
			return  reply.charAt(0)+'+reply.substring(1)+'">'+reply.substring(1)+'';
		});
		statusHTML.push(''+status+'');
	}
	document.getElementById('twitterStatus').innerHTML = statusHTML.join('');
}
script>
Twitter

Practices Developers Should Avoid

Cory Fischer — Sun, 18 Jul 2010 06:04:57 +0000

Every developer wants to create the best site for their users. Though often times they will add functionality, use formats, or design in a way that matches what they perceive to be a “standard”. Unfortunately, just because something is used often doesn’t mean it’s a good thing. In this article I’ll discuss some common practices that you may have done without even considering other options.

Complicated comments

We often times think that our site is the center of the internet universe, but unless you’re Facebook or Google you may want to rethink how your users interact with your site. Unless you’re using a common interface such as DISQUS for your comments, you should leave your site’s comments open and not force them to register for an account. It’s unlikely someone will want to go through the registration process (especially if they have to wait for a confirmation email) since people don’t generally like to have dozens or even hundreds of accounts littered across the internet.

There are exceptions to the rule, and certainly large corporate websites are a part of that exception (e.g. Facebook & Google), however unless you specifically want to cause your users to go through an arduous process, you should let them comment freely or use a cross-site commenting system such as DISQUS.

SEO over usability

SEO is an important aspect of any site’s push to get more visitors. The problem comes when you program to SEO so much that it causes problems with your users ability to complete their intended tasks. If you’re modifying your navigation or presentation layer to accommodate SEO, you’re doing it wrong.

Truth-be-told about SEO: content is king! You want good search results you need to worry about two things.
1) Get links from other, important sites. If Google (or Yahoo, Bing, and others for that matter) see that you’re getting links to your site from other important sites, your search rankings will skyrocket to the top. That’s 90% of the battle.
2) Write good content. You want your site to be found with the proper search terms. To do this, write good content. Between this and getting important links, that’s 99% of the battle. Everything else, including proper

tags and presentation layer changes, is just fluff. They can help, but only marginally and usually at the expense of your users. The same ones you were so desperate to get to your site through a search engine I might add.

As far as presentation layer is concerned, you may be asking yourself, “does Flash really inhibit my SEO attempts?”. The answer is… maybe! If you’re not using Flash properly, you can. Google has a harder time reading flash files, and only wants to index text. if you don’t have a mechanism to accommodate search engines suddenly content, which is king, becomes second fiddle. There are many ways to fix this, but the best and easiest way is to hide your content in a
tag that has javascript replace the contents with your flash file. This is often confused as Cloaking. Cloaking refers to when you give the search engine different content then what you present to your users. If the content is the same, you’re golden. However, if you present “Storing MySQL Data in Hierarchical format” to your users, and give Google search terms like “Brittany Spears, Lebron James, and Lindsey Lohan” that’s when you get blacklisted.

Remember to use rich media responsibly, that content is king, and usability trumps SEO.

Video Player Controls

This is a simple one that many people often do because other people do it. Just because everyone does it doesn’t make it a standard. Well, maybe that’s not true, but if Apple jumps off a bridge and all that.

It is quite common to see video players with audio controls. This is absolutely unnecessary as all computers have audio controls. Many have dedicated buttons, while Windows has it right in the bottom corner. Most users already have their audio at a level they already want, so why reinvent the wheel? If you must include an audio control in your video player, please remember to default it to max volume. This is something that, for example, CNET does poorly. Because most users have their audio at a level they already want, having your video player at something less than max means they must either 1) adjust it on the video player for each video they watch, or 2) adjust their system audio to start watching, then adjust their system audio back after watching it.

Targeting New Windows

This is a simple one but an important one. It is generally a good idea to target links to other websites to a new window, and links within your own domain to the same window. This makes sure that users are still connected to your site. The exception to the rule that says “target links within your own domain to the same window” is when you’re in a large article (much like this) and you have links in the middle. This disrupts the flow of your article. Target references in a new window so users can come back to your article, or save the links for when they are finished.

Fancy Dates

Fancy dates often look like this: “About an Hour ago” or “14 hours ago”. Instead of generalizing your time, let your users know the exact date and time something happened. Users are likely more interested in knowing what time something happened than knowing how long ago it was. It is less specific to use fancy dates, because your users don’t know if “2 hours ago” really means “1 1/2 hours ago and I rounded up” or “2 hours and 59 minutes ago and I don’t count minutes”. Give your users the full story!

Multi-page Articles

Click here to read page 2. Ok, not really, but this is often the case on many sites. Is it really that hard to put a large article on one page? It is much easier to scroll than to traverse through a half dozen pages. Even if your page has a lot of rich media content, ask yourself if it’s really necessary to use multi-page layouts first.

Overuse of Ajax

Ajax is an excellent tool to make your internet applications richer. I am personally a huge fan of Ajax, but if used improperly it can be a huge liability to users. Ask yourself one of these questions:

Will my users want to access this content through a URL?

Will my users want to use the back/forward buttons in their browser?

If the answer is yes to either of these, then reconsider using Ajax to present your content. There are obvious ways around this, but they are generally clunky and not always worth the effort.

Conclusion

These are just a few examples of how not to do things. They all have a common theme: Users First! If you always remember to question each thing you do, even if it seems to be a standard, you’ve got at least one notch in your “I’m a good developer” belt.

The Truth About HTML5

Cory Fischer — Sun, 06 Jun 2010 08:50:38 +0000

There are many misconceptions out on the web about what HTML5 actually is, so I want to address this first. Most misconceptions surrounding the discussion are that HTML5 is an Adobe Flash replacement or even just a video player. Although it supports embedded video without the need for plugins, HTML5 is not just about video in your browser. It is a specification that will eventually be a recommendation from the W3C (World Wide Web Consortium) to browser makers on how HTML code should be rendered. Normally, recommendations such as HTML 4.01, the most commonly used specification today, stops there. However, HTML5 also has specifications on how to make GPU and other OS-level elements available to the HTML5 API so that web developers can program their sites once and end up getting the same visual results in any browser on any platform from the same code. At least that’s what’s supposed to happen. Once the W3C released a recommendation, it is considered a “standard”.

Everyone is pushing for HTML5 now as if it’s already been recommended by the W3C. The iPhone and iPad are already supposedly supporting it. The problem is, this specification is still in draft. Browser makers like Mozilla, Google, and Apple are all trying to add the functionality to their browsers to “beat each other” to punch by supporting things before they are a standard. This makes for a bad recipe when a recommendation changes before it becomes a standard. Currently, many browser makers are doing the same thing with CSS3. Since things such as rounded corners are only supported in the CSS3 specification, browser makers add their rendering engine’s prefix before the current specs code so that if the recommendation changes, they’re still only screwing up their “proprietary” version of the code. For example, if you wanted the top left corner of a div element to be rounded, the specification currently reads that your code should look like this:

div {
	border-top-left-radius: 5px;
}

That’s one line of code that should render identically in every browser. Unfortunately it doesn’t render in any browser since CSS3 is still in draft and not a standard. So to get this working in modern browsers, Mozilla, Google, and Apple have added their own versions:

div {
	-moz-border-radius-topleft: 5px;
	-webkit-border-top-left-radius: 5px;
}

Both have the browser-specific prefix (-moz for Gecko engines like Firefox, -webkit for webkit browsers like Chrome and Safari), but are also different in name. Instead of ending with one word “topleft” webkit uses “top-left” before the word “radius”. The reason is because the Gecko engine supported the border radius property before anyone else, and then the recommendation changed the format for the border-radius property. Now we have two ways to make rounded corners for two different browser engines that don’t match and neither one of them is true CSS3.

HTML5 is in the same situation as CSS3. That is, it’s a mess. Not HTML5 per-se, but browser implementation of it. The video streaming, for example, was first supported by Firefox using the Ogg Theora codec. This was the spec in the HTML5 recommendation until Apple decided to raise a stink about it and said they would only support the H.264 codec. Apple shipped Safari 4 with quasi-support for the HTML5 video element and the H.264 element. Google said they would ship Chrome to support both codecs, while Microsoft said they will only support H.264 once IE9 comes out. Because all these browser makers started throwing their weight around and refusing to support something before it became a standard, the specification to require a specific codec was dropped.

I personally believe there should be one codec for all browsers to support. That is the whole idea of an “open standard” after all. Another aspect of an open standard is… umm… openness. The Ogg Theora codec is completely open source, meaning anyone can use it for any reason in any product without paying royalties. H.264 is currently maintained by ISO Moving Picture Experts Group (MPEG) and anyone who wants to license H.264 has to pay a hefty price. Two of its patent holders: Microsoft and Apple. So if Mozilla, Google or anyone else wants to build a browser that supports HTML5, they would have to pay Microsoft and Apple (among others) to use it. Now, there are some benefits to H.264. It delivers DVD quality video at twice the bit-rate with low processor usage that could benefit mobile devices. These benefits do not out weigh the implications of an open standard in my opinion. If any browser maker has to pay a few corporations, that essentially means that a few companies will “own” the internet.

This does not necessarily mean that Ogg Theora is the right choice, but I believe it is best for the web at this point. Is the argument about mobile devices supporting this codec valid? Well, that brings up another issue with HTML5. The specification is not due to become a W3C recommendation until 2022. By then, mobile devices will be more powerful than todays laptops.

There are so many things that can change between now and 2022, and the browser makers will come up with their own proprietary “standards” in the years to come. Meanwhile, web developers will still have to code for multiple browsers and not be able to take advantage of the open standards that HTML5 is supposed to represent.

So, will HTML5′s video element (as well as the canvas element) kill Flash? It’s hard to say at this point, but since there are currently many limitations to the HTML 5 draft, such as full-screen mode, no interactive elements, etc. I don’t foresee this happening any time in the near future. Also, Flash is currently installed on 98% of computers world wide, while Internet Explorer has 65% of the browser usage share and doesn’t have a version that supports any HTML5 in one form or another. Until developers can program to 90% of the web, they will generally stick with what’s supported. So once IE9 comes out, which will supposedly support the video element, it will slowly have to gain usage share while older versions such as IE6, IE7, and IE8 fizzle out of usage. If you think that’s going to happen quickly, you’ll be interested in knowing that IE6, which was released in 2001, still currently holds around 17% of the market. Until this happens, developers would have to use something such as Flash as a fall back for older browsers, and since Flash currently does a better job than HTML5, developers will likely stick with Flash for a very long time instead of programming two versions of the same video.

My hope is that HTML5 will pick up steam and allow us developers to take advantage of features that are traditionally only seen in desktop applications. The web is fast moving, and HTML5 will have to become a recommendation well before 2022 if it is to succeed. If there is not a recommended codec for the video element, it would be better to allow multiple codecs so that open ones can exist. The web should remain open and stay away from any sort of proprietary ownership.

Is HTML5 the future? Yes, I believe it is. Lets just hope it’s not the far future.

Hierarchical Data in MySQL

Cory Fischer — Fri, 07 May 2010 07:03:03 +0000

For the purposes of this post, we’ll use a fake website and use the hierarchy in the image below.

Managing a site structure, or any other structure, in a database often gets overlooked. It’s pretty simple and a common standard to use what’s called the Adjacency List Model without sitting down and thinking about other possible solutions. The Adjacency List Model in a database table consists of an ID and a Parent ID. The Parent ID usually refers to the ID of another record in the same table. Here is an example of what one might look like:

------------------------------------------------
|  ID  |      page_name          |  parent_id  |
------------------------------------------------
|  1   |       Home              |  NULL       |
|  2   |       Books             |  1          |
|  3   |       Best Sellers      |  2          |
|  4   |       Cloths            |  1          |
|  5   |       Toys              |  1          |
|  6   |       Electronics       |  1          |
|  7   |       iPods             |  6          |
|  8   |       Computers         |  6          |
|  9   |       Desktops          |  8          |
|  10  |       Laptops           |  8          |
|  11  |       Netbooks          |  8          |
------------------------------------------------

Notice the Home page has an ID of 1 and no parent_id. Best Sellers has the parent_id of 2 which corresponds to the ID of 2 which belongs to Books. Desktops, Laptops and Netbooks all have a parent_id of 8 which corresponds to the ID of Computers, and Computers has 6 as the parent_id which corresponds to Electronics, and finally Electronics has 1 as its parent_id which corresponds to Home. This is the most common form of hierarchical data storage in a database. First thing you’ll notice is that it’s pretty simple. If I’m on a page and I need to find it’s parent, the SQL is easy.

Assume $page_id is the variable that corresponds to ID in the database. Lets retrieve its parent.

SELECT * FROM pages
WHERE page_id = (
	SELECT parent_id FROM pages WHERE page_id = $page_id
)

Similarly, we could get all the child pages

SELECT * FROM pages WHERE parent_id = $page_id

Now you might be thinking there’s nothing wrong with this. After all, it’s very simple and easy to understand, but what if you want to find the entire hierarchy? Suppose you’re on “Laptops” and you want to get the crumbtrail all the way up to the Home page. (For simplicity and clarity, I’ll use the page “name” rather than the ID’s in my examples).

SELECT pageLevel1.page_name AS lev1, pageLevel2.page_name AS lev2,
	pageLevel3.page_name AS lev3, pageLevel4.page_name AS lev4
FROM pages AS pageLevel1
LEFT JOIN pages AS pageLevel2 ON pageLevel2.parent = pageLevel1.page_id
LEFT JOIN pages AS pageLevel3 ON pageLevel3.parent = pageLevel2.page_id
LEFT JOIN pages AS pageLevel4 ON pageLevel4.parent = pageLevel3.page_id
WHERE pageLevel1.page_name = 'Home';

The result is:

---------------------------------------------------------------------
|   lev1         |   lev2         |   lev3         |   lev4         |
---------------------------------------------------------------------
|   Home         |   Electronics  |   Computers    |   Laptops      |
---------------------------------------------------------------------

There’s a problem with this. Besides the large amount of joins, this assumes we know the depths of the hierarchy. In a website application we generate the crumbtrail, navigations, and hierarchy trees dynamically. This Adjacency List Model is too flat and has no easy way of getting the depth of a node.

The Adjacency List Model is great for finding parents of nodes, but not so great at finding grand parent nodes, great grand parent nodes, even second-cousin-once-removed nodes, or the uncle-no-one-talks-about nodes.

Luckily, there is another solution to our database hierarchical structures. It’s called the Nested Set Model. This model uses two fields to determine the spot in the hierarchy as oppose to a single parent_id: a left and a right. Our example above would look like this in a database using the Nested Set Model:

----------------------------------------------------
|  ID  |      page_name          |  lft  |	rgt	|
----------------------------------------------------
|  1   |       Home              |  1    |	22	|
|  2   |       Books             |  2    |	5	|
|  3   |       Best Sellers      |  3    |	4	|
|  4   |       Cloths            |  6    |	7	|
|  5   |       Toys              |  8    |	9	|
|  6   |       Electronics       |  10   |	21	|
|  7   |       iPods             |  11   |	12	|
|  7   |       Computers         |  13   |	20	|
|  9   |       Desktops          |  14   |	15	|
|  10  |       Laptops           |  16   |	17	|
|  11  |       Netbooks          |  18   |	19	|
----------------------------------------------------

If you’re unfamiliar with the Nested Set Model, this might be confusing. All you have to remember is that if an item’s “lft” and “rgt” fall between another rows lft and rgt, it is a sub-node of that row. (Notice this is not “left” and “right” as they are reserved words in SQL, so I used lft and rgt).

Here is a diagram of the the same site with the numbers added to give you an idea of the structure.

Since Home is the top of the hierarchy, it starts it’s lft with the lowest number, and rgt with the highest number; therefore any page that has an lft that’s GREATER than 1 and an rgt that’s LESS than 22 is a sub node of Home. Likewise, Books has 2 and 5 with Best Sellers having 3 and 4 making it a sub node of Books. We can then know that Best Sellers is also a sub node of Home because 3 and 4 fall between 1 and 22. Hopefully this is making some sense. If you need help following the flow of the above chart, follow the blue line of this chart. Notice the blue lines go in numeric order.

This might seem quite a bit tougher than the Adjacency List Model, and you’d be right. The implementation is tougher, but once you start using it in your application you won’t be sorry. Let’s try to get a crumbtrail from “Laptops” again.

SELECT parent.page_name
FROM pages AS node,
pages AS parent
WHERE node.lft BETWEEN parent.lft AND parent.rgt
AND node.page_name = 'Laptops'
ORDER BY parent.lft;

This creates the same results:

------------------
|   name         |
------------------
|   Home         |
|   Electronics  |
|   Computers    |
|   Laptops      |
------------------

This was much simpler. I just found “Laptops” lft and rgt and I’m basically saying “find me all the pages where the left is less than Laptop’s left and the right is greater than Laptop’s right”. This just works its way right up the hierarchy.

What about finding the entire depth of a tree with this model?

SELECT lft,rgt,page_id,page_name,
    (
		SELECT COUNT(page_id) FROM pages
		WHERE lft < node.lft 
		AND rgt > node.rgt
		AND node.lft <> 0
	)
	AS depth
FROM `pages` AS node
WHERE lft <> 0 ORDER BY lft

The subquery (line 2 above) is an easy way to find how deep inside the hierarchy we are. We can use this in our code to create a full tree:

 
$last_depth	= -1;
$output		= '';
$i		= 0;
$pageList = mysql_query("SELECT lft,rgt,page_id,page_name,
	(
		SELECT COUNT(page_id) FROM pages 
		WHERE lft < node.lft 
		AND rgt > node.rgt 
		AND node.lft <> 0
	) AS depth 
	FROM `pages` AS node 
	WHERE lft <> 0 ORDER BY lft");
 
while($row = mysql_fetch_array($pageList)) {
	if ($i==0){
		$liOffset = $row['depth'];// first (and therefore highest) in the hierarchy
	}
	$current_depth = $row['depth'];
 
	if ($last_depth > $current_depth) {
		$output .= str_repeat('',($last_depth-$current_depth));
	}
 
	$output .= "]."'>".$row['page_name'];
 
	if ($row['rgt'] - 1 != $row['lft']) {
		$output .= '';
	} else {
		$output .= '';
	}
	$last_depth = $current_depth;
	$i++;
}
while($last_depth >= ($liOffset+1)) {
	$output .= '
';
	$last_depth--;
}
echo $output.'';

This will give you a perfect

SELECT node.page_name, (COUNT(parent.page_name) - (sub_tree.depth + 1)) AS depth
FROM pages AS node,
	pages AS parent,
	pages AS sub_parent,
	(
		SELECT node.page_name, (COUNT(parent.page_name) - 1) AS depth
		FROM pages AS node,
		pages AS parent
		WHERE node.lft BETWEEN parent.lft AND parent.rgt
		AND node.page_name = 'Home'
		GROUP BY node.page_name
		ORDER BY node.lft
	)AS sub_tree
WHERE node.lft BETWEEN parent.lft AND parent.rgt
	AND node.lft BETWEEN sub_parent.lft AND sub_parent.rgt
	AND sub_parent.page_name = sub_tree.page_name
GROUP BY node.page_name
HAVING depth <= 1
ORDER BY node.lft;

The results are:

----------------------------
|   name         |  Depth  |
----------------------------
|   Home         |  0      |
|   Books        |  1      |
|   Cloths       |  1      |
|   Toys         |  1      |
|   Electronics  |  1      |
----------------------------

If you familiarize yourself with this model, many things will become second nature to you. You may ask yourself, “how do I add a new node in the Nested Set Model?”. I will leave you not with the SQL, but with the theory. If you wanted to add a node “Servers” under “Computers” to the right of “Netbooks” you would just make the new “lft” be 20 and the “rgt” be 21, and every other node that has an lft or rgt greater than or equal to 20 should have 2 more added to it.

The downside of the Nested Set Model

Although I feel that in many cases the Nested Set Model is superior to the Adjacency List Model, there are some negatives.

1. It is not widely used, therefore support when you run into issues is a limited. You may find some help online, but the vast majority of users and existing applications are not using this model.

2. Data corruption is possible. If you make a mistake in your code when you add a new node, move a node, delete a node etc. you may cause data corruption. For example, what happens if you try to move a Node to be a child of one of it’s on children? If you create your script objects right and always use them after extensively testing to make sure they always work, you should be fine. However if someone goes straight into the database and changes some random pages “lft” value to zero, the repercussions could involve restoring your database to a previous backup or meticulously going through the hierarchy in the back end and manually updating the nodes yourself. This is not a fun task!

3. It can be hard to learn. It’s a steep learning curve. I had to read the same article probably ten times before I fully grasped the SQL queries, but eventually it all clicked, and now that I use it in my applications I feel confident in many of the custom requests I receive.

Rewriting URL’s with mod_rewrite

Cory Fischer — Sat, 01 May 2010 09:27:12 +0000

Rewriting URL’s from difficult-to-read addresses to simple, clean addresses can make your website much more user-friendly and SEO-friendly, but it can be difficult to accomplish without the proper tools. Enter mod_rewrite! Mod_rewrite is a part of the Apache web server that acts as a proxy before content from the web server is returned to your browser. If your server isn’t running Apache then there are also extensions available for IIS such as the URL Rewriter.

What is the benefit of this? Well, how many times have you come across a link like this while browsing?

http://www.example.com/some-dir/index.php?page=animals&animal=dogs&action=videos

First thing is, this is not very descriptive and hard for users to remember. Second, this isn’t going to get you very good search results in Google or any other search engine. Search engines much prefer URL’s to look something like this:

http://www.example.com/animals/dogs/videos

Coding in mod_rewrite

So how does mod_rewrite work to turn the nasty looking first link into the super nice looking, SEO-friendly second link? It all starts with the .htaccess file (yes, it starts with a dot “.”). I’ve always been better at learning-by-doing or learning-by-seeing, so lets just jump into the code. First, I’ve created a file in the root of my site called .htaccess. In it are some simple lines of code like this:

<IfModule mod_rewrite.c>
    RewriteEngine On
    # send any non-files or non-folders to the root index.php file
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.*)$ /index.php?page=$1 [QSA,L]
IfModule>

This starts off with a conditional statement checking to see if Apache has the mod_rewrite module at all to avoid errors. Inside the ‘if’ statement is the actual code. The first line “RewriteEngine On” simply says we’re activating mod_rewrite. This is a quick and easy way to turn URL rewriting on/off without having to delete or comment out a bunch of lines of code. The second line starts with a hash mark. This is how you do comments. The next two lines start with “RewriteCond”. These are conditions for the last line which is the actual rule. The condition lines aren’t even read unless the RewriteRule is met. The RewriteRule is read like this: “if the path of the URL matches my regular expression, then check the RewriteCond”. The “path” here is the part of the URL between the domain and the query string: http://www.example.com/some/directory/file.php?querystring. If you are unfamiliar with Regular Expressions, you can familiarize yourself with them with Regex cheat sheets online. I got the following from addbytes.com.

With the above regular expression: ^(.*)$ says “match 0 or more characters”, which any URL would match, so in this case the RewriteRule will always be met, and will always look at the RewriteCond. Since there is no other RewriteRule between the two RewriteCond they both apply to this rule. The first condition is making sure the file name (through %{REQUEST_FILENAME} ) is not a directory (the exclamation mark meaning *not* and the -d meaning directory). If it meets this condition, the second condition is ignored and we go back to the RewriteRule. The second RewriteCond is similar to the first except that “-f” makes sure this is not an existing file. The reason for this is that we don’t want to redirect every file on the server. If there currently exists a directory or file “/animals/dogs/videos” on the server then we don’t want to write over it, because remember our rewrite rule already said “any path” which would include the file we’re going to end up redirecting to. This would lead to an infinite loop of redirects, and nothing would be accessible.

Once one of these conditions is met, the rewrite engine goes back to the RewriteRule and looks at the URL that comes after the first regular expression. The regular expression and the URL are separated by a white space. So, our new URL that we want to go to is /index.php?page=$1. This would be a PHP file that exists in the root of the site. The PHP page has a parameter call “page” with the $1. This dollar sign and number means that anything that is within the first set of parenthesis in the first part of our RewriteRule should be put in this spot. In this case, the entire path. So in our example above, when someone tries to go to example.com/animals/dogs/videos, they will get redirected to example.com/index.php?page=/animals/dogs/videos. Now, you won’t see any of this in the browser, but that’s what your web server sees in the background. So in your index.php file you can do something like:

if (isset($_GET["page"])) {
    $page = $_GET["page"];
    // display code based on this variable
}

The above is obviously very simplified, but you get the idea. The last part of the RewriteRule is this: [QSA,L]. This is where you can specify extra information about he URL you are redirecting to. The QSA means that any additional query string parameters should be added to the new URL, and L means that this is the last rule that should be met; any other rules after this are ignored.

Click here for a mod_rewrite cheat sheet, thanks to addbytes.com for the list.

What about other, more complex redirects? Suppose we want to redirect old URLs to a new domain or website? Lets look at RewriteMaps.

RewriteMaps are variables that include an external text file to map certain keys to certain values. It might look like this:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteMap sections_var txt:/usr/local/apache/conf/sections.txt
    RewriteCond %{QUERY_STRING}% section=([0-9]+)&item=(\w+)
    RewriteRule ^/store/(.*) http://www.example.com/store/${sections_var:%1|no-section}/%2/ [QSA,R=301,L]
ifModule>

We also have a sections.txt file that looks like this:


1    Electronics
2    Music
3    MP3 Players
4    Televisions
5    Laptops
6    Netbooks
7    Desktops
8    Books
9    Software
10   Accessories

So for this example, we want our old, out of date URLs that people may have saved to their favorites, or Google may have indexed, to know where our new links are.

We reference this text file with the RewriteMap variable called sections_var. We then identify where the file is located by writing out the full path to the text file. In our RewriteRule we are checking to see if the incoming URL is a string starting with /store/ followed by zero or more characters. The RewriteCond checks to see if the query string has section=(one or more numbers) and also: &item=(one or more words). An example of this kind of URL is: site.com/store/index.php?section=10&item=cases.

Once we return to the RewriteRule, our first variable we come to is ${sections_var:%1|no-section}. Lets look at the first part: ${} . This means a variable with code inside the braces. Inside the braces are two sets of information separated by the pipe character. If the part to the left of the pipe character isn’t found, the part to the right of the pipe character is read. The sections_var:%1 takes the first parenthesis value from the RewriteCond and finds the value in the sections.txt map. Look at the sections.txt map above, and if, for example, the number 7 is in the URL, the value “Desktops” is returned to the RewriteRule. Again, if no value is found the default “no-section” is given. The %2 is the second parenthesis given in the RewriteCond. Notice, a percentage is used here as opposed to the dollar sign like we used in the first example. When you “backreference” to the first part of the RewriteRule you use the dollar sign. When you backreference to the RewriteCond you use the percentage.

Last thing we have in this example is the R=301. This is telling the browser and all the little Google bots out there that this is a permanent redirect and it should update all the indexes it has.

Tips

If you want different sets of rules for different directories in your site, you can put the .htaccess files in lower directories as well. The farther down your directory it is the more it takes precedence. If you have an .htaccess file in /site/animals/ then any rules in that directory will run before any the rules of an .htaccess file in /site/ or the root directory.

Other issues you may have with rewriting URLs is that if you rewrite to a new domain, the URL will resolve to the new address. This may not be the desired effect. In this case, you would use the P flag in your brackets at the end of your RewriteRule:

<IfModule mod_rewrite.c>
    RewriteEngine On
    # any incoming URL on http://oldsite.com gets redirect to http://newsite.com
    RewriteRule ^(.*) http://www.newsite.com.com$1 [QSA,P]
ifModule>

This will take any incoming URL from oldsite.com and forward it to newsite.com, but you’ll still see “oldsite.com” in your browsers address bar. I have found this useful for things like redirecting my crossdomain.xml file when my application server is down and I have web services that need the crossdomain file.

Optimizing Your Rewrite Code

It is important to remember that every request made to your server comes through your mod_rewrite file. If you load a single page, it and all of its images, css files, javascripts and other files will go through this file separately. This could lead to slow load times if you have a lot of rules. Mod_rewrite is a powerful tool if you spend the time to optimize it properly.

Remember to utilize maps when possible. Maps won’t be queried if a rule’s conditions aren’t met.

Familiarize yourself with regular expressions to create fewer rules. Complex rules are harder to write but the rewrite engine can go through them quickly. Just remember, the fewer rules the better. When you need to write maps, they don’t get called as often as the rules, but try to keep them short anyway. A list of 10, 20 or 50 shouldn’t be too much for it, but if you create a map with 500 items you could see a big slow down.

Lastly, try to rely on your server-side code like the first example if possible. If you can send large chuncks of data to your PHP file and parse your the data with your PHP code (or whatever language you’re using) the PHP code can run a lot faster than your rewrite code.