Note: CodeIgniter Reactor 2.0.2 has a bug in it’s core Loader class that breaks package config file loading (and sparks, sadly). It’s recommended that you use 2.0.1 OR the latest at https://bitbucket.org/ellislab/codeigniter-reactor OR make the following change in your 2.0.2 installation: https://bitbucket.org/ellislab/codeigniter-reactor/changeset/c461483c8ca0 . The last option is the best.

It took a month longer than expected, but we’ve finally released the Beta version of GetSparks.org, the repository and package manager system for CodeIgniter.

Here’s the very minimum of what you need to know:

1. A spark.info file is now required in every spark
2. You should upgrade your spark system as soon as possible (just follow the usual installation instructions)
3. If you submitted a spark prior to the beta, we’ve normalized all version numbers to x.x.x. So curl 1.0 becomes curl 1.0.0. Check the site.

The alpha was all about weeding out fundamental issues in the initial repo and system. The big problems we had were:

• Some spark developers wanted the ability to specify dependencies
• There was no difference between a repository tag and the spark version — They had to be in sync
• Users wanted RSS feeds for latest spark overall, and latest versions of individual sparks
• There was no way to rate a spark
• There was no way to tell if your spark was going to be accepted by the spark processor prior to submission

We knocked those out and a little more with the beta. Going forward:

All sparks must have a spark.info file at the root. Details are here.

• You can now specify spark dependencies in spark.info
• The spark manager will now install a spark and all of its dependencies
• The spark manager can now self-update
• All spark versions must be in x.x.x format. Sparks released prior to the beta had their version numbers change in the system (tags remained the same though)
• Versions and tags are now decoupled. The version is pulled from spark.info
• Sparks are now ratable via a simple “Love”, “Like”, or “Hate” rating
• A Spark-SDK is available to spark developers in order to validate their sparks prior to submission

It is necessary that you upgrade your spark manager with the release. The current version is 0.0.4. You can upgrade by removing the ‘tools’ folder from your webroot and following the same installation instructions at: http://getsparks.org/install

Send any issues to ohcrap@getsparks.org!

Now that the structural changes have been made, we can focus on tightening up the spark system’s feature set. This is what you see in the coming weeks:

• The ability to install sparks in configurations where the application folder is not in your webroot (possible now, but not automated)
• The ability for sparks to act more like modules — ie, sparks will be able to override core libraries and more

Also, we plan to:

• Put a news section on the site (long overdue)
• Configure our notification system better so its emails don’t land in your spam folder

And don’t forget the ultimate goal: Integration in CodeIgniter Reactor.

Check out the beta and ping the team with any problems! ohcrap@getsparks.org

Note: I wrote this over a month ago, but decided not to publish it until now.

I was talking with the Co-founder of a pretty cool start-up in DUMBO the other day about why the non-PHP development world generally has such disdain for PHP and the community surrounding it. He brought up an interesting point that stuck with me, largely because I hadn’t heard it before.

If you’re unaware of the usual beef most developers have with PHP, it tends to revolve around:

1. Ugly syntax
2. Lack of some necessary features that other languages have (prior to 5.3, namespacing, closures)
3. Inconsistent function naming, usage, and other quirks
4. Mix of procedural and OO-ness
5. The fact that 80-90% of PHP projects are probably gigantic piles of shit

But his problem with PHP was a little different. He didn’t say the actual language was poor — he said it was the general culture surrounding the language, which is usually iconified by a language’s founder, that seems to encourage bad practices. That is, PHP code bases tend to be hacky and unmaintainable.

The concept that the community surrounding a language or framework embodies an author’s philosophy seems to be true. He brought up Ruby and Matz. Matz wanted a language that was easy to read and write, and enhanced programmer productivity. Don’t Ruby developers seem to harp on rapid application development and the elegance of their language?

Then DHH and the Rails came up. Then Guido and Python. So I thought: What about Rasmus?

Rasmus Lerdorf is an interesting figure. He created the original version of PHP, continues to contribute, is widely considered a demigod in the community and the authority on almost anything PHP. He steals masses of attendees at conferences, gets hired by big internet places, and garners the respect of everyone despite one glaring property: Rasmus represents what most non-PHP developers hate about PHP.

Rasmus generally promotes abstention from using frameworks, and the use of PHP as more of a templating language. To him, this translates to raw speed and scalability (load-wise). To everyone else, this translates to piles of procedural spaghetti code, and unmaintainable projects. For roughly 10 years following the birth of PHP in 1995, this was how PHP projects were written.

But another issue cropped up:  In it’s pizza-faced adolescent years (pre-5.0), PHP gained a serious following among novices. The language has a fantastically low barrier to entry, so anyone could get started in 2 minutes by downloading some self-extracting *AMP stack for Windows. Additionally, the acceptance of the MVC paradigm hadn’t really occurred yet in web development. What do you get when you mix n00bs and a lack of best practices? Unmaintainable garbage. And that’s what proliferated.

Don’t get me wrong — there were some great PHP developers around, even back then. But like I said, unrefined n00b-sauce was all around. When cowboy PHP developers with no standards got together to build a project, it came out looking like PHPbb, PHPNuke, or some other gnarled mash of .php3 files. But can you singularly blame PHP developers? No! The other web language giants, ASP and Perl, were also gross as hell and promoting the same spaghetti-code practices.

So why does PHP get a bad rap? Because of its legacy. And most old-time PHP devs who have fled to Python, Ruby, and Java haven’t really looked back to see what kind of development has happened in the language since the introduction of MVC on the web. Additionally, there were super-outspoken critics like “Ruby guy” Zed Shaw complaining of developers with “PHP-Infected Brains”, and the distribution of stuff like this on RubyInside.

PHP was a ghetto.

But the development of frameworks like Zend and CodeIgniter have greatly pushed the language development into the right direction. In fact, it’s been pushed in the opposite direction of where Rasmus would probably like to see it. Check out the Zend or CodeIgniter frameworks and tell me it’s not some of the best documented, most well-written code you’ve seen.

When most developers learned Ruby, they were learning Rails and MVC  at the same time. PHP was in use for a full 10 years before that. So there really wasn’t a period of time when heinous Ruby was being written by novices. There was an established standard in place for Rails, and the barrier to entry was a much higher, typically keeping less experienced developers out.

The fact is, a PHP applications can be as well-written as an application in any other language, and probably have the additional advantage of speed. The widespread use of MVC-style development in the PHP world is a relatively recent phenomena though, and admittedly, we can probably thank Rails for it.

So what does PHP have going for it now?

1. Standards (not universal, but generally a flavor of MVC for most projects, and little procedural crap)
2. A very low barrier to entry
3. Speed & Scalability (maybe the best among script-based languages)
4. A great unit testing framework
5. Arguably the best documentation for any language

Additionally, it’s behind some of the internet’s most influential websites and tools, like Facebook, Digg, Wikipedia, WordPress, Drupal, etc. I’d bet that having a solid understanding of it opens more doors for a developer than any other.

If you don’t agree with the above, comment on this post, or email me — I’d like to hear why you don’t think so.

I’m no PHP fanboy — in fact, I’m very language-agnostic. I write PHP more often because, you guessed it, people pay me to. So it all comes down to this:

If you are capable of making wise software design decisions, PHP is a great choice to build your web application with.

By the way, if I just convinced you to build your next webapp in PHP, check out CodeIgniter. It’s the lightweight, no magic, ultra fast framework for PHP. When it comes to CodeIgniter, I am a fanboy.

4/4 Edit: I’ll be presenting a tool I wrote in Perl called ‘divvy‘ tonight at Hack and Tell in NYC. Here’s a video of a previous event (about 10 minutes in) where a developer plugs in his laptop to reveal Windows XP on his desktop (strike 1.5) and then admits to being a “PHP Coder” (strike 4). Is it me, or can you feel the judgment in the air? Maybe it’s me.

On Twitter, I’m @_kennyk_.

In my initial post after joining the CodeIgniter Reactor team (over Thanksgiving weekend ’10), I went as far as to saying that you could hold me responsible for the quality of the CodeIgniter documentation. Here’s a universal truth: Documentation > magic.

CodeIgniter 2.0 was released last week, with an announcement on the EllisLab news feed. The “Reactor” project is now considered to be “CodeIgniter”. All official forum references, downloads, and docs referring to CodeIgniter are referring to the project sitting here at BitBucket.

You can check out the lengthy change-log here: http://codeigniter.com/user_guide/changelog.html

Version 2.0 was released while some big features were still in development (but not 100% ready for the limelight). My favorite upcoming feature (by no coincidence) is something I’m currently putting together: A user-contributed note system for the user guide.

Everyone who keeps a place in their heart for PHP (as hard as it may be at times) knows how helpful php.net is. It’s a single place for clear PHP documentation, code examples, and user-contributed notes.

Some would argue that the most helpful parts of the PHP docs are the notes that PHP developers contributed themselves. A long-time request for CodeIgniter has been to add something similar the user guide.

Since late December, I’ve worked been working on (and have pretty much completed) a full-featured versioned user-contributed notes system for the docs. You can the see the progress in the BitBucket ‘user-notes’ branch:  https://bitbucket.org/ellislab/codeigniter-reactor/src/a6f52cbc8229

The actual application used to host the notes is being developed at: https://bitbucket.org/katzgrau/ci-userguide-notes

And of course, some screenshots of notes in action:

Adding a note looks like a clone of the contribution page at PHP.net:

You can check out a live instance of a user guide with the new system here (link not guaranteed to work into the future!): http://ci-notes-exp.katzgrau.com/user_guide/libraries/benchmark.html

Like I mentioned, the notes are versioned. That means notes can be promoted from older versions to newer versions, and they can be pruned from newer versions as they become obsolete, without affecting the old. Best of all, the user_guide will still come distributed in HTML format with the framework. The notes are pulled in and loaded via some simple Javascript.

I’ve always thought the CodeIgniter docs were the best among any framework that I’ve come across. I think this’ll take them even further.

Look for them at the end of Q1 2011.

There are a lot of people out there who call themselves “LAMP” developers — short for Linux, Apache, MySQL, PHP. That’s the standard configuration for production PHP applications. Recently, I ended up having to build a CodeIgniter application on Windows, IIS, Mysql+MS-SQL, and PHP. Sound like there are bound to be issues? You bet, and it especially hurts because now I’m a real-live WIMP developer.

And what made it even more interesting was that due to constraints, I had to develop the application in Ubuntu and deploy to Windows for production.

Please keep in mind that I didn’t opt for this setup by choice. The servers to be used were already in place, and well, it just had to be this way. I’d imagine this unholy mix can be found on server farms somewhere around the seventh or eighth circle of hell.

Anyway, the point of this post is to document a few “gotchas” that came up along the way. At this point I should say that application is now happily humming along in production. I knew from the start that mixing all of the above would be a headache, but luckily, things worked out without sapping too much time. I should also say this before I start: Thanks to some great work done by others over the past 2 years, this setup was actually possible.

Gotcha #1: PHP and MSSQL on Ubuntu

Thank god this was so easy. In order to use the standard database functions like mssql_connect, mssql_query, etc, all I needed to do within my existing LAMP installation was run:

$ sudo apt-get install php5-sybase

And sh-bang, I could connect to SQL Server without an issue. From CodeIgniter, I set the database to use mssql as my driver, and I was home free.

Gotcha #2: PHP and MSSQL on Windows

For years, forums and IRC rooms were filled with hopeless requests to get these two to mix. In 2008, Microsoft wrote a driver to help PHP developers seamlessly connect to SQL Server. For Win-PHP installations, just install this gem: http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=80e44913-24b4-4113-8807-caae6cf2ca05 . Also, read up on the docs.

That driver uses a different API then the regular PHP mssql_* functions. In fact, it uses sqlsrv_* functions instead. So CodeIgniter can’t work with it out of the box. Luckily I found an excellent 2 and 1/2 year-old post by a guy who wrote the CodeIgniter driver to work with the Microsoft drivers. Just download the code, and drop it into system/database/drivers. Read up here: http://www.kaweb.co.uk/blog/mssql-server-2005-and-codeigniter/

One thing you will have to do to make it work with the latest version of CodeIgniter is create a dummy function in sqlsrv_driver.php. Just drop ‘function db_set_charset() { }’ somewhere in the class declaration.

As a side note, that driver would make a great addition to CodeIgniter Reactor.

Gotcha #3: mssql vs. sqlsrv

Notice that I had to use different drivers for connecting to SQL Server between Ubuntu and Windows. This is why it’s handy to use some sort of database abstraction class like the one that comes with CodeIgniter. All I had to do to switch between drivers when I deployed to the new environment was edit the configuration.

Also, query result fields that are fetched using PHP’s mssql driver come back as strings. The Windows sqlsrv driver gets fancy and hands back field values as objects. Your code won’t have to change for the most part, but beware that MSSQL ‘datetime’ fields come back as native PHP DateTime objects using sqlsrv_*, not strings. In the code, I ended up doing something like: if($date instanceof DateTime) return $date->format(…); else return $date;

Edit: Commenter Brian Swan and Twitter user Juozas Kaziukėnas have pointed out that the sqlsrv_ PHP driver will take a boolean ‘ReturnDatesAsStrings’ option in sqlsrv_connect() that specifies whether datetime fields come back as strings or objects. More info is here: http://msdn.microsoft.com/en-us/library/ee376928%28v=sql.90%29.aspx. Thanks guys!

Gotcha #4: File Permissions and Logging

As the author of WPSearch, a Wordress search plugin which does heavy work with the filesystem, I can say that the number one cause of broken installations is that the permissions are too strict. On Windows, files can appear to be 777 when viewing them via FTP, but on the Windows end, they are set as ‘Read Only’, or ‘Archive’. This throws a serious wrench into things.

When I first deployed to the production server, all I would get for responses was a blank page. Worst of all, the log wasn’t writing anything. After a trace through the CodeIgniter bootstrap, I found that the application died when the logging class was loaded. If your application’s logging threshold is set to write anything, and the logging fails, the application might just crap-out. This is different behavior than I’ve seen on linux, where you just won’t see logs appear in the logging directory, but the overall application still works fine.

Just turn off logging to make sure that empty responses aren’t the result of file permission errors.

Gotcha #5: No .htaccess Fo’ You!

I hadn’t known that I needed to drop the final application into an IIS instance from the start. I learned about that 1 day prior to the launch. Before that, I though I’d be rollin’ on Windows/Apache. Wrong!

IIS doesn’t use silly .htaccess files, which I had only used for standard URL rewriting/prettified URLs. I was forced to decide between converting the .htaccess to an IIS web.config file, or just ditch the prettified URLs. Since it was a small application, I went with the latter. Here’s a nice StackOverflow thread discussing a translation though: http://stackoverflow.com/questions/702526/translating-an-apache-htaccess-file-to-an-iis-web-config

Edit: Juozas had some comments here too: “@_kennyk_  about .htaccess vs web.config – you can actually import .htaccess in IIS URL Rewrite section.”

Gotcha #6: 2 Databases, 1 Application

This is more of a CodeIgniter issue. For this application I had to lookup data in a MySQL database, then use some of that information to pull rows from a MSSQL database. The hurdle revolves around having two concurrent database connections open in CodeIgniter.

You can’t really load the database in traditional CodeIgniter style, like $this->load->database(), or by autoloading it. Something like this doesn’t work:

$this->load->database('mysql-group-name');
# Do Stuff with mysql
$this->load->database('mssql-group-name');
# Do Stuff with mssql

Instead, you have to ask CodeIgniter to hand you back the actual database object with each connection. It’s a good idea to encapsulate each within the models where you’re using them. Do something like this:

class SomeMySQLModel extends Model {
  # the mysql db instance
  private $_db = NULL;

  function SomeMySQLModel() {
    $this->_db = $this->load->database('mysql-group', TRUE);
  }
}

class SomeMSSQLModel extends Model {
   # the mssql db instance
   private $_db = NULL;

   function SomeMSSQLModel() {
     $this->_db = $this->load->database('mssql-group', TRUE);
   }
 }

Gotcha #7: Mysterious lag time

IIS can exhibit some odd behavior regarding response times. When I was initially testing the application on my SliceHost dev server, I was getting pretty speedy responses in about ~100ms (keep in mind that I was connecting to both MSSQL and MySQL across the internet, not locally).

But when I moved to production, requests were taking 6 seconds each. Confused, I thought there must be some sort of bottleneck in the application. I used CodeIgniter’s profiling and benchmarking classes to investigate. I benchmarked the MySQL and MSSQL connections, queries, and overall application execution time.

The app was reporting that it was finishing responses within 100ms. So where was this strange delay coming from?

I then thought there might be some sort of redirect loop going on. From the shell, I ran:

$ time curl -v [address]

And something else became apparent. The full content of the page would come back, but there was a 6 second lag before the connection would finally close. After a little research on the Google, it turns out I’m not the only one who’s observed this with PHP and IIS: http://www.google.com/search?q=iis+php+response+lag

I would like to say I figured out why IIS was being so lame, but the in-house net-admin suggested moving the application to another production server on hand, where the issue mysteriously dissappeared. At that point, everyone was happy, so I didn’t look into it any further. I did read about some resolutions regarding DNS settings, and skipping name resolutions in the db configurations. That might have been it, but I’ll never know.

Recap:

Don’t willingly try this. That being said, everything worked out much better than it could have, and there were relatively easy solutions to the bumps along the way. If you have to put together a project like this, I highly suggest using some sort of database abstraction (or a framework like CodeIgniter), and planning out how you’ll work out URLs, permissions, and differing environments. Thinking these things over will save you a lot of time, and hopefully leave the stakeholders a whole lot happier.

A couple of weeks ago, Derek Jones of EllisLabs announced that the CodeIgniter Core would officially be branched so a community-driven version of the framework could be created. This new branch will be called CodeIgniter Reactor.

The plan, which was clarified today, is focused on allowing a select group of CI engineers to actively commit new features to the codebase, while taking feature requests via a uservoice forum. Over time, any proven, time-tested features in Reactor will be merged into the Core.

This announcement comes after a two year period of relatively few changes to CodeIgniter — something the community has had mixed feelings about. For some, it meant CodeIgniter would remain a remarkably stable platform to build on in contrast to frameworks like Cake or Kohana. For others, it meant CodeIgniter was probably dead.

So is that why frameworks like Kohana, once a fork of CodeIgniter, exist?

That, and also: A frustrating reality for many in the CodeIgniter community is that although it is the fastest, lightest MVC framework for PHP, it still lacks some important features. Many once-smitten CodeIgniter evangelists grew out of the framework and began looking for something that was more full featured. That is, they wanted an ORM, more libraries, more robust implementations of unit testing and caching, and a style of usage that actually conforms to the definition of MVC.

Why does CodeIgniter still have such a loyal following if more full-featured frameworks are around?

As mentioned, CodeIgniter is considered one of the fastest script-based frameworks available. It takes a “load only what you need” approach — something other platforms definitely don’t do. Rasmus Ledorf, the original author of PHP and skeptic of PHP frameworks in general, has even singled out the framework as something he might actually use.

As a side note, while at Yahoo! I’ve had the chance to read into exactly why Rasmus doesn’t like MVC via the internal wiki pages that he wrote himself — and guess what, he’s right. But that’s a topic for another post.

The other big (if not HUGE) reason that CodeIgniter has remained popular is the documentation. You cannot find a framework with more clear or concise documentation. Of course, it’s easy to keep the docs current when the framework hasn’t changed for two years!

So what the hell is going to happen with CodeIgniter Reactor?

I’m one of the engineers on the project, and I’ll be clear that CodeIgniter Reactor will pick up where CodeIgniter left off. Although the official roadmap hasn’t been laid out just yet, you’ll find Reactor will still have a high level of dependability and outstanding documentation. If Reactor’s docs fall into dismay, hold me personally responsible. I won’t let it happen.

There is a feature request forum for Reactor set up at UserVoice — a place where CI users who want to suggest additions can go.

Of course, that doesn’t mean every feature suggestion, whether it’s the popular or not should actually be implemented. The Reactor team will make intelligent decisions about what and what not to change, remaining in line with the original principals of CodeIgniter: Light footprint, low barrier to entry.

If I had to underscore my own objectives in implementing new features, they are speed, efficiency, and power. That is:

• Does it belong in the framework?
• Can this feature be built efficiently?
• At the highest level of efficiency, will it be fast?
• Is it comparably robust, and powerful in the face of similar features in other frameworks?

A word to the skeptics:

A tweet I saw immediately following the Reactor Announcement was “Death by bureaucracy is the fate for CodeIgniter shame … Good riddance, it never matured”. This really embodies the sentiment most users have about other frameworks. It’s arguably the fate that Kohana and Cake have seen.

What I’m saying is: stuff away the cynicism until the first quarter of 2011. You’re going to see a ton a positive changes in the framework, and CodeIgniter users will have plenty to brag about without sacrificing usability and documentation.

There are 6 engineers on the dev team — one who I know very well, and a few others who I’ve met just recently. In all honestly, I’ve been very impressed by everyone, and I couldn’t imagine anything but a bright future for the framework. The other devs are:

• John Crepezzi
• Ed Finkler
• Mark Huot
• Kenny Meyers
• Phil Sturgeon

Six developers can approach the “too many cooks” scenario, but I think you’d also have to recognize that CodeIgniter is supported by volunteers, and a project needs 5 or 6 people in order maintain consistently high activity when they’re generally working on it part-time.

Okay, so now what:

Over the next few weeks, the Reactor team will be getting organized and planning out the future of CodeIgniter. Check back here or at my Twitter feed, because I’ll definitely post a podcast outlining exactly what the Reactor team plans to do. Also, I tend to lurk around the IRC #codeigniter channel on freenode as _kennyk_ .

If you have any questions, comments, or general skepticism, comment below. I’m all-ears.

I’m definitely not the type to evangelize something I don’t think is useful. Ask friends of mine, and they’ll likely tell you how I went through phases where I endlessly promoted things like Notepad++ for Windows, Netbeans IDE (PHP), Sequel Pro, Gnome-Do, Thinkpads, Macbooks, Toy Story 3, iPod Touches, and Visual Studio ’08. I just can’t help it. When I get excited about something, I have a hard time stfu-ing.

But holy crap. Let me tell you — if you think the WordPress default search sucks — as I did 2 years ago, try WPSearch. It’s (IMO) the second-most-useful plug-in that has ever existed for WordPress. It also has the slickest admin UI I’ve ever used in WordPress (a big thanks to JQuery and it’s plug-ins, of course).

I wrote WPSearch, and I 100% recommend it to anyone running something that isn’t a traditional WordPress blog. Run a recipe catalog? Use It! Review engine? Use it! Shopping engine? What the hell are you doing without my plug-in on your site? Your customers can’t find shit!

Fun scenario: Let’s imagine you sold laser printers on your site along with other computer peripherals, and every product page on your site had “[brand] [model] Printers” in it’s title, like “Dell 1100 Laser Printer”. If someone searched for “printers” with default WordPress search, they wouldn’t get those products back.

Does that scare you? It should. Get WPSearch now. And if you don’t like it, tweet @_kennyk_ and be brutally honest. I can take it.

Side note #1: When I wrote WPSearch, I was also trying to cash-in big time at a programming contest where the grand prize was a trip to RailsConf and the runner-up would get a set of steak knives (presumably to stab whoever won the grand prize).

Side note #2: Toy Story 3 is an intellectually provocative and introspective masterpiece!

Side note #3: Askimet is #1, but is tragically taken for granted

If you want to read about the background of the WPSearch Search plug-in for WordPress, read below. But if you just want the gist of this post, here it is:

WPSearch is the best search plug-in for your WordPress blog. It is a stemming, stop-word blocking, fast, relevant, fulltext search for WordPress. There isn’t a single plug-in in the WordPress repository that can do what it does.

You can get it here: http://wordpress.org/extend/plugins/wpsearch/ — or just install it through the WordPress plugin administration backend. Just search for WPSearch.

If you run any sort of monetized blog, you might be losing sales or readership if users can’t find what they need on your site. According to Adotas, 43% of users hit the search box first.

Background:

I’ll admit that I’ve been pretty vocal already about WordPress’ lacking search functionality. I’ve been vocal about it for 2 years, and it was the original impetus to get this blog up and running. It was also the reason that I wrote wpSearch, the original version of WPSearch — the lucene-based search plug-in for WordPress. wpSearch was the first true fulltext search for WordPress, in my opinion.

But the problem with wpSearch was that it wasn’t highly engineered. I had written it for a programming contest, and I was on a tight deadline — and when projects are rushed, the quality of code goes down. And because of that, bugs made their way out of the woodwork over the next two years.

Consumed by college work and my job, I didn’t have much time to address those issues. In fact, I had declared wpSearch unsupported a year after it was released in 2009.

A year following that, Daniel Hay at Pixelberry in New Zealand had requested that I add the ability of searching within a category for a client of his. That’s when WPSearch 2 development began.

I wanted to give wpSearch a full rewrite, and correct several mistakes I made with the first version:

• Rename it from wpSearch to WPSearch
• Change the listing name in the WordPress repository to WP Search, so searches for ‘search’ would bring it up
• Follow and MVC pattern (it’s a complex plug-in)
• Build a configurable search driver framework, so any driver could be written to search and index
• Build better logging
• Evangelize it to no end (wpSearch was barely promoted)
• Give it a UI independent of the default WordPress stylesheets, and also make it stylish

The biggest change is the configurable driver part. The free version of WPSearch contains a driver that uses Zend Lucene in the background. Any driver, however, can be written to work with WPSearch. Drivers for SOLR, sphinx, the Google Search Appliance, or name-your-own-search-product could be written.

I did that because PHP is not the best language to write a search engine in. Since Zend_Search_Lucene is the backend driver of the free version, there is an upper-bound of scalability on the plug-in. After all PHP is a scripting language, and I doubt Zend ever really imagined someone would make the ludicrous decision of indexing tens of thousands of posts in PHP. I found the breaking point to be about 20,000 docs. At that point, I ran into memory issues, slow mid-indexing optimizations, and slow first-hit (non-cached) searches.

So the point is that I poured everything I had into WPSearch 2, and I want to tell everyone about it. I did this project under the umbrella of OConf, my start-up, with business partner John Crepezzi. John’s an ex-engineer at Sun Microsystems, and he spends his days at Patch.com now. He also wrote the backend driver for WPSearch Pro, an alternate driver for WPSearch 2 which can handle up to 500,000 docs.

John and I gave a talk at Wordcamp NYC, where we officially launched WPSearch 2.The topic was on the default WordPress search, and why avoiding a remedy to it can lose you both readers and money. If you run a shopping engine, people can’t find your products. If you run a news site, readers can’t find your content.

If you don’t think people even use the search box — heads up, the advertising gurus at Adotas say 43% of users who find your site do.

You can check it out here: http://wordpress.org/extend/plugins/wpsearch/ . In it’s short time in the repository (about 9 days), it’s already had around 1,000 downloads, and I’ve had a lot of positive feedback coming in. I also dropped a comment about it on WordPress founder Matt Mullenweg’s site, where he mentioned a search product for Drupal. Hopefully he’ll check it out and let me know what he thinks.

Check it out!

CodeIgniter’s output caching mechanism — at least in my opinion — has limited usefulness. It can be used to cache the final payload sent to the user for a given number of minutes. But sometimes clearing the cache for all pages or a specific page can be useful, especially if keeping the user from seeing stale data is important.

This post will discuss what caching is, and how it is used in CodeIgniter. The next post will provide a few helper methods for clearing caches for specific pages (or really, URIs). I think this will make CI’s caching feature a lot more usable.

In a nutshell, this is how (and why) output caching is typically implemented:

Suppose we have a page at http://mysite.com/stats/website which loads and shows all the statistics about the users on my site, activity, files posted, etc. In other words, a lot of SQL queries (maybe 40 or so) and calculations go into this page. That takes significant time and resources.

Now imagine that this page is particularly popular. Maybe a heavily-read blog or bookmarking service links to your website, and you receive an unexpected burst in traffic. All of those page loads are re-running your calculations and queries, and that is consuming serious memory and CPU cycles. The number of requests your website can serve per-minute is now falling through the floor!

The idea of caching involves using the fact that our page probably doesn’t need calculate those statistics or run those queries every time. Perhaps gathering all of that data once an hour would suffice.

“Caching” involves generating the statistics page like we normally would, but saving a copy of the output we were going to send to the user for a certain amount of time. During that time period, everyone who requests that page will get the same page output that we generated x minutes ago.

After that time period passes, and we receive another page request, we recalculate the stats, and send them to the user. Of course, we’ll save the newer stats for a given amount of time for subsequent requests.

This basic caching — “page caching” — is the kind that CodeIgniter offers.

When you are inside a controller, you can tell CodeIgniter that you want to cache any requests to the current URI for n minutes. For that time period, you only want to send the output your are about to generate.

This is how it looks:

class Stats extends Controller
{
  function website()
  {
    // 40 SQL Queries (Maybe a little overboard)
    // Serious, ultra-precise calculations
    // More intensive code that isn't good for fast page loads
 
    // Tell CI you want to cache the output of this page for 10 minutes.
    // This means that for the next 10 minutes, when CodeIgniter receives
    // a request for stats/website, the CI system will not even consult this
    // method or instantiate this controller. It will serve a static page containing
    // the exact content that you output here.
    $this->output->cache(10);
 
    $this->load->view('stats/website', $data);
  }
}

And that’s all you have to do! In this example, CodeIgniter’s output caching works great.

So then, what were the qualms I mentioned earlier?

Suppose we also cache user profile pages, which might be located at a URI similar to http://mysite.com/users/profile/kennyk. For whatever reason, we will assume that caching this page could be very beneficial due to heavy traffic on the site.

If a user profile is cached for 10 minutes, and a user updates his profile in some profile editing area, he will still see the older version of his profile page when he tries to view it. This can be confusing and inconvenient.

Other MVC frameworks (like Rails) handle this issue in a snap by allowing the developer to “sweep” caches at will. CodeIgniter does not offer any functionality similar to this.

My next post will document a cache helper I’ve whipped up to allow CodeIgniter to sweep caches for a given URI. Look out for “Flushing CodeIgniter’s URI-based Cache (Part II).”

A very cool part of CodeIgniter is its ability to give you the ‘profiling’ information for page loads. That is, if you add:

$this->output->enable_profiler(true);

In your controller before you load a view, CodeIgniter will give you information regarding how fast the page loaded, how many SQL queries executed, the content of each query, and the running time of each query. This is incredibly useful when you are trying to debug your application, or simply see how quickly things are loading.

There’s only one problem: To enable profiling, that line of code above must be present. What if you want to profile several pages, or even your whole web application? In that case, you have to start thinking:

• I could put that line in the constructor of my controller, and then of of that controller’s methods will be profiled.
• I could put that line in each method I want to profile.

These methods start to get ugly. And of course, you don’t want to comment out each profiling line when you don’t need them.

My method to tackle this problem involves using CodeIgniter’s Hooks feature to enable or disable profiling for the entire web applications based on a value in the configuration file. In the end, I can turn on profiling for my entire website via a config value by setting it to true or false. This is my method:

1. Enable hooks in your CodeIgniter Application by going to config/config.php end setting the flag to true:

$config['enable_hooks'] = TRUE;

2. Create a file in the config directory named hooks.php if it does not already exist. Inside it, place:

$hook['post_controller_constructor'][] = array(
                                'class'    => 'ProfilerEnabler',
                                'function' => 'EnableProfiler',
                                'filename' => 'hooks.classes.php',
                                'filepath' => 'hooks',
                                'params'   => array()
                                );

3. Create a folder in your application directory named hooks (If it does not already exist). Inside it, create a file named hooks.classes.php. Inside it, put:

class ProfilerEnabler
{
	function EnableProfiler()
	{
		$CI = &get_instance();
		$CI->output->enable_profiler( config_item('enable_profiling') );
	}
}

4. Finally, add this line to your application’s main configuration file:

$config['enable_profiling'] = false;

Setting this value to true will enable profiling across your entire website. Now, you can zip through pages, checking the load times of each without any trouble!

Note: I’ve always thought that an additional way to do this would be to extend the Controller class and enable profiling from there. But this method, I think, has the least impact on existing code.

After an exhausting week and a half tracking down the source of a mysterious bug in wpSearch, I think I can finally close the book on the “null result” issue that had me pouring over the source code.

wpSearch 1.5.0.5, the first official release after the 1.5 landmark, brings to the forefront some of the features and fixes slated in the last post. wpSearch 1.5 has had the following features implemented:

• Comment Searching
• A behind-the-scenes event logger for easily figuring out user issues
• An upgrade to the underlying Lucene Search
• An upgrade to the underlying StandardAnalyzer (used for relevancy)

And these fixes:

• No more null results after a post is edited
• Foreign character support (or simply indexing content with ‘UTF-8′ encoding
• Memory issues for content-heavy posts

wpSearch 1.5.0.5 is a rock-solid release that is starting to make a name for itself in the WordPress world. The new ‘Phone Home’ feature in wpSearch allows users to report their copy of wpSearch. A few of the blogs with wpSearch currently in use are listed here:

• Building The Ergonomic Guitar
• ComputerBob
• Savoring Kentucky

Patrick Cushing at the EnterVenture blog wrote a very detailed comparision of the default WordPress search’s relevancy vs. wpSearch’s. This article ended up at digg.

Of course, as far as wpSearch has come in its short lifespan, there exists a set of users that deserve credit for pointing out issues and keeping me informed of bugs, needed features, etc.. So, in no particular order, I would like to thank:

• ComputerBob, at ComputerBob.com for pointing out the first instance of the empty result issue. He has thoroughly documented his usage with wpSearch at his blog, in a fair and balanced fashion. Furthermore, he has sent his index data back with detailed comments when most users would simply give up on wpSearch. Thanks ComputerBob.
• Robert Irizarry, who has kept the wpSearch thread at the WordPress repository stuffed with feature ideas and issue notices.
• Olivier, who’s 6000 posts provided the first failed scalability test for wpSearch. His pointing out of this issue led to a change to allow for greater scalability — in other words, wpSearch 1.5 was tested successfully up to 7,000 posts. Great dedication to detailing these issues has helped wpSearch greatly.
• Karl Heigl, who first mentioned the fact the wpSearch was not handling German accents, and subsequently all foreign (to the U.S.) characters. This also ended up affecting Olivier. This bug was fixed in 1.5.0.5. Thanks Karl!
• A user named Brian, said, “Thanks for the update. If you need any other information or even help testing, I’d be happy to assist. Just let me know. ” Thanks for your support Brian.
• And to all those who have donated to this project so far!

So, wpSearch 1.5.0.5 wouldn’t be at it’s current status if it weren’t for those supporting it.

Features coming up for wpSearch include result highlighting, contextual snippets, and a progress meter for index building.  I encourage everyone who is reading this but hasn’t installed wpSearch yet to try it out, and see the awesome blog search that you’ve been missing.