I’m definitely not the type to evangelize something I don’t think is useful. Ask friends of mine, and they’ll likely tell you how I went through phases where I endlessly promoted things like Notepad++ for Windows, Netbeans IDE (PHP), Sequel Pro, Gnome-Do, Thinkpads, Macbooks, Toy Story 3, iPod Touches, and Visual Studio ’08. I just can’t help it. When I get excited about something, I have a hard time stfu-ing.

But holy crap. Let me tell you — if you think the WordPress default search sucks — as I did 2 years ago, try WPSearch. It’s (IMO) the second-most-useful plug-in that has ever existed for WordPress. It also has the slickest admin UI I’ve ever used in WordPress (a big thanks to JQuery and it’s plug-ins, of course).

I wrote WPSearch, and I 100% recommend it to anyone running something that isn’t a traditional WordPress blog. Run a recipe catalog? Use It! Review engine? Use it! Shopping engine? What the hell are you doing without my plug-in on your site? Your customers can’t find shit!

Fun scenario: Let’s imagine you sold laser printers on your site along with other computer peripherals, and every product page on your site had “[brand] [model] Printers” in it’s title, like “Dell 1100 Laser Printer”. If someone searched for “printers” with default WordPress search, they wouldn’t get those products back.

Does that scare you? It should. Get WPSearch now. And if you don’t like it, tweet @_kennyk_ and be brutally honest. I can take it.

Side note #1: When I wrote WPSearch, I was also trying to cash-in big time at a programming contest where the grand prize was a trip to RailsConf and the runner-up would get a set of steak knives (presumably to stab whoever won the grand prize).

Side note #2: Toy Story 3 is an intellectually provocative and introspective masterpiece!

Side note #3: Askimet is #1, but is tragically taken for granted

If you want to read about the background of the WPSearch Search plug-in for WordPress, read below. But if you just want the gist of this post, here it is:

WPSearch is the best search plug-in for your WordPress blog. It is a stemming, stop-word blocking, fast, relevant, fulltext search for WordPress. There isn’t a single plug-in in the WordPress repository that can do what it does.

You can get it here: http://wordpress.org/extend/plugins/wpsearch/ — or just install it through the WordPress plugin administration backend. Just search for WPSearch.

If you run any sort of monetized blog, you might be losing sales or readership if users can’t find what they need on your site. According to Adotas, 43% of users hit the search box first.

Background:

I’ll admit that I’ve been pretty vocal already about WordPress’ lacking search functionality. I’ve been vocal about it for 2 years, and it was the original impetus to get this blog up and running. It was also the reason that I wrote wpSearch, the original version of WPSearch — the lucene-based search plug-in for WordPress. wpSearch was the first true fulltext search for WordPress, in my opinion.

But the problem with wpSearch was that it wasn’t highly engineered. I had written it for a programming contest, and I was on a tight deadline — and when projects are rushed, the quality of code goes down. And because of that, bugs made their way out of the woodwork over the next two years.

Consumed by college work and my job, I didn’t have much time to address those issues. In fact, I had declared wpSearch unsupported a year after it was released in 2009.

A year following that, Daniel Hay at Pixelberry in New Zealand had requested that I add the ability of searching within a category for a client of his. That’s when WPSearch 2 development began.

I wanted to give wpSearch a full rewrite, and correct several mistakes I made with the first version:

• Rename it from wpSearch to WPSearch
• Change the listing name in the WordPress repository to WP Search, so searches for ‘search’ would bring it up
• Follow and MVC pattern (it’s a complex plug-in)
• Build a configurable search driver framework, so any driver could be written to search and index
• Build better logging
• Evangelize it to no end (wpSearch was barely promoted)
• Give it a UI independent of the default WordPress stylesheets, and also make it stylish

The biggest change is the configurable driver part. The free version of WPSearch contains a driver that uses Zend Lucene in the background. Any driver, however, can be written to work with WPSearch. Drivers for SOLR, sphinx, the Google Search Appliance, or name-your-own-search-product could be written.

I did that because PHP is not the best language to write a search engine in. Since Zend_Search_Lucene is the backend driver of the free version, there is an upper-bound of scalability on the plug-in. After all PHP is a scripting language, and I doubt Zend ever really imagined someone would make the ludicrous decision of indexing tens of thousands of posts in PHP. I found the breaking point to be about 20,000 docs. At that point, I ran into memory issues, slow mid-indexing optimizations, and slow first-hit (non-cached) searches.

So the point is that I poured everything I had into WPSearch 2, and I want to tell everyone about it. I did this project under the umbrella of OConf, my start-up, with business partner John Crepezzi. John’s an ex-engineer at Sun Microsystems, and he spends his days at Patch.com now. He also wrote the backend driver for WPSearch Pro, an alternate driver for WPSearch 2 which can handle up to 500,000 docs.

John and I gave a talk at Wordcamp NYC, where we officially launched WPSearch 2.The topic was on the default WordPress search, and why avoiding a remedy to it can lose you both readers and money. If you run a shopping engine, people can’t find your products. If you run a news site, readers can’t find your content.

If you don’t think people even use the search box — heads up, the advertising gurus at Adotas say 43% of users who find your site do.

You can check it out here: http://wordpress.org/extend/plugins/wpsearch/ . In it’s short time in the repository (about 9 days), it’s already had around 1,000 downloads, and I’ve had a lot of positive feedback coming in. I also dropped a comment about it on WordPress founder Matt Mullenweg’s site, where he mentioned a search product for Drupal. Hopefully he’ll check it out and let me know what he thinks.

Check it out!

After an exhausting week and a half tracking down the source of a mysterious bug in wpSearch, I think I can finally close the book on the “null result” issue that had me pouring over the source code.

wpSearch 1.5.0.5, the first official release after the 1.5 landmark, brings to the forefront some of the features and fixes slated in the last post. wpSearch 1.5 has had the following features implemented:

• Comment Searching
• A behind-the-scenes event logger for easily figuring out user issues
• An upgrade to the underlying Lucene Search
• An upgrade to the underlying StandardAnalyzer (used for relevancy)

And these fixes:

• No more null results after a post is edited
• Foreign character support (or simply indexing content with ‘UTF-8′ encoding
• Memory issues for content-heavy posts

wpSearch 1.5.0.5 is a rock-solid release that is starting to make a name for itself in the WordPress world. The new ‘Phone Home’ feature in wpSearch allows users to report their copy of wpSearch. A few of the blogs with wpSearch currently in use are listed here:

• Building The Ergonomic Guitar
• ComputerBob
• Savoring Kentucky

Patrick Cushing at the EnterVenture blog wrote a very detailed comparision of the default WordPress search’s relevancy vs. wpSearch’s. This article ended up at digg.

Of course, as far as wpSearch has come in its short lifespan, there exists a set of users that deserve credit for pointing out issues and keeping me informed of bugs, needed features, etc.. So, in no particular order, I would like to thank:

• ComputerBob, at ComputerBob.com for pointing out the first instance of the empty result issue. He has thoroughly documented his usage with wpSearch at his blog, in a fair and balanced fashion. Furthermore, he has sent his index data back with detailed comments when most users would simply give up on wpSearch. Thanks ComputerBob.
• Robert Irizarry, who has kept the wpSearch thread at the WordPress repository stuffed with feature ideas and issue notices.
• Olivier, who’s 6000 posts provided the first failed scalability test for wpSearch. His pointing out of this issue led to a change to allow for greater scalability — in other words, wpSearch 1.5 was tested successfully up to 7,000 posts. Great dedication to detailing these issues has helped wpSearch greatly.
• Karl Heigl, who first mentioned the fact the wpSearch was not handling German accents, and subsequently all foreign (to the U.S.) characters. This also ended up affecting Olivier. This bug was fixed in 1.5.0.5. Thanks Karl!
• A user named Brian, said, “Thanks for the update. If you need any other information or even help testing, I’d be happy to assist. Just let me know. ” Thanks for your support Brian.
• And to all those who have donated to this project so far!

So, wpSearch 1.5.0.5 wouldn’t be at it’s current status if it weren’t for those supporting it.

Features coming up for wpSearch include result highlighting, contextual snippets, and a progress meter for index building.  I encourage everyone who is reading this but hasn’t installed wpSearch yet to try it out, and see the awesome blog search that you’ve been missing.

After its first week in the wild, wpSearch has been run on a number of different versions of WordPress and PHP, highlighting some places to improve aspects of its core. wpSearch 1.5 has just been released, with a completely rewritten search mechanism to bring search speeds into the milliseconds.

Certain features available in wpSearch 1.x.x.x have been removed in favor of tighter integration with the WordPress core and raw speed. The search popup is no longer an option, removing the need for 2 javascript libraries, 2 CSS files, and 4 images. This decreases the page load time by close to 500 ms for a first-time page view over a broadband connection.

wpSearch now integrates its results into the WordPress search using pure WordPress API. Here are some statistics from version 1.5:

These stats were gathered on a non-dedicated development server with 1 GB Ram, 3.2 Ghtz Hyperthreaded Intel P4, with Windows XP, and a WAMP installation without any sort of code caching (like Zend Optimizer). Needless to say, this server isn’t the quickest, but it still turns out very impressive search times.

wpSearch 1.5 is also completely compatible with the latest release of WordPress, 2.6.

Get it here: http://wordpress.org/extend/plugins/wpsearch/

Keep the comments coming!

katzgrau@gmail.com

wpSearch (more info in my previous post), the lucene-powered search plugin for WordPress, has officially been accepted into the WordPress plugins repository. You can view and download wpSearch here:

http://wordpress.org/extend/plugins/wpsearch/

The latest version as of right now is 1.1.0.0. Several major features have been added since the original beta release.

• Seamless integration of wpSearch into your blog. After you activate wpSearch and build your blog’s search index, the search box on your blog will now be configured to use wpSearch for searches.
• You can now decide whether you want search results in the page ( the standard ), or have them loaded into and AJAX search pop-up. (Originally, the AJAX pop-up was the only way to view results ). This option is configurable via the WordPress admin screen.
• Bloggers can now tweak the importance of things such as title, content, and tags in a blog search. This effectively allows control over what is considered relevant in a blog search.

So what’s next for wpSearch?

More searchable content: It’s no secret that the best content on a blog is sometimes in the comments. This is especially true for bloggers of tech and programming sites where blog readers often put useful contributions in comments.

The opening of the source: At SourceForge! Sure, PHP is inherently open-source (it’s a scripting language, after all!). But the best future for wpSearch would entail its placement into SourceForge.NET where the coding community can have the opportunity to contribute to the wpSearch project. wpSearch is already registered at SourceForge, and has a project page at:

http://wpsearch.sourceforge.net/. (Right now, there isn’t much setup up).

I plan to have wpSearch developed at SourceForge, and have stable releases be uploaded to the plug-in repository at WordPress.

There are some other features I plan to add to wpSearch very shortly, one of which is contextual search result content, so you can see the words around the matching content of a search result.

I can’t think of the others off the top of my head. What I would really like to know is if anyone finds wpSearch to be of value so far, and whether they are having any difficulties.

I read on another blog that blogs get xx% more comments if the words “Have your say” are at the end of a post. I think I’ll try that.

Have your say!