On Versioning Local Packages Like node_modules and sparks

A recent trend in package management is to install packages locally to an application. Composer for PHP is doing it by default, npm for node.js is doing it by default, and a few others are following along.

You know who did that schnazz before both of them? GetSparks. Yep, the GetSparks team did it before it was cool.

Now, I’m not going to say “I told you so,” but there were some serious doubters — there were even dissenters on the team. But it ultimately boiled down to that fact that global/system-level packages are often painful to deal with. I honestly think those opposed to local packages just wanted to stick to what they were comfortable with.

I understand versioning someone else’s packages seems a little redundant at first, but in reality, what other qualms are there? If you install system-level packages, you invite yourself to dependency hell — and all of the sudden you need extra hacks to make it work right.

Reality: rvm for ruby and virtualenv for python are workarounds for a problem.

I love when things “just work.” If I can check out a repository and just run `node app.js`, I’m happy. If I’m putzing around with rvm, virtualenv, bundler, or pip, I’m annoyed.

Looking back, I’m glad we made that choice with GetSparks — way back in January 2011, before node and Composer.


Package code becomes your responsibility, and it shoudn’t be treated like a black box. Having packages installed locally invites you to poke around and understand what’s going on, instead of just opening an issue on github when problems arise.

PPS: I agree with this guy when it comes to versioning node packages. Many still don’t.

PPPS: What, is the npm registry going stay online 30 years from now, when your code is mission-critical legacy crap that nobody except $250/hr consultants will touch? Version those modules.

Posted in CodeIgniter, PHP Development | Tagged , , , , , , | Leave a comment

Startups, College, and Blue-Collar Ideas of Risk

At the start of this year I left my gig at Yahoo! (before the meltdown) to go off on my own. Most people who have taken “the plunge” (I’ve taken it once before) are met with a series of doubts and feelings of disbelief from family and close friends. “But what will you do for money!? What about healthcare?!” Yada, Yada, Yada.

The point of this rant(ish) piece is to highlight some very backwards ways of thinking about Startups, Risk, and College in the traditional middle class sense.

Coming from blue-collar past, I’ve realized that a lot of my friends and family come from union-type vocations and upbringings that essentially rely on an employer to provide for them. Sell your labor during the day, take home a paycheck at the end of the week, and hope you don’t get laid off or go on strike.

Who wants that? Well, I guess a lot of us. But it’s boring ride where your earnings are capped, and the only excitement comes from the occasional fear of whether you’ll need to look for a job other then the one you’ve held for 20 years. But other than that, it’s safe. Work for the postal service, utility company, or take up some other trade, and you’ll get by.

So that “money” question you might get asked when you think about joining a startup? It’s rooted in fear and the general idea of not having some sort of safety net. I think those safety-concerned folks hedge their bets to such a degree that they ultimately hurt themselves. And do you know what is generally viewed as the safest bet of all? Education. One or many degrees will become safety nets for life.


So that blue-collar crew I was just profiling — an education for them, specifically a college education, is the ticket out of the paycheck-to-paycheck, debt-ridden hell that defines the class. Work hard, get a degree, get a good job somewhere, and you’ll be alright.

Those same people who advocated that you keep your job and play it safe also suggest that you take what is actually a massive gamble at the age of 18: taking out student loans to get a college education. But this is pitched as an “investment.”

Whenever I talk to some high school student about college, I wish I could impress on them the insanity of big student loans. You might be taking out 6-digit unforgivable chunk of cash, hoping you can pay it back alongside your regular living expenses post-college. That’s not easy (watch that clip).

So let’s compare that to joining a startup. Let’s say you make $100k / year at your current job, and you leave to join a startup where you need to go half salary, either funded by the startup, or by freelance work to keep you afloat. You aren’t saving any cash, but you can afford living expenses at 0% margin.

At the startup, you are getting a real-world education that no business school can provide, and the only cost is opportunity cost. For the sake of argument, your startup didn’t do so well, and after 4 years your loss in opportunity cost is $200-250k, but you owe $0 to anyone for your time.

Bachelor’s degrees, Master’s degrees, etc: They don’t entitle you to anything, and you generally acquire them at the cost of tens of thousands in debt. You gave up 2-6 years of your life learning at a fantastically slow rate, only to find an exit where you:

  • Owe some loan shark of a company like Sallie Mae $20-250k (depending on the degree(s))
  • Potentially have the same loss in opportunity cost
  • Need to go find a job, because it turns out degrees aren’t tickets for jobs
  • Have no practical experience anyway, so people will hire you at a minimal salary if and when you do find a job

The point is: College is one of the biggest gambles you’ll ever take, and the risk involved in joining a startup is laughable compared to it. Next time you think about quitting your job and going back to school, join a startup instead. And if you didn’t go to school yet — put some serious thought into how much of a financial setbeck you really want when your 4 years are up.

Posted in Jobs, Life, Soap Box, Startups | Tagged , , | 6 Responses

Is Your WordPress Slow? Come See My Talk at Wordcamp NYC

WordCampNYC – June 9-10

I’ve always had a thing for the ops side of building apps — automating, optimizing, and building for scale (did it for iVillage and Yahoo among others). And let me tell you, a standard WordPress installation leaves plenty of optimization and scaling to think about.

It’s not that the WordPress core itself is poorly written (although I wouldn’t exactly like to put my name on it). With all the functionality WordPress offers (initializing themes, loading active plugin, providing and array of simple helper functions that actually do complex things), it’s not really surprising that a standard installation gets only 30-ish requests per second out of the box.

And when you start adding plugins, that number can easily drop to the single digits. At that point, there are probably a hundred or so MySQL queries executing every request — That’s madness, but it’s an ugly reality of modular CMSs like Drupal and WordPress. But there’s hope: With appropriate caching solutions, smart management of plugins, and server-side tuning, we can get some wind under those WordPress wings and see it fly.

I’m giving a talk at Wordcamp NYC, and the title of the session is “Making WordPress Fly: Optimizing Your Servers for Performance”. It’ll be along the same lines as my LAMP performance talk at CICON, but obviously centered around tuning specifically for WordPress.

If you’ll be there, and you’re interested in making your instance resilient in the face of traffic spikes, swing by! Also, feel free to drop me a line and ask me any performance questions ahead of time so I can work some answers in ahead of time.

Posted in CodeIgniter, PHP Development, Wordpress Development | Tagged , , | Leave a comment

Startups: Gold Diggers vs. Real Entrepreneurs

NYC Tech is exploding (or has exploded), and I’m constantly hearing about new startups and their ideas. What’s really strange is that, at least on the surface, a lot of these ideas don’t sound like great ones. So for you soon-to-be entrepreneurs out there, I want to make my case for building a real business as opposed to “gold digging”.

I have about 10 ideas for a new app or project per week, and only occasionally will one stick in my mind as a world changer. I’ll think about it all day, draw out some stuff on paper, and do my best to dream about it when I sleep. What do all of those ideas have in common? They’re all things I need; solutions to problems I have.

How do I boil down those ideas to a few that I really care about? It takes about a 30 seconds each:

  1. Do other people need it?
  2. Will the people who need it buy it or at least generate enough revenue to make it worth the effort?

If the answer to #1 is no, then I really don’t go much further. I’ll definitely have ideas that satisfy this requirement, and why drain any energy doing something that is “neat,” but not life changing, and appears pretty risky in terms of ROI?

Need is an important thing. If I can’t feel the need instinctually, I have trouble focusing my energy canon on it. If I can’t see other people needing it, same deal.

#2 kills a lot of ideas. If something is needed, but it won’t make any money, I’m usually not interested. There are exceptions, like open source projects. Sometimes you just want to do something good for the world, and seeing other people use your project (and say nice things) is a much-appreciated bonus. And if it’s not a black hole of time or money, why not?

But not everyone, understandably, has the same filter. And there’s a temptress in tech right now: The allure of the 2 year old, 1 billion dollar startup. Thanks, Instagram.

It’s so common right now to hear tales out of the mouth of some high-flying “entrepreneur” who’s just in it to win it via acquisition. Some bozo of a founder who’s out for prestige and a payday, planning to make their own miraculous exit. They talk as if attracting a billion users is only a matter of time. Of course, they probably don’t let investors see that side.

Those guys bother me, and there’s a lot of them. It’s like they really don’t care about the actual product, or the need it fulfills, or whether it makes any money. It’s like they don’t care about building and managing a real, profitable business.

Are you starting a business? Think about your motivations. Do you truly, sincerely believe in what you’re doing? Could you see yourself doing the same thing in 5 years and sustaining yourself if you aren’t acquired? Do you see a better world for the space you’re in? If the answer to any of these questions is “no,” then you might just be a gold digger. It’s hard to build a company, recruit a team, and change the world when you’re one of those. Your lack of passion will bleed through, and the quality and effort you put into your product will reflect it.

So here’s the culmination of all this: If you want to start a company, build a real business. Make sure you’re passionate about what you do. Be genuine when dealing with everyone involved — your employees, investors, and the end users. Be sure that 100% of you is aligned with what you’re doing, and that you aren’t putting on a show for anyone. If you can do this, you’re much more of an entrepreneur than any of those gold diggers are.

PS: Be sure to follow through.

Posted in Jobs, Soap Box | Tagged , , | 2 Responses

Professor Thiel, I Have A Pitch for You

An article posted on Reuters today has announced that Peter Thiel, a famous opponent of the traditional university education — proposed, and is now instructing a course in entrepreneurship at Stanford — CS 183: Startup.

If you’re not familiar with Thiel, read up on him. He’s a serial entrepreneur whose resume is way too long to talk about here. In short, he co-founded Paypal and is “Don of the PayPal Mafia” — a group ex-PayPalers who seem to start billion-dollar businesses in their sleep.

Search for Thiel’s name in Google, and you’ll quickly become acquainted with his views on college: that it’s a big risk of time and money better spent starting a business and changing the world.

I agree. I agree so much that I have evangelized this exact idea to many-a-high-schooler before I ever knew who Peter Thiel was.

Only 7-years ago, I was an 18-year old kid born to blue-collar, union-member parents. I struggled over the idea of taking on 6 digits in student loans to attend college. I ultimately went to the state-school that offered me the most amount of money instead, and graduated with $0 to repay (and a near-perfect GPA, just saying).

I’ve shunned the thought of ever stepping foot on another campus to acquire more fancy paper. I subscribe to the idea that imagination, hard work, planning, and persistence trump all, and that ultimately, those factors will decide who moves and shakes.

That idea has its vicious opponents, as I’ve since found. Post-college, you are forced to grapple with the reality that a degree, or more specifically one from an elite school, offers its advantages. Too much bias exists when you don’t have a degree at all, and additionally, students from elite schools are recruited the most heavily, leverage powerful personal networks out of the gate, enjoy a lifetime of instant credibility, and maintain exclusive brotherhood with other alumni.

Having met and dealt with all that, I still strongly embrace Thiel’s general ideas. I don’t exactly think college should be skipped altogether, but I do think it is wildly overvalued and a big risk. It’s pitched by guidance counselors everywhere as a smart $xxx,000 “investment”, when it’s essentially a speculative gamble (sure hope I can pay back these loans once I break onto the art history scene!).

I believe prospective students should invest more time and effort believing in their own ideas before embarking on a four-year training course for the rat race and fitting themselves with financial handcuffs. But in their teenage years, who has that sort of clairvoyance?

So here is my question (and comments) for Peter:

Why are you teaching a course at Stanford?

What you’re doing is reenforcing the idea that the best opportunities will come from spending four years at an elite private school. If there are any high-schoolers who know who you are, you’re surely helping convince them that college is the right place to be.

I have a modest proposal — something you may have thought of already: Make CS 183 course content viewable via iTunes, YouTube, a blog, or some other source. Show everyone outside of Stanford that they didn’t lose out. Help spread the idea that hard work and dedication is most important — not spending time in a classroom.

What do you think?

PS: I’d rather this was some sort of course managed by the Thiel Foundation, but I also understand the visibility benefits if this is done through Stanford.

Posted in Life, Soap Box, Startups | 3 Responses

New PHP Meetup Group: Pro PHP NYC

Keeping in line with my insistence that PHP can be an excellent choice when developing an app, John Crepezzi and I have decided to organize a new Meetup Group in NYC that focuses on achieving high levels of quality when building applications for the LAMP* stack.

We felt that the other handful of PHP Meetup groups in the area cater to those just starting out more often than not. The engineers who are serious about building professional, enterprise-level applications don’t really have a group where they can consistently expect presentations on those topics.

We plan to build a group around subjects like testing, architecture, scaling and performance, optimization, and ops — the topics that are actually relevant to high-traffic, revenue-generating applications. The world has plenty of those applications built on PHP, but not enough Meetup groups for the developers who build or maintain them.

The group page is here: http://www.meetup.com/pro-php

You’re highly encouraged to join if quality and performance are your thing. The first Meetup will be at the end of March.

If you don’t know me and John, we’re both core committers to the CodeIgniter framework. We also founded GetSparks.org and did our time writing for scale at places like Yahoo and AOL. We can be found at fine eateries such as Dos Toros.

* A better acronym for what we’re targeting would probably be: L**P

Posted in Uncategorized | Leave a comment

The Top 10 CodeIgniter Sparks of 2011

John Crepezzi and I launched GetSparks.org (the CodeIgniter package manager and repository) a little under a year ago, and the response we received from the community was overwhelmingly positive.

The best part of GetSparks isn’t the site itself. It’s a moderately simple app that provides a vehicle for quickly dropping other developers’ code in your codebase.

Many of the packages on GetSparks are very well maintained. I am continually impressed by the amount of effort spark developers pour into their submissions when I peruse the site and try new packages out. One example would be WanWizard‘s DataMapper-ORM: Look at how well thought out it is, and how detailed the docs are.

Another would be codeigniter-payments. I rolled in to the second day of CICon 2011 NYC to find @calvinfroedge talking about a new payment spark he had made — one of the most in-depth, flexible payment libs I’ve ever seen.

GetSparks has almost clocked 50,000 package downloads at this point, but there are handful of sparks that have really stood out in terms of popularity. They are the year’s top 10:

name contributor installs
php-activerecord machuga 1806
curl philsturgeon 1361
template philsturgeon 1010
ion_auth benedmunds 990
restclient philsturgeon 931
DataMapper-ORM WanWizard 748
gravatar_helper seejohnrun 688
assets bstrahija 636
fire_log dperrymorrow 623
markdown katzgrau 622

And of course, there are people whose contributions have made GetSparks.org the success it is. A special thanks goes out to:

  • Beau Frusetta — First team member (retired), and the guy who made GetSparks look halfway decent
  • Mike Pauley — Team member
  • Sean Downey — Team member
  • Joe Auty — Team member
  • Jake Ingman — Who made us a sweet logo
  • Joel Cox — For some fine features and great sparks
  • Balsamiq — Who donated a copy of their mockup software
  • Spicer Matthews of CloudManic, who contributed the “forking” feature (which I have big plans for)

Happy New Year everyone. Keep up the awesome work, and keep knocking out those timeshanks when developing a new app with new sparks (things like payments, emailing, auth, assets, etc).


Posted in CodeIgniter, PHP Development, Tools | 4 Responses

Your Awesome Hack That Nobody Ever Heard Of

Something that has always come naturally to me (I’m not saying it’s something I’m great at) is promotion of my projects. When I wrote chip a couple months ago, I wanted to spread the word and evangelize it. I presented it at Hack and Tell in NYC, made a screencast, and pretty much inundated my co-workers at Yahoo! with tips and tricks on how to use it. I submitted it to Linux Journal, and now I’ll apply to couple sys-admin confs to present the next version.

Y! folks: `yinst install chip` :)

I did all that because I was excited about it and I wanted everyone who had ever had the same problems that chip solves to use it. Isn’t that why a lot of us make our projects open source — because other people might find it useful?

I think there are a few different reasons developers might open-source their projects, but I think it ultimately whittles down to this: You’ve solved some problem or made something easier, and you never want to see anyone without a solution (your solution in-particular) again.

But a lot of developers stop short of promotion. They hack something together that they know other people would find useful, release a few versions, and leave the project in the wilderness of github for someone else to stumble upon. That might — but probably never will — actually happen.

In a lot of ways you can find the exact same thing among startups. A couple developers get together with a great idea and a whole lot of initial excitement. They feverishly build whatever the hell it it is and put out an initial version. Then when they realize the software does not in-fact sell itself, the project begins its slow decline into obscurity and float away in the river of bad excuses.

Has this ever happened to you? Did you hack something that’s awesome and should be used by everyone, but no one has ever heard of it? Did you try and start a company, build an entire product, and start to lose interest in it when the time for marketing and promotion came around?

Following through on projects is one of the hardest things to do, but you owe it to yourself. I’m guilty of stumbling on this too. But remember: You didn’t spent 4 months of your life on 3 hours of sleep so you could look back and say “Yea, I should have followed through on that, but I …”

The point: If you can’t close on a project, you can’t close shit. You are shit. Hit the bricks pal, and beat it.

If you get the reference, it’s from a great movie named Glengarry Glenn Ross. A good friend of mine turned me on to it. The relevance? It’s time to sell. To investors, to customers, to anyone who will listen. Get off your ass and go do it.

The success of you and your app depends on your ability to work hard and follow through. Gary Vaynerchuck refers to it as “The Hustle.” It’s how great products, projects, and brands are built.

So yea. Let me end on a quote. This is difficult, because I hate inspirational quotes — but Method Man and Biggie tell it like it is: “Everything ya get you gotta work hard for it.

Posted in Life | Leave a comment

GetSparks.org Beta Released, Big Changes

Note: CodeIgniter Reactor 2.0.2 has a bug in it’s core Loader class that breaks package config file loading (and sparks, sadly). It’s recommended that you use 2.0.1 OR the latest at https://bitbucket.org/ellislab/codeigniter-reactor OR make the following change in your 2.0.2 installation: https://bitbucket.org/ellislab/codeigniter-reactor/changeset/c461483c8ca0 . The last option is the best.

It took a month longer than expected, but we’ve finally released the Beta version of GetSparks.org, the repository and package manager system for CodeIgniter.

Here’s the very minimum of what you need to know:

  1. A spark.info file is now required in every spark
  2. You should upgrade your spark system as soon as possible (just follow the usual installation instructions)
  3. If you submitted a spark prior to the beta, we’ve normalized all version numbers to x.x.x. So curl 1.0 becomes curl 1.0.0. Check the site.

The alpha was all about weeding out fundamental issues in the initial repo and system. The big problems we had were:

  • Some spark developers wanted the ability to specify dependencies
  • There was no difference between a repository tag and the spark version — They had to be in sync
  • Users wanted RSS feeds for latest spark overall, and latest versions of individual sparks
  • There was no way to rate a spark
  • There was no way to tell if your spark was going to be accepted by the spark processor prior to submission

We knocked those out and a little more with the beta. Going forward:

All sparks must have a spark.info file at the root. Details are here.

  • You can now specify spark dependencies in spark.info
  • The spark manager will now install a spark and all of its dependencies
  • The spark manager can now self-update
  • All spark versions must be in x.x.x format. Sparks released prior to the beta had their version numbers change in the system (tags remained the same though)
  • Versions and tags are now decoupled. The version is pulled from spark.info
  • Sparks are now ratable via a simple “Love”, “Like”, or “Hate” rating
  • A Spark-SDK is available to spark developers in order to validate their sparks prior to submission

It is necessary that you upgrade your spark manager with the release. The current version is 0.0.4. You can upgrade by removing the ‘tools’ folder from your webroot and following the same installation instructions at: http://getsparks.org/install

Send any issues to ohcrap@getsparks.org!

Now that the structural changes have been made, we can focus on tightening up the spark system’s feature set. This is what you see in the coming weeks:

  • The ability to install sparks in configurations where the application folder is not in your webroot (possible now, but not automated)
  • The ability for sparks to act more like modules — ie, sparks will be able to override core libraries and more

Also, we plan to:

  • Put a news section on the site (long overdue)
  • Configure our notification system better so its emails don’t land in your spam folder

And don’t forget the ultimate goal: Integration in CodeIgniter Reactor.

Check out the beta and ping the team with any problems! ohcrap@getsparks.org

Posted in CodeIgniter, PHP Development, Tools | Tagged , , , | Leave a comment

Video: chip — A Log File Monitor & Multiplexer

If you’ve ever had the need to investigate production issues in a load-balanced setup, you’d know why having a tool to pull down all of your remote log files into a single filtered stream can be handy. I think splunk is great at this, but in many cases, overkill.

I wrote chip to accommodate this demanding use case:

I have n production servers. From my local machine, I want to view the web application logs across those servers on a single terminal, while highlighting error lines in red. Additionally, I want those errors emailed to me.

That’s what chip does. It’s a powerful log file monitor (like swatch) and multiplexer. That is, it combines local or remote logs into one stream that you can see. Watch the screencast:

A full description of what chip can do is at the github project page, but the gist is:

chip starts up and tries to open log file(s), either local or remote. If one isn’t specified, it reads from standard input. It goes through each line looking for patterns that you have specified as arguments. When it finds a match, chip does whatever you told it to do with that match.

I use chip daily to watch production logs. When I’m investigating issues, I set up an additional pattern and handler to send certain log lines to my inbox. Just think: An instant monitor set up to watch all of your production logs while you do more important things. Useful for just tailing logs, splitting them, colorizing them, monitoring them, or any combination of the above. You can also use it as a more flexible alternative to `tee` when piping in input.

Anyway, now I can get back to working on the GetSparks.org project :)

chip is still technically a beta, but has been stable throughout my use of it. Check out chip at https://github.com/katzgrau/chip.

On Twitter, I’m @_kennyk_.

Posted in Uncategorized | 1 Response