a day in the pit my view from inside

2May/110

Work Flow & Process… Building side projects

Many people have asked me how I have time to work on personal projects while juggling a full-time gig and having a social life. Friends will often say things like "you're crazy man, seriously when are you not working on something?" to which I typically respond "I'm always working on something that applies to me." It's really a simple idea: I build for myself. I find a problem I have, then I solve it. This alleviates the need to impress anyone but myself, and allow my creative juices to spill as they please. At the end of the day, I love building tools and I've always been pragmatic so I want to build useful things. Here are some examples of items I've worked on in the past.

  1. HelloBirthday - A Facebook application that manages and delivers personalized birthday messages to my friends. I always forget birthdays and desperately needed a way to stay on top of my online social network.
  2. SimplyFor.us - A stand-alone web application that allows me to privately maintain a list of my ideas, and voluntarily share those ideas via time limited access codes.
  3. ThatWasMean - A web application for posting and scoring things people did that were mean. It was a joke ;) but it kept my co-workers and I laughing for hours.
  4. ThinkEffect's CMS - A web application I build to spawn easy to customize web applications for clients. I did this for fun on the side and later incorporated it into my small design group.
  5. AnswerWise - A web application, and mobile app, that crowd-sources answers to any kind of question. So I could be sitting around in some new city and see something cool, snap a picture, and ask "What the heck is that?"
  6. SyncedHost - A small web application that queued requests for free web hosting applications and allowed me to approve or deny the apps. I ran a small web hosting company and was inundated with requests via e-mail and needed something to track applicants and automatically create web hosting accounts and e-mail details out.

There are plenty of other, less glamorous, projects but these highlight a solution to a need. If other people also find them to be useful, that is just a bonus.

The reason I'm such a stickler about building for me is that my motivations come and go and as they do I like to have projects that fill different areas of interest. Context switching is what I'm all about. Further, it's a dive into how I think. Technology should benefit the one who is building it. If a secondary market pops up (i.e. everyone else) then that is wonderful and I will happily introduce new features by popular vote. But for now, it's all about solving lots of little problems.

One common problem that people run into is that they can't think of anything to work on. It's really easy: look at what you do every day and ask if there is something that you'd want to do differently. More likely than not you'll have a project in no time. Looking at my projects above, there are even a few that have been solved by other people but I still want to build my own. The motives behind building my own could be that I am looking for just one feature out of many or a simplified version of a specific feature. It's not out of the question to rebuild something just to see how to do it! In fact, I often search for what I have in mind before I start building to see how others solved the problem. It keeps me honest and provides one of the most valuable insights to developing solutions: not all problems are easy to solve; respect people who have solved them and made the solution easy to use and look pretty ;)

OK. So we now know that the projects I build are mostly point-solutions to my day to day routines. Great, so finding time to work on them actually makes sense: I'm investing in saving time. And sure enough, it works. Take SimplyFor.us as an example, I used to carry around a notepad to write down my ideas during the day. Eventually I would somehow lose or misplace that notepad and all of the ideas written inside of it. Creating a web application to store all of my ideas solved the main problem, losing ideas. Additionally I created an easy way to share my ideas with others which solved my other annoyance of having to send e-mails to people requesting feedback. Now I store my idea on the go and share it whenever I want. I can update my ideas at any time, allowing them to truly evolve, and also keep notes on who else is working on a similar idea or what progress I have made on any of them. I bet I used to spend at least an hour a week messing with tracking and managing ideas that popped into my head. SimplyFor.us took about 8 hours to build and deploy. I've been using it for months. It solves for my needs.

Fair enough. How did I go about building it? This is the fun part. SimplyFor.us actually integrates into my build stack nicely. When I feel like working on something I first check my "bank of ideas" and if nothing pops out worth tackling I go back to the drawing board and work on something I've already built.  In the case where I want to build something new here are the steps I follow:

  1. Write a few sentences describing the goal of what I'm building.
    1. What's it supposed to solve?
    2. How will it get done?
    3. What's the one use case that I could ship this tool after completing.
  2. Next, design the user experience around that one use case.
    1. Get into Balsamiq, OmniGraffle, or some other UI mock tool and design.
    2. Share the design with my friend Jon Kupferman and see what he has to say. If he OK's, on to step 3.
  3. Build the use case. I use Ruby on Rails. Yes, I always do.
    1. Start an SVN/Git repository and check in the Rails base
    2. Write a few tests (not all the time. I'm trying). Make those tests pass.
    3. Style up a front-end worth looking at.
    4. Make sure the use case is solved.
    5. Push it to one of my web hosts and give it a domain.
  4. Use the damn thing for a few weeks. Sharing with friends as I go, if I think it will help people or if I want some more feedback.
    1. But never push it onto people. I don't spam it on Reddit. I rarely do any promotion outside of posting to my own blog. It's for me after all.
  5. Iterate, if necessary.

Pretty straight forward. I'm no prodigy, genius, mastermind, etc. I just stick to a pattern, rinse and repeat. If I get bored, I context switch until I'm entertained again. Eventually I will go back to a project that gets left in the dirt, brush it off, and continue. I call this strategy nothing and it consists of identifying what would be great, designing what is necessary, and building what works.

(I'd like to post some mockups from the flow I described above and hopefully will get around to that one day soon)

Tagged as: , No Comments
29Jan/110

Ask Mike: How do you learn a new programming language?

I found myself sitting on a couch the other night explaining to one of my good friends how to learn a programming language. His background was in computer engineering; most of his experience, which was scarce to begin with, was focused on low-level C-like languages.

So, here my friend sits trying to learn a language that is totally different from C, C++, and Java: Ruby. Ruby was my first choice because it illustrates a fundamental learning concept to a language: syntax. In Ruby, we see a lot of unique conventions, short hand notations, and other syntactic sugars and that seems to be the biggest hurdle in learning a language for me.

So here's my list of concepts and information I seek when I'm treading unknown soils:

  1. Define a class, instantiate it, and make it do something
    • This exercise will give you the opportunity to toy with inheritance, class vs. instance methods, public and private scope, and optionally instance-specific functions like constructors and destructors.
  2. Find an article on "ten neat tricks to " ... or .. "ten things I wished I knew about"
    • Explore why others like this language... What is cool to them about it? Do these tricks show you anything about the language that the book(s) or tutorial(s) you're following do or don't do?
  3. Read about the language data types
    • Are the data types what you'd expect to see? Do floats auto-magically grow? Are strings immutable? Sometimes by looking at the data types you'll see some interesting features of a language. For example, in Ruby you can take an integer and call methods on it like .days.from_now (5.days_from_now).
  4. Find an article highlighting the ten worst things about the new language
    • Yup. Understanding the language's perceived weaknesses is very important. In Ruby, the MRI interpreter supports green threads -- this sucks in some cases. But along with the criticism often comes a solution. For the case of parallel threads, using the jRuby interpreter solves that issue by supporting OS-level threads.
  5. Code something fun
    • For my friend, he is building a small BlackJack dealer simulator. Perfect. The important thing here is to pick an idea that you can articulate well, understand how you'd theoretically build it, and would enjoy spending time working on.
  6. Optional: Go read some "Why I Switched from Lang 1 to Lang 2 " or " Lang 1 to Lang 2 side by side " articles.
Tagged as: , No Comments
26Jan/110

PHP 5.2 mysql_connect w/ same parameters does not return a new connection!

This is subtle... I was writing some "reconnect" logic for a DB Class to catch and handle MySQL error code 2006, Server Gone Away. After a solid 5 hours of wasted time I traced the poor logic on my part back to the mysql_connect() function. I was thrown off track at first when the code was hung up on a mysql_select_db() function. Turns out that mysql_connect(), if called twice in an execution series with the same parameters, will take the old link and return that! What The French. So, yes, I read the manual and a fourth param (boolean) can be passed into the function to say I want a NEW connection.

mysql_connect()

The fourth param is called new_link, and the excerpt from php.net is as follows:

If a second call is made to mysql_connect() with the same arguments, no new link will be established, but instead, the link identifier of the already opened link will be returned. The new_link parameter modifies this behavior and makes mysql_connect() always open a new link, even if mysql_connect() was called before with the same parameters. In SQL safe mode, this parameter is ignored.

Good to know. Better to remember. Necessary to understand. Fail on my part.

Tagged as: , , , , , No Comments
19Oct/100

PHP array_diff vs foreach: a battle for speed

1000 runs w/ 1000 data elements in the two arrays (php array diff): 2.7389E-5 seconds
1000 runs w/ 1000 data elements in the two arrays (php foreach): 1.085E-7 seconds
php array diff slower by 2.728E-5 seconds

There were two arrays for this test: $big_set, which had 3147 string elements and $to_diff, which had 1581 string elements. We needed the difference of the two arrays and found that array_diff is slower on average. The nicety to using a foreach(){} is that we can now do other computation within the loop, as opposed to having to loop a second time after the differential set is found. Code below...

<?php
print_r($argv);
$runs = $argv[2];
echo "$iterations\n";
$data_size = $argv[4];
echo $data_size;
$big_set = array();
for($i = 1; $i < $data_size; $i++){
  $big_set[] = "Hello there ".rand(500,5000);
}
$to_diff = array();
for($i = 1; $i < $data_size; $i++){
  $to_diff[] = "Hello there ".rand(500,5000);
}
 
for ($i=0; $i<$runs; $i++) {
   $time = 0;
   $start = microtime(true);
   $diff = array_diff ($big_set, $to_diff);
   $t = microtime(true) - $start;
   $time += $t;
}
 
echo $runs, ' runs (php array diff): ', sprintf('%2.9f',$time/$runs), ' secs', BR;
 
for ($i=0; $i<$runs; $i++) {
   $time2 = 0;
   $start = microtime(true);
 
   foreach($big_set as $p) {
        unset($to_diff[0]); // cost of an associative array lookup & delete
   }
   $t = microtime(true) - $start;
   $time2 += $t;
}
 
echo $runs, ' runs (forloop): ', sprintf('%2.9f',$time2/$runs), ' secs <br />';
 
if($time2 > $time) {
    $seconds = $time2 - $time;
    echo "forloop slower by ". $seconds/$runs ." seconds";
}
else {
    $seconds = $time - $time2;
    echo "php array diff slower by ". $seconds/$runs ." seconds";
}

Execute via command line :/>php file.php -runs -data_size <# of items in array>
Must be in that order -- I did not make it handle fancy arguments.

Update: Adding PHP Bench for other cool benchmarks

Tagged as: , , No Comments
16Oct/100

Facebook Development: Hello Birthday V2 Open Graph

Well, it has been a few months since my original post on Developing Facebook applications (Hello Birthday's original post). During this time, Facebook has introduced a new API called Graph (Graph Docs). A big change is the authentication protocol moving to OAuth 2.0. Yay for OAuth, because it definitely simplifies things...

Facebook continues supports the legacy Rest API, which is good because the new Graph API is still changing slightly here and there... If you have Rest API keys stored, you can easily upgrade them using one of the new API's function calls.

The most exiting feature that Graph introduces is the ability to subscribe to user attributes, like friends, application permissions, and current location.

As you can imagine, this has huge performance gain implications and reduces the need for overly complex web apps with polling-like data integrity processes... Read: No longer do I have to ask Facebook if a user has given me permission, because I know now what I have when they sign up and if it changes at any given time.

Facebook is pushing iFrame Canvas apps. Beware of this. It is a difficult, and hack-prone, situation where you have to find a way to authenticate a user within an iFrame and use Javascript to do a top.window redirect. Facebook Developers say better support is coming. For now, JS is your friend.

Let's move on though... The new API is great and all, but I'm more interested in talking about some of the insights gained from my first real FB App. Here are some high level lessons learned from my non-iframed canvas app (v1):

  • Javascript support is limited
  • FBML is really a sad attempt at a markup language, I'd rather see HTML sensitive attributes that do the same thing FBML does, without he need for a whole new language
  • Global styles were less-than-easy to work with due to FB rendering my page within theirs
  • Rendering pages requires all of Facebook's shell to render. Boo
  • Porting the app to be a stand-alone web app is not trivial, plan out your adventure ahead of time. If you think your app might become a standalone product plan for it (the best you can)
  • Unable to fine-tune performance (you <strong>can use memcache though!</strong>)
  • As the previous note hints at: use memcache it makes life so much easier if you're dealing with a lot of data and constantly using lists of friends and information about each friend
  • Get a friend to work on the app with you. It makes testing and continued efforts so much easier. Thanks Joe :)
  • Get a good set of friends to bounce ideas off of. Thanks Jon, Erica, and others whose names escape me right now
  • Ask your USERS for feedback early & often
  • It's nice to have my app feel as though it belongs in the Facebook layout. I think this is a big win to Canvas as a whole
  • The API has worked very well for me and I've experienced little to no hiccups
  • Love FQL
  • Facebook developers are somewhat easy to get a hold of but problem solving may take time.

During development of V1.5 (the transition to the Graph API) some of the changes made were:

  • Move to an iFramed app
  • Clean up transitional code that helped me get from boot-strap to version one
  • Remove FBML mark up
  • Implement more personalization options to the user experience
  • Create more of an "experience" for that matter... Adding descriptions next to links to give context. An ongoing effort, by the way
  • Became more transparent in implementation details: I told my users more about how Hello Birthday worked, what features were being built, and what bugs I found.
  • Listened to feedback and added: Personalized Messages on a global level (defining a set of default personalized messages that are randomly selected during the wishing process)
  • Listened to feedback and changed: Birthday messages to be delivered during the day at user specific time ranges.
  • Fixed issues related to local time zones
  • Identified users who have privacy settings that don't allow Hello Birthday to retrieve their birthday *big one*

One of the points above is really important (bullet point 6) and it hovers around the idea of creating an open dialogue between you and your users so that they feel comfortable complaining, suggesting, and critiquing features. My users are the debuggers, like with most applications, and just by letting people know when I was making changes, what they were, and screenshots of what to expect I was able to generate immediate feedback. The best part is, that feedback was both good, bad, and off topic! I was receiving (as comments to the posts on the App Wall and via e-mail) bug reports, thumbs up, and thumbs down on different ideas/feature enhancements. By continuously increasing the number of posts (to a certain degree) I was essentially getting into the daily lives of Hello Birthday's users and it caused an awesome feedback loop to begin where there now exists a few users who love to let me know when things break! Awesome.

I'm very close to having the version two of Hello Birthday launch. As you can imagine, there is a bit of hesitation around the new authentication protocol and making sure my legacy API key's of my users transfer over. I'll write a post on that (if it ends up not being trivial). At any rate, I'm looking forward to seeing the feedback on version two. There has been a lot of subtle changes and feature enhancements that I'm sure (because of feedback received already) will be welcomed nicely.

You can expect a post after the launch of V2 where I will go over some of the insights Facebook provides to developers for their apps. There's a lot of interesting data around conversion rates and what permissions people grant your application that I want to share! Thanks.

Tagged as: , , , , No Comments
6Oct/100

Years later, my blog becomes ADayInThePit.com

A short post indeed: I've renamed my personal blog to "A Day In The Pit." I think it properly reflects where most of my article inspiration stems from. Further, I'm selfish and want to keep ephekt.com to myself. (It will still point for now). Update your bookmarks, all 5 of my faithful & patient followers, and look forward to my next part on my series of Facebook Development posts.

Tagged as: , No Comments
30Sep/100

GMail Conversation Threading can now be disabled!

This is a follow-up post for those who read my previous blog entry on preventing GMail threading. Google Mail now has the option to disable "Conversation View". Go to your Gmail, click Settings in the upper right corner, then you should land on the General tab and about half way down the page is the option to turn off "Conversation View". Hope this helps those of you out there who have had issues related to e-mailing account owners of your web apps... Or just plain get tired of accidental threading.

Filed under: email, Information No Comments
10Jul/100

Just a quick time comparison of PHP’s preg_replace vs. str_replace

Here's a short and succinct comparison of running time to replace multiple characters in a given string. I ran the test 20 times each, so the numbers you will find are average running times:

$start = microtime();
$str = "23ilrj23oirj23iorj o23irj23klfj23lkjr4ocimior 4r ioj234roij234r io34jrio4jrio34r jio4jr o34jr oi4jr io34 r";
$new_string = preg_replace('/[\w2]/',',',$str);
$end = microtime();
echo($end - $start);
echo "\n";
$start = microtime();
$str = "23ilrj23oirj23iorj o23irj23klfj23lkjr4ocimior 4r ioj234roij234r io34jrio4jrio34r jio4jr o34jr oi4jr io34 r";
$new_string = str_replace(array('2',' ',"\t"),',',$str);
$end = microtime();
echo($end - $start);

The results:

regex: 0.000608
tr: 0.00024099999999999

Filed under: Code, Information, PHP No Comments
6Feb/100

Facebook Application: Hello Birthday

About a month ago it was my birthday. On that day I was receiving countless e-mails from Facebook notifying me of Happy Birthday messages when I realized many of the messages followed a particular pattern (i.e. "Happy Birthday(!| <name>,| <endearing term>). Don't get me wrong, I enjoy and appreciate all of my birthday wishes. But being an Engineer, my brain finds patterns and attempts to "code up solutions." However, I had not played with the Facebook API yet and had been waiting for a good reason to come around so I took the idea of a Happy Birthday messenger to work and put together a version one of my Hello Birthday application. The application follows a few goals: to be easy to understand, easy to use, and it should just work. With that, here are the criteria I came up with to take a crack at achieving these goals:

  1. Automatically message friends on their birthday
  2. Allow enabling and disabling of such said feature
  3. Use a generic message that more than 50% of users observed currently use when wishing a friend "Happy Birthday"
  4. Manage an exclusion list, containing the friends Hello Birthday should not automatically message
  5. Post onto the friends' wall as if it were the user

With plans to continue development, I have a more robust friend list module going in soon and the ability to set up custom messages for particular friends. Eventually, some form of gift giving may be in order. To bootstrap development I followed a few principles:

  1. Prototype over masterpiece
  2. Design impacts implementation, but implementation makes design possible
  3. Low budget: use open source and free resources

To prototype the project I chose to use Ruby on Rails and SVN. To bridge my rails application and Facebook, I added the Facebooker rails plugin to my application. Facebooker is a pretty neat extension to Rails that provides all kinds of helper methods and an extremely easy to use interface wrapping the Facebook API. It also provides nice view helpers for generating FBML (the Facebook Markup Language) and FBJS (Facebook JavaScript). Developing version one locally saved quite a bit of time by realizing change immediately as opposed to any form of deployment. I then set-up a Facebook application (in Sandbox mode) for development (i.e. hello_birthday_dev) that only I had access to. Facebook offers test developer accounts, so after registering 6 or 7 of these to use with tests I was able to enumerate all of the states that Hello Birthday would need to recognize and transition.

When it came time to deploy to production I needed a production server and Facebook application. I registered the Facebook application (again, in Sandbox mode) for production (i.e. hell0_birthday) and then signed up for hosting with Heroku.com. Heroku is a Ruby (mostly Rails) cloud computing platform as a service. Basically, you get one free "cpu worker" and as you need more they sell scalable features. This, however, is perfect for what I need: a Rails host that allows cron jobs and makes deployment super fast & easy. Sure enough, Heroku makes deploying an application as easy as:

  1. git add .
  2. git commit -a
  3. git push heroku master

A slug of my app is created and deployed to my birthday.heroku.com production server. And to make sure I can monitor errors, I installed the hoptoad error monitoring service. Hoptoad app provides me with web-based access to error reporting and resolution management (organizes errors based on environment; test, production, and or development).

The training wheels were taken off: Sandbox mode disabled on Hello Birthday. I submitted my application to the Facebook directory a week or so ago and it's been smooth sailing since. I currently am servicing a little over 40 people. No complaints have been submitted thus far.

In summary, this is my first Facebook application and I have tried to utilize a lot of the latest technology in the process to illustrate how quickly a scalable solution to an easy problem can be solved. With Heroku & fairly efficient code, I do not foresee a problem handling a potentially high growth rate. I will try to bring back updates to this article if anything goes terribly wrong.

You can check out the Facebook application at http://apps.facebook.com/hello_birthday

Filed under: Code, Information, Ruby on Rails, Software No Comments
24Sep/090

AnswerWise — The power of humans

Over the last few weeks I have run into numerous spiders and being the inquisitive type of person I am I always want to know what I am looking at. Thanks to my senior capstone project at the University of California at Santa Barbara, I finally have an answer to pretty much any question I could formulate... And the best part about AnswerWise is that it supports images. Win win. Here is the latest question asked and it's answer. (AnswerWise had the answer in under 10 minutes)

What kind of spider is this and is it poisonous?

spider

Answer: This is a Phidippus johnsoni (Red-backed Jumping Spider) its bite results in swelling and pain at the bite site but its not poisonous. But of course if you are allergic it will be worse.

Pretty cool huh? I think so. A little bit about our senior project statistics: Over 800 questions have been asked with an average 15 minute response time and 80% or more of the asked questions include images.

Filed under: Information No Comments
 « Newer Entries  Older Entries »

Ephekt/Mike Rose/Smasher5

Twitter: smasher5

Sections

Archives