Installing iPython Notebook on a Mac OS X Mountain Lion

Installing iPython Notebook on a Mac OS X Moutain Lion 10.8.1 for Development and Testing

The “Quickstart” is anything but that leaving off details like the recomended dependencies are basically required for a functional notebook. Here is the sequence that worked for me.

[apps]$ mkvirtualenv ipythonvm –distribute
[apps]$ workon ipythonvm

at this point pip freeze shows

[apps]$ pip freeze
distribute==0.6.28
wsgiref==0.1.2

next up the big instAll

[apps]$ easy_install ipython[zmq,qtconsole,notebook,test]

the instructions suggest running iptest. Don’t bother yet. It will fail the tests without more dependencies like “nose’ installed. Keep on. Also note the “easy_install” in front of readline is specific to set the sequence to pass the tests. Not sure why. No time to question today and I’m in a VM so it can’t do much harm. Proceed with.

[apps]$ pip install nose
[apps]$ easy_install readline
[apps]$ easy_install pexpect
[apps]$ easy_install ipython[zmq]

At this point checking packages shows

(ipythonvm)LOCAL:~/dropbox/code/apps
[apps]$ pip freeze
distribute==0.6.28
ipython==0.13
nose==1.2.1
pexpect==2.4
pyzmq==2.2.0.1
wsgiref==0.1.2

but

[apps]$ ipython qtconsole

fails. Keep trying.

pip install Tornado

Tornado works, but still errors on ZeroMQ. It comes down to this. We need ZeroMQ and PyZMQ. PyZMQ is installed (see pip freeze lista bove) but is missing the dependencie ZeroMQ doesn’t work with pip or easy_install as far as I can tell. Now we go old school. I also didn’t read the docs and was a bit tired so rather than specifying to intsall in the virtualenv I accepted the defaults and installed it in /usr/local/bin globally.

for ZeroMQ, I downloaded the Mac latest stable release from this page:
http://download.zeromq.org/

On 9/15/2012 I downloaded this one and extracted it:
zeromq-2.2.0.tar.gz 14-Apr-2012 09:53 1.8M

Unpack the .tar.gz source archive and cd into that directory. Remember I installed globally for this one package after the battle so first I had to “deactivate” in the virtual environment I was in (iphythonvm for me). Then change into that directory someplace you have “write” rights. For me I copied it from downloads to “code/contribs” which is where I put random stuff I haven’t modified but may or may not be using. Thus the next command was:

cd zeromq-2.2.0/

Run ./configure, followed by make.

But remember, outside of a VM we are back to sudo so this looks like:

sudo ./configure
password:
sudo make
sudo make install

switch back to my VM

[apps]$ workon ipythonvm

the environment now looks like this:

(ipythonvm)LOCAL:~/dropbox/code/apps
[apps]$ pip freeze
distribute==0.6.28
ipython==0.13
nose==1.2.1
pexpect==2.4
pyzmq==2.2.0.1
wsgiref==0.1.2

now we try again.

[apps]$ ipython notebook

Success! Now let’s go get somethign to look at. I created a folder in my apps folder to put the downloads and grabbed a git repository from blogger Titus of Living in an Ivory Basement http://ivory.idyll.org/blog/teaching-with-ipynb.html

mkdir iPythonNotebooks
cd iPythonNotebooks
git clone git://github.com/ngs-docs/ngs-notebooks.git

Now we test it again:

[iPythonNotebooks]$ ipython notebook
[NotebookApp] Using existing profile dir: u’/Users/eschipul/.ipython/profile_default’
[NotebookApp] Serving notebooks from /Users/eschipul/Dropbox/Code/apps/iPythonNotebooks
[NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888/

I then imported the ngs-10-blast notebook. There are codeblocks in the notebook, so for a proof of concept I just picked one that imported “blast” knowing I had not imported it. Inside of Chrome, inside of the notebook, I can click on a codeblock that begins:

import csv
import blastparser

# open the output file for reading
fp = open(‘out.txt’)

and then I select “cell run” and it runs it as if I was in Eclipse. Properly giving an error of

—————————————————————————
ImportError Traceback (most recent call last)
in ()
1 import csv
—-> 2 import blastparser
3
4 # open the output file for reading
5 fp = open(‘out.txt’)

ImportError: No module named blastparser

That is huge. Think about it. Running code inline in the middle of a web page with a compiler and debugging. Not javascript by Python in a sandbox that can import modules and do everything else you would do with idyl. And saved in JSON based notebooks that can be shared and used for testing. It sort of blows your mind.

I’m not suggesting this is a replacement for a good training video. But it is a great addition to the educators arsenal of tools for online learning in richer environments with greater interactivity. I’m impressed to say the least.

Other randome take-aways. I did not know about the pexpect package and it is pretty compelling if you work at a company. It removes the need for the C libraries for builds which means that not everyone needs to install xCode if they have to /configure, make, make install, etc.

Thus I recommend you take a look at what pexpect can do as it was new to me. Pretty cool actually.
http://www.noah.org/wiki/Pexpect#Description_of_Pexpect

Managing The Fire-Hose of Ideas

As the company has grown over the years, I have hit a number of tipping points that were unexpected and hurt the company. Bears AttackUnfortunately I have been unable to find a book that predicts these moments accurately and I know few people who have organically grown a 30+ employee 13 year old technology and marketing firm. We’re a bit odd. So while there are many sage leaders in the city of Houston, few have ever been “in my shoes” so to speak and most aren’t really sure what we do. In other words, sometimes I am flying blind and changes are clear only in hind sight. I am having one of those moments now and it involves ideas and a very motivated, skilled, enthusiastic and hard working group of employees.

The problem is ideas. Too many of them.

My management philosophy has always been the same. “Hire good people. Train the hell out of them. Let them run.” There can be problems with this. If you train them and they run off to a field to pick daisies, you fire them. But my experience has been that people are a LOT more motivated when they are given the tools and the freedom to do their jobs.

I once worked at a large corporation where I needed a Vice President’s sign off to get a $30 book I needed to do my job. And I had to write up a justification about why I needed the book. I called it a “pre-book-report” at the time. Anyway, I come from a family that consumes books like other people consume Doritos so this blew my mind. My manager, her boss, the VP and I spent way more than $30 in salaries debating the merits of said book. Most of which was made up because none of us had read it. So while the CEO said we were there to “maximize shareholder values” the rest of us didn’t get the memo. So I kept reading books and just paid for everything I needed out of my own pocket.

And I vowed I wouldn’t cripple my employees that way when years later I started a company.

Back to ideas. Thanks to our clients we get to eat. And we help them make a profit using our technology and processes. Everything is thanks to our clients. They expect and deserve the best possible service at the best possible value that we can deliver. That takes training. And I am committed to training. So far this year we sent 11 people to SXSWi, 2 people to SMX, 2 people to DrupalCon, 2 people to NTEN’s NTC, 2 people to the TSG Summit, 2 people to PyCon, we have 4 scheduled to attend Tufte, etc….  And it’s only March 19! Perhaps I shouldn’t share this because my competitors can see that the secret sauce over here is training. But I’m not that worried as investor led companies tend to maximize profits for the quarter and therefore lack the discipline and will to invest so heavily in training. Particularly if all of those expenses hit you in the same quarter.

Now, all of those employees are back from cities all across the United States and they are walking in to my office with idea after idea. After idea. After idea. And ideas are good. But it’s too much.

This is compounded because ideas are pretty cliche. You can’t patent an idea, you can only patent an implementation of an idea. Ideas only have value when you take action on them. It is results that create value, not ideas. Yet all of us in life want to provide the ideas and have them get done, usually by someone else. And we take it personal when someone shoots our idea down, and people know that, so the more ideas you throw out there the more people nod their heads in agreement. “Why yes, that is a good idea.” And it might be. But we’ll never know unless someone prioritizes it and commits the resources to implement the idea and then evaluates the results.

And some ideas are just bad. For example Ethan Watters expressed these emotions about one idea:

The idea of going to a Shriners meeting and listening to some high school student read her award-winning essay on the value of democracy seemed like an activity that I might encounter in the first ring of hell.

Nothing against the Shriners, but that is an idea that if you told me you were doing that I would say “hmmm, sounds interesting.” Yet I would be thinking: “No, that does NOT sound like a good idea for me and NO I do not want to test that idea.” But I wouldn’t say that.

A few years ago I judged a Tech-Transfer event for MBA students who presented a case on if an academic patent should be commercialized for the university where the research was done. I kid you not – this one patent was for a nanotech etching machine that was less than half the size and more expensive than one that was commercialized and in use in industry. It is hard not to look at that idea, shake your head, and think “was that just some dude who wanted to frame a patent for his wall?” I guess it’s academia so they have more wiggle room, but sheesh. This is an example of a bad idea that wasted time and money.

Testing ideas is expensive.

As a CEO your dream is someone walks up and says

“I had this idea so I prototyped it and the initial results look promising. Can we schedule a time to go over the results?”

And sometimes that happens. It really does. And those people get promoted at our company much faster than others. But more often than not you are presented with an idea like it is a sacred object and expected to immediately commit resources to test it. And there are simply too many ideas. And never enough resources.

(Sidebar: You actually get a LOT of innovation from the sales team (yes really) because they talk to prospects and see actual needs before people who only work with products we already support. Because no one within the company already knows X new product, a sales person with initiative will self install (read: prototype). That is how we started offering WordPress and Drupal as new product lines and THEY ARE GREAT!)

I was pondering the expense of organizing and testing all of these ideas while on a long walk with the dog this morning. A few possible solutions came to mind:

  1. Set up a DIGG type ranking system for idea submission and have employees vote the ideas up or down.
    1. They talk about this a little in Groundswell. But Idunno, I rarely see committees find the best possible idea. They usually blend everything until you get a compromised version of mush. Or whoever can write the best python script wins the vote. I love Amazon reviews, but I rarely write one. Does that mean my ideas don’t have value because I won’t use that particular tool?
  2. Require employees to write-up the idea and present it in an organized fashion at a scheduled time.
    1. This would stop the revolving door in my office of people presenting great ideas. Yet as I recently blogged about visionaries, it is the Eureka moments that lead to big discoveries. I am not sure a global “you must write it up” filter is in the best interest of the company.
  3. Schedule office hours.
    1. This is probably something I should do as a CEO as I am a little too accessible at times which prevents me from getting my work done. But again, will I miss a Eureka moment? What is it that I do that could possibly be more important than working with our employees?
  4. Say “no” to everything.
    1. Saying “no” to everything has actually worked well for me in the past. If the employee  isn’t motivated enough to overcome the first “no” then they aren’t that committed to the idea. Or so goes the thinking. But people are very different culturally. Extroverts ask me the same question 10 times while introverts won’t ask at all! Won’t this method bubble the “squeaky wheel” ideas up to the top? I doubt those are the best ones.
  5. Make them run it by their manager first.
    1. Otherwise known as the “hide behind hierarchy” method. Would this not break the spirit of an employee if they felt the CEO was inaccessible? What if they had an issue with their manager at a personal level, but had a good working relationship, but didn’t want to share? And do I really want to be the type of founder who is unwilling to talk to any employee? The answer to that is a resounding “no!” I spend more time with employees than I do with clients because I know developing our employees is what it takes to get to great customer satisfaction!

I’m at a loss here. I see we have hit this point. I feel like I am drinking from a firehose and I can’t keep up. While ideas alone are worthless, the implementation of a good idea has definite value!

My question to anyone who has made it this far in the post is “do you know of a system that has been tested and works for a CEO of a high growth company to handle employee ideas?” And I specifically do not want ideas. What I need is knowledge of a system that has been tested and works. Even if that system is a behavior modification on my part.