Upgrade Linux kernel on Ubuntu 14.04.3 to Mainline v4.3.4-wily

linux
The Linux Foundation

Upgrade Ubuntu 14.04 LTS to the newest kernel just in case you want to play with later versions of Dockers or systemd on an LTS release.

DISCLAIMER – MESSING WITH THE KERNEL CAN BE BAD. RUN YOUR BACKUPS AND SNAPSHOTS AND DON’T PLAY WITH LIVE AMMO!

Step 1 – First check “kernel.ubuntu.com/~kernel-ppa/mainline” to find out the current mainline kernel. As I type this it is 4.3.4 but you may not want to copy and paste these as it has probably changed by the time you read this. It is software, right?

sudo su
 mkdir -p /home/ubuntu/kernels
 cd /home/ubuntu/kernels/
 wget kernel.ubuntu.com/~kernel-ppa/mainline/v4.3.4-wily/linux-headers-4.3.4-040304_4.3.4-040304.201601230132_all.deb
 wget kernel.ubuntu.com/~kernel-ppa/mainline/v4.3.4-wily/linux-headers-4.3.4-040304-generic_4.3.4-040304.201601230132_amd64.deb
 wget kernel.ubuntu.com/~kernel-ppa/mainline/v4.3.4-wily/linux-image-4.3.4-040304-generic_4.3.4-040304.201601230132_amd64.deb

sudo dpkg -i linux-headers-4.3.4*.deb linux-image-4.3.4*.deb
 sudo reboot

NOTE: I have hit an error on this upgrade several times similar to this:

Errors were encountered while processing:
linux-headers-4.3.4-040304-generic

The fix was to run these

apt-get -f install
apt-get autoremove

Next you MUST REBOOT.  Then log back in and check what kernel you are running

uname -a

# from that you get a long string and in the string you should see something like this:

4.3.4-040304-generic

Keep googling for more, or duckduckgo-ing. My biggest advice would be to create a throw-away VM in the cloud to test this stuff. VMWare isn’t great for networking by “sox” imho.

Ubuntu Linux
Linux of the Ubuntu variant

systemd – I’ll deal with it like I deal with Pepsi in the North East

OPINION: The topic is init scripts. The part of a computer that determines what starts first and next and next etc. Most of my readers, and I thank you both, will want to close this tab in your browser and come back on a non-geeky day.

System Init. – Do you need your keyboard before your monitor? Nah, we’ll bring up the monitor before the keyboard.. And yet we have bigger issues like when to initialize the CPU, RAM, HDs, USVs peripherals, etc.

So I was udating my automatic services in Windows… oh wait, no it was in Linux on 14.04.3 and everything kept telling me the same thing.

zOMG why are you using upstart when systemd is the bright-new-shiny!?!?

I know I’ll have to give in but this thing smells like something between SELLinux and WindowsNT’s implementation of POSIX.

setting up vmware fusion 7 for tendenci development

Tendenci install docs – latest version – are in the git repo here: https://github.com/tendenci/tendenci/blob/master/docs/source/installation/installation.txt  Please note that Tendenci is a FAST moving project and you should update your install before submitting issues given the rapid pace of change.

I am adding this as a blog post instead of a help file as things change so quickly it might be outdated in a week. Anyway, part 1 of probably several on setting up vmware fusion with Ubuntu 14.04 to develop Tendenci Open Source Software for Non Profits and NGOs. (disclosure – I work there, blah, disclaimer, blah, etc….)

First – install vmware Fusion 7. They have a 30 day trial so you can start with that.

Second – download ubuntu 14.04 from the official distro. Given Ubuntu appears to be heading towards the dark side lately, you will be fine just using Debian Linux as almost all commands are identical. In fact as one of the lead developers on Tendenci I can tell you I’ll be testing it on Debian, CoreOS in Dockers.

(As an FYI – you can also spin up a demo site of tendenci right now at https://create.tendenci.com if you just want to kick the tires. But I’d much RATHER see you join the community to provide future-tools to change the world.)

Now in pictures setting up vmware with ubuntu (14.04.2 and 14.04.3 are both fine – 14.04.3 will save you time on updates. So do that first after install:

sudo apt-get update && sudo apt-get upgrade -yy

1-vmware-fusion-tendenci

2-vmware-fusion-tendenci

  1. 3-vmware-fusion-tendenci

5-vmware-fusion-tendenci

 

This is mostly a useless graphic saying to install your preferences as you see fit. Other stuff happens before and after this. Carry on.

6-vmware-fusion-tendenci

 

Not a useless graphic. VMware’s console is a pain so you are definitely going to want to have openssh installed. 9-vmware-fusion-tendenci

stuff

7-vmware-fusion-tendenci

more stuff8-vmware-fusion-tendenci

confirm you did stuff

4-vmware-fusion-tendenci

now do more stuff – in this case install vmware tools so you can share the file system and don’t have to suffer through VI/Nano or whatever. Sublime is way better.

10-vmware-fusion-tendenci

In this image it says “reinstall vmware tools” but on yours it will probably say “Install vmware tools” – pick that one.

Next, what SHOULD be simple is definitely NOT simple. You have to configure Ubuntu guest machine to support vmware tools. Vmware exposes the tools into the guest OS but you still have to mount the drive, install, configure. What a pain in the arse. Anyway, Some helpful commands and from there you have to google your way.

Note at this point I’m assuming you are SSH’ed into your VM on the local and have done a “sudo su” so you are operating as root.

In the properties of your virtual machine make sure the CDROM drive is set to “mount”. In the guest OS you should then be able to do this:

sudo mount /dev/cdrom /media/cdrom/

and

cd /mnt/
ls
cdrom hgfs

This mounts your drive from vmware into the host. The fact that you don’t have a cdrom drive on a mac is irrelevant – it still maps it and gives you access to source files for vmware.

Next add your debian package keys or nothing else will work. Again, it is up to YOU to check if the paths are still correct as time changes everything. As I type this today these are correct

wget -qO - http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub | sudo apt-key add -
wget -qO - http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub | sudo apt-key add -

Verify they installed correctly

apt-key list

Add/create your files. Maybe check if they exist first so you don’t overwrite but the basics are:

touch /etc/apt/sources.list.d/vmware-tools.list
nano /etc/apt/sources.list.d/vmware-tools.list

Next edit your vmware-tools.list file to include the repo

deb http://packages.vmware.com/packages/ubuntu precise main

NOTE- if you aren’t on “precise” that won’t work so check your distribution (‘trusty’ etc…). Now install.

apt-get install open-vm-tools-deploypkg
root@tendenci:/# cd ~
root@tendenci:~# ls
vmware-tools-distrib
root@tendenci:~# cd vmware-tools-distrib/
root@tendenci:~/vmware-tools-distrib# ls
bin doc etc FILES INSTALL installer lib vmware-install.pl
root@tendenci:~/vmware-tools-distrib# ./vmware-install.pl

From there I take all of the defaults except for this one which I answer yes (it’s just vm after all.)

Would you like to enable VMware automatic kernel modules?
[no] yes

Check your mount points if needed

ls mount-point

If everything worked, which is unlikely out of the box as vmware has unfortunately not gotten along with Mac’s lately, but hey, if you got lucky you should now have a mounted drive at

cd /mnt/hgfs/ with vmware in it.

NOTE – several points in there, it doesn’t hurt to reboot your vmware host os. I apologize for not being more precise in this post but I know if I get the basics out there google or ddg will solve it for you. That is my only goal – to point you in the right direction if not perfect.

Lastly if the above, or a derivative of the above worked, then you should be close to doing something like this.

Clone the git repo down to your mac. For me I put it on my Mac in ~/Documents/code/webapps/repos/

then inside of the guest machine, assuming the above worked for you (test test test people) then you should be able to FIRST install tendenci the normal way using the instructions here

THEN, and only THEN, move your dist repo on the host (after you can pull up the site in a browser) to a different folder and sym link in your cloned repo.

For my dev environment this is done with:

ln -s /mnt/hgfs/webapps/repos/tendenci/tendenci /usr/local/lib/python2.7/dist-packages/tendenci

I’ll try to tighten these instructions up over time but wanted to get them out there as I honestly lost hours and hours getting just the basics working with VMWare. I get that it is faster than vagrant, but almost not worth the grief with the difficulty of getting the vmware tools working.

Helpful source links used in the creation of this wonderful blog post. Categorize this one as “something is better than nothing” and use your google fu (or is it “foo”?) to fill in the details. I’m only sharing it because my google fu ain’t bad, and it was a challenge for me. So…. helpful further links:

Tendenci issues: https://github.com/tendenci/tendenci/issues

VMWare file sharing: http://hplgit.github.io/teamods/ubuntu/vmware/mac.html

Ask Ubuntu notes: http://askubuntu.com/questions/29284/how-do-i-mount-shared-folders-in-ubuntu-using-vmware-tools

sublime text 3 packages – knowledge share

Rebuilt my laptop and have been reconfiguring and upgrading stuff ever since. I forget how much customization goes into a dev box. These are mostly notes to myself but if they help someone else, cool.

You can get sublime text 3 at http://www.sublimetext.com/3 – it is my text editor of choice. And like wordpress plugins or tendenci add-on modules, it is extensible. Make it do what you want, right?

ST3 packages and installing them is a bit different from sublime text 2 so I thought I would share what I had to do to get things installed.

Go to this link
https://packagecontrol.io/installation
Then copy the code on the left on that page
Open the sublime console by typing CTL+` (that last thing is a “tick” mark – top left on a mac keyboard)
Paste the code you copied into the sublime console and hit enter.
Then restart sublime text 3.

Next hit COMMAND+SHIFT+P and type in “Package Control: Install Packages”
You should now see a list of packages to install.

Some common plugins that don’t require much thought to decide on. After that install what you need, but no more, to keep things tidy and fast.

HTML5
Makes HTML look pretty and easier to work with by color coding.

Python Improved
To make this your default Python, open a `.py` Python file.
Select `View -> Syntax -> Open all with current extension as… -> PythonImproved`
and you should be all set.

Color Highlighter
Shows CSS codes in the color that is chosen FFFFF for example.

GitGutter
Shows changes since your last commit by putting small visual cues on the left hand side.

sublime-adding-packages

Screen Shot 2015-08-15 at 2.00.23 PM

markdownlight-sublime-plugin

Search for more packages here: https://packagecontrol.io/  They have some good ones for css and less/sass/etc… I’d look for the ones with the MOST downloads as an indicator of if it is a stable plugin.

django errors using boot2docker tutorial

If you are doing the docker-compose / boot2docker tutorial on the docker site at:

https://docs.docker.com/compose/django/

There are a few typos and items that will stop you, at least on a Mac although I suspect they are an issue on a PC as well. These are my notes, out of context but hopefully they will help someone else having problems with errors and configuration of Django Dockers for their dev environment.

boot2docker up
docker-compose up
# the following line runs it in the background
docker-compose up -d
To connect the Docker client to the Docker daemon, please set:
 export DOCKER_HOST=tcp://192.168.59.103:2376
# this tells you your environment variables
docker-machine env
http://192.168.59.103:5000/
# The following is incorrect in the docs
docker-compose run web python composeexample/manage.py syncdb

Rough notes, but hopefully enough to help you google for more if you are hitting the same wall I did with boot2docker and docker-compose for Django.

Update: also helpfule https://github.com/ambitioninc/container-transform

boot2docker – simplify working with docker containers on a mac

/Geek Warning – all others flee and hide to avoid boredom./

Boot2Docker is a wrapper that makes working with dockers on a local computer just a bit easier.

That’s the goal. Keep it simple and it works. If we really want to keep it simple we now have kitematic to run docker containers with a GUI.

GUI for Docker Container Development
GUI for Docker Container Development

Plus people in programming mode sometimes get insecure and like to (seriously) debate Vi or Emacs as if those who don’t know it are of a lesser kind. I believe in letting compilers deal with my long select-case/kwargs/collections/dictionaries/whatever you want to call them. Thus I LOVE seeing tools that make programming EASIER. Especially visual ones even though I’m back to preferring the command line.

But back to boot2docker for working with containers locally. On a Mac, it’s nothing most of us haven’t done already with Virtual Box (free) or VMWare (not free but faster than VB) sharing a local folder into the VM. We all do this so we can use our dev tools without even more junk into the VM itself.

Number 1 reason to use it if you use a Mac? It doesn’t make me install homebrew (not a fan – sorry) and mess up the $PATH on my Macs. And Containers are just cleaner than virtualenvs as there are no dependencies, only attached containers. ‘Tis beautiful.

OK, the point of this post – if you are having any problems installing boot2docker on your Mac as I did, first it probably is NOT your fault. OK, well maybe it is because we all like bright shiny things and have a ton of junk installed. For me I would install it and it just would not start up. I found the documentation sort of glazed over a few things. So here you go.

  1. Upgrade to the latest Virtual Box BEFORE installing boot2docker. For me it just hung and didn’t give me a clue what the problem was. Do this first.
  2. Let b2d put its .pem and other keys where it wants to. You don’t need to even put these in your .ssh folder either.
  3. I did put it’s three environment variables in my bash because I’m lazy.
  4. You don’t have to sudo when you call “boot2docker up”. It doesn’t like that.
  5. You don’t have to make an entry in your /etc/hosts file for the dev server. Maybe more convenient but the IP works just as well.
  6. SSH – I made this waaaay 2 complex. it’s like vagrant. Just type “boot2docker up” followed by “boot2docker ssh” and you are in.

boot2docker-self-sufficient-runtime

Again – keep it simple and don’t make life complicated like I did. Install the boot2docker app for mac from the site. Install the package for Mac. Then from terminal:

boot2docker init

You only have to run that once. Then from anywhere:

boot2docker up
boot2docker ssh
exit
boot2docker down

Of course you might want to try out some of the exercises after the “boot2docker ssh” line when you are in the environment. Just trying to keep things simple, but no simpler than needed.

state

State
Impure functions are often more efficient but also require that the programmer “keep track” of the state of several variables. Keeping track of this state becomes increasingly difficult as programs grow in size. By eschewing state programmers are able to conceptually scale out to solve much larger problems. The loss of performance is often negligible compared to the freedom to trust that your functions work as expected on your inputs.

Maintaining state provides efficiency at the cost of surprises. Pure functions produce no surprises and so lighten the mental load of the programmer.

http://toolz.readthedocs.org/en/latest/purity.html

ImageMagick and Redmine on Ubuntu 12.04 LTS

Notes from the dungeon: ImageMagick and Redmine 2014-03-23

# abbreviated notes on installing redmine gems with imagemagick on ubuntu 12.04
# some of the following notes came from http://www.underworldproject.org/documentation/LibpngDownload.html
# which is an open source library for subsurface exploration like “magma” (yes, I said “magma”)

sudo su
cd tmp
wget http://sourceforge.net/projects/libpng/files/libpng16/1.6.10/libpng-1.6.10.tar.gz
tar zxvf libpng-1.6.10.tar.gz
cd libpng-1.6.10
./configure --prefix=/usr/local
make
make install

# imagemagik still doesn’t install the gem. Found this on StackOverflow but lost the link:

apt-get install graphicsmagick-libmagick-dev-compat

/rant/ – global variables like PATH are needed. Windows has the registry. In this case for ImageMagick to work with PNGs we need to tell it the home globally defined as PNG_DIR. There are several ways to do this in unix and in Ubuntu/debian. I personally don’t like variables stuck in .bashrc if the variables are really user specific. The .bashrc for your profile is NOT global. And skeleton profile files for unix named user profiles are a one-time thing that you can’t update everyone globally later once their profile is created. GLOBAL_VARS in .profile or .bashrc make (specifically you the network admin) your life convenient. But it’s not all about us. .bashrc variables or custom entries in /etc/hosts suck. Magical profile specific path variables break the python rule of “no magic” which means that in every other way they completely suxor. Let’s be honest – it just tempts you to run applications as root, right? None of us have ever seen that. HmmmmHMMMM? /end-rant/

Better option: profile.d applies to all bash prompts as well as any GUI that gets launched so it is currently my preferred method. (note “preferred methods” for me frequently last either 48 hours or 20 years, one or the other. Educate me if you have a better way.)

In this case we need to specify one variable only. But it will expand so let’s do it the right way. We’ll make a .sh file in /etc/profile.d. In practice I usually put these with my project without execute rights and then sym link them into the /etc/profile.d/ folder with “chmod +x myscript.sh” etc…. But to keep it simple – let’s just make our shell script as follows.

cd /etc/profile.d
sudo touch redmine_globals.sh
chmod 755 redmine_globals.sh
sudo nano redmine_globals.sh
# for bash put this in the script file
export PNG_DIR=/usr/local

# exit. or call it manually with “source /etc/profile.d/redmine_globals.sh”

PROGRESS? Let’s check it.

http:// your-domain-name-for-redmine/admin/info

NOoooooOOOOOoooo… Alas I still get a fail when I refresh the redmine page on the availability of ImageMagick. Major FAIL. I read a bunch more as it still didn’t work. Found this command and it executes properly.

sudo apt-get install libmagickwand-dev

#but did it work? Hmmmm.

bundle install --without test development

# still no

# Then tried this:

sudo apt-get install imagemagick libmagickwand-dev ruby-rmagick

# restarted

root@redmine:/var/data/redmine# touch tmp/restart.txt

SUCCESS!!!!!

Screen Shot 2014-03-23 at 3.12.46 PM

render magma – for the evil geniuses out there

MAGMA GIS simulation software Underworld

I have absolutely no need to calculate anything about magma. But it holds a sacred place in my mind and while fighting with RMagik gems I found a few helpful notes on the underworld site (like sub-surface GIS people, not the Mafia. Sheesh.)

Thus I recommend you check out just how far the open source community has come in terms of GIS and ESRI including subsurface water simulation, magma and much more. Check it here: http://www.underworldproject.org/models.html#Magma

Screen Shot 2014-03-22 at 5.37.12 PM

Brackets editor – new to me

Brackets editor by Adobe. OK, I assume they purchased it, but it’s under their brand. Link:

http://brackets.io/

After download on the Mac, if on Maverick, it will prevent you from running it. Instead of using Alfred or Spotlight, try instead:

1) Download
2) Install in apps (click the dmg from the step above)
3) Go to apps, find brackets.app, and hold CTL while you click it. This should bring up the option to “Open” as well. Do that.

Screen Shot 2013-11-25 at 2.08.36 AM

Apple’s switch to SMB2 with Maverick and Developing with VMWare and Ubuntu

On Tendenci development configuration…. Through one of the thousands of sources of input that hit me in a given week between websites, newsletters, other programmers, employees and random people I talk to, it finally clicked with me the significance of Apple switching to SMB.

OK, to back up for non-geeks. Computers talk to each other and devices like printers using common protocols. Microsoft, going back to modifications to DOS has used SMB. (skipping a bunch of history here.) Fast forward to a few weeks ago when Apple released OSX Maverick for free. In the release of Maverick everyone talked about how it was FREE. They are giving away the software counting on us to buy the hardware. OK, I get that.

Tendenci Open SourceWhat they also did was change from their own network protocol, called Apple Filing Protocol, and switched to Microsoft’s protocol SMB. Wait, what? Why?

Well, first Apple made their OSX Server software $50 in the app store. A comparable server software package from Microsoft is $2500. So I purchased a Mac Mini server. Sadly with even 35 users it wasn’t that fast. AFP is slower that SMB I’m told. But they could have improved AFP. Instead Apple made the switch to SMB. This not only speeds up their server but most importantly it allows MACs to connect to local area networks managed by Microsoft Servers without any extra software or tech support needed.

Apple is moving into corporate America folks.

Apple owns the home/consumer market in my opinion, even if I have an android phone our house is full of mostly Macs. 1 or 2 PCs or Linux but mostly Macs. Our company is already fully switched to Macs and Linux and the Cloud. But a lot of companies have not. I’ll leave predictions of Apple’s strategy to break into the Fortune 500 to reporters far more qualified than me.

What I do know as a programmer is that my life just got a LOT easier.

/back to geek speak/ We program on Linux but use Mac laptops. So we are always connecting back and forth which is a pain. And developing locally, on an airplane for example, I need virtual linux machines that run on my local computer. For that I used to use open source Virtual Box by Oracle, but it’s too slow on a Mac IMHO. I tried VMWare Fusion 6 and apparently they have a deal with Apple allowing direct access to the hardware. All is know is that VMware is MUCH faster than virtual box or vagrant. And I’m impatient so I’ll pay the $70 ish for VMWare Fusion.

Previously to share folders between my local computer (Mac) as the host computer running a virtual Linux computer on VMWare (Guest) required me to set up sharing through VMWare. This gets complicated. Your host folders are mapped to /mnt/hgfs/ inside of linux. If you symlink into a project and install software, given it is a symlink that means your files will still install in the /mnt/hgfs/ folder. For example:

Project folder path to virtualenv inside of Linux 12.04 LTS might be:
/var/www/projects/mydjangoproject/venv/
Linking from VMware Fusion you would create a share perhaps similar to
/mnt/hgfs/shares/projects/mydjangoproject/venv/
that pointed to your virtual environment folder.

Because this is a sym link, if you install a virtualenv for example the path maintains the linux path. So a “which python” gets you something in the /mnt/hgfs/shares/projects/ folder instead of the /var/www/projects/mydjangoproject/venv/. This makes portability a problem.

Samba to the rescue. The above method required configuration of the virtual machine through VMWare fusion, which slows down designers. And doesn’t easily port to VirtualBox or Vagrant. You can make magic happen by using Samba:
https://help.ubuntu.com/12.04/serverguide/samba-fileserver.html

Installed in the guest OS, for me 12.04 Ubuntu, and setting up your /etc/samba/smb.conf file with something similar to this:

[www]
path = /var/www
browsable =yes
writable = yes
guest ok = yes
read only = no

Restart you VM and magically in Apple’s Finder you will now see the local VM in your “shared” portion of finder.

Lastly for the programmers out there, do a bit more research before using anything other than NAT on your local for security. You have to configure file sharing security on the Mac host. Samba sharing security via smb.conf. And chmod/chown security on the folders and files inside the linux guest. While it might be tempting to just blow down the house with 755, remember that whoever takes that image might bridge the adapter and…. well, that would be your fault. So be careful out there kids.

Still, loving the fact that my directory structures can be identical, that I can pass off a vmware image to a colleague and it JUST WORKS. Dreamweaver edits, bash, git, whatever. Between SSH and the adobe suite you are now all powerful to make better looking applications using better software. Rock. On.

OS X Mavericks – 3 items of note for web devs

Apple’s next OS release has 3 items that I find very interesting for developers. Some good and some bad.

Website Push Notifications – OSX (not IOS, but the desktop) – people rarely install another browser on their iphone. Website push in their words: –

“Keep users up-to-date with news and other alerts using Apple push notifications. Once users have signed up for notifications from your website in Safari, you can send them push notifications that appear just like Mac app notifications, even when Safari isn’t running.”

Linkedin – it looks like they are getting married with LinkedIn with LinkedIn single sign on. Since everyone else is using Google Apps for single signon, or leaning that way, I think this brings a lot more power to linkedin.

Developer Tools – they SAY, they are going to auto-download developer tool dependencies. I read that as GCC will download without having to go and update the add-ons in xcode every time a new release of Python comes out or you get a net Mac. The jury is out for me on this one if it would even work or if it is a good idea.

https://developer.apple.com/osx/whats-new/

simplify, because I can see why Cal hates Django even if I love it

Yes I love Django and Tendenci has been rewritten fully on the Django framework. Open Source as well. (Go install it and tell us what you think.)

But there are days…. like right now…. like right this instant…. there are days when instead of doing big things I’m fighting inconsistent name spaces between apps for no reason. We have PEP. But nobody thought to go “ya, let’s just always call this one “title_block” in the base.html template? Or no hook to a global template on a site? Or standard theme definitions. For all the beauty I see in Django somedays I feel like I’m revisiting 2001 with setting up conventions. We don’t have to lock in like Rails, we don’t have to lose flexibility to gain efficiency.

You still can’t easily sequence the loading of installed apps for packages as well as your local apps. (better get ReportLab in the right spot or else!) So you roll your own. Days that I struggle with deployment even though those smarter than me have written scripts to help. New employees just deploy straight to databases on Heroku or use SQLLite locally, never getting the errors in South that a real user would get trying to set up their own server somewhere between the cloud and non-production.

And then there are the days I see 100+ queries go by on the debug toolbar on a page that used to only need 7. And everyone just says “cache” it as if the Calomine lotion will cover up the rash of inefficiency.

Don’t get me wrong, I’d rather walk on broken glass than convert to PHP. (yes let the flame war begin). But …. but …. I’m a big enough kid to find myself laughing out loud at this keynote called “Why I Hate Django” by Cal Henderson.

Oh yes I really do believe the future is Python and Django. I’ve bet the farm on it as they say. And if Facebook has to compile PHP basically into C++ then all of us will face these problems at Scale.

Yet can we all just admit that we can and should improve our conventions, our name spaces, our theme standards, remove the incentive for programmers to keep removing the local_settings.py file from the root of the /conf/ or site folder. Use manage.py runserver or foreman start? That’s kind of a big question if it changes the way you load your environment variables? We are creating a learning curve that will send all but the most stalwart young programmer back to PHP.

I don’t even consider myself a “real” Django programmer yet as I keep having to look up syntax. But having written one web framework that has stood the test of time, and now working with the team who is rewriting it, I see danger in them hills. The danger of not keeping things simple enough.

gdrive on the mac refuses to let you change the associated account

Ah google drive. Not exactly a Drop Box killer in any way, but with the conversion of google docs to the google drive many of us inherited it if we wanted it or not.

The first strike against this new imposition of gdrive rules is that apparently it doesn’t work so well with Google’s own Cloud Connect Product which integrates with Microsoft Office. But that wasn’t our particular problem with gdrive this evening. My problem with gdrive was that I was trying to cascade a laptop to one of my sons and it refused to let him log into the Gdrive with his gmail address. Even though all traces of the former owner had been removed, or so I thought…. (yes we uninstalled, reinstalled, Chrome, gdrive, rebooted, software updated, etc, etc….)

Mind you this isn’t google apps for business. This is a regular blah@gmail dot com type of email account that is also used for itunes etc. We uninstalled Chrome. Removed my other sons name from anyplace it appeared in System Settings on the Mac (OSX Lion 10.8.3). In particular I was sure to remove the former owner from all services under:

System Preferences / Internet & Wireless / Mail Contacts & Calendars

At this point after uninstalling Chrome and Gdrive both. Reinstalling Gdrive numerous times and changing everyone’s passwords I have been reduced to adding new aliases in my /.bash_profile file and my youngest son is like “Dad, it’s not that big of a deal I can just access it on the web site.” But no….. now I’m ticked.

Here is the error message and then the solution.

And for the search engines the text of the dialog box reads

"Google Drive. The account you entered does not match. Please sign in again with your ______@gmail.com account to proceed."

The catch is that account doesn’t exist anymore. Google drive was uninstalled as was Chrome. And gmail and web services are working fine on the new account when logged in to the web using Chrome or Firefox. It is JUST the Mac locally installed G Drive that won’t allow access.

The solution cobbled together from various blog posts. Using terminal with a sudo prefix:

sudo cd ~/Library/Application\ Support/Google/Drive
Password:

and you will see something like this:

LOCAL:~/Library/Application Support/Google/Drive
[Drive]$ ls -a
. CrashReports lockfile sync_config.db
.. cacerts snapshot.db

Just kill it all. From there is should be a matter of doing

cd ..
sudo rm -rf Drive

and then launching Gdrive on the mac again. This time it should prompt you to login with the new account associated with that computer. I can’t recall if we did a reboot in-between this step or not so consider that as well. And empty your Trash before you reboot because you probably need to do that anyway.

Two side notes learned over the years dealing with the Mac-n-Cheese-version-of-shiny-BSD-Aqua-Unix we call Macs (?rant? POSIX absolutely SHOULD be acceptable in WWF. Grrrrr.).

1) Do not EVER change your primary user admin home directory. Ever. You can change anything else on a Mac, but it will be a living hell trying to find every reference to the home directory baked into the OS. So if you buy a laptop from bob smith and his home directory is ‘bsmith” then either get OK with it, or rebuild the machine.

2) Gdrive, and google docs, are fine for personal use IMHO. So is DropBox. But neither have scaled for us at work even using google apps. Two alternatives I would encourage you to look at for business file sharing and syncing are SpiderOak and SugarSync. We are testing both although I am leaning heavily towards SpiderOak because it is lower cost and has better security. Of course like TrueCrypt or 1Password if you lose your key there is no plan B, but that also means your data is truly secure and you should invest in a lock box at a bank and (gasp) right down the authentication.

Installing iPython Notebook on a Mac OS X Mountain Lion

Installing iPython Notebook on a Mac OS X Moutain Lion 10.8.1 for Development and Testing

The “Quickstart” is anything but that leaving off details like the recomended dependencies are basically required for a functional notebook. Here is the sequence that worked for me.

[apps]$ mkvirtualenv ipythonvm –distribute
[apps]$ workon ipythonvm

at this point pip freeze shows

[apps]$ pip freeze
distribute==0.6.28
wsgiref==0.1.2

next up the big instAll

[apps]$ easy_install ipython[zmq,qtconsole,notebook,test]

the instructions suggest running iptest. Don’t bother yet. It will fail the tests without more dependencies like “nose’ installed. Keep on. Also note the “easy_install” in front of readline is specific to set the sequence to pass the tests. Not sure why. No time to question today and I’m in a VM so it can’t do much harm. Proceed with.

[apps]$ pip install nose
[apps]$ easy_install readline
[apps]$ easy_install pexpect
[apps]$ easy_install ipython[zmq]

At this point checking packages shows

(ipythonvm)LOCAL:~/dropbox/code/apps
[apps]$ pip freeze
distribute==0.6.28
ipython==0.13
nose==1.2.1
pexpect==2.4
pyzmq==2.2.0.1
wsgiref==0.1.2

but

[apps]$ ipython qtconsole

fails. Keep trying.

pip install Tornado

Tornado works, but still errors on ZeroMQ. It comes down to this. We need ZeroMQ and PyZMQ. PyZMQ is installed (see pip freeze lista bove) but is missing the dependencie ZeroMQ doesn’t work with pip or easy_install as far as I can tell. Now we go old school. I also didn’t read the docs and was a bit tired so rather than specifying to intsall in the virtualenv I accepted the defaults and installed it in /usr/local/bin globally.

for ZeroMQ, I downloaded the Mac latest stable release from this page:
http://download.zeromq.org/

On 9/15/2012 I downloaded this one and extracted it:
zeromq-2.2.0.tar.gz 14-Apr-2012 09:53 1.8M

Unpack the .tar.gz source archive and cd into that directory. Remember I installed globally for this one package after the battle so first I had to “deactivate” in the virtual environment I was in (iphythonvm for me). Then change into that directory someplace you have “write” rights. For me I copied it from downloads to “code/contribs” which is where I put random stuff I haven’t modified but may or may not be using. Thus the next command was:

cd zeromq-2.2.0/

Run ./configure, followed by make.

But remember, outside of a VM we are back to sudo so this looks like:

sudo ./configure
password:
sudo make
sudo make install

switch back to my VM

[apps]$ workon ipythonvm

the environment now looks like this:

(ipythonvm)LOCAL:~/dropbox/code/apps
[apps]$ pip freeze
distribute==0.6.28
ipython==0.13
nose==1.2.1
pexpect==2.4
pyzmq==2.2.0.1
wsgiref==0.1.2

now we try again.

[apps]$ ipython notebook

Success! Now let’s go get somethign to look at. I created a folder in my apps folder to put the downloads and grabbed a git repository from blogger Titus of Living in an Ivory Basement http://ivory.idyll.org/blog/teaching-with-ipynb.html

mkdir iPythonNotebooks
cd iPythonNotebooks
git clone git://github.com/ngs-docs/ngs-notebooks.git

Now we test it again:

[iPythonNotebooks]$ ipython notebook
[NotebookApp] Using existing profile dir: u’/Users/eschipul/.ipython/profile_default’
[NotebookApp] Serving notebooks from /Users/eschipul/Dropbox/Code/apps/iPythonNotebooks
[NotebookApp] The IPython Notebook is running at: http://127.0.0.1:8888/

I then imported the ngs-10-blast notebook. There are codeblocks in the notebook, so for a proof of concept I just picked one that imported “blast” knowing I had not imported it. Inside of Chrome, inside of the notebook, I can click on a codeblock that begins:

import csv
import blastparser

# open the output file for reading
fp = open(‘out.txt’)

and then I select “cell run” and it runs it as if I was in Eclipse. Properly giving an error of

—————————————————————————
ImportError Traceback (most recent call last)
in ()
1 import csv
—-> 2 import blastparser
3
4 # open the output file for reading
5 fp = open(‘out.txt’)

ImportError: No module named blastparser

That is huge. Think about it. Running code inline in the middle of a web page with a compiler and debugging. Not javascript by Python in a sandbox that can import modules and do everything else you would do with idyl. And saved in JSON based notebooks that can be shared and used for testing. It sort of blows your mind.

I’m not suggesting this is a replacement for a good training video. But it is a great addition to the educators arsenal of tools for online learning in richer environments with greater interactivity. I’m impressed to say the least.

Other randome take-aways. I did not know about the pexpect package and it is pretty compelling if you work at a company. It removes the need for the C libraries for builds which means that not everyone needs to install xCode if they have to /configure, make, make install, etc.

Thus I recommend you take a look at what pexpect can do as it was new to me. Pretty cool actually.
http://www.noah.org/wiki/Pexpect#Description_of_Pexpect