Recently, a bash script that I've used many times on lots of different types of servers was giving very strange results. Typically the script is called as ./script.sh info.cfg and proceeds to run through several processes. On this occasion, I was left with : command not found. When I hit the script with bash -x I could see that both the script and the config file were giving strange results and choking on line-breaks and curly braces. I checked the bash version, which was indeed up-to-date enough. The problem turned out to be that the files were being interpreted in the wrong format!? Solution?: Run :set ff=unix on all of the files in vim. Nothing too complex, but this had me scratching my head for a few minutes.

While on the topic of vim, I took a few swings on Vimgolf a little while back. As a generalist in the editor sphere, I found this to be a really neat way to gain exposure to advanced vim techniques without the overhead of RTFM.

While on the topic of Github-integrated learning applications :), I also dipped my toes over at Exercism.io. This is such a cool implementation! Polyglotism, here we come.

To change topics, I was working on a Drupal site that showed several custom modules enabled but not present in the site codebase. It sometimes happens that a site maintainer/developer will delete a module without properly disabling it in the website configuration. This can cause unexpected performance and behavioral issues for a site, so it's important to resolve this by reinstating the missing modules and following proper uninstallation procedure. In this case, the missing modules were in the repo history. Here's how to retrieve them.

Find the commit where a module was deleted:

git rev-list -n 1 HEAD -- <file_path>

Checkout the revision just before the delete commit:

git checkout <delete_commit>~1 -- <file_path>

Note: You can go back further than 1 commit, substituting the desired number for '1', and using the delete commit as an anchor.

I've dabbled with docker containers a few times using Vagrant and VirtualBox per the documentation for OSX. What isn't clear is how to easily access docker deployments from the OSX host, but after poking around the webs, I eventually stumbled on this little nugget. When firing up the VM that will host the docker deployments, rather than running vagrant up, insert a little snippet on the front of the command to specify port forwarding as follows:

FORWARD_DOCKER_PORTS=1 vagrant up

If your VM is already running, you can s/up/reload :)

bash tricks accumulated over the last several weeks:

Argument expansion

Need to move or copy a file under a long directory tree?

cp /this/is/a/really/long/path/to/{file-01,file-02}.txt

Git filter-branch

Need to get rid of large media files in your Git repo throughout the entire history of the project?

git filter-branch --index-filter \
    'git rm --cached --ignore-unmatch ./path/to/file/*.ext' \
    --tag-name-filter cat -- --all

Supports wildcard matching!

History Expansion

There are several techniques here, but my current favorites are sudo !! (re-run the last command as sudo), and !<string> (run the most recent command starting with string).

Long-running Processes with screen

Running a multi-hour/day process and don't want your computer to be tied to the task? Use screen!

  • Create Ctl-a Ctl-c
  • Switch Ctl-a n, Ctl-a p
  • List Ctl-a "
  • Name Ctl-a A
  • Kill Ctl-a k
  • Detach Ctl-a d

To re-attach you can list screens with screen -list. Then re-attach with screen -r {SCREEN-ID}.

Git stash

Just because it's not entirely clear how this works, but it really is this simple:

git stash save "Short description"
git stash list
git stash pop stash@{0}

If you're on "branch-A" and want changes to apply to "branch-B", simply checkout branch-B and then stash.

Remove Pesky Windows Linebreaks

I frequently observe Windows-style line-endings in code files. These can introduce confusion into the repo history, as inconsistency in line-break handling will cause lines that may have no visible changes to be recorded as a change. Here's a little vim magic to remove these cleanly:

:%s/<Ctrl-V><Ctrl-M>//g

I've been playing with Hubot a bit lately, and decided to up the ante on the endeavor by creating a Hubot Docker container. There were a couple of misadventures before landing on a stable source container from which it will now be crazy easy to extend and deploy.

I set up Docker using the instructions here.

You may notice from reviewing the Dockerfile that line 6 imports a 'Universe' apt source. This is to support dependencies associated with installing a newer version of Node.js (v0.10.19) in a container. To facilitate deployment speed, the container doesn't include this repository by default. Further down on line 9, another apt repository is added to point to a newer Node.js package. You can find more information about this here.

The Dockerfile includes several different conventions, including RUNning commands, and ADDing files and ENVironment variables to the container.

I had initially tried setting up an S3 Brain with the aws2js Hubot script. I got it working several times, but ultimately found the npm package handling for this and a few other Hubot scripts to be far too brittle to recommend currently. Most often, package retrieval would hang, fail, or occasionally fail silently :/ In each case the following error would appear:

Error: Cannot find module 'aws2js'

Same problem, one tool... (this would be a great time to set up an S3 media bucket to show results). In addition to ImageMagick, I installed ghostscript with brew install ghostscript to enable working with eps vector images.

Starting with a square profile photo:

identify nhoag-bw-sq.jpg
/path/to/nhoag-bw-sq.jpg JPEG 480x480 480x480+0+0 8-bit sRGB 65.5KB 0.000u 0:00.000

Generate a black rectangle to match the width of the profile photo:

convert -size 480x100 xc:black black.jpg

Reduce the opacity of the new rectangle:

convert black.jpg -alpha on -channel alpha -evaluate set 25% shade-25.png

Overlay the modified rectangle on the photo:

composite -gravity center shade-25.png nhoag-bw-sq.jpg nhoag-shade.jpg

Convert black text eps file to png:

convert -colorspace RGB -density 300 black_text.eps -resize x95 black_text.png

Reduce the opacity of the black text png:

convert black_text.png +flatten -alpha on -channel A -evaluate set 10% +channel black_text_opac.png

Create a slightly larger canvas for the black text png (to accommodate blur):

convert -size 475x110 xc:transparent black_text_canvas.png

Overlay the black text png on the new canvas:

composite -gravity center black_text_opac.png black_text_canvas.png black_text_opac_canvas.png

Blur the black text png:

convert black_text_opac_canvas.png -blur 0x4 black_text_opac_canvas_blur.png

Overlay the black text png on the profile photo:

composite -gravity center black_text_opac_canvas_blur.png nhoag-shade.jpg nhoag-shade-black_text.jpg

Convert white text eps file to png:

convert -colorspace RGB -density 300 white_text.eps -resize x95 white_text.png

Overlay white text png on the profile photo:

composite -gravity center white_text.png nhoag-shade-black_text.jpg nhoag-shade-black_text-white_text.jpg

Generate a new png with the text 'profile':

montage -background none -fill white -font Courier \
    -pointsize 72 label:'Profile' +set label \
    -shadow  -background transparent -geometry +5+5 \
    profile_text.png

Overlay the 'profile' text on the profile photo:

composite -gravity center -geometry +0+90 profile_text.png nhoag-shade-black_text-white_text.jpg nhoag-shade-black_text-white_text-profile.jpg

You can view the finished product on my Acquia Google+ profile.

I recently modified the Google profile image on my work Google+ account to make it more clear that it's a work profile rather than a personal profile. I added a few layers to my existing profile image including a work logo and a couple of simple transparencies. In the past I would have relied on Photoshop, GIMP, or another heavy image manipulation GUI software to accomplish this task. This time around, I decided to make use of the OSX Preview program in conjunction with the command line tool, ImageMagick.

To get ImageMagick running on OSX is as simple as running brew install imagemagick with Homebrew. This gives you tons of image manipulation powers that are described under the command man convert.

Preview is pretty much a terrible image editing interface. But with patience you can use it to minimally mash up several layers of images. To start, I had a profile JPG image, and a logo PNG file. To add to the mix I created a couple of lightly shaded transparencies to encapsulate the logo. Following are a set of commands to generate a simple shaded transparency.

Generate a black rectangle:

bash New Black Rectangle convert -size 20x80 xc:black black.jpg

Add transparency to the black rectangle:

bash Add Transparency convert black.jpg -alpha on -channel alpha -evaluate set 25% black-25.png

Add more transparency:

bash More Transparency convert black-25.jpg -alpha on -channel alpha -evaluate set 50% black-12.5.png

Now to mash the images together, open them all up in Preview, 'select all' for an image you want to overlay on the profile, paste the image into the profile, then resize and/or re-orient the overlay image dimensions as desired.