Below is a Github Gist with a map comprised of GeoJSON data, along with the bash command that was used to create it.

Github automatically generates maps for GeoJSON files in Gists and repositories. This is well documented at the following article: https://help.github.com/articles/mapping-geojson-files-on-github

The map below pulls data from eBird API, transforms the data structure into GeoJSON format using jq, and then POSTs to a Github Gist.

Previously, I covered the use of the git filter-branch for removing large media assets from a repository's history.

Recently I had a new opportunity to perform this task on whole directories. Here is the command sequence I used to clean up the repo history and to shrink the pack file:

# Remove a directory from history in all branches and tags
$ git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch path/to/dir' --tag-name-filter cat -- --all

# Shrink the local pack
$ rm -Rf .git/refs/original
$ rm -Rf .git/logs
$ git gc
$ git prune --expire now

# Push up changes to the remote
$ git push origin --force --all
$ git push origin --force --tags

Additionally, it turns out I only had a partial list of assets to remove from the repo. I was able to track down the rest with a couple of additional Git commands:

# Show the 20 largest items in the pack
$ git verify-pack -v .git/objects/pack/pack-{hash}.idx | sort -k 3 -n | tail -n 20

# View a pack object
$ git rev-list --objects --all | grep {hash}
{hash} path/to/pack/object.ext

Rinse and repeat.

I've been using Oh-My-Zsh and Powerline for vim for a while now, but just recently converted to Prezto and vim-airline, repectively, on word of the usability and performance improvements. Already, the benefits have been well worth the minimal effort required to set these up.

  • Note: anecdotally, the original switch from bash to zsh was quite easy - I understandably hear questions about this often.

With Prezto, two improvements stand out markedly:

  1. Using the Oh-My-Zsh git plugin, I would often see a long lag time (sometimes 5+ seconds!) between cd'ing into a git repo and the appearance of the prompt - I'm kind of surprised I never traced that =/ In any case, Prezto is consistently near-instantaneous.
  2. I had a reverse history search with Oh-My-Zsh that would check for the string ahead of the first space (i.e. cd th would show me any previous cd commands via the 'up' arrow). The Prezto reverse history search plugin does two or three better by letting me search for strings with spaces, 'cd' will match any command from history that had 'cd' anywhere in the entire command, and there is a nice highlight on each match.

With vim-airline, set-up and customization were an absolute breeze - especially compared to powerline, which is notoriously difficult to install.

So as you can tell, I'm sold. I haven't been this excited about the shell in at least months :)

Following are the Git commands and Git family of tools that I use in my day-to-day.

Before writing this post, I generated a report of my most-used git sub-commands with the following:

history | awk '$2 == "git" {print $3}' | sort | uniq -c | sort -nr | head -n 20

Most of my Git usage is bread-and-butter Git commands, but I'll highlight some of my favorite Git features below.

Say you've just added and committed a change, and only then you spot a typo in your changes. Rather than adding a whole new commit for that, simply fix the typo, git add the fix, and then git commit --amend to incorporate the typo fix in the last commit.

You massively messed up a file or directory and just want to start over on that one thing:

git checkout ./path/to/messed-up-thing

Need to pull in a file from another branch in your repo?:

git checkout branch-name -- path/to/file

Need a new branch that's completely devoid of historical context?:

git checkout --orphan branch-name

When cloning a new Git repo, you don't always want to clone the 'master' branch, or to use the repo name as the directory name in your local clone. Here's how to get around both of these in one swoop:

git clone -b not-master repo-address.git local-name

Need to see a diff for a single file?:

git diff ./path/to/file

Want to add new files, modified files, and deleted files all in one go? Try git add -A. You can also refine this command by specifying, ./path/to/directory-or-file.

For those times when you really need to reset, nothing beats git reset --hard. But wait a minute... all the new untracked files are still popping up with git status!! Sure we can grep | awk | xargs rm that cruft, but Git has a convenient (yet dangerous!) feature:

git clean [-f (-d)?]

I use git log often enough, and the prettification options make the command very useful, but my go-to for Git log review these days is good ol' tig. It's interactive, has great defaults, and is very easy to type.

Why do I use git remote so much?! The main reason is that I am usually connected to multiple remotes from a local clone - one for the canonical repo, and a second for my fork. git remote -v shows what my clone is connected to. This way I can easily rebase from upstream and ensure that my fork is in sync.

While merges are useful, I generally dislike merge commits. Aside from rebasing and setting git to auto-rebase on merge, another of my preferred methods for sharing commits between different branches is git cherry-pick commit-hash. With cherry-pick, you're essentially distilling a commit down to a diff and applying that diff to another branch, all in one awesomely concise command. cherry-pick even supports commit ranges as follows:

git cherry-pick 'older-commit-hash^..newer-commit-hash' # the '^' stands for inclusive

Certain types of repositories are particularly well suited to the subtree command. I happen to work with an open source project called Drupal, that when configured as a multisite is an excellent candidate for the subtree workflow. In this case you might find the following commands to be of use:

git subtree add --prefix sites/site-name multisite-repo-name branch-name --squash
# Then to update
git fetch remote-repo-name remote-branch-name
git subtree pull --prefix sites/site-name multisite-repo-name branch-name --squash
# And to push
git subtree push --prefix sites/site-name multisite-repo-name branch-name

In Drupal, subtree is particularly useful with sites that share a common profile or a common module or theme composition. It's also handy for keeping aspects of a multisite secret, or to retain control over the release of new features or sites to production.

Reset a git repo to zilch with the following (careful now!):

mkdir dirname
cd dirname
git init
git remote add remote-name repo-address
git push --mirror --force remote-name
# Data destroyed!

Need to migrate an existing Git repo to a new repo address? Here are the commands:

git clone --mirror src-repo-address migrate
cd migrate
git remote add dst dst-repo-address
git push --mirror dst
cd ..
rm -rf migrate

I flip a lot of websites around between servers, and often haven to migrate SSL certificates as part of the process. The below openssl commands represent most of what I encounter in terms of validations and encoding conversions.

Primer

First, a couple of basics on certificate components. The four SSL components to know are the Certificate Signing Request (CSR), the key, the certificate, and the Certificate Authority (CA) intermediate certificates. When starting fresh, you'll first generate a CSR and a key. The CSR can then be taken to a Certificate Provider to generate the certificate, while the key should be protected at all costs.

Finding SSL Components

Primer over, let's get down to the business of moving existing certificates. Depending on permissions and ownership, you may need root access for some of the following.

To find a certificate on an unfamiliar web server, you'll want to reference the VirtualHost (vhost) file. If you don't know the location of the vhost, you'll want to run something like the following command to get a list of vhosts on the server:

/usr/local/apache2/bin/httpd -S

Once you've pinpointed the vhost location, scan the file for the SSLCertificateFile and SSLCertificateKeyFile references:

<VirtualHost ipaddress:443>
        SSLEngine on
    # snip
        SSLCertificateFile      /path/to/ssl.crt
        SSLCertificateKeyFile   /path/to/ssl.key
    # snip
</VirtualHost>

We can now proceed with the referenced ssl.crt and ssl.key files from the vhost.

Certificate Encoding

In validating certificate components, you may encounter an unable to load certificate error. This means the certificate component is not encoded in the expected format. Following are a few commands for testing and transforming the encoding of certificate components.

Test if the key is PEM or DER:

# PEM
openssl rsa -inform PEM -in ssl.key
# DER
openssl rsa -in ssl.key -inform DER -text -noout

Test if the certificate is PEM or DER:

# PEM
openssl x509 -inform PEM -in ssl.crt # PEM
# DER
openssl x509 -in ssl.crt -inform DER -text -noout # DER

Convert the key to a new encoding:

# PEM
openssl rsa -in key.der -inform DER -out key.pem -outform PEM
# DER
openssl rsa -in key.pem -inform PEM -out key.der -outform DER

Convert the certificate to a new encoding:

# PEM
openssl x509 -in ssl.der -inform DER -out ssl.pem -outform PEM
# DER
openssl x509 -in ssl.pem -inform PEM -out ssl.der -outform DER

Validate SSL Components

Do the certificate and key match?

# LONG FORM
(openssl x509 -in ssl.crt -noout -pubkey; \
openssl rsa -in ssl.key -pubout)

# SHORT FORM
(openssl x509 -noout -modulus -in ssl.crt | openssl md5; \
openssl rsa -noout -modulus -in ssl.key | openssl md5)

Check the dates that the certificate is valid:

openssl x509 -noout -in ssl.crt -dates

View the entire contents of the cert:

openssl x509 -in ssl.crt -noout -text

You may not have the CA intermediate certificates. These can be acquired from vendors by certificate type. As an example, you might see CN=DigiCert High Assurance CA-3 in the certificate contents. This would lead you to the Digicert Root and Intermediate Certificate webpage, where you can reference the table for the appropriate CA intermediate certificate download.

Verifying the certificate chain:

openssl verify -CAfile chain.crt ssl.crt
  • Note: You may need to also convert the encoding of the intermediate cert.

Conclusion

Once everything is validated and converted to the expected encoding, you can proceed with installation of the SSL Certificate on firm footing.