goaccess

ncdu for the Win While reading through some internal tool enhancement tickets at work the other day, I happened accross a quick mention of a command line tool that I’d not yet seen, but which proved to have immediate value. The tool is ncdu, ‘NCurses Disk Usage’, which as the man page states, “…provides a fast way to see what directories are using your disk space.” In the process of onboarding new sites to Acquia Cloud, it’s not always clear where the lines have been drawn with regard to separating out code and media assets for a site....

I made a bunch of big changes to the GoAccess shell plugin including improved argument handling, greater agnosticism, and greater configurability with time filtering. I converted the plugin from sourced script to regular bash script. This meant the script components had to be ordered into a sequential configuration, but this also makes the script more portable and easier to fire up. The script options now support short and long call values, courtesy of Stack Overflow....

I made some good progress on the GoAccess plugin over the past few days. Many of the kinks have been ironed out and making access.log reports has never been so easy :) It’s a pretty simple plugin, but it does the job well. My favorite parts of the script are a fun bit of regex that’s just aesthetically pleasing, the awk date filter, and the overall flow of execution. yes='^1$|^([y|Y]([e|E][s|S])?)$' cmDATE=`date -u -v-"$goINTVAL"H +\[%d/%b/%Y:%H:%M:%S` # OSX date command format (-v)....

An excellent log analysis tool that I picked up recently from a blog post by my colleague, Amin Astaneh, is the GoAccess interactive web log analyzer. Out of the box, you can unleash GoAccess on raw or piped log data to reveal an array of interesting traffic patterns that might otherwise take some serious piping skills to crack - I covered some of these in a recent blog post.