Category Archives: Open Source

Tiny linux disk usage reporting script – dux.bash

I decided to write a very small disk usage reporting script that provides some extra information than does just using ‘du‘ directly.  The script of course uses du, among other command line tools and parsing commands to generate the little report.  It suits my needs for the moment. Hopefully others out there will also find it useful.

Firstly, here is the script output to give you an idea of how to execute it, what it does, and how it looks:

$ ./dux.bash /home/jbl /home/jbl
Building extended du reports for /home/jbl in /home/jbl ...
[du extended report]:
13G     ./Personal
4.7G    ./DATA
2.4G    ./Pictures
1.4G    ./Downloads
Total (GB): 21.5
 
373M    ./core
260M    ./tmp
73M     ./game-saves
37M     ./new-music
33M     ./new-books
32M     ./vim-env
24M     ./random-tools
15M     ./stuff
Total (MB): 847

The script takes two arguments, the directory you want to analyze, and the directory where you want to store the reports.

As you will see, the script provides me an at-a-glance look at the following characteristics of a particular directory structure:

  • Grouping and separation of larger (GB sized) directories from smaller (MB sized) directories.
  • Directories sorted by size in each group.
  • Total sizes for each group of files and directories.

With this output I can see clearly which directories are causing the most contention, and how much of an impact they have compared to other directories.

The script is very crude, and probably needs some work and error correction (accounting for files off or root, etc.)   It also creates some temporary text files (used to construct the report), which is the reason for the second argument to the script.  However for now it’s doing what I need, so I figure it’s worth sharing.  Here it is:

#!/bin/bash
echo "Building extended du reports for $1 in $2 ...";
cd $1
du -sh $1/* > $2/du-output.txt
cat $2/du-output.txt | egrep '([0-9][0-9]M)' > ~jbl/du-output-MB.txt
cat $2/du-output.txt | egrep '[0-9]G'> ~jbl/du-output-GB.txt
cat $2/du-output-MB.txt | sort -hr > $2/du-output-MB-sorted.txt
cat $2/du-output-GB.txt | sort -hr > $2/du-output-GB-sorted.txt
echo '[du extended report]:';
cat $2/du-output-GB-sorted.txt
echo -ne "Total (GB): " && cat ~jbl/du-output-GB-sorted.txt | perl -pe 's/^(\d+\.+\d+|\d+)\w*.*/$1/g' | paste -sd+ | bc
echo ""
cat $2/du-output-MB-sorted.txt
echo -ne "Total (MB): " && cat ~jbl/du-output-MB-sorted.txt | perl -pe 's/^(\d+\.+\d+|\d+)\w*.*/$1/g' | paste -sd+ | bc

I’m not sure what more I will need from it going forward, so it may not get much love in the way of improvements.  Since it would be nice to have it on each system I’m using, I may convert it to a single script that has no need to generate temporary text files.  

Nuff said!  Hopefully you find either the script, or portions of it useful in some way!

For convenience, I’ve also added this script as a public gist so you can download dux.bash from github.

Cheers!

How to Backup an Ubuntu Desktop (12.04, 14.04)

Source: http://askubuntu.com/questions/9135/how-to-backup-settings-and-list-of-installed-packages

Warning: Read about caveats in the link above before use

#——————————————————-

## The backup script
 dpkg --get-selections > ~/Package.list
 sudo cp -R /etc/apt/sources.list* ~/
 sudo apt-key exportall > ~/Repo.keys
 rsync --progress /home/`whoami` /path/to/user/profile/backup/here
## The Restore Script
 rsync --progress /path/to/user/profile/backup/here /home/`whoami`
 sudo apt-key add ~/Repo.keys
 sudo cp -R ~/sources.list* /etc/apt/
 sudo apt-get update
 sudo apt-get install dselect
 sudo dpkg --set-selections < ~/Package.list
 sudo dselect

#——————————————————-

Who is this for: users that have normal regular use of their computer, that have done minimal or no configuration outside their home folder, did not mess up startup scripts and services. A user that wants to have his software restored to how it was when he installed it with all customizations being done and kept in their home folder.

Who this will not fit for: servers geeks, power users with software installed by source (restoring the package list might break your system), users that have changed the startup script of some application to fit better their needs. Caution: there is a big chance any modifications outside home will be over written.

Ansible Playbooks – Externalization and Deduplication

Image result for ansible

Externalization and Deduplication

Developers who understand the concepts of modularity and deduplication should immediately recognize the power behind being able to include settings and commands from external files.   It is seriously counter-productive to maintain multiple scripts or playbooks that have large blocks of code or settings that are exactly the same.   This is an anti-pattern.

Ansible is a wonderful tool, however it can often be implemented in counter-productive ways.  Lets take variables for example.

Instead of maintaining a list of the same variables across multiple playbooks, it is better to use Variable File Separation.

The Ansible documentation provides an excellent example of how to do this.  However I feel that the reasoning behind why you would want to do it falls short in describing the most common use-case, deduplication.

The documentation discusses the possible needs around security or information sensitivity.  I also believe that deduplication should be added to that list.  Productivity around how playbooks are managed can be significantly increased if implemented in a modular fashion using Variable File Separation, or vars_files.   This by the way also goes for use of the includes_vars module.

Here are a list of reasons why you should immediately consider a deduplication project around your Ansible playbook variables:

Save Time Updating Multiple Files

This may seem like a no-brainer, but depending on the skills and experience of the person writing the playbook, this can become a significant hindrance to productivity.   Because of Ansible’s agent-less and decentralized manner, playbooks can be written by anyone who wants to get started with systems automation.  Often, these can be folks without significant proficiencies in programmer-oriented text editors such as Vim, Emacs, or Eclipse – or with bash scripting experience around command-line tools like awk, sed, and grep.

It is easy to imagine a Java developer without significant Linux command-line experience opening up one playbook at a time, and modifying the value for the same variable, over and over… and over again.

The best way for folks without ninja text-editing skills to stay productive is to deduplicate, and store common variables and tasks in external files that are referenced by multiple playbooks.

Prevent Bugs and Inconsistent Naming Conventions

In a perfect world, everyone would understand what a naming convention was.  All our variables would be small enough to type quickly, clear enough to understand its purpose, and simple enough that there would never be a mis-spelling or type-o.  This is rarely the case.

If left un-checked, SERVER_1_IP can also be SERVER1_IP, Server_1_IP, and server_1_ip.  All different variable names across multiple files, referencing the same value for the exact same purpose.

This mess can be avoided by externalizing this information in a shared file.

Delegate Maintenance and Updates to Variables That Change Frequently

In some environments, there may be playbook variables that need to change frequently.  If these variables are part of some large all-encompassing playbook that only some key administrators have access to be able to modify, your teams could be left waiting for your administrator to have free cycles available just to make a simple change.  Again, deduplication and externalization to the rescue!  Have these often-changing variables externalized so that users who need these changes immediately can go ahead and commit these changes to very specific, isolated files within your version control system that they have special rights to modify.

Cleaner Version Control History (and therefore Audit History)

If you have the same variables referenced by multiple files, and you make changes to each of those files before you commit them to version control, then your version control history can become a complete mess.  Your version control history will show a change to a single value affecting multiple files.  If you come from a software development background, and are familiar with the concept of code reviews, then you can appreciate being able to look at a simple change to a hard-coded value (or a constant), and see that it only affects one or two files.

I hope the reasons above convince some of you to start browsing your playbook repositories for possible candidates for deduplication.  I really believe that such refactoring projects can boost productivity and execution speed for individuals and teams looking to push changes faster while minimizing obstacles around configurations shared by multiple systems.  Send me a note if this inspires you to start your own deduplication project!

Examples of recursion, in Perl, Ruby, and Bash

This article is in response to the following question posted in the Perl community group on  LinkedIn:

I’m new to PERL and trying to understand recursive subroutines. Can someone please explain with an example (other than the factorial ;) ) step by step, how it works? Thanks in Advance.

Below, are some very simplified code examples in Perl, Ruby, and Bash.

A listing of the files used in these examples:

blopez@blopez-K56CM ~/hello_scripts 
$ tree .
 ├── hello.pl
 ├── hello.rb
 └── hello.sh
0 directories, 3 files
blopez@blopez-K56CM ~/hello_scripts $


Recursion example using Perl:

– How the Perl script is executed, and it’s output:

blopez@blopez-K56CM ~/hello_scripts $ perl hello.pl "How's it going!"
How's it going!
How's it going!
How's it going!
^C
blopez@blopez-K56CM ~/hello_scripts $

– The Perl recursion code:

#!/usr/bin/env perl
use Modern::Perl;
my $status_update = $ARGV[0]; # get script argument
 
sub hello_world
{
&nbsp;&nbsp;&nbsp; my $status_update = shift; # get function argument
&nbsp;&nbsp;&nbsp; say "$status_update";
&nbsp;&nbsp;&nbsp; sleep 1; # sleep, or eventually crash your system
&nbsp;&nbsp;&nbsp; &amp;hello_world( $status_update ); # execute myself with argument
}
 
&amp;hello_world( $status_update ); # execute function with argument


Recursion example using Ruby:

– How the Ruby script is executed:

blopez@blopez-K56CM ~/hello_scripts 
$ ruby hello.rb "Doing great!"
Doing great!
Doing great!
Doing great!
^Chello.rb:7:in `sleep': Interrupt
&nbsp;&nbsp; &nbsp;from hello.rb:7:in `hello_world'
&nbsp;&nbsp; &nbsp;from hello.rb:8:in `hello_world'
&nbsp;&nbsp; &nbsp;from hello.rb:8:in `hello_world'
&nbsp;&nbsp; &nbsp;from hello.rb:11:in `'
blopez@blopez-K56CM ~/hello_scripts $

Note: In Ruby’s case, stopping the script with CTRL-C returns a bit more debugging information.

– The Ruby recursion code:

#!/usr/bin/env ruby
status = ARGV[0] # get script argument
 
def hello_world( status ) # define function, and get script argument
&nbsp;&nbsp;&nbsp; puts status
&nbsp;&nbsp;&nbsp; sleep 1 # sleep, or potentially crash your system
&nbsp;&nbsp;&nbsp; return hello_world status # execute myself with argument
end
 
hello_world status # execute function with argument

Recursion example using Bash:

– How the Bash script is executed:

blopez@blopez-K56CM ~/hello_scripts $ bash hello.sh "..nice talking to you."
..nice talking to you.
..nice talking to you.
..nice talking to you.
^C
blopez@blopez-K56CM ~/hello_scripts $

– The Bash recursion code:

#!/usr/bin/env bash
 
mystatus=$1 # get script argument
 
hello_world() {
&nbsp;&nbsp;&nbsp; mystatus=$1 # get function argument
&nbsp;&nbsp;&nbsp; echo "${mystatus}"
&nbsp;&nbsp;&nbsp; sleep 1 # breath between executions, or crash your system
&nbsp;&nbsp;&nbsp; hello_world "${mystatus}" # execute myself with argument
}
 
hello_world "${mystatus}" # execute function with argument

Managing Your SMTP Relay With Postfix – Correctly Rejecting Mail for Non-local Users

Image result for SMTP postfix

I manage a few personal mail relays that I use for relaying my own mail and for experimentation purposes (mail logs are a great source of unique and continuously flowing data that can you use to try out different ideas in GUI, database, or parser development).  One of them was acting up recently.  I got a message from my upstream mail-queue host saying that they’ve queued up quite a bit of mail for me over the last few weeks, and that I should investigate, as they do want to avoid purging the queue of valid mail.

Clearly I wanted to avoid queuing up mail on a remote server that is intended for my domain, and so I set out about understanding the problem.

What I found was that there was a setting in my /etc/postfix/main.cf that, although it was technically a valid setting, was incorrect for the role that mail-server was playing.  Specifically the mail server was supposed to be rejecting email completely for non-local users, instead of just deferring it with a “try again later” message.

In this case, I’m using Postfix v2.5.5. The settings that control this configuration in /etc/postfix/main.cf are as follows:

  • unknown_local_recipient_reject_code
  • local_recipient_maps

local_recipient_maps

local_receipient_maps defines the accounts that this mail server will accept and relay mail for. All other accounts would be “rejected” by the mail server.

However, how rejected mail is treated by Postfix depends on how it is configured, and this was the problem with this particular server.

For Postfix, it is possible to mark a message as “rejected”, but actually have it mean “rejected right now, but maybe not permanently, so try again later”. This “try again later” will cause the e-mail message to be queued on the upstream server, until it reaches some kind of retry time-out and delivery is once again attempted. Of course this will fail again, and again.

This kind of configuration is great for testing purposes, because it allows you to test the same messages over and over again without losing them, or to queue them up so that they can be reviewed to ensure they are indeed invalid e-mail messages. However this is not the state you want your mail server to be in permanently. At some point once things are ready for long-term (production) use, you want your mail server to actually reject messages permanently.

unknown_local_recipient_reject_code

That is where unknown_local_recipient_reject_code comes in. This configuration property controls what the server means when it “rejects” a message. Does it mean right now, or permanently?

The SMTP server response code to reject mail permanently is 550, and the code to reject mail only temporarily is 450.

Here is how you would configure Postfix to reject mail only temporarily:

unknown_local_recipient_reject_code = 450

And here is how you set Postfix to reject mail permanently:

unknown_local_recipient_reject_code = 550

In my case, changing the unknown_local_recipient_reject_code from 450 to 550 is what solved the problem.

In summary, if you ever run into an issue with your Postfix mail server where you believe mail is set to be REJECTED but it still seems to be queuing up on your up-stream mail relay, double-check the unknown_local_recipient_reject_code.

# Local recipients defined by local unix accounts and aliases only
local_recipient_maps = proxy:unix:passwd.byname $alias_maps
 
# 450 (try again later), 550 (reject mail)
unknown_local_recipient_reject_code = 550

References
http://www.postfix.org/LOCAL_RECIPIENT_README.html
http://www.postfix.org/postconf.5.html#unknown_local_recipient_reject_code

My Fun With Necrolinguaphilia

Last night I attended a talk given by Dr. Damian Conway (of Perl Best Practices fame) titled “Fun With Dead Languages“.  Although this is a talk that Damian had given previously, it is the first time that I heard it, and I’m so glad I did!

I was the first to arrive at the Mozilla office building at 366 Adelaide, and so was able to score a sweet parking spot right across the street (no small feat in downtown Toronto).

I arrived and introduced myself to Damian as he was preparing for his delivery shortly before a herd of approximately 70 hackers (according to Mozilla) from all language and computing backgrounds started pouring through the meeting room doors to be seated.

Damian has a very energetic style of presentation, and was able to hold our attention while covering everything from the virtual extinction of the Gros Michel Banana, to the benefits and efficiencies of stack-based programming (using PostScript as an example).  He compares many, very different languages including Befunge, Brainfuck, Lisp, and Piet, and suggests that a great place to look for new ideas is what he calls the “Language Morgue”, where he includes languages such as Awk, Prolog, Cobol… and even C++ as examples of dead languages and language paradigms.

Mr. Conway also dived into excruciating detail on how the Latin natural language can be used as an effective computer programming language, and has even gone so far as to write a module called Lingua::Romana::Perligata, which he has made available on the CPAN.

I also had the special treat of sitting right behind Sacha Chua who brilliantly sketched notes of the entire talk in real-time.  I haven’t had the pleasure of formally meeting Sacha just yet (didn’t even say “hello”, my bad!) as I didn’t want to distract her.  Aside from having my mind blown by Damian’s talk, I was also being mesmerized by Sacha’s artistic skills, and so I do feel somewhat justified in keeping my mouth shut just to absorb everything that was going on right in front of me (front-row seats FTW!).

20130806 Fun with Dead Languages - Damian Conway

Sacha has made her “Fun With Dead Languages” sketch notes publicly available on her blog for everyone to review and enjoy, and has placed it under a Creative Commons license, so please share freely (and drop her a note to say “thanks!”).

Overall, I learned a lot from this talk and appreciate it immensely.  The energy of the audience made the discussion that much more enjoyable.  If you are interested in programming languages or language theory in general, I suggest you attend this talk the next time Damian decides to deliver it (or find a recording if one happens to be available?).  Damian Conway’s insights and humorous delivery are well worth the brainfuck ;)

Installing CentOS 6.4 from a Net Install Image on a Virtual Host

An Opportunity To Play Around with CentOS

One of the personal projects that I’ve always had itching away at the back of my mind was the urge to revamp my home network monitoring and security.  One of the tools that I love using for network monitoring is Xymon.  However, this gives me an opportunity to do things slightly different.  I have decided to give CentOS a go instead of my typical choice of Debian for a Linux distro in a server environment.  I am curious to see what advancements have been made in the RPM world, and I’d like to keep my Red Hat skills up to date.  What better way to do so than to set up a CentOS server with some production tools and services on it :)

Pre-installation Setup

So here we are, I have the CentOS Netinst (Net Install) image loaded into a VM, and I boot up the guest.

Since this is a fresh install on a 20GB virtual disk, I’m going to select “Install or upgrade an existing system” here.

I press “enter” and lots of console logging and scrolling action takes place.

Eventually I am prompted to “test the media”.  Usually this is referring to a physical CD typically used to install the OS on a physical server.  To me the phrasing feels a bit antiquated in this day of cloud services.

In any case, I still say yes, hoping that it will catch any errors in the ISO image file before I run into a bug during the installation process.  Better safe than sorry.

After the virtual disk is “successfully verified” to be OK, I try to move forward with the installation.

Be sure to note that after your virtual disk is verified OK, that the installer may decide to eject your CD media, in order to give you an opportunity to test other media.

Since I have no other media to test, this is actually kind of annoying. In order to continue with the installation, I have to go into the VM settings and re-connect the CDROM to the VM.

Select your language and keyboard options if the defaults are not suitable.  Otherwise, just  move past these dialogues by selecting “OK”, or hitting enter.

 

When you are asked “What type of media contains the installation image?”, select “URL”.
Continue reading Installing CentOS 6.4 from a Net Install Image on a Virtual Host

What is the “Cloud”?

The “Cloud” Will Save Us!

You hear about it every day, “cloud services”, “cloud storage”, “the cloud as a platform”. But what is the “cloud” really? The definition of what the “cloud” is, is different for everyone.  Some believe it is the implementation of a certain group of technologies, such as web servers, virtual hosts, and GUI frameworks.  Others believe it is a philosophy for modern software development and implementations – in particular web-based and mobile implementations. Others still see the “cloud” as simply a way of out-sourcing infrastructure – yet still somehow see the need to have dedicated “Cloud Administrators”.

So what is the “Cloud” really?  I offer my humble opinion below.

In With Old, Out with the New

Virtualization has been around for a very long time, so has Software as a Service (SaaS) and Platform as a Service (PaaS).  These technologies have been with us in different forms and iterations since the time of X11.  Of course, these technologies have evolved significantly over time, but that does not make them revolutionary, merely evolutionary.

I keep hearing phrases and comments to the effect of “the cloud changes everything”, when in fact it really doesn’t.  It is simply another form of outsourcing.  The real benefit of todays’ “cloud” technology is that it makes (or seems to make) management of infrastructure easier.  But convenience always come with a price.

Easier? Maybe Not So Much.. Especially For Seasoned Professionals

The easier things are, the more often you are likely to do them.  If it becomes easier to deploy apps via Amazon EC2/S3, or to a DotCloud instance, then there is a strong likelihood your organization will deploy more of them.  Instead of managing infrastructure, you are now concerned with managing deployment practices, configuration standards, and code-bases. Not to mention the human resources required to maintain those applications going forward.

The infrastructure “problem” doesn’t go away, it’s just relocated – it’s now someone else’s problem.

Over-Reacting and Under-Utilizing

When organizations frantically down-size their teams in a drastic attempt to remain modern, it bothers me; saddens me really, because deep down I know that the new cloud-based technologies these organizations are hoping to take advantage of are simply re-iterations and re-implementations of the same technologies they’ve always had to deal with.  HTTP, CSS, SSH, and Linux, for example.  It is quite likely that most companies with significant IT resources already have people who are skilled enough to rip through the implementation of “cloud” technologies, armed only with their previous experiences, and the core “problem-solver” attitude that they’ve always had, that doesn’t go away with time.

“Not enough Cloud experience.” Really? Do you mean using a GUI web interface to setup a remote host?  Or perhaps you mean the command-line configuration that needs to be done to YAML formatted text files in order to get a Rails application up and running?  Of course old-hat Systems Administrators or Web-Application developers don’t know “precisely” how it all works – the first time around.  But after the effort is put in to get the application up and running, to document the setup and check it into version control, and to automate as much of the time-consuming or repetitive manual tasks as much as possible, the rest is, as they say, “cake”.  What you need to focus on is developing the kind of people who can do all of this, and have fun with it.  This is how you effectively re-train.  This is how you retain good talent.  You have to allow the people you have to show you they can adapt.  It is a waste of experience to let people go because their experience is not up-to-date.  That’s not their fault.

More Of The Same, Spot The Patterns

Newer scripting languages and frameworks are being hyped as if they can do things that have never been done before.  I’ve seen this with the likes of Ruby, Python, and Perl. Despite the fact that Perl has one of the largest, organized, stable, and well-tested libraries  of any programming language to date (the CPAN), it doesn’t get the same kind of love that newer languages like Ruby and Python do, especially in corporate environments.  Sometimes it in fact does pay to re-invent the wheel, but most often it does not.

In Conclusion

If you are still trying to figure out what the “cloud” really is, know that it is simply a string of technologies that have been around for a long time, re-branded to look new and cool (for marketing purposes), and bundled with some new management tools and remote storage to make things “easier”.

To sales and marketing folks, it could simply mean trendy and cool.  To developers, it may mean LAMP or MEAN.  To systems and infrastructure people it could mean hyper-visors, virtual machines, and software containers.  To DevOps folks, it may involve Puppet, Chef, and Ansible automations, or Continuous Integration.

To recruiters and hiring managers, it often means Amazon AWS and Spring Framework Experience.  And to end-users, it typically means anything they can access from all of their phones, laptops, tablets, and PCs simultaneously.

The “Cloud” means many things to many different people.  My humble opinion? At it’s core – at the heart of the all the technology and implementation that has made it all possible; are tools, software, and individual experience that have been around since the beginning, and it is ALL based on the concept of Open Communication, and the spirit and foundation of Free and Open-Source Software.

Just Another Perl Hacker

Sometimes writing small snippets of code can be meditative.

The other day, I realized that, even though I happen to be, among other things, Just Another Perl Hacker, I never bothered to write my own JAPH signature.  So I went ahead and wrote up a very simple (but effective) one.  Once it was complete, it dawned on me that other perl hackers may appreciate the ability to generate a signature like my own.

Since perl is all about code reuse and sharing, I figured I would write up a JAPH signature generator so that anyone can have an awesomely obfuscated JAPH signature like I do.

Firstly here’s my JAPH signature:

1
2
$_='Kvtu!Bopuifs!Qfsm!Ibdlfs-!K/!Cpccz!Mpqf{';
@_=split//;foreach(@_){print chr(ord()-1)}

You can run the JAPH signature by copy/pasting it to a text file (e.g., japh_sig.pl), and running with

1
perl japh_sig.pl

Which returns:

Just Another Perl Hacker, J. Bobby Lopez

You can also run the JAPH signature straight off the command line (with ‘perl -e’), but you have to replace the single quote characters in the string with double-quote characters, for example:

1
perl -e '$_="Kvtu!Bopuifs!Qfsm!Ibdlfs-!K/!Cpccz!Mpqf{";@_=split//;foreach(@_){print chr(ord()-1)}'

This is all just for fun of course, but if you do end up using my JAPH signature generator, please let me know by sending me a quick message on Twitter to @jbobbylopez.

Have Fun!

The JAPH Signature Generator

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/usr/bin/env perl
##############################################
# USAGE:
#   perl japh.pl This is my awesome signature
#
# OUTPUT:
#   Your JAPH Signature:
#       $_='Uijt!jt!nz!bxftpnf!tjhobuvsf';
#       @_=split//;foreach(@_){print chr(ord()-1)} 
#
#   Returns:
#       This is my awesome signature
#
# AUTHOR: J. Bobby Lopez &lt;jbl@jbldata.com&gt;
##############################################
 
use feature say;
 
my $offset = 1;
my $signature = join (" ", @ARGV);
my $obfus_sig;
 
my @obfus = ();
my @sig = split //, $signature;
 
foreach my $c (@sig)
{
    push @obfus, ( chr ( ord($c) + $offset ) );
}
 
$obfus_sig = join ("", @obfus);
 
say &lt;&lt;"OUT";
 
Your JAPH Signature:
\t\$_='$obfus_sig';
\t\@_=split//;foreach(\@_){print chr(ord()-$offset)} 
 
OUT
print "Returns:\n\t";
@_=split//,$obfus_sig;foreach(@_){print chr(ord()-$offset)};
say;
1;

Why You Shouldn’t be Sharing “Live” Documents by E-mail

 

If you and your team, in 2013, are still sharing Microsoft Office (Word, Excel) documents via internal corporate e-mail, I’ve got news for you.  You’re doing it wrong.

“Live documents” are documents that are actively being updated and collaborated on by multiple people.  Collaborating on these documents by e-mail is a process that you should avoid.  It is a process that can eat away at your team’s productivity precious minutes at a time, and can severely impede your team’s work-flow and ability to stay synchronised.

I’ve been involved with projects where this method of collaboration was adopted.  Whenever I recognize this to be the case, I would immediately share my concerns, and try to suggest better ways of getting the team organized. There are always better ways to do it.

One of the biggest problems with e-mail document sharing is that there is no tracking or accountability.  There is no way to easily know what version of the document you have in your possession.  Is it the latest?  Perhaps it’s new enough?  Ever had to find an email with a document attachment, and ended up trying to craft clever little search terms to search your inbox?  Even if you find the document you were looking for, there is no way for you to know whether it is the last official revision, unless a system is implemented to allow “official” versions of the document to reside in a central location.

If there is a point-person in charge of managing this kind of set-up (for example, a simple system implemented with shared folders), and the maintainer ends up leaving the company for any reason (vacation, short-term disability, lay-off), then you still end up in a bad situation.  Without someone actively maintaining the structure of the document store, things will end up getting messy very quickly.  Users will begin storing documents in arbitrary locations (whatever feels right at the time), and before you know it, you will have to start yet another document archive clean-up project.

Version control is ubiquitous, and it is here to stay.  Any company (in any industry, not just IT) not seriously considering a process for document revision control should at least make it a point to have the discussion at least once a year.  You may find that your current document handling processes are actually a significant time waster, and that implementing a document management system could save you a lot of time (or money) over the long run.

There are many document sharing and collaboration technologies available today.  Some of the more popular include Sharepoint (if you are a Microsoft shop) or Documentum.  There are also many open source (free) packages, such as Drupal, Joomla, and Liferay.  There are even projects like Etherpad that make collaboration just plain fun.  You can also roll-your-own (if you are so inclined) by developing a custom system on top of foundational version control software such as Git or Bazaar, as I personally have done in the past.

Do your research when considering a content management system.  Some important considerations you might want to make include:

  • Is it easy to set up?
  • Is it easy to use?  Does it blend well with our team’s work-flow?
  • Is it safe?  Is it easy to make backups?
  • What kind of security mechanisms does it have built-in?
  • Is it easy to get our data out of the system (strong import/export functionality), in the event that we decide to move to another system in the future?
  • Is it cross-platform, or does it tie us to a specific platform (operating system)?
  • Is the cost worth the investment for a company our size?

The important thing here is to start thinking about it.  Be open to evaluating multiple products before you decide on a system that blends best with your organization’s work-flow. Software is about solving problems, which includes eliminating routine and time-consuming tasks.  If your company is not continually looking at new ways to improve efficiencies via clever (and practical) software implementations, then it will eventually be left in the dust as more efficient start-ups and entrepreneurs bring their shiny new productivity platforms to the game.

Adventures with Ubuntu 12.04 and Linux Mint 14 (Nadia)

Over the last week I’ve been playing around with Ubuntu 12.04 (Precise Pangolin) and Linux Mint 14 (Nadia).  Although I can appreciate Linux Mint (it is indeed very elegant), I think I will be sticking with Ubuntu 12.04 LTS for the time being.

My affection for the Unity interface that comes with Ubuntu 12.04 stems from the fact that I’ve been a heavy user of Mac OSX over the last year.  Before that, I was using Ubuntu 9.04, but the UI was heavily modified and stripped down, as I was a heavy user of the Xmonad window manager.

Having that experience with Xmonad, which is essentially a high-productivity, tiling window manager; and later working with the MacBook Pro (Late 2011) OSX environment, I’ve come to appreciate how important it is to have a powerful desktop UI that also gets out of your way.  The Unity interface follows that line of thinking, and is a real treat to work with once you start getting the hang of it.

There are some drawbacks to Unity, especially with regard to how applications are organized within the launcher, however I find that overall it will be a very rewarding environment to work in.

Luckily I have all my Vim and GNU Screen configuration files checked into version control, so it was easy enough for me to get GVim and all my other cross-platform apps up and running in my new desktop environment with minimal fuss.

Some screen shots of my desktop environment below:

The only real problems that I ran into with Ubuntu 12.04 were problems that were really hardware related. I’m running an ASUS S56CM Ultrabook, which has an oddly integrated Nvidia GT635M GPU.   So for now, I need to run my graphics intensive (OpenGL) applications via Bumblebee v3.0, however once that was set up, everything worked fantastically!

The Apache Software Foundation Celebrates the 17th Anniversary of the Apache HTTP Server with the release of v2.4

World’s most popular Web Server powers nearly 400 million Websites across the globe

Numerous enhancements make Apache HTTP Server v2.4 ideally suited for Cloud environments. They include:
•    Improved performance (lower resource utilization and better concurrency)
•    Reduced memory usage
•    Asyncronous I/O support
•    Dynamic reverse proxy configuration
•    Performance on par, or better, than pure event-driven Web servers
•    More granular timeout and rate/resource limiting capability
•    More finely-tuned caching support, tailored for high traffic servers and proxies.

Read the full press release at The Apache Foundation’s blog.

Playing with Factorials, Haskell, and Perl

I’m currently making may way through a book called “Seven Languages in Seven Weeks” by Bruce A. Tate.  So far it’s been an interesting read, but I’m far from finished.

One of the things in the book that caught my eye was a recursive factorial function in Haskell, which seemed so simple, that I had to see what it would look like in perl.

So I wrote up the following perl snippets to calculate factorials.  There are, of course, multiple ways to do it as I’ll describe below.  There are also (likely) many other ways which I haven’t thought of, so if you have an interesting solution, please share.

One of the things that really caught my attention was how simplistic the syntax was for writing somthing so complex.  Recursion is a fairly simple idea once you’ve seen it in action – a function that executes itself.  However, the implementation of recursion in a given programming language can be somewhat difficult to comprehend, especially for new programmers or those without programming experience.

Although I haven’t dived into Haskell quite yet, it seems to make implementing a factorial function so simple, that I kind of stumbled when trying to understand it, thinking I was missing something.. but it was all there in front of me!

Firstly, let’s clarify what a factorial is (from wikipedia):

In mathematics, the factorial of a non-negative integer n, denoted by n!, is the product of all positive integers less than or equal to n. For example,

5 ! = 5 \times 4 \times 3 \times 2 \times 1 = 120 \

 

So the factorial of 5 is 120.  Or 5! = 120.   Lets look at the Haskell example from the book.

let fact x = if x == 0 then 1 else fact (x - 1) * x

The above line is saying “if x is 0, then the factorial is 1 – otherwise, call myself with (x – 1), multiplied by x”

Lets look at this in ghci (the Haskell console):

[jbl@watchtower tmp]$ ghci
GHCi, version 7.0.3: http://www.haskell.org/ghc/&nbsp; :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Loading package ffi-1.0 ... linking ... done.
Prelude&gt; let fact x = if x == 0 then 1 else fact (x - 1) * x
Prelude&gt; fact 5
120
Prelude&gt;

After seeing how easy it was to implement the recursive factorial function in Haskell, here are my attempts in perl.

Firstly, using a loop:

#!/usr/bin/env perl
 
use strict;
use warnings;
use feature "say";
 
my $nni = $ARGV[0] ? $ARGV[0] : 5;
 
for my $i ( 1..($nni - 1) )
{
    $nni = $nni * $i;
    say $nni;
}

This first example doesn’t implement a function, and is really just bad (but still working) code. It requires that your base number be global and alterable, in this case $nni.

Now, lets try it with an actual function:

#!/usr/bin/env perl
 
use strict;
use warnings;
use feature "say";
 
my $nni = $ARGV[0] ? $ARGV[0] : 5;
 
sub fact 
{ 
    my ($nni) = @_;
    return !$nni ? 1 : fact( $nni - 1 ) * $nni;
}
 
say fact($nni);

This second method works similarly to the Haskell implementation. It implements a function that calls itself, without any looping required.

However, it’s still not as concise as the Haskell version, so lets try again:

#!/usr/bin/env perl
 
use strict;
use warnings;
use feature "say";
 
my $nni = $ARGV[0] ? $ARGV[0] : 5;
my $fact;
$fact = sub { my ($nni) = @_; !$nni ? 1 : $fact-&gt;( $nni - 1 ) * $nni };
say $fact-&gt;($nni);

Aha, now we’re getting somewhere. In this third example, the fact() function is anonymous, and we’re assigning it to $fact via reference. This allows us to use $fact like an object with a single method that does the factorial calculation.

Although this is pretty much as concise as I was able to get it while taking readability into account, here’s a final example that goes a step further:

#!/usr/bin/env perl
 
use strict;
use warnings;
use feature "say";
 
my ($nni, $fact);
$nni = $ARGV[0] ? $ARGV[0] : 5;
$fact = sub { !$_[0] ? 1 : $fact-&gt;( $_[0] - 1 ) * $_[0] };
say $fact-&gt;($nni);

This last example uses perl’s pre-defined variable @_ which automatically holds a list of function arguments by default. I usually avoid doing this, since it hurts readability, especially for those who don’t live and breathe perl on a daily basis.

To my surprise, it would seem that Haskell has Perl beat (at least in this example) as far as readability + conciseness is concerned.

I haven’t spent much time playing golf here to reduce the number of lines or characters beyond the last example, but if anyone does come up with a tighter solution, please let me know!


Edit (20111005T22:43:50): Here’s a version I found that uses the Math::BigInt module

#!/usr/bin/env perl
 
use strict;
use warnings;
use feature "say";
use Math::BigInt lib=&gt;'GMP';
 
my $b = Math::BigInt-&gt;new($ARGV[0]);
say $b-&gt;bfac();

This version is likely much faster, since the Math::BigInt package is intended to be used in situations where large integers are being handled.

Here’s the post I found with examples written in other languages as well: Factorial Challenge: Python, Perl, Ruby, and C