Tag Archives: Productivity

Ansible Playbooks – Externalization and Deduplication

Image result for ansible

Externalization and Deduplication

Developers who understand the concepts of modularity and deduplication should immediately recognize the power behind being able to include settings and commands from external files.   It is seriously counter-productive to maintain multiple scripts or playbooks that have large blocks of code or settings that are exactly the same.   This is an anti-pattern.

Ansible is a wonderful tool, however it can often be implemented in counter-productive ways.  Lets take variables for example.

Instead of maintaining a list of the same variables across multiple playbooks, it is better to use Variable File Separation.

The Ansible documentation provides an excellent example of how to do this.  However I feel that the reasoning behind why you would want to do it falls short in describing the most common use-case, deduplication.

The documentation discusses the possible needs around security or information sensitivity.  I also believe that deduplication should be added to that list.  Productivity around how playbooks are managed can be significantly increased if implemented in a modular fashion using Variable File Separation, or vars_files.   This by the way also goes for use of the includes_vars module.

Here are a list of reasons why you should immediately consider a deduplication project around your Ansible playbook variables:

Save Time Updating Multiple Files

This may seem like a no-brainer, but depending on the skills and experience of the person writing the playbook, this can become a significant hindrance to productivity.   Because of Ansible’s agent-less and decentralized manner, playbooks can be written by anyone who wants to get started with systems automation.  Often, these can be folks without significant proficiencies in programmer-oriented text editors such as Vim, Emacs, or Eclipse – or with bash scripting experience around command-line tools like awk, sed, and grep.

It is easy to imagine a Java developer without significant Linux command-line experience opening up one playbook at a time, and modifying the value for the same variable, over and over… and over again.

The best way for folks without ninja text-editing skills to stay productive is to deduplicate, and store common variables and tasks in external files that are referenced by multiple playbooks.

Prevent Bugs and Inconsistent Naming Conventions

In a perfect world, everyone would understand what a naming convention was.  All our variables would be small enough to type quickly, clear enough to understand its purpose, and simple enough that there would never be a mis-spelling or type-o.  This is rarely the case.

If left un-checked, SERVER_1_IP can also be SERVER1_IP, Server_1_IP, and server_1_ip.  All different variable names across multiple files, referencing the same value for the exact same purpose.

This mess can be avoided by externalizing this information in a shared file.

Delegate Maintenance and Updates to Variables That Change Frequently

In some environments, there may be playbook variables that need to change frequently.  If these variables are part of some large all-encompassing playbook that only some key administrators have access to be able to modify, your teams could be left waiting for your administrator to have free cycles available just to make a simple change.  Again, deduplication and externalization to the rescue!  Have these often-changing variables externalized so that users who need these changes immediately can go ahead and commit these changes to very specific, isolated files within your version control system that they have special rights to modify.

Cleaner Version Control History (and therefore Audit History)

If you have the same variables referenced by multiple files, and you make changes to each of those files before you commit them to version control, then your version control history can become a complete mess.  Your version control history will show a change to a single value affecting multiple files.  If you come from a software development background, and are familiar with the concept of code reviews, then you can appreciate being able to look at a simple change to a hard-coded value (or a constant), and see that it only affects one or two files.

I hope the reasons above convince some of you to start browsing your playbook repositories for possible candidates for deduplication.  I really believe that such refactoring projects can boost productivity and execution speed for individuals and teams looking to push changes faster while minimizing obstacles around configurations shared by multiple systems.  Send me a note if this inspires you to start your own deduplication project!

Team Meetings at the Workplace

I’ve been to many team meetings (or “all hands”) that have, in the end, provided no real value to the team in attendance.  It is very possible (and likely quite common) that most meetings are actually a waste of time (as many of my closest colleagues and friends would agree).  However, this does not always have to be the case.

Meetings are important to have.  They provide an opportunity for team members to share not just the facts, but also how they feel about the current situation for a given project.  Team meetings can often provide context for the day-to-day work that each of us is involved with.  Just as importantly, it is an opportunity for leaders to maintain an awareness of how their team members are feeling, in order to better estimate how the team might respond given additional stress.

Meetings have to be run with intention in order to be successful.  Topics should be specific, and every topic discussion should end with an actionable deliverable assigned to a specific individual.  This may sound a bit restricting, but it goes a long way toward saving your team more time for actual work, along with giving them (and yourself) more time to relax.

There are many meetings that I have attended where discussions were essentially a one-way dialogue between the presenter and the rest of the team, where feedback was neither elicited nor volunteered.  When the host (usually a team lead, manager, or PM) asks the ever gripping question “..any questions or comments on this?”, and a conference call of about twenty or so people falls dead-silent.. well, I would consider that a problem.

Often the problem is that the meeting is too broad, involving too many people who don’t work with each other on a regular day-to-day basis.  Smaller meetings are the key to success.  Meetings should be specific, covering 3-5 of the most important items or active projects.  Meetings should also be quick (standing meetings are often very effective).  Questions and responses should be clear and to the point.  “What’s the status of the X project?”   “What’s changed from last week?”  “What do you need to keep this on track?”

Team meetings shouldn’t get too technical either.  That’s what e-mail, white-boards, and dedicated meetings are for.  Having technical discussions at team meetings will rarely be beneficial, and will often just turn into an echo chamber or a religious debate.  These types of discussions are good to have, but not when leadership is at the table.

When leadership is at the table (Managers, Sr. Managers, Directors),  the focus of the meeting should be for the leadership team to identify the most critical issues to address, and do all that is necessary to raise the collective spirit of the team; with feedback that is directly related to the work that they are doing.  The only way to do this is to be aware of the team’s accomplishments since the last meeting.  Every little inch matters.  Every little unit of work and motivation you can squeeze out of your team will benefit your project, and your stakeholders.   I am not suggesting micro-management, nor am I advocating unnecessarily cracking the whip.  Instead, what I would suggest is that teams work together to develop processes and routines that will continually provide positive, targeted feedback on the work that the staff has accomplished.


WebPIM: A Custom, Web-based, Personal Information Manager

I’ve always wanted a web-based application to help me manage all my stuff. “WebPIM” (as I’ve nick-named it for now), is currently one of my main personal projects that I have been working on.  I started this project back in 2003 as a simple web-based file manager, and have been slowly hacking away at it in my spare time ever since. “WebPIM” can act as a central reference point for all personal or project information. The way I’ve implemented my custom PIM is purely based on the way I work, so it may not be to everyone’s liking. However, I think it could really help individuals who need a way to organize tasks, projects, documents, and general files in a free-form, yet hierarchical and accessible way. Much of the thinking behind the way WebPIM is being developed relates to GTD ((Getting Things Done – David Allen)), and how to get “stuff” off your mind, and into a system.

Here’s the general idea – you have a lot of “stuff” – stuff that’s just sitting around on scraps of paper, on your hard drive, in your e-mail, and every other place you can’t seem to remember. This may be un-important stuff, or it may be severely important stuff – but none of it is organized into any kind of easily reference-able and “trusted system” ((GTD terminology)).

You have several options; the first of which is to do nothing. Unfortunately, ignoring the problem and hoping it will go away won’t solve the problem. Lets assume you want to change your situation, and we’ll use my experiences as a baseline for discussion.

I have tried many personal information managers over the years, and all of them have been incomplete in one way or another. Also, with the new wave of hosted applications like Google’s GMail, Calendar, and Google Docs, I am becoming more and more uncomfortable storing all my stuff on a remote, corporate server over which I have no control ((This has become more and more of a concern for me, having accounts on Google, Facebook, and others. Maybe I’m just paranoid.)).

My solution to this dilemma has been to write my own PIM, and so far, I’ve been happy with the results.

The way WebPIM currently works is by operating as a front-end to a linux based file-system. From WebPIM, I can create directories, create text files, upload files from my local hard drive, and move files around from one directory structure to another. This is the simple stuff that I think any web-based file manager should be capable of. More than this however, WebPIM provides the following features:

  • Move multiple files from one directory to another (batch move)
  • Text-dialog editing of all files (you can edit HTML and XML files in the interface)
  • Full path display when traversing directories, which allows you to go directly to any directory within your current absolute path via a hyper-link
  • Web-download functionality allowing you to download a copy of your favourite web page or web-accessible file into your current directory.
  • Recursive web-download, so that you can download an entire website for later reference (implemented using HTTrack ((www.httrack.com/)) in the back-end).
  • Project short-cuts, so that you can create short-cut groups to access multiple directory structures on the same interface. This allows you to access general reference information, along with specific project information all within the same interface, and without disrupting your overall PIM hierarchy.

I think the idea can be better explained with a screenshot of the main interface:

WebPIM Interface
– WebPIM Interface (Click on the image for a larger view) –


Obviously there is still a lot of polish required before this becomes useful to the general public, but I really do believe there is a market for it.  If anyone is interested in trying this out, leave a comment and let me know.  I can probably set up a demo, or provide the source code as-is so that you can give it a shot on your own system.

WebCalendar – An Open Source Shared Calendar Solution

Two years ago, I discussed a shared calendar solution called Monket Calendar. In the discussion I mentioned that I was considering Monket Calendar as an alternative to PHP iCalendar, which was another (read-only) web based calendar solution I was using at the time.

I no longer use PHP iCalendar, and I haven’t since looked at Monket Calendar. The only calendering solution that I use now, and that has all the features I need, is the absolutely wonderful WebCalendar by k5n.

Continue reading WebCalendar – An Open Source Shared Calendar Solution

Ear Plugs

I’m a big fan of ear plugs. I use them while working, reading, and sleeping. When I wear them, I find it takes very little effort to concentrate on what I’m doing. Using ear plugs while sleeping helps me fall asleep faster, and allows me to get a full night’s rest while almost never waking up in the middle of the night.

Being a big fan, I’m always on the lookout for better ear plugs, such as plugs that block out more sound, are more comfortable, or even more portable. Part of my search lead me to an article (2005) by which gives a great overview of the different types of ear plugs available, and a rating as to how effective they are in comparison to each other.

If you’ve ever wanted to increase your concentration at work or at home, or have had trouble getting a full night’s sleep, you should read the article above. A simple pair of ear plugs could change your life!

Productivity Tools and Systems

I’m currently working on an article to introduce one of the productivity tools that I’ve been working on for a while. I’ve come across many different types of tools for various purposes, such as Freemind, Cmap Tools, iGTD, Google Calendar, etc. I currently have a set of tools which I use regularly (some of which I developed myself) and was curious what others out there are doing to manage their time and the large amount of information they have to process on a daily basis.

Along with PIM ((Personal Information Management)) tools, what productivity methodologies (if any) do you follow? I’ve found that a mix of ideas from “The 7 Habits of Highly Effective People” ((Book: “The 7 Habits of Highly Effective People” by Dr. Stephen R. Covey)) and “Getting Things Done” ((Book: “Getting Things Done” by David Allen)) works well for me. However, I know there are other, potentially better systems out there ((I say “potentially better” since what might be better for some people may be worse for others)), and I’m sure my system could be improved by methods I’m not currently aware of. For example, I recently came across an article on Lifehacker ((Lifehacker is an excellent blog/website created by Gina Trapani which focuses on productivity tools and processes, with a slant towards computer power users)) by Brad Issac which discusses how Brad received productivity advice from Jerry Seinfeld. The advice Seinfeld gave Issac outlines a simple, but brilliant way of motivating yourself to get things done.

Jerry Seinfelds advice is one I’m sure I’ll be trying out as part of my own productivity system; but I’m sure there are also many other tips and tricks out there. Whats in your productivity toolkit?

Monket Calendar: Web-based iCal Calendar Editor

In my recent searches for an alternative to PHP iCalendar, I came across a very cool application called Monket Calendar by Karl O’Keeffe. After reading through the description of the application, I was very impressed by what it proposes that it can do. Some of the items include the following:

  • Ability to View *and* modify iCal calendars
  • Drag and drop interface to move calendar events
  • Change start/end dates by re-sizing the event without a page refresh

A lot of these features are very attractive because it makes Monket Calendar a direct competitor to would be killer apps such as Mozilla Sunbird, for it’s ability to modify remote calendars, and PHP iCalendar, which is great for viewing remote calendars, but does not provide an ability to make changes to them.

Some may argue that Mozilla Sunbird still has the advantage of being a stand-alone application that does not require a webserver or services and libraries such as PHP and AJax to run. I would argue that what makes Mozilla Sunbird such a great application is its ability to interact with shared calendars located on remote servers via WebDav or FTP. Of course, the overhead of having to install and configure Apache, PHP4, and Ajax support would probably turn most users away from Monket Calendar. However, if a calendar can be accessed and updated from the website it is hosted in the first place, it wouldn’t make much sense to run a local application to download the calendar’s events, unless you have dial-up or have a requirement to work offline for long periods of time.

Further Reading:

Monket Calendar

Mozilla Sunbird Project

PHP iCalendar Project

More Than Just Another Todo List

Recently I whipped up a simple PHP/SQLite based Todo List application for myself, as I didn’t like the Todo Lists available on my Blackberry, or on Mozilla Sunbird.

In passive discussion with some of my colleagues at VMware, the book “The 7 Habits of Highly Effective People” came up. The book is a great read if you take it with a grain of salt. I haven’t read the book in a while, but it does contain many good time management and life management tidbits.

In any event, we discussed one of the topics in the book pertaining to tasks, and ranking or prioritization of tasks that needed to get done. In the book, Stephen Covey discusses charting tasks (or Todo Items) in an Urgency/Importance graph. Urgent tasks, or tasks that needed to get done quickly, but were not overly important, would rank higher on the Urgency scale. Tasks that were Important, but did not necessarily need to be done right away would rank higher on the Importance scale, and lower on the Urgency scale.

The Todo List I wrote utilizes this underlying methodology for organizing Todo List items, and so, Items that are both important and urgent would appear higher than those of less importance or urgency.

I’ve only had to use my custom Todo List for a while to realize how much more functional it was compared to the standard Todo Lists available today on most PDAs and Personal Information Management systems.

I have to do some code cleanup before I release the application for public consumption. I just wanted to start a discussion on the topic to guage interest, and to see if anyone else has taken it upon themselves to develop a more usable alternative to the standard linear Todo Lists currently available.

Designing Finance Management Software

I’ve been playing with the idea of developing finance management software. Mostly this idea stems from my own need to manage my finances better. Beyond that however, I am interested in expense tracking, savings, and projections.

Businesses usually employ chartered accountants or excel savvy office administrators to do a lot of what I’m talking about. And I understand that its not just a matter of fancy software, but staying on top of your tracking and expense reporting. However, I believe that a system can be developed that can force (or motivate) it’s users to manage their finances better.
Continue reading Designing Finance Management Software

Getting Organized At Home

Man is it hard to keep organized! I’ve been trying to clean up my home office but just keep uncovering more stuff that I’m not sure I want to throw away. I think I’m a bit of a pack-rat.

I have several pieces of computer equipment that I don’t even use anymore, but are still very useful. I’m thinking I should put up a Buy N’ Sell section on my website so that I can get rid of some of this stuff. I might also have to advertise in the classifieds, and probably on E-Bay.

Anyway, keep an eye out for some stuff for sale!

At Work, on The Road, at Home… Practical, Non-intrusive Solutions to Data Syncing

Data syncing has been a recurring problem for me for as long as I can remember. Reading and modifying documents at work, and then trying to access these documents at home can be a pain in the ass unless you get into the regular habit of copying these documents to your USB key, or to your laptop. When you are ready to work on these documents at home, you have copy these documents off your device, do your work, then save them documents back to the device so that you will have an updated copy when you go back to work. Continue reading At Work, on The Road, at Home… Practical, Non-intrusive Solutions to Data Syncing