Rules of online privacy etiquette

This is a modified version of an opinion piece from my ITPro column originally published in The Age.

So you’ve been careful with your online profile. Obfuscating your details, giving false birth dates, leaving all the fields as blank or nonsensical. You have kept your posts on blogs professional and free of private life tidbits. If you belong to a social network your status updates have been moderate and cryptic. 

Then one day you sign into Facebook and see that your family and friends have wished you a happy birthday, or congratulated you on your new job or worse, tagged you in a photo. Suddenly your data, which you’ve carefully curated, is slowly exposed by what is ultimately accidental politeness and social convention.

It would be fairly trivial (for who? who is doing this?) to backfill missing data points (such as day and month of birth) by searching for the right congratulatory key phrases. Even more devastating if someone actually says “Happy 21st” because now they have your year of birth. 

We are social creatures, it is in our nature to create connections and then use our knowledge and shared experiences as a kind of social currency. The digital community member that shares nothing about themselves, just watching and not participating (called lurking) is viewed with an element of suspicion.

Digital communities aren’t bad, they are good and healthy extensions of human society. An Internet without community forming tools and services would be a dull and dark place. We know the positive power of the Web’s ability to create spaces free from censorship and geographical constraints to give voice and a place to belong.

The darker side of this belonging to a community is that it is easy to be lulled into a sense of complacency about your private information. Ill framed, throw away comments or that photo or slip of detail about yourself or loved ones can have lasting unintended consequences. There are no take backs on the Internet.

There are plenty of cases in recent history in caches of leaked documents, communications and memos from private intelligence agencies showing that data dredging (finding relationships in large volumes of data) is alive and well and can be sold to governments and big businesses. It is also standard fare for security researchers and tech savvy marketers to continually come up with better strategies for identifying individuals in last generations anonymized data sets.

So while the bulletin boards, chat sites and social networks that you frequent may be run with the best intentions and by people you trust, third party “intelligence” gathering and surveillance can still discover facts that you never intended to go beyond that small circle.

You may think “but I’m not important enough to worry”. That is not the point. If say, you are an active member of a sporting message board, that also has as a person of interest to some foreign multinational, you may find that by association, your details are dredged and relationships inferred by lazy statistical models. There are more than a few instances of these inferred relationships between innocent people and criminal activity in internet search results.

Gossip, especially on the Internet, has powerful and far reaching implications. Even if it’s done in good fun or would be considered otherwise harmless that information may become yet another data point against someone in a dredging exercise.

A colleague remembers that in her mother’s Family Circle magazines there was always some kind of brag column or agony aunt which enabled readers to write in and boast or gossip about their “DH” (dear husband) or Little Wife, condescending naming practices that is still alive and well. Imagine if intelligent semantic web engines could link “DH” to your profile?

Remember when it was the norm to put a notice in the paper announcing the birth of a child or marriage etc? Well these papers may one day be digitised and put on the web and made discoverable through search engines. Little Jane Doe, who is now 16 years old has her date of birth, suburb and parent’s name now ripe for collection. That funny little thing that you wrote into the magazine is now internationally searchable and it probably has at least your first name and suburb tagged against it.

It doesn’t even really matter if the details aren’t complete enough to identify you. Data mining techniques are very good at looking for correlations in large and diverse sets of information. Relationships can be inferred, correctly or incorrectly,  that might follow you about even in benign ways.

Digital community service providers, be they social networks or even a tiny message board, absolutely have the onus of protecting their users privacy, no ifs or buts. However even the greatest levels of privacy protection can’t protect you when you or your peers leak your own information.

I will leave you with a few points that I think should become a kind of privacy etiquette.

  1. Don’t offer other peoples’ information, even inadvertently. Before you wish someone a happy birthday or anniversary, check to see if they have set that in their profile. If in doubt say nothing.
  2. Don’t get specific. 
  3. Don’t tag photographs. Leave it up to the person to tag themselves if they wish.
  4. Don’t break the scope of the message (No Gossip clause). If a story or piece of information was shared between a small group do not re-share or cut and paste it into other networks.
  5. Check your profile regularly and see if any fields that you have set as empty have since been “helpfully” filled in by the software making inferences.
  6. Agitate against the rule that real names must be used.

A common refrain is “Don’t join the network in the first place”, to which it is my opinion that I would rather know what people are saying about me so that I can correct it or obfuscate it rather than just hide my head and hope that no one is talking about me. At the very least I’d rather claim my name or handle to stop others from using it maliciously.

Key loggers

This opinion first appeared in my ITPro column in the Sydney Morning Herald. This content deals with issues of domestic abuse. If you or anyone you know may be experiencing domestic violence please contact

Last week I was asked for advice about a person that thought that their partner was digitally tracking them through the use of key loggers. The more I heard the more convinced I was that key logging was occurring. This grown woman would be grilled about her internet usage, her partner would discover her passwords and log into her social networks and post as her and even take her phone being taken away as “punishment”. The level of surveillance being described was abusive, pure and simple.

Key loggers are hardware devices or computer programs that record keystrokes (some will even track mouse coordinates and clicks). The stream of characters are logged somewhere accessible for the program installer to study.

Key logging software and hardware is not illegal. Like most technology it can be used to help or harm, however dwells in a very grey ethical and legal area. Most often key logging is the domain of malware installed without the users consent with the sole purpose of harvesting usernames and passwords to bank accounts, email and other sensitive information.

Even the “legitimate” uses when installed on your own hardware are questionable at best. The installer is not doing it to track their own movements after all. Key logging software doesn’t tend to advertise itself to the other users of the computer – its effectiveness lies in it being undetectable.

By the time key logging solutions are seen as a valid next step, trust is already gone.

Even at the corporate level, the use of key loggers is a double edged technology. Key logging is generally indiscriminate in its approach. The proof that an employee is misusing their workstation or internet access may well have also in the log the login details of their bank account or other private data that can constitute the key components of identity theft. Use of key loggers with the best intentions may still make data available to you that you have no legal right to store.

Proponents of key logging activities as a form of protection, often do not disclose that there are better and more constructive ways of preventing  misuse of technology. I think the thrill is in catching people after the fact rather than proactively trying to create a reason for people not to behave that way. 

When it’s all said and done, I don’t believe key loggers are very effective at stopping anything, but they are wonderful at creating environments based on fear and maintaining a power imbalance where the technically capable can spy on those less technically savvy. Even the best justifications that I have read for key loggers (usually made by the private-eye-esque developers) are frankly creepy.

The Australian Institute of Criminology notes that key logging is dubious legal proof at best, since it only tells you what was entered, not who entered it.

It can be very hard to tell if there is a key logger installed. Good, up to date virus detectors should be able to scan for the known key loggers. Hardware key loggers sit between the keyboard lead and the back of the computer, so you may be able to see it and disconnect it.

I have read suggestions that most key loggers can be thwarted by using on screen keyboards (a fairly standard accessibility feature of the modern operating system).

Ultimately if you have to use a computer that you can’t trust you can use a “live” Linux desktop (such as Ubuntu) which boots from a USB stick if you want to make sure that a key logger is not running.

Finally, if you think that you have a key logger installed on your machine by a scammer you can get information from

If you fear that your partner is using key logging and other technical forms of surveillance to track and control your behaviour, please seek help. Digital stalking creates anxiety, depression and can completely undermine your relationships. 

For those people that install key loggers on their partners computers, please go and read a modern definition of Domestic Violence and seriously reconsider your behaviour. Do not fool yourself by thinking that it is just harmless “protections”.

For more help see

Legendary Bystander template

legendary bystander


Download this png.

Grab photos of the people playing in your game session.


Put in the point value in the orange dot.

Size and put them into the background layer.

Print to 63x88mm.

Cut out and use one of the blank filler cards as the backing and slip it into a sleeve the same colour as your villain cards.

Introducing MosaicaFantastica

MosaicaFantastica - Gorilla at Toronga Zoo, Sydney

Gorilla at Toronga Zoo, Sydney

MosaicaFantastica is a Java program that attempts to recreate an image using recursive colour sampling. The effect is to create a new image which approximates the source/target image in a style reminiscent of impressionism or pointillism.

MosicaFantastica recursively sub-divides the target and using a basic colour distance formula decides whether it can get away with using a basic shape (rectangle or oval) to fill the available space. If the colour difference is too great then it sub-divides and tests again.

Ultimately you get an image which is made up of basic shapes arranged using random parameters such as rotation and size variation to make interesting and unique interpretations.

This is a work in progress.

Installing Rails RVM on Ubuntu 11.10

Using as a basis a previous post I have updated it to run on Ubuntu 11.10.

sudo apt-get install curl git-core build-essential bison openssl libreadline5 zlib1g zlib1g-dev libssl-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libmysqlclient-dev mysql-client mysql-server autoconf

curl -L > rvm-install-head
bash rvm-install-head

echo ‘[[ -s “$HOME/.rvm/scripts/rvm” ]] && . “$HOME/.rvm/scripts/rvm” # Load RVM function’ >> ~/.bash_profile

source ~/.bash_profile

rvm install 1.9.2
rvm use 1.9.2 –default
rvm 1.9.2 –passenger
rvm 1.9.2
gem install rails mysql passenger
apt-get install libcurl4-openssl-dev
rvmsudo passenger-install-nginx-module

gem install rails

rm rvm-install-head

Ruby utility script – csv2json

A script that I wrote and am using more and more.

Arguments are “filename.ext”

require ‘json’

source = ARGV[0]
if source.nil?
puts “Missing filename argument”
puts “USAGE: #{$0} filename.ext”

unless source && FileTest.exists?(source)
puts “#{source} not found!”

puts “Processing ‘#{source}'”


file =,’r’)
ext = File.extname(source).to_s.downcase

outname = source.sub(ext,’.json’)
out =,’w’)

sep = case ext
when ‘.csv’
when ‘.psv’
when ‘.tab’
# assume comma since it is *CSV*2json
puts “Extension: #{ext} using ‘#{sep.source}'”

lines =\n\r?/)

header = lines[0].split(sep)
data = lines.drop(1).map {|item| item.split(sep)}

outdata = []

data.each do |item|
h =
for i in (0..(header.size – 1))
h[header[i]] = item[i]
outdata << h end out.write outdata.to_json out.close puts "File written to " + out.path

Ages of the Internet

As the web matures, the commercialisation pattern is changing – new monetisation opportunities arise while the incumbent “cash cows” get commoditised and the margin is squeezed. Each of the “Ages” below has its heroes that created the technology and the business model to succeed and then dominate. Each age was made possible by the standardisation and commoditisation of the prior ages.

The current age is Clouds, we are at the very beginning of this age and it is likely to dominate technical thinking for the next 2 to 4 years. Clouds will drive the cost of scaling and the barrier to entry of minor media players down. The battle for media in the age of clouds is differentiation and agility in maximising content reuse.

The “Product” The “Currency” Key Players
Age of Pipes • Hardware
• Protocols
Bandwidth • Sun
Age of Portals • Publishing Platforms
• Online Classifieds
• Personal Homepages
Content • AOL
• Big Pond
• Alta Vista
• Yahoo
• Geocities
Age of Search • Search Engines
• Recommendation
• Crowdsourcing
• Google
• Yahoo
• Bing
• Wiki
Age of Clouds • Software as Service
• Distributed Systems
• Always On/Ubiquitous Computing
Scale • Amazon
• Google
• Microsoft
Age of Semantic • Context
• Entities
• Sentiment
• Relationship
• Intuition/Personal User Agents
• Big Media?
• Facebook
• Google
Age of Trust • Security
• Privacy
• Identity
• Authority
Reputation • Big Media?
• Standards Bodies
• Cultural Institutions
• Knowledge Markets

Why should Big Media bother?
Because each age moves from engineering towards social, the technical challenges become less about “is it possible?” and more “how can it be used?”. Big media companies are in a unique position because they have made the investment in hardware, platforms, content and traffic. Big media has an established audience and, maybe more importantly, audiences untapped (niche interest, hyper local, international, etc.). The Age of Clouds will drive hosting costs down, increase computing power for small competitors to differentiate products through “good enough” brute force methods (eg. Content scraping, data mining, machine learning and business intelligence techniques). In the age of semantic web, the incumbents will be those that understand the content and the audience best – this is why Google, Apple and Microsoft are falling over themselves to introduce social aspects because relationships, entities and sentiment are engagement glue. Once the volume of content is too big, and the user can access it on any device, any time, any where the products that will succeed in this age are the ones that turn mass data into information and knowledge and relevance, which is actually the key competency of journalism.

When a consumer has access to the world media and as domestic and international territories blur, breaking news won’t be so much as speed to publish, but rather, speed to discover and then socialise.

The Age of Semantic changes the questions from: “What happened?” to:
• What does it mean?
• What is hidden?
• Why did it happen?
• Will it happen again?

System design and architecture

Some thoughts I had regarding effort. Too often I think that people spend more time on getting the implementation set in stone, rather than making sure that the higher order issues are well defined.
This is the order of emphasis on permanence and therefore effort in getting it right.
Design Pattern – correctly identifying the “shape”. The design pattern should only change when the function of the system changes.
Architecture/Design – correctly selecting the parts to fit the “shape”. Architectures should only change when advantage can be gained (or the design pattern changes)
Implementation – correctly building the parts. Should be reviewed and changed as required.

Installing Rails 3 on Ubuntu 11.04

One of the more challenging problems with the opensource world (and joys in my opinion) is the rate of change of the software and systems. Unfortunately it means that every year I seem to be having to learn new ways of installing the latest tools. Every 6 months when I install the latest Ubuntu and Rails, something needs hand holding. Here is my attempt to get Rails running on Ubuntu 11.04.

Firstly, credit goes to yasi8h from theKindOfMe blog – see the original post here

Below is a variation of the process outlined in the blog post, however I have changed things to make it work for a fresh Ubuntu 11.04 install.

Run the following in sudo/root

apt-get install curl git-core build-essential bison openssl libreadline5 libreadline5-dev zlib1g zlib1g-dev libssl-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libmysqlclient-dev mysql-client mysql-server autoconf

curl -L > rvm-install-head
bash rvm-install-head

Read and follow the instructions given as shell output after  rvm-install-head to edit your .bashrc. You will need to remove the return statement and wrap the whole code block in if’s and append the command to load the rvm into the shell environment. 

OR suggests running:

echo '[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm" # Load RVM function' >> ~/.bash_profile

rvm install 1.9.2
rvm use 1.9.2 –default
rvm 1.9.2 –passenger
rvm 1.9.2
gem install passenger
apt-get install libcurl4-openssl-dev
rvmsudo passenger-install-nginx-module

gem install rails

rm rvm-install-head