In PostgreSQL, sorts larger than a certain size will get performed on disk instead of in memory, and this makes them much slower as a result. Ideally all sorts should be done in memory (except for the ones that are genuinely too big to fit into your available RAM, because swapping to virtual memory should be avoided at all costs).

Seeing which sorts are happening on disk

To see which sorts are being performed on disk, the following parameters need to be enabled in /var/lib/pgsql/data/postgresql.conf:

log_destination = 'syslog'
log_temp_files = 0

PostgreSQL syslog logging goes to the ‘local0′ facility by default, so in /etc/syslog.conf make sure that you have ‘local0.none’ added to the line for /var/log/messages, and add the following line:

local0.* /var/log/pgsql

Then restart syslog and reload PostgreSQL:

service syslog restart
service postgresql reload

You should now start seeing details of your disk-based sorts in /var/log/pgsql. Each one will start with a line containing the phrase ‘LOG: temporary file’ and will show other details below this line including the SQL of the query which contained the sort. After a while you will see patterns emerging.

Making sorts happen in memory instead of on disk

If you have disk-based sorts occurring frequently, you can find the size of the largest of the frequent queries and then set this as the standard threshold for sorts. The parameter for this in /var/lib/pgsql/data/postgresql.conf is ‘work_mem’. For example, if you had frequent disk-spaced sorts using up to 64MB, you would set the following in /var/lib/pgsql/data/postgresql.conf:

work_mem = 64MB

Be very careful, however, because this setting gets used by each query in each thread, so if you set this too high you could rapidly run out of RAM and cause the system to start swapping, which would be disastrous. The best thing to do is increase this bit by bit until as many as possible of your frequent queries are sorting in RAM but without running out of available memory. To see how much RAM is actually available on your system, use the command ‘free’. As Linux tends to use up the majority of available RAM for its disk cache, the number you actually care about is the ‘free’ column in the ‘buffers/cache’ row – this is how much RAM is really available. For example, here we can see that there are just over 11 GB of free RAM available:

# free
total used free shared buffers cached
Mem: 12300988 12013536 287452 0 277788 10801424
-/+ buffers/cache: 934324 11366664
Swap: 4192956 240 4192716

Once you’ve dealt with sorts for frequent queries, you might find there are still the occasional disk-based sorts happening for a small number of particularly intensive queries. For those, you can increase the sort memory threshold on a per-query basis and then put it back to the standard setting once the query is finished. To do this, prepend your query with:

SET work_mem = '500MB';

changing ’500MB’ to whatever size is appropriate for you, then append the query with:

RESET work_mem;

to return it to the standard threshold. Again, be careful that you don’t use up all the system’s available RAM because then the dreaded swapping will occur.

Once you’ve done all this, as many sorts as possible will be happening in memory instead of on disk, and your PostgreSQL installation should be performing considerably better as a result of this tuning.

Getting information from the logfile into a more useful format

When PostgreSQL logs temporary files (disk-based sorts), the log entries can be rather messily distributed in the logfile, especially if you’re logging other things, such as slow queries, at the same time. I knocked up the following Bash script to quickly extract the details of the sorts into a more useful format:

#!/bin/bash

rm -f $TMPFILE1 $TMPFILE2

It’s a great space so go along and check it out! There are prints and books of my work for sale there. They also serve great coffee and tea and delicious vegetarian food.

Launchpad: Tedious scrolling through multiple screens of apps in order to launch one of them? No thanks. Dreadful.

Mission Control: It seems like a good idea in principle to combine Exposé and Spaces, but in doing so some useful functionality is lost: minimised windows are no longer shown; multiple desktops are now one-dimensional rather than two-dimensional; I no longer have a number in the Menu Bar telling me which desktop I’m currently on; and it’s harder to see the contents of the other desktops. All a bit of a shame really.

Full-Screen Apps: This seems initially useful, but: it doesn’t make use of multiple monitors; the keyboard shortcut only seems to work some of the time; it disables my hot corner to quickly get to the Desktop; switching between windows within a fullscreen app (e.g. Mail) is sometimes impossible; and it adds confusion because all windows already have a Zoom button, so now they all have two maximise buttons instead of one.

Resume/Auto Save/Versions: This seems like a good idea in theory, but I doubt many of the apps I use will implement this for a long time, if ever.

AirDrop: I was really looking forward to this one but didn’t realise that it only works on some recent Macs, i.e. none of mine. So that was very disappointing.

Mail: The redesign is quite nice both aesthetically and ergonomically. Nothing ground-breaking, but nice. I’ve already become very dependent on the conversation view to the point where I now wish they’d hurry up and enable it for my iPhone too.

Address Book/iCal: The horrible graphics in these are ugly and ridiculous, and Address Book has much less space for useful information as a result. I thought Apple had stopped this sort of nonsense years ago and didn’t expect to see it come back now. Also, iCal is having severe problems talking to my work calendar, and the work calendar is iCal Server running on OS X Leopard Server, so it’s particularly annoying that that doesn’t work properly [now fixed by using SSL for CalDAV]. Having said all that, I do like the new ‘planner’ pane on the ‘Day’ view – that’s now replaced ‘Week’ as my standard view as a result.

Front Row: Instead of bringing this up to date, which would have been sensible especially now that Mac minis have HDMI outputs, they’ve just removed it instead. Great. Thanks, Apple. I don’t use it that much in comparison to Plex on my Mac mini-based media centre, but it would have been nice to have functionality added instead of the whole thing being taken away. Very poor show indeed.

Terminal: I love the new ‘Blur’ option for window backgrounds, and I’ve come up with a really nice combination of blur and opacity for my active and inactive windows. But why, when I open new tabs, do they start SSHing into random servers when I don’t want them to? I presume this is something to do with Resume but it’s bloody annoying and I want it to stop! [Turns out it was necessary to change the 'New tabs open with' settings in the Startup tab in Terminal Preferences.]

UI changes: Being able to resize a window from any edge is very welcome – I wish they’d done this years ago. It’s nice to see the back of the oversized Aqua scroll bars at last (but desirable to change the scroll bar setting to show ‘Always’ in the General System Preference because this is a computer, not an iPhone). The overall look and feel is like a more subtle, streamlined version of Snow Leopard, which is quite nice generally. I worried about the icons in sidebars (in Finder and Mail most noticeably) being stupidly oversized, but thankfully I then found the ‘Sidebar icon size’ option in the General System Preference – setting that to ‘Small’ makes everything OK again. And the increased use of animation is quite enjoyable and might occasionally be vaguely useful for new computer users. All the new gesture stuff is irrelevant to me because I don’t have any of the new Macs with big trackpads, and I prefer using mice anyway (though not Apple mice because they’re an ergonomic nightmare). Oh, and suddenly finding scrolling going in the wrong direction wasn’t particularly useful, but thankfully that was easy to turn off in the Mouse System Preference.

System stuff: Make sure you have ‘Show indicator lights for open applications’ ticked in the Dock System Preference, partly because that’s a traditional behaviour which helps to actually make the Dock useful, and partly because if you don’t then Lion will randomly quit open apps without asking. Another bit of weirdness is the ‘Mail, Contacts & Calendars’ System Preference, which seems to serve no useful purpose whatsoever other than to confuse users by duplicating settings which you’ve already got in Mail, iCal, etc.

So there we go. Once you’ve turned all the annoyances off it’s basically just a neater, slicker version of Snow Leopard with a few things missing, so I’m not exactly blown away by this new OS. I think the main problem is that trying to mix Mac OS X and iOS causes a conflicting mess of too many different UI paradigms. Apple should either have stuck with tweaking the Mac OS X foundations or rebuilding the whole thing from the ground up, not ending up with something in between the two which doesn’t quite work.

Edit: It’s quite interesting to compare the rather dark tone of this review to the optimistic positivity of my Snow Leopard and Leopard reviews. It makes Lion feel like even more of a step back than a step forward.

Edit: Reading back on my reviews of previous versions of OS X, I’m reminded of what a mess Apple made of To Do Items in Mail when that was added to Leopard. I see they’ve fixed that by simply removing them again, which I guess is fair enough really.

As well as making it possible to take photographs at any time, the iPhone’s operating system, iOS, provides the photographer with a portable processing lab in the form of third party ‘apps’ downloaded from the App Store. Other smartphones have photo apps too, but the range available on iOS is far superior to what’s available on Android or other platforms. Quite a few people are now familiar with the popular Hipstamatic app, but this is only one of many. The first iPhone photography apps tended to offer simulations of Lomography cameras and were often quite ‘lo-fi’, which was perhaps natural given the relatively poor quality of the cameras on the first iPhones. However, the camera has been improved on each new iPhone model, and the iPhone 4 has a camera which is on a par with some compact cameras. At the same time, the photography apps available have gone beyond just lo-fi effects and into much more professional territory, offering advanced editing features, layering and masking, a huge range of filters and textures, and other facilities such as HDR. The range of editing possibilities for iPhone photography is now quite incredible.

In addition to the camera itself and the software, there are increasing numbers of hardware accessories for iPhoneography, such as the Owle Bubo mount and stabiliser, and the Rollei telephoto lens for the iPhone 4. The fact that hardware manufacturers such as Rollei are taking the iPhone so seriously is a very good sign for the future of the medium. Another very good sign is that the iPhone 4 has just overtaken the Nikon D90 as the most popular camera in the Flickr community. Many people are now finding that the sheer power of a good camera that’s always available, combined with the advanced editing features offered by the huge range of apps, makes for photographs which are both spontaneous and highly artistic. This has led to a new photographic and artistic subgenre referred to by some as ‘iPhoneography’, which is seen in many ways as highly representative of the current zeitgeist of art meeting technology. iPhone photographs can encompass all photographic genres from street photography to landscapes, but their spontaneity and artistic processing gives them a style of their own. The influence of iPhoneography can be seen increasingly on more traditional photography, as well as on film and TV, where iPhoneography-style processing such as blurring, tilt-shift effects, vignettes and colour filtering is increasingly in vogue.

There is a vibrant community of incredibly talented iPhoneographers on the Internet who are pushing the boundaries of this medium and sharing their results with each other and the rest of the world, and this even includes some professional photographers such as Chase Jarvis who have embraced the iPhone as a valid and exciting part of their toolkit. Many of these iPhoneographers use Flickr to share photos and to give feedback. This makes sense because Flickr has been the most popular photo-sharing site for some time. However, some iPhoneographers use other sites in addition to, or instead of, Flickr. Tumblr has been popular due to its simple microblogging format which is ideal for posting photos and getting feedback on them, although recent technical glitches have made it a less attractive proposition. Some maintain photo blogs using WordPress or Blogger. Twitter is also very popular with iPhoneographers because it enables them to share thoughts and tips, as well as to link to new photos no matter where they are hosted.

In addition to these generic sharing sites, however, there are sites which are specific to mobile photography. Taking the iPhoneography world by storm currently is the mobile photo-sharing phenomenon known as Instagram. Despite its low-resolution square-format restrictions, it enables mobile photographers to share photos and get feedback in a very simple and streamlined way which makes it very addictive for many. Other sites dedicated to mobile photography are EYE’EM and MobiTog. Instagram, EYE’EM and MobiTog cater to all mobile photographers, not just iPhone photographers, so those using Android phones and Blackberrys can also join in the fun.

A rather different approach to photo sharing is taken by P1XELS. This is a curated site which takes the form of a blog, and at P1XELS they are keen to emphasise the importance of iPhoneography as an art form rather than just as a method for taking ‘snaps’. Only photographs taken on iPhones are welcome here, which is probably indicative of how much more advanced the iOS platform is for photography in comparison to other mobile platforms. Anyone remaining cynical about the worth of iPhoneography only has to browse the beautiful images on this site to find out what incredible works of art can be created from photographs taken on the iPhone.

There are various other blogs devoted to showcasing talented iPhoneographers and which also provide news and information about the iPhoneography scene. Glyn Evans’ iPhoneography blog, founded back in 2008 when Glyn coined the term ‘iPhoneography’, is usually the first port of call for information about what’s happening in the iPhoneography world. Other excellent iPhoneography blogs include Marty Yawnick’s Life in LoFi, Edgar Cuevas’ iPhoneogenic, and the bickr blog. As well as these blogs, there is also the very worthwhile iPhoneArt site which has a thriving community and is growing and getting even better all the time, and also the extremely useful iphoneographyCentral site, recently set up by Nicki Fitz-Gerald, which provides iPhoneography tutorials and other handy resources.

iPhoneography exhibitions and events are occurring around the world. Some of these have been arranged by P1XELS, but there have been quite a few others too, including several at various Apple Stores worldwide. More are appearing all the time, such as the HIPSTAMATICS exhibition in London earlier this year, and also the London iPhoneographers: Street Photography event which I’ve helped to organise at the Apple Store on Regent Street in London. As mobile photography becomes ever more prevalent, no doubt we will see even more of these events and exhibitions appearing in the future.

A similar scenario is occurring with books about iPhoneography, which are increasing in number all the time. Current examples of traditionally-published books are The Art of iPhoneography by Stephanie Roberts, Create Great iPhone Photos by Allan Hoffman, iPhone Photography & Video for Dummies by Angelo Micheletti, and iPhone Obsessed: Photo Editing Experiments with Apps by Dan Marcolina. There are also now many self-published books by iPhoneographers on Blurb, including my own iPhoneography.

After reading this article, I hope those skeptical about the future of mobile photography, and iPhoneography in particular, are a bit less cynical. I also hope that those who are already pursuing an interest in this type of photography have found some useful information and have been encouraged to take their interest further. There is much fun to be had, and many great images to be created. My own interest in iPhoneography began in August 2009 (when I wrote a blog post about my new-found enthusiasm for it), and since then, as already mentioned, I’ve helped to organise an iPhoneography event and have published a book of my iPhone photos. I’ve also been showcased and reviewed in various places, been successful in photo contests, been featured on Flickr Explore, and been approached to have one of my photos used for the cover of a novel (though unfortunately this didn’t happen in the end, but it was nice to be considered). Additionally, I run the London iPhoneography Group which I formed in September 2010. You can find me on Flickr as matt_brock, you can check out my Tumblr-based iPhoneography site or look at my work on P1XELS should you wish to do so, and you can find me on Twitter as _mattbrock. My interest in iPhoneography continues to grow, and given the exciting nature of the medium I’m quite sure it will do so for some time to come.

At 7pm on 11th July, the Apple Store on Regent Street will be hosting an iPhoneography event to highlight four talented members of the London iPhoneography Group who will be showing their work and discussing how they use their iPhones for street photography in London: Daniel Holland, Robson Santos, Matthew Burlem and Leyla Bile.

We’re very excited about this because it will be the first iPhoneography event in an Apple Store in England, and also because it will coincide with the first ever London Street Photography Festival which runs from 7th to 17th July. There have been superb iPhoneography events in other Apple Stores around the world, and we’re very pleased to be able to do one of our own in London.

As administrator of the London iPhoneography Group and a keen iPhoneographer and Londoner, I’m providing support for this event (but not showing photos this time).

Useful Links

Daniel Holland (Danny Dutch)
Robson Santos (iPhoneography London)
Matthew Burlem
Leyla Bile

London iPhoneography Group
London Street Photography Festival

Update (18/07/11)

The event was extremely successful and I’ve written a review of it on the iPhoneography blog. Below are some photographs from the event.

Firstly, EC2 is much more flexible than traditional colocation. Our infrastructure can now be expanded or altered, quite radically if necessary, in hours or even minutes, just by typing a few commands or clicking a few buttons. Previously, making similar changes would have taken weeks of planning, ordering, delivery, physical installation, software installation, etc.

Secondly, EC2 works out cheaper. EC2 server instances tend to work out cheaper than their physical colocation equivalents, especially when you’re able to plan ahead with pricing by using Amazon’s ‘Reserved’ pricing instead of their standard ‘On-Demand’ pricing. Also, because of EC2′s flexibility, it’s only ever necessary to run precisely the infrastructure required at the present moment, so you’re never paying for more than you need. With a traditional colocation setup, the lack of flexibility means that you will always have more hardware than you need at the present moment in order to cope with possible traffic spikes, hardware failures, etc., and that extra hardware increases the cost even further.

Getting up and running with EC2 is pretty straightforward. You just need to provide payment details, set up security details, and then you’re free to fire up server instances and try it out. There are various instance types to choose from depending on the hardware spec you need. Much of this can now be done via the Amazon Web Services (‘AWS’) Management Console, although you can also install the EC2 command line tools in order to operate EC2 via the CLI. I prefer using the CLI, probably because I’ve been doing it that way for longer, but also I guess there are still some things you can do via the CLI which you can’t do via the AWS Management Console.

Any decent system administrator will want to set up and configure instances according to his own tastes and requirements, and the best way to do this is to start an instance from an image that’s closest to what you want, then modify it accordingly, then create your own image from that running instance. I originally started up an instance from a CentOS image provided by RightScale, then modified it heavily according to my own preferences, then created an image of my own from it. This image is what I called the ‘base’ image.

I then created several different instance types for each of the different types of servers we have in our infrastructure. For each of these I started from the ‘base’ image, installed the applications required for that instance type, then created a new image from it, so that I ended up with several images: one for web server instances, another for mail server instances, and so on. Each instance type has a certain set of events which need to occur when that instance is first run, e.g. working out its own external hostname, setting up certain directories that are needed, etc., and for this I have a startup script in /etc/rc.d/init.d on each instance which works out whether it’s a brand new instance and, if so, performs all these tasks.

Once all the required instance types have been created and made available as images which can be started as required, it’s almost possible to start up a full web application infrastructure with all the components required for it. However, there are a few extra things to think about first.

Firstly, it’s necessary to consider disaster recovery. Conveniently, most of our instances do not contain any unique data which would be missed if the instance died, and if an instance dies then it’s normally just a case of firing up a new one to take its place. Obviously requirements will vary from one company to another, but in our case only the database instance contains unique data, i.e. the live database. I solved the disaster recovery requirement for the database instance by utilising an AWS storage solution called Elastic Block Store (or ‘EBS’). An EBS volume is a storage volume which can be connected to a running instance and which can also be snapshotted. So, all the data for our live database is on an EBS volume connected to the database instance, and this gets backed up and snapshotted every night. I have a Bash script which does this automatically from cron, and which also removes any shapshots older than 28 days.

Therefore, if the database instance dies, I can just fire up a new instance and connect the EBS volume to the new instance. In the unlikely event that the EBS volume dies, I can just create a new one from the latest snapshot and connect it to the database instance. This is therefore a very handy DR solution for the live database, and EBS volumes have the added bonus of being faster than the standard instance-based storage, which is obviously ideal for a database.

All instance images and EBS snapshots are stored on S3. S3 is an exceptionally safe form of storage, so the chances of anything getting lost on it are incredibly low, and this very low level of risk might be reassurance enough for many companies. In our case, however, I’m exceptionally paranoid so I also take backups of everything we have on S3 (including all our media which is also stored there). Every month I use the rsync-style functionality within the very handy s3cmd tool to incrementally backup all our S3 buckets to a disk on a server in our office.

Another consideration is how to load balance the front-end servers. Amazon provide a great solution for this called Elastic Load Balancing. This can be used to quickly and easily create a load balancer which can be configured to distribute whatever traffic you like across whichever instances you need. It even has support for sticky sessions. The only problem I’ve found with ELB is that it requires you to point a CNAME at the load balancer rather than an A record, which means that you can’t use root domains (i.e. apple.com instead of www.apple.com) because the DNS spec does not allow a root domain to be a CNAME. There are various hacks to get around this, but none of them are perfect, and it’s a problem I hope Amazon will solve soon. In the meantime, this thread in the AWS developer forums gives more information.

It’s also necessary to consider the delivery of email. Unfortunately, due to the ease with which server infrastructures can be created on EC2, a lot of spammers use it to send email, which means that EC2 IP addresses are heavily blacklisted. For each EC2 instance you wish to send email from, therefore, you need to follow a process which helps Amazon to ensure that you’re not sending spam. Firstly, you need to assign an Elastic IP to the instance. An Elastic IP is an IP address you can request and then keep for your own use, assigning it to whichever instance you like at any given time instead of using whichever IP address that instance happened to have by default. Once the Elastic IP is assigned to the mail server instance, you just need to tell Amazon to set up reverse DNS for that Elastic IP and remove its email sending limitations. This process works well, and we haven’t had any problems sending mail from our mail server instance since completing it.

As with the general EC2 functions, EBS, ELB and Elastic IPs can be controlled via the AWS Management Console, but the CLI tools tend to be more powerful than the console. CLI commands for EBS and Elastic IPs are included within the EC2 command line tools, but ELB has its own set of CLI tools.

The actual migration of our infrastructure to EC2 was similar to other infrastructure migrations I’ve done in the past, but with the added bonus of EC2′s flexibility. This means that you can set up the infrastructure roughly how you think it will need to be, and then tweak it as you go along by starting and stopping instances, changing instance types, etc., instead of having to predict things as best you can then hope for the best. Since the migration, our entire infrastructure has been successfully running in the cloud for several months, and we continue to be very happy with Amazon Web Services. We closed our account with our colo provider some time ago, then sold all our physical servers.

For those of you who haven’t yet started using EC2, or who haven’t got very far with it yet, lots of useful details on the practicalities of hosting on EC2 can be found in this excellent series of blog posts written by my friend Paul Norman.

There’s a lot more you can do with EC2 which I haven’t covered, but hopefully this post gives an idea of how easy it is to set up a web infrastructure on there, and by reading the documentation and following the updates from Amazon, you can easily work out which additional details you need.

As I have a MobileMe account, there is a heavy focus on that service, but MobileMe does cost a significant amount of money each year which may not be ideal for everyone, so I’ve given alternatives wherever possible.

Email

Email is the first of several things that I use MobileMe for. On the whole, MobileMe provides a decent, reliable email service with plenty of storage, and it works very smoothly with Mail on iOS and Mac (as you would expect). Also, the MobileMe System Preference enables you to sync all your mail accounts, rules, signatures and smart mailboxes across all your Macs, saving you a lot of configuration hassle. Additionally, MobileMe webmail is very nice for those rare occasions where it’s needed.

A free alternative is Gmail, but this doesn’t work as nicely as MobileMe in Mail on Mac or iOS – in fact it’s pretty messy in Mail on Mac – plus you can’t automatically sync your mail accounts, rules, signatures and smart mailboxes across all your Macs.

Contacts

I’m always amazed when I receive one of those Facebook updates from a friend saying that they’ve lost or changed their phone, and could everyone please contact them to remind them of their number? Now that we all use multiple computers and smartphones, surely nobody should have to manually re-add all their contacts into their new phone.

I use MobileMe to keep contacts synchronised automatically and wirelessly between Address Book on my Macs and Contacts on my iPhone. It works beautifully.

If you’re using Gmail for email then you’ll presumably want to access your Google Contacts. You can do this from iOS with Google Sync, which works OK. It’s harder to get Google Contacts working with Address Book on Mac, though. It generally seems to be pretty glitchy and, at least for certain versions of Address Book, the sync will only occur when you physically connect and sync your iPhone, which is pretty silly. On the whole, this is a somewhat messy and undesirable option from what I’ve seen.

Calendars

Again, I use MobileMe to keep my calendars in sync, and it does a great job of keeping calendar events synchronised across iCal on my Macs and Calendar on iPhone. As well as my regular calendars, I also subscribe to calendars on other sites such as Last.fm, and MobileMe keeps all these calendar subscriptions in sync too, both on my Macs and my iPhone. It even maintains the proper calendar colours across all my devices! Excellent stuff.

The obvious free alternative here is Google Calendar. Using Google Sync this works reasonably well with iPhone, but getting synced with iCal on Mac is fairly complicated and potentially rather unreliable. Also, as far as I am aware, Google Sync doesn’t provide any way of letting you sync calendars you’ve subscribed to on other sites. Again, not a great solution.

To Do Items

MobileMe infuriated me for quite some time with its inability to sync To Do Items from my Macs to my iPhone. However, the new MobileMe calendar enables third-party developers to make apps which ‘plug in’ to the MobileMe calendar to fill in the missing gaps. One such app which has recently been created for iOS is BusyToDo. I’ve just started making use of this app and I can report that it works wonderfully and is fantastic value for money. I can now sync my To Do Items back and forth between my iPhone and iCal on my Macs, and because I make extensive use of ‘to do lists’ I’m extremely pleased about this.

If you’re not a MobileMe subscriber, the best solution I’ve found is to use Google Tasks, which is free. On iPhone, iPad or iPod Touch, go to gmail.com/tasks in Safari, then just log in with your Google account and you’re ready to go. If you click the arrow at the bottom and choose ‘Add to Home Screen’, it will create a nice Google Tasks icon so it can be accessed like a regular app.

Unfortunately, setting up Google Tasks as a Mac app is a little more complicated, but it’s not too complicated and it works well when it’s done. The first thing to do is download and install Fluid, a very handy program which enables you to turn web apps into Mac desktop apps. Once installed, launch Fluid to create a new ‘Site Specific Browser’. The URL should be https://mail.google.com/tasks/ig and the Name should be ‘Google Tasks’, then click Create. You can then go into your Applications folder and run Google Tasks just like a regular app, logging in with your Google account to get to your tasks. To make it look a bit nicer in your Dock, you might want to add a more appropriate icon. I suggest downloading the excellent Fluid Mac Icons from Fairhead Creative Goodies page (and double click to unzip it once it’s downloaded). Then go into Preferences for your Google Tasks app, click the Change button next to ‘Application icon’, and select the ‘Gtasks-Icon.rsrc’ from the ones you just downloaded. You might need to restart the app and possibly re-add it to your Dock for the new icon to take effect.

When the app is running, you should see something which looks like this:

The only problem with this solution is that it doesn’t work when you don’t have any Internet access, but in practice I haven’t found that too much of a problem.

There is also a task list management application called Things, but that’s stupidly overpriced for a glorified task list in my opinion, and it doesn’t even do over the air sync between iOS and multiple Macs.

Notes

I make extensive use of notes for keeping lists of music and films I want to get, making meeting agendas and action lists, writing down ideas for songs, and all sorts of other random things. Notes are, therefore, pretty important for me, and I use MobileMe to keep my notes synced between Mail on Mac and the Notes app on iPhone. In my opinion it’s a bit odd having notes in a mail application, but there we go; it seems to do the job fine, and it helps to keep as many things as possible within MobileMe, which is neater for me.

If you don’t mind in-app adverts and a limited amount of storage for syncing data, a free alternative to consider is Evernote. You can install the Evernote app on multiple Macs and iOS devices and it does a fair job of keeping things in sync wirelessly. I found, however, that the wireless sync on the iPhone was pretty ropey, but they may have improved it since then. One benefit of Evernote is that it allows many types of media as well as just written notes, so you can take a photo with your iPhone and use that as a note, or you can save a web address from Safari and use that as a note, etc. That’s quite cool.

Documents

It’s obviously very useful to have your documents kept in sync between all your devices. I use MobileMe’s iDisk for this. On my Macs I choose the option to sync my iDisk, which keeps a local copy of the iDisk on that Mac to speed things up. On my iPhone I use the iDisk app which enables me to view documents (though, annoyingly, it doesn’t work with OpenOffice documents) and share them. In general, this solution works well for me.

A popular alternative is Dropbox, which has apps for Mac and iOS. When I’ve used Dropbox I’ve found it to be a little clunky, but many people seem very happy with it. Dropbox is free for a limited amount of storage, but prices increase significantly when you add additional storage. Another solution to consider is Google Docs, which allows you to edit documents in the browser on both Macs and iOS devices. As far as I know there are no desktop apps for it, although you could use Fluid to make a desktop app out of it like we did with Google Tasks (above). Google Docs is free, however.

Bookmarks

I depend quite heavily on bookmarks and find it frustrating when I’m using a browser which doesn’t have all my bookmarks in it. By far the simplest way of dealing with this problem is to use Safari on all your Macs and iOS devices and sync bookmarks between all of them using MobileMe. This works well for me because I have a MobileMe account and I consider Safari to be a decent browser.

It’s not easy finding alternatives to this system. Other browsers offer free ways of keeping bookmarks in sync, e.g. Google Chrome features a bookmark sync facility using your Google account, but that only works with Chrome on the desktop and will not sync your bookmarks to Safari on your iPhone/iPad/iPod Touch. If you use MobileMe but you prefer to avoid Safari on the desktop, you can get around this by using Xmarks to sync your bookmarks from your preferred browser to Safari on the desktop, then use MobileMe to sync from desktop Safari to mobile Safari – this is a messy solution, but it does work.

News

RSS is a fantastic way of keeping up to date with your favourite news sites, and it becomes even more useful if you can keep your RSS feeds in sync between all your devices. You can sync RSS feeds over MobileMe but in order to do this you have to use the RSS facilities in Safari which are fiddly and limited, and you can’t sync to iOS, so all in all this is a pretty poor solution.

Google Reader is the best thing to use here, and it’s free. I don’t much like using the Google Reader web app on my Macs, but there are Mac desktop apps which will sync well with Google Reader and provide a good all-round experience. I use NetNewsWire, which has always worked well for me, although I’m not keen on the in-app adverts. There are other options on Mac such as Gruml, but I’ve never felt any huge need to switch.

NetNewsWire is also available on iOS but, like many others, I’ve found it to be increasingly unusable on my iPhone. There are much better alternative (paid) apps which many people swear by such as Reeder and Byline, but at the moment I actually just use Google Reader in Safari because it’s free, it has a decent mobile interface interface, and you can create an app icon for it on your home screen just as we did for Google Tasks (above).

Twitter

Yes, it’s actually possible to keep Twitter in sync across multiple devices. If you’re a fairly heavy Twitter user, you might find it quite annoying that your desktop Twitter app doesn’t know about all the tweets, direct messages, replies, etc. that you dealt with earlier on your iPhone. You can solve this problem by using Echofon on all your Macs and iOS devices. It’s not perfect, but it does a reasonable job of remembering where you were on your iPhone and transferring that across to your Mac, and vice versa. Echofon is free if you don’t mind seeing in-app adverts.

Edit (08/02/11): modified ‘To Do Items’ to reflect the availability of BusyToDo for syncing MobileMe To Do Items to iOS.

Towards the end of the year I created the London iPhoneography Group and I’m hoping to find the time to arrange a meet for the group sometime soon – it will be great to meet some fellow iPhoneographers in real life. I also produced a book of the best iPhone photos I took with my iPhone 3G, and I’ve received lots of nice feedback from people who have bought the book which makes it really worthwhile. During 2011 I hope to produce a book of photos taken with my iPhone 4.

During 2010 I’ve been featured in various places and had some success with contests during the year, details of which are available here (when Tumblr is working). I hope to continue this success in 2011, but the most important thing for me is to keep trying to create interesting iPhone photos and share them with others, and also to keep being inspired by the amazing things other iPhoneographers are doing. Long may it continue!

Never having used software RAID before, this was quite a steep learning curve for me, especially given that the Xserve G5 is a ‘New World’ class of Mac, which means that it uses the Apple Partition Map partitioning scheme, which is fairly rare in a Linux world that is largely involved with master boot records. I use Yellow Dog Linux on our Xserves because it’s based around Red Hat, and we use Red Hat-based distributions as standard. Yellow Dog Linux 6.2 actually makes it very easy to set up RAID as part of the installation process, although there are one or two aspects I prefer to do a different way as a result of experimenting with this process. Also, manipulating existing arrays and making new disks bootable took me some time to figure out.

So, let’s discuss the method which ended up working for me. The Xserve has three disks, and I wanted two of the disks to contain RAID 1 partitions with the third disk as a spare. I used RAID 1 because I wanted a RAID level which would continue to work well even if only one disk was left functioning on the server. RAID 5 requires a minimum of two disks with Linux software RAID, and it’s slower than RAID 1, so it’s generally less desirable for me. Also, you can’t use a RAID 5 array for your /boot partition, so since you’ve got to use RAID 1 for /boot then I figure you might as well use it for everything else.

Before starting the Yellow Dog Linux installation, I firstly used mac-fdisk in Finnix to partition the disks myself, because this gives more control over the process, and I found that odd and unhelpful things happened when trying to do the initial partitioning in Yellow Dog Linux. Finnix, incidentally, is a superb live sysadmin Linux distribution which still has PowerPC support in version 93.0 and is thus ideal for working on Xserves. mac-fdisk must be used because it’s the only Linux partitioning tool I’m aware of which understands the weird partition types needed for things to function on a New World Mac.

So, boot into Finnix, for which you’ll need to choose ‘finnix64′ on these G5 machines. Next we’ll do the partitioning. In mac-fdisk, for each of your disks, firstly press I to initialise the partition scheme. This will remove all existing partitions then create the partition map, which is a special partition that must always be the first partition on each disk. Then press B to create the special Apple bootstrap partition, which should ideally be the second partition on the disk (specify 64 as the first block for this partition so that it follows on from the first partition). Then press C to add each Linux partition that you will use as part of your RAID partitions. I normally make the first one of these the /boot partition with a size of 100MB, then the second one the / partition to use up most of the disk, then finally a partition to use as swap. Don’t forget to press W to write the new partition map. After that, make a note of the exact scheme and partition sizes you have used on the first disk, then do the same on the other two disks.

Next, boot off the Yellow Dog Linux install CD. When you get to the partitioning screen, choose ‘Create custom layout’ and continue. You should now see a representation of the partitioning scheme you just created within Finnix. It will probably say ‘ext3′ or ‘Foreign’ for each of the Linux partitions you created. Edit each of these on /dev/sda and /dev/sdb these so they are all formatted as ‘software RAID’. Just leave /dev/sdc alone for now – we’re not going to do anything with this third disk as it’s just there as a spare. Actually, Yellow Dog Linux does allow you to set this up as a hot spare for your RAID array partitions, but I found in practice that this seemed to cause more problems than it solved, so I avoid that now.

Then click the ‘RAID’ button and choose to ‘Create a RAID device’. Assuming you’ve followed a similar partitioning scheme to me, choose ‘/boot’ as the mount point, set the RAID level to ‘RAID1′, and tick the boxes next to ‘sda3′ and ‘sdb3′. This will create the RAID device /dev/md0 for the /boot partition. Then do the same for the other RAID partitions you need, including the swap partition (if you have one). Then proceed with the rest of the install, configuring the other options as you want them.

Once Yellow Dog Linux is installed, boot your Xserve and log in. When you do watch cat /proc/mdstat, you should see information about your RAID arrays showing a healthily active state and containing the disk partitions you specified in the installer. Some of your RAID partitions may still be resyncing; don’t worry about this – it’s just a normal part of the process. So now you can start using your Xserve, and you won’t need to worry about the RAID setup again until one of the disks fails.

In Yellow Dog Linux, when a disk fails, you will get notified via email because mdadm is automatically started up at boot time as a daemon in monitor mode. For these notifications to work you just need to make sure you have your root email from this server going somewhere useful. When you get a notification like this, you’ll see that the partitions on the failed disk will be marked as failed, e.g. if it’s the second disk which has failed then mdadm --detail /dev/md0 will show that /dev/sda3 is active but that /dev/sdb3 has failed. You should now remove all the failed partitions with mdadm /dev/md0 -r failed and equivalent for each of your RAID partitions. Then shut the machine down, take out the failed disk, and replace it with the third disk which we allocated as a spare. It’s best to put the spare disk into the slot where the failed disk was, so that we don’t encounter any confusion with drive ordering. Then start the server up again.

As we already partitioned this new disk earlier on, we should be easily able to add the partitions to our RAID arrays with mdadm /dev/md0 --add /dev/sdb3 and equivalent for each partition. We can then watch the new partitions being synchronised into the arrays with watch cat /proc/mdstat. Once that’s done, we’re back up and running with full redundancy and only a tiny amount of downtime to swap the disks around.

We still need to make the new disk bootable, however, so that we can boot the server if the original disk fails. This is a case of setting up the yaboot boot loader properly on the Apple bootstrap partition on the new disk. As you may remember, the bootstrap partition is the second partition on the disk which we created as part of the partitioning process. For the sake of these examples we’ll assume the new disk is the second disk (/dev/sdb), but if it’s the first disk then just change /dev/sdb to /dev/sda in each example.

So, firstly we need to make sure that /etc/yaboot.conf is set up correctly. The ‘boot’ parameter must be set for the correct partition on the correct drive, so in this case this line should be boot=/dev/sdb2. There shouldn’t be any need to edit anything else in this file. We then format the bootstrap partition with mkofboot -b /dev/sdb2. And then we install the yaboot bootloader with ybin -b /dev/sdb2. You should then be able to boot off the new disk, although it’s worth testing that just to be on the safe side.

We can now add a new third disk to the Xserve and partition it in the same way as we did earlier, so this disk will act as a spare next time a disk failure occurs. And there we have it – full disk redundancy on an Apple Xserve G5 using Linux software RAID.