Tag Archives: Fireworks

Our First AIR App

Posted in Development on by Paul Boag

With Adobe AIR now up to 100 million downloads, being utilised by big name players such as the BBC and increasingly invading our desktop Headscape decided to give it a go.

This is a guest post by two of the Headscape developers – Craig Rowe and Dave McDermid

Setting up

Adobe Air is Flash + WebKit + SQL Lite on the desktop. As a Flash developer you can dive right in and use the Air extension for Flash to publish your beautifully crafted swfs and AS code into an installable cross platform desktop app. However, the flash projector has been around for a while doing similar things and we wanted to put our hard earned web development skills to productive use. So we went the HTML/Javascript route.

The SDK is free for download but Aptana provides a rather neat eclipse based IDE in which to work. Handily the new project wizard allows you to import a multitude of Javascript libraries making it incredibly easy to get a new project up and running with your preferred initial setup (we went for jQuery).

The Problem

To make the exercise worth while we needed a real world problem to solve. Trivial examples usually do very little other than increase your ability to copy/paste example code and do your best fireworks night ‘ooo’ ‘ahh’ impression at it. Instead, we chose to add to our server admin experience…

As loving, caring web developers we actively monitor all our servers, and most of our live websites. For a while this was a DOS script, this was then migrated to a Bash script on a Cron job. It worked great, but required a computer science degree to maintain. So here was our problem: we needed a pretty, maintainable and reliable app to monitor websites. All it had to do was let us know when one disappears or throws a nasty error.

Download and try our site watcher AIR application

The Journey

Step 1: The wireframe

Paper Wireframe

A new Adobe AIR project in Aptana comes with an html file named after your application which acts as your main program window. Creating a basic layout with a few buttons, titles etc can be done in a snap. The html, css and javascript are dealt with in exactly the same way as if intended to be deployed as a website (with no browser compatibility worries as we are only targeting the Adobe AIR WebKit rendering engine).

Step 2: The magic

The wireframe identified the main viewer as comprising of an unordered list of sites, each with their current status and edit/check/remove links. This list needed to be persisted, but editable by the user.

If this was a website we may be looking to server side scripting and a database of some nature. However, we had only jQuery and the air libraries. Although SQL Lite was an option we decided it was an over complication for what is a relatively simple, first AIR app. So, knowing that we could use jQuery to manipulate a DOM and the air libraries to save and open local files we opted for XML as a data source.

<?xml version="1.0" encoding="utf-8"?>
<sites>
<site address="http://www.headscape.co.uk/" status="200" frequency="30">
Headscape&#146;s website
</site>
...
</sites>

Looking back into our application html file we can see that we are given a readLocalfile() function that returns the string contents. This can then be passed into a jQuery object and manipulated in the usual way.

[The canny among you will notice that this readLocalfile() function is merely a few calls to the flash filesystem classes (using the AIR aliases). In fact at some points I directly call the flash library rather than using the AIR alias. There’s no functional difference, I’m just used to the flash namespaces]

With this ability added to the jquery ajax capabilities the application flow could be easily envisaged as follows:

  • On DOM Ready read the local xml file
  • For each site element in the xml create an LI element with the appropriate display and action elements
  • Fire off a jquery ajax call for each site
  • Use the response code to formulate a class for the LI to change its display
  • Fire off any notifications if the response code has changed i.e. e-mail, notification window, twitter post
  • Set a timeout before checking again
  • On window close parse the unordered list back into xml and write it to the persistent file

The Stumbling Blocks

Viewing the source of an installed AIR app can be done by nipping into program files (or Applications for the MACs amongst us) and looking in the application name directory. Here you will see the html, css and javascript files that make up the app (so we can continue to learn from others deployments just as we would with a website).

A brief look at the sitewatcher source and the flow described above becomes immediately clear. ‘Sitewatcher.html’ is our main form and it includes script.js as the main driver of the application with the #sites ul as the main containing element. The rest is GUI. ‘PopulateUIfromXML()’ directly completes steps 1-2 and fires off the 3-6 process via ‘CheckSite()’. However dispersed within this are the unusual non-front end website development bits, so we’ll look at those now:

Acting as a System Tray App

Many AIR apps, particularly those to do with notification (twitter, yammer etc) seem to want to run as system tray applications, we were no different.

The process of doing so is relatively easy, and encapsulated in the appropriately named SetUpSysTray() function of script.js. Essentially what we need to achieve is an override of the minimise behaviour, the setting of an appropriate icon and the associated window toggling behaviour.

Window.nativeWindow gives us access to the OS window holding our html window, and we can listen to events on it in much the same way. ‘nwMinimized’ is set to fire on display state changing and, if being minimized, instead docks (hides) the window and prevents the default behaviour. 

if(air.NativeApplication.supportsSystemTrayIcon)
window.nativeWindow.addEventListener(runtime.flash.events.NativeWindowDisplayStateEvent.DISPLAY_STATE_CHANGING, nwMinimized);
function nwMinimized(nativeWindowDisplayStateEvent) {
if(nativeWindowDisplayStateEvent.afterDisplayState == runtime.flash.display.NativeWindowDisplayState.MINIMIZED) {
nativeWindowDisplayStateEvent.preventDefault();
Dock();
}
}
function Dock() {
window.nativeWindow.visible = !window.nativeWindow.visible;
}

This works fine but on its own will actually just hide the window from view/the taskbar leading the user to have to use task manager to close it. To ensure that an icon for your application sits in the system tray we need to set an icon for the nativeApplication (the mere presence of which will cause windows to display the app in the systray).

Back to SetUpSysTray() and we’re using a content loader to load the icon graphic into memory, with an iconLoadComplete handler waiting to do the work:

var iconLoader = new runtime.flash.display.Loader();
iconLoader.contentLoaderInfo.addEventListener(air.Event.COMPLETE, iconLoadComplete);
iconLoader.load(new air.URLRequest("../icons/AIRApp_16.png"));
function iconLoadComplete(event){
if(air.NativeApplication.supportsSystemTrayIcon){
air.NativeApplication.nativeApplication.icon.bitmaps = new Array(event.target.content.bitmapData);
air.NativeApplication.nativeApplication.icon.tooltip = "SiteWatcher";
air.NativeApplication.nativeApplication.icon.menu = new air.NativeMenu();
// Create Menu Items
var openCommand = new air.NativeMenuItem("Toggle");
openCommand.addEventListener(air.Event.SELECT,function(event){
Dock();
});
var sep = new air.NativeMenuItem("", true);
var exitCommand = new air.NativeMenuItem("Exit");
exitCommand.addEventListener(air.Event.SELECT,function(event){
air.NativeApplication.nativeApplication.exit();
});
// Add Items to menu
air.NativeApplication.nativeApplication.icon.menu.addItem(openCommand);
air.NativeApplication.nativeApplication.icon.menu.addItem(sep);
air.NativeApplication.nativeApplication.icon.menu.addItem(exitCommand);
}
}

We simply set the native application icon to be the bitmap data before finalising the sys tray setup by creating a menu to appear on click.

Note for MACs: The minimised event listener is not applied if the system does not support system tray icons. This is to avoid confusion on MACs where a minimise goes to the MAC dock anyway.

Notification Windows

So the app can now run in the system tray and use jQuery to check the sites listed in an XML file at regular intervals. However we need a process of notification. The App could be running on an actively used machine in which case we want messenger style pop-ups. Or it could be running on a separate machine/server from where we want it to send e-mail/other notifications.

In the case of window notifications we took a short cut and used some example code from Adobe Developer Center. This is encapsulated within the DisplayManager.js and Message.js files. Display Manager acts as a queue, dequeuing and displaying on a timed basis if the user is present. This is an important requirement as you do not want a user missing a notification prompt merely because they were away from their desk for a while. It can be easily achieved via the USER_IDLE and USER_PRESENT air events – in this case stopping and starting the poller.

‘CheckSite()’ simply queues message HTML when the response code received is different from the previously stored code. When the queue poller is running (the user is present) the message is dequeued and displayed via Message.js.

At this point it is worth remembering that notification windows are no different to any other native window. It’s just a name we’re giving to the way we are using native windows. They therefore have their own events, contents, position etc and this can be seen by the Message.js code where a new chromeless, transparent native window is created and its contents loaded from the message.html template.

The display process is then as follows:

  • Message.js stores the message content in a variable on the new window
  • MessageScripts.js, then running once the message.html dom is ready, sets a listener on the HTML_BOUNDS_CHANGE event before setting the message variable content as the body – ultimately firing the bounds change event
  • This event is handled by setting the native window height to match before firing the layoutComplete event
  • On hearing this Message.js makes the window visible, finds an appropriate resting position (in relation to any other messages) and animates it in.
E-mailing

A key feature of any site watcher is to let us know when something bad happens. The combination of emails and an email-to-text service allows us to be notified the minute we spot trouble. This was easy enough in the bash script, using sendmail on the Mac. Not so straight forward for an AIR app. We can’t run sys commands and there is no built-in SMTP server. The solution is to use sockets in AIR. A little hardcore, but it keeps the solution nice and tight.

Anyone who’s sent an email with telnet will know that the principle of SMTP is, as the acronym suggests, simple. Adobe gives us plenty of clues for opening sockets and listening for messages. All we had to do was make sure we sent the right info. There are some restrictions in opening sockets from scripts outside the application sandbox, but for our purposes it worked a treat. With a little trial and error we were firing off emails left right and centre.

The icing on the cake was adding twitter support. With a one-line AJAX call in JQuery and a little config it was a no-brainer. This allows us to keep an online audit in the form of a private tweet stream. For people who check twitter more frequently than their email, it’s handy for notifications. If Twitter let us UK folk receive updates via SMS again then we can ditch email-to-text in favour of Twitter.

Step 3: The makeover

One of the nice benefits to working in the web-kit world is being able to use some CSS3 styling such as rounded corners. So we went to town. The more design that is CSS based and the less that is image based the better.

JQuery UI allowed us to make the entire list sortable in a sweep of the mouse, and the prefs popup tabbed in a blink (suddenly there was heaps to customise!).

The End

Hopefully this post has given you an understanding of how quick and easy it really is to make a useful AIR application. We’ve shown how you can implement a system tray application that utilises notification popup windows and sends e-mails as well as uses local files as a data store. This is not intended as a best practice discussion. It was our first AIR app and developed in a very small amount of time as a proof of concept and so that we could share our experiences with you. We welcome any feedback.

Download and try our site watcher AIR application

144. Scale

Posted in Classic shows on by Paul Boag

On this week’s show Paul talks to Joe Stump from Digg about scalable websites, we review the best apps for web designers and investigate services for sending bulk emails.

Play

Podcast: Download (28.3MB)

Download this show.

Launch our podcast player

News and events

How much should you charge?

If you are starting your freelance career the number one question you will have is ‘how much should I charge?’ It is important and yet strangely it is not something you are taught at college. Perhaps they don’t teach it because it is a damn hard question to answer. It is certainly something we have avoided talking about on this show.

Fortunately an article entitled ‘Six things to consider when setting your freelance rate‘ has been released this week. Although the article does not give a magic number, it does provide 6 valuable insights that will inform your final decision. These include…

  • Young freelancers and recent grads almost always ask for too little.
  • You can do things your clients can’t.
  • Your rate influences your perceived value.
  • You don’t get to keep it all.
  • An hour worked is not an hour billed.
  • The higher you start, the less you’ll need to increase.

I couldn’t agree more with everything said in this article. However, the one that really resonated with me was ‘You do not get to keep it all.’ Your rate has not only got to cover your billable hours but the cost of sales and marketing, as well as your various overhead. The article has a link to a superb rates calculator that helps you work out your chargeable rate based on these various costs. Definitely worth checking out.

A plethora of accessibility posts

With the implement arrival of WCAG 2.0. we are seeing a resurgence of interest in accessibility. This has led to a plethora of accessibility posts over the last few weeks. These include…

  • Writing good ALT text – This is a simple post about the use of the ALT attribute. It suggests two rules of thumb when it comes to writing ALT text. First, if you were to describe the document to someone over the phone, would you mention the image or its content? If you would, the image probably needs an alternative text. Second, does the alternative text of any images in the document make sense if you turn off the display of images in your web browser? Simple advice, but well worth remembering.
  • Designing for Dyslexia – This is a series of 3 in depth articles that look at the subject of Dyslexia. It asks what Dyslexia is and how we as web designers can improve our sites to accommodate the needs of Dyslexia users. Its interesting stuff. Part 1 defines what Dyslexia is. Part 2 looks at some of the conflicting requirements with users who have visual impairments. Part 3 suggests some specific things you can do to improve the legibility of your designs.
  • Accessible forms using WCAG 2.0. – This extensive post aims to provide web developers and others with practical advice about the preparation of accessible HTML forms. It compares the WCAG 1.0 accessibility requirements relating to forms with those contained in WCAG 2.0. Important stuff but not a 5 minute read!
  • Too much accessibility – The RNIB explains how the LEGEND tag can cause more harm than good if not concise and relevant. The reason? LEGEND text isn’t read at the start of the FIELDSET, it is read at the start of the label. It repeats at the beginning of every single text label in that FIELDSET.

A business case for deleting content

I find myself using the word ‘simplify’ a lot when I talk to clients these days. So many website owners are constantly wanting to add features or content to their site. However, in reality we should be removing not adding to our already bloated websites. This is particularly true for large institutional websites. However it does also apply to smaller sites.

Take for example the Headscape website. When we started the redesign process for our site, I sat down and really thought through what information prospective clients wanted. The answer was very little. In the end our large text heavy website was reduced to a single page. That is the power of simplicity.

Gerrry McGovern summed it up perfectly this week in his post entitled the ‘Business case for deleting content‘. He wrote:

The more you delete, the more you simplify. The more you simplify, the more you increase the chances of your customers succeeding on your website.

We may think that we cannot delete content because ‘somebody might want it’ or because we believe ‘it will help our search engine ranking’. However, bloated sites bring complexity and with complexity comes confusion. The more content on your site, the less chance a user will be able to find the content they need.

12 principles for keeping your code clean

We finish today with a great post for those who need help with their HTML code. Whether you are a student learning HTML or a designer who is more comfortable in Photoshop than Coda, this is a very useful article.

The post provides 12 excellent tips for keeping your code clean. These include…

  • Use a strict doctype
  • Set your character set and encode those characters
  • Indent your code
  • Keep your CSS and JavaScript external
  • Nest your tags properly
  • Eliminate unnecessary divs
  • Use better naming conventions
  • Leave typography to the CSS
  • Add a class/ID to your BODY tag
  • Validate
  • Order your code logically
  • Just do what you can!

The article explores each of these points in depth and communicates clearly current best practice in coding HTML. Well worth the read even if only as a reminder.

Back to top

Interview: Joe Stump on Building a Scalable Site

Paul: Ok, so joining me today is Joe Stump from Digg. Good to have you on the show Joe.

Joe: Oh, good to be here.

Paul: Have we had you on the show before?

Joe: Ah, not that I’m aware of.

Paul: Oh, wow, well we need to rectify that then. It’s good to have you on. Well, I have to say, this interview was arranged by Ryan, who is our producer. And he’s a developer, and I’m a designer. And he suggested we got you on the show, not that I wouldn’t like you on the show, obviously. That we got you on the show, obviously about scaling websites. Now, I’m going to be out of my depth very quickly here, so you’re going to have to be very gentle with me Joe.

Joe: Sure

Paul: So, in fact, it was so bad, that as I sat down to write questions I thought: "I don’t know what I’m doing here" , so I went and talked to some of the developers at headscape, and I asked some of the Boagworld listeners, and so we’ve got a little selection of questions for you, that, hopefully we can learn a little bit about how you go about doing things at Digg. Lets start off, what’s your job title, what is it that you do at Digg?

Joe: Ah, I have a really fancy job title that doesn’t mean a lot of anything, but ah, my official job title is "Lead Architect" and um, I think what best describes it, is that I manage the technical implementation on the code side.

Paul: OK

Joe: So, Digg’s broken up into a lot of different arenas on the tech side, we’ve got, R&D, which is headed up by Anton Cast, we’ve got operations, which is headed up by Scott Baker, and then under that are the people that I work with, ah, probably most closely in implementing solutions for Digg. Ron Gorodetzky is our lead systems engineer, Tim Ellis, also known as timeless, is our chief DB wonk, and then, Mike Newton is our lead network guy. So I think us four kind of steer the technical implementation along. The managers, ah, the manage, and handle the strategy and partners, and stuff like that.

Paul: You managed to say the word manager with real distain.

Joe: Oh, no actually, I have a great manager, John Quinn, he’s our VP of engineering, he’s by far the best direct manager I’ve probably ever worked with. Yeah, he’s really good.

Paul: OK, well lets go back in time a little bit. And start by, well, when was the point when you realized, that you were going to start having scaling issues with Digg? When did you start thinking about the whole subject of scaling?

Joe: Um, well Digg was pretty big when I came on board, so Digg was about 10 – 12 million uniques when I joined on.

Paul: Wow.

Joe: And I think we’d just cleared 35 million last month. So scaling was obviously an issue, but the big difference is that, I think sites generally go through a few different levels of scaling, where like the first one’s like, "I’m just going to throw it on a virtual server, or an Amazon server, you know, you’re basically just seeing if things are going to just "stick to the wall", and then they do. Ah, so the first thing you normally do is start breaking services off onto separate boxes. I want to put my DB on one box, my server on another box, and maybe memcached on each of them. And then you hit, read saturation on that one DB server, so then you go to the kinda next level of scaling. Which is where Digg was when I started, where you start dangling, a whole bunch of read slaves, off of your DB master, so, and for those who are not familiar with the master / slave terms, you send all your writes to one database server, and then that disseminates those writes to a whole bunch of slaves, and then you send all your read traffic to those slaves. So that’s where Digg was when I started. It’s write http traffic across a whole bunch of servers, its read traffic across a whole bunch of slaves, and then we have one master. And we’re now going through, what I think is the third phase, where you hit write saturation on your master, which is a bigger problem, because you then need to start sending some write traffic to some masters, and we’re actually going with a strategy that’s common with Facebook, and Flickr, and those kind of websites, where it’s called horizontal partitioning, where you put some of your records on one server, and the other records on another, so it’s like, you can say, for users, all users whose names start with A through J, would go on this database server, and K to Z live on this other database server. So we’re in the middle of implementing the first swipe at that. So we’ll be pretty aggressively into where everything will be federated and partitioned across a whole bunch of servers.

Paul: OK, one of the questions which kinda came up, which kinda relates to that, is, once you start spreading things across multiple servers, how do you handle things like user sessions, which have obviously got to be persistent.

Joe: Aha, so there are a couple of ways to handle that which, I’d say most people are handling it by.. There’s two ways, probably that you can do it easily. One of them, is if you have what they call "session affinity" on your load balancer, so the load balancer will say: "Oh, well this person, last time I had them here, they went to server A, so we’ll send it back to that server". So the session always lives on only one box. That’s one way to do it, we don’t do that here, we have a custom session handler in PHP which sorts the session in Memcache, and that allows you to.

Paul: Can you just clarify what memcache is, for idiots like me who don’t know.

Joe: Sure, memcache is a distributed caching system that’s actually, basically what it allows you to do, is expose a machines RAM over the network, and cache stuff into another machines RAM across the network.

Paul: Ah, OK

Joe: Yeah, it’s insanely fast, it was developed by Danga back in the day, and Brice Fitzpatrick, yeah so it’s heavily used by anyone whose scaling with LAMP, even a lot of people who aren’t. They all use memcached.

Paul: Wow

Joe: So, yeah, we store all of our session data in memcached, so PHP creates a unique session id, and we just stuff session data into that in memcached, and we can distribute that across, I don’t know, 50 or 60 memcached servers, and what not.

Paul: So how many servers do you guys have, it must be a staggering number by now.

Joe: Um, yeah, it’s kinda funny, every time I ask Ron that, he’s always like "Ah, I don’t know"

Paul: Laughs

Joe: Because we really can’t I mean, I couldn’t give you a specific number because on any given day, we’ll pull or push into production, a dozen servers, so, hundreds, there’s definitely hundreds in production. So.

Paul: I mean, with that many servers, so obviously you’re talking about taking servers on and offline, and all that kind of thing, I mean, making updates to the site, when Daniel comes up with some stupid idea, that you’ve got to apply to the site, of a new feature that he wants to apply on the site, and you’ve gone through the process of making it work. And you’ve then got to push it live.

Joe: Aha

Paul: How does that work? How do you go about pushing something like that live when there are so many servers involved.

Joe: So we have Ron Gorodetzky our lead systems engineer guy. So us developers have a bunch of M4 make files, that, when you check the code out, you run basically Make, Install, and it, for lack of a better word, it builds or compiles the website into a cohesive package, and then Ron pushes that to each server, I think he is still doing it by rsynch, but I know we are migrating over to Puppet, so it may happen via Puppet soon. The production side of things, is something that’s handled completely by operations, so I couldn’t tell you specifically how it happens, but generally, we make a tag of the website, and tell Ron, we need you to push "9.4.15" or something like that, and he does a checkout, builds it, and pushes it to all of the different servers.

Paul: So is that – do you actually have to take the site offline to do those updates? How do you minimize the downtime that’s involved with that.

Joe: Oh, well there’s a bunch of different ways. Um, we don’t bring the website down normally for pushes, it depends on the size and complexity of the push. But like, day to day pushes, we probably push I guess, a minimum of once a day, just little bug fixes and stuff like that. And those happen generally in the middle of the day, and nobody notices, it’s no big deal. Ah, the outages these days, are completely dependant on database activity, you know, like database fixes and stuff like that. And the ways that we’re getting around that these days, is using a different method of partitioning called vertical partitioning. Where, like for instance, like I think our comment Diggs table, of like, who’s dug a comment, up or down, that’s like 15 billion records in it.

Paul: Wow

Joe: that’s like, yeah, if you’re like to alter that table, you’d probably crash mysql, but if you were, it would probably take a week to alter it. So instead we probably create another table, where we have like comments, and then we have another one called comments_made_up, or something like comments_diggs, comments_diggs_beta, and that has a couple of extra fields in it. And so we’ll say, OK, we’re about to push the code, at the end. When we push the code, the first comment id that was added to the table was 15,000,000,001, so then you start at 15,000,000,000 and work my way back in the table. So we do some of that live as well. For the next push that we’re doing, we’re using a migration table, which will tell us how far along each record is, and which records we’ve actually migrated, and stuff like that. And then we’re going to use this package called "GearMan" which is a queuing system which we’ve had in production for a while. And we’re basically turning our servers into a giant BotNet, so we’ll back fill all those partitions quickly.

Paul: Wow, that kind of amount of data, it must create huge problems, even adding a new piece of functionality onto the site, to actually code it in a way that’s not going to have a momentous impact on the database. This must be something that’s always constantly on your mind I guess?

Joe: Yeah, I’ll tell you a really funny story that highlights that perfectly, we have these little green badges that are on the Digg button, and they indicate, that a friend of yours has dug that story. And when you hover it shows the last four friends to dig it or something. So that’s a pretty knurly query, against a very big table, and we’ve actually had to, what I would call "dial it down a bit", so that it only shows up on the stories that are 48 hours old, and it won’t show up if there are more than 500 diggs or something. So the features fairly usable, but it’s not like… Well before if someone went to the top of 365, it was basically crashing our servers. So we’ve been rewriting that, and we basically, the way that we’re rewriting it is: If you write something, we take that Digg and we drop it into each of your followers buckets. So we make a bucket for each story for each person. Any time one of their friends digs it, we drop that dig into their bucket, but the problem is, like Kevin Rose is followed by 40,000 people, so every time he digs something, I need to drop 40,000 things into 40,000 different buckets. And we did the math, and just to get that feature up and running in a vast sane manner, so that it will scale, so we can bring it back in full capacity so everyone can use it all the time. We need 1.25 GB of storage, and we need to be able to sustain 3000 writes per second in order to keep just that small feature online.

Paul: So that really kind of illustrates that a relatively small feature that someone comes up with, has massive ramifications from your point of view.

Joe: Yeah, this is something that has kind of been something that I always talk about. I mean even back when I was doing consulting, I’d kind of have clients come to me, and it would be: "Check out this awesome design", and I would be like "that designs awesome, but that little feature down there, that’s going to cost you know, $100,000 in servers, and 500 man hours. But it’s, like, well the designers think of sizes and shapes, and so Daniel always jokes around and says: "Well I can make it purple" if that will make it easier for you" you know, it’s like…

Paul: Laughs

Joe: Laughs – well that doesn’t make it easier!

Paul: Well, we’re going to get you and Daniel back on the show to talk about this whole design / developer relationships, so you can lace your side of it now, and get your side in first. Before he defends himself.

Joe: Sounds like a plan.

Paul: So are you at the point now where you’ve got an architecture that’s kind of infinitely scalable, or are you going to have to go back to the drawing board if Digg just goes even more, you know off the scale than it already is?

Joe: Yeah, well we’re actually at the drawing board right now.

Paul: Yeah?

Joe: Yeah, Ron, myself, and some of the other senior peeps, about 8 or 10 months ago, we started putting together… well we knew that we were going to start to have serious limitations, especially since we were going to be scaling internationally. You know there is a problem with latency. With you guys across the pond hitting the west coast and other things like that. So we want to be in multiple data centers. We want to be actively serving traffic from multiple data centers, so we’re are, well we need to horizontally partition, we need to make sure we can do that. And we need to live in two different data centers. We need to be able to survive one data center disappearing. So we spent basically a week in front of the white board, and we created this thing called IDDB, which is kind of an elastic storage engine, built on top of SQL, and memcachedb, that we’re going to be putting the first bits and pieces into production, probably over a month or so. And really, the whole partitioning thing isn’t the difficult thing to figure out. The difficult thing is basically spanning multiple data centers, and also we’ll have a couple hundred gigabytes of data, and I need to spray that across, you know, a couple dozen different servers, without bringing the site down. So we have a couple million – 3 or 4 million users, and I need to take all of their records, and all of their auxiliary records, here’s like your user record, and there’s also a bunch of cruft related to that. So I need to take all of that, and migrate it to different partitions. But I need to do that while the site’s still up and running, and I need to do that without breaking the site for you. So, that’s the more complex problem at this point.

Paul: I mean you talk about spreading across multiple data centers, and if one of those data centers goes down, the site does too, and whatever. How are you currently handling redundancy? How are you making sure the site stands up at the minute?

Joe: Right now, our only single point of failure at this point, is our actual data center, so if the data center falls off the planet, then we’ve got problems. But we’ve got a general architecture. We’ve got a couple of general balancers up front. And those two have, what we call a "heartbeat" between them, and if one of them falls off, the other instantly takes over traffic for it. And that balances traffic across, I couldn’t even tell you, dozens and dozens of web servers, and of course, the load balancer does help checks on those, so if any of those falls over, it will drop it out of the pool. Behind that, we have, I think, 4, I guess our masters are technically single points of failure, but we have multiple masters as well, and we have dozens of read slaves hanging off of them. I think right now it takes about 10 minutes to bring a new master into production if one fails. So, and then we have, to store our files, we have a thing called MogileFS, which is a distributed web dav storage engine of sorts, and we can loose multiple nodes on that, and not have any problem with that as well.

Paul: Yeah, so at the moment it’s a physical problem that you have, that if your data center gets hit by an earthquake or whatever, then you have problems. Please tell me it’s not in San Francisco?

Joe: It’s not in San Francisco.

Paul: Ha ha, yeah, you’re not actually going to say where it is are you?

Joe: No I can say we have one on the west coast, and we have one on the east coast.

Paul: Oh, well that’s at least something. Um, I mean so far we’ve concentrated a lot on scaling technology, but there’s kind of another side to this, as well, where you get something like Digg, that has grown incredibly rapidly, over a very short length of time, and that is, kind of scaling the team behind it. You know, I don’t know how many developers were working on Digg when you joined it, but there would certainly be a heck of a lot more now. And I’m quite interested in how you went about growing the team. And how you deal with that kind of rapid growth, and making sure everyone knows what they’re working on, and not overwriting others work, and all the complexity that goes along side of that. How have you dealt with that?

Joe: Sure, I guess, to put things into context a little bit, when I was hired, we had both Kurt Wilms and I, were both hired on the same day, and were respectively employees 18 and 19, and developers, I think there were 7 of us. So, now we’re at the low 20′s as far as developers, and we’re in the mid 80s, as far as total employees, and that’s been in a year and 9 months. So as far as scaling the teams go, some of the building blocks were well in place by the time I got here. Like, source repository, stuff like that. But I think the crucial things that we’ve done, since I’ve come on board, that were, we had some coding standards that were out there, but they weren’t really in force. And then we had, we didn’t really have any frameworks in place. I think one of the problems, you know, when Jay, our CEO, was asking, where do we find more senior developers, I told Jay, like that’s not the right question, the right question is like, how do we get the code, and how do we get the technology, in a position, where we don’t have to hire really senior people. So I think the keys to that are, we do have pretty strict coding standards, so we do enforce those rigorously, through continuous integeration environment, and code reviews. Every piece of code that gets pushed to production has to be reviewed. And that’s literally 4 or 5 coders, sitting in a room, going line by line through change sets, and stuff like that. And that sounds really time consuming, but without question, on every code review I’ve sat in on, we’ve found one show stopper bug. So, those have been crucial, in getting things put together. The other things we did as we grew, is we broke the team up into smaller teams, so we have a development team of 20 – 25 developers, but that’s broken up into teams of between 3 and 5 developers. This really helps in a couple of areas. 1, it enforces code ownership. So everybody has this problem. I code this, then I go and work on something completely different. And 6 months later I come back to this code and I’ve forgotten. I don’t remember any of that. Where as if you’re always working in the same area of the sites, not only do you remember a lot more, you’re a lot more familiar with that. But also, you feel a bit more of a sense of ownership over that. You’re not just this hired gun that’s come in and ploughs through this feature then moves on to something else. You have your own territory that you need to keep track of. You need to keep really nice and what-not. So we did that, and then we have a bunch of core frameworks, that we’ve built. We have a small application framework, we have an AJAX framework, and now, we have this data access layer that we’ve been building up called IDDB. So I think those are pretty crucial too. It’s difficult to find coders that are intimately aware of memcached, and race conditions that exist in memcached, and um, we have to be very selective about what fields we add indexes on in mySQL. We also have to be very selective about how we store that. Normalization flies totally out the window, if you’re a DBA guy. A lot of concepts, they are not bad developers, by any means, they do great AJAX work, they do great application level PHP work, but if you don’t have frameworks in place for them to not have to worry about the super-super complex stuff. It’s going to be really difficult to hire, and it’s going to be really difficult to, you know, get a lot of stuff running in parallel and stuff.

Paul: Wow.

Joe: Yeah, and then we also, another one of the things we’ve adopted, is the agile SCRUM. So we’re doing sprints, and we’re running those in parallel now across all the teams. So right now we have 4 major projects going on right now.

Paul: Ok. So you talk about a sense of ownership there, and the developers are split down into multiple certain areas. You know, does that not get boring, for the developer, having to work on the same bit of code long time, or do you rotate people?

Joe: Well, we don’t currently rotate people, the team structure’s been in place for about 4 or 5 months now. And you know, most of the work they get is fairly immediate, and we’re moving major projects like Facebook connect, so the "Tools and integration team", their doing facebook connect now, and after this, they will maybe work on a new version of the API and so on. It’s written out to wide swaths of the site, so that we have "Site Apps" which does like, all the different applications on the site. And then we have "Tools and Integration" where we have the external projection of Digg, then we have "Core Apps" which is like, search, R&D stuff like recommendation engine, and what not, and then we have "Core Infrastructure".

Paul: So it’s probably enough to be interesting?

Joe: Yeah, we have pretty broad teams, and also, when we put people on those teams, even if someone has an amazing core infrastructure background, but they say, look, like, one of our UI guys, used to be really heavy into core infrastructure stuff when he worked at Quest, and managed massive warehouses, but he says, like, "That’s not what gets me up in the morning any more". It’s like, "Javascript UI interfaces are". So we try to put people on the teams where, you know, where their passions lie. And that’s kind of another thing that people need to recognize. And that’s like, not all developers are driven by, or interested in the same things. We have some, what I would call "UI / Frontend" developers, where like, they could care less about PHP, but we have PHP guys who could care less about Javascript. So I think, recognizing strengths and weaknesses, and capitalizing on those, is pretty important too.

Paul: OK, one last question to finish off, and that is, well you know, the kind of things that you’ve been talking about are fascinating to hear, about the kind of challenges that you have to face with the size of Digg, and the amount of traffic you have to handle. But obviously a lot of people who are listening to this podcast, aren’t at that stage. They are not working on massive projects like that. So I’m really interested in what advice you would have, for those who are just beginning to suffer from scalability problems, and they feel that it’s coming, and it’s something they need to be paying attention to. But it’s not on the enormous scale that you have to deal with. What things can they do right now to prevent problems down the line?

Joe: Um, I think, the easiest things you can do, is you need to re-think the LAMP acronym, because that stack is actually no longer really that stack. I would take Linux, and I’d take Apache out of that stack, and it doesn’t matter, as long as you’re running on a Unix. And as far as Apache goes, Lighty and EngineX are much better at getting a lot more money out of your box, as far as scalability goes. The two areas, that I think people, they sound hard, but they are really easy. One of them is installing and using Memcached, and the other one is installing and using a queuing system of some sort. And I think, like, recently I went through this with a little side project, called "Please Dress Me" which AJ and Gary Benashack and I did.

Paul: Oh, yes yes.

Joe: And we had a very small virtual server at MediaTemple, that survived pretty crushing blows from TechCrunch, Digg, BoingBoing, with almost no load. And that was like, beforehand, memcached is so second nature to me at this point, that I was just like, "Oh, well I’m just going to cache everything in memcached", and it was literally just like this RAM spewing machine. It didn’t actually hit the database. Actually my sysadmin at MediaTemple was like "Something’s really weard, MySQL is only doing like 1 query a second, and you’re doing like 500 requests per second from BoingBoing. So I’m cached. Yeah memcached is just like, it takes literally 10 minutes to install, and probably another hour or two to implement.

Paul: Wow, that sounds excellent, that’s really good advice. Joe, thank you so much for coming on the show, and I can’t wait to get you and Daniel fighting with one an other in the not too distant future. I’m hoping to get a good violent argument about designers and developers, just because I can.

Joe: Laughs.

Paul: I was a little bit disappointed when you guys sat down at Future of Web Design, were far too nice to one another, compared to the evening before, when you’d had a bit to drink, and you were talking in the restaurant. That’s the kind of conversation I want, that real vicious session.

Joe: OK, I’ll make sure that Daniel and I get liquored up before coming on then.

Paul: Yeah, that’s the answer. Ok, thank you so much Joe, that’s really good advice, and we’ll talk to you soon.

Joe: Thanks Paul, bye.

Thanks goes to Nathan O’Hanlon for transcribing this interview.

Back to top

Listeners feedback:

Top web designer applications

Often this section of the show consists of questions for myself and Marcus. However for a change, I thought we should ask the questions. Via the forum, the boagworld site and twitter I recently asked you to vote for your ‘favourite web designer application‘. The response was overwhelming and you can see the complete list of suggestions on UserVoice. However, here are the top 5…

  1. Firebug – Firebug is a Firefox addon that puts a wealth of development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page. A less popular suggestion was the Web Inspector in Safari.
  2. Web developer toolbar – The Web Developers toolbar is a Firefox addon that provides a variety of web development tools. You can disable CSS and Javascript, visually highlight elements, manage cookies and much more. A less popular alternative was the IE developers toolbar.
  3. Adobe Photoshop – A professional image-editing and graphics creation software from Adobe. It provides a large library of effects, filters and layers. This is the grandfather of such applications and many (like myself) use it out of habit more than anything else. Less popular suggestions included Fireworks, Illustrator and Inkscape.
  4. Coda – Coda is a one window development environment for the mac. It includes FTP, text editor, browser preview and even terminal window. A beautifully designed app it appeals to the more visual web designer. Simple, easy to use and elegant.
  5. TextMate – TextMate is a powerful text editor for the mac with an extensive plug-in architecture. From its code highlighting in near endless languages to support for numerous version control systems, TextMate is probably the most powerful text editor out there.

If you disagree with the Boagworld Listeners top five or want to see the other entries then head on over to UserVoice and vote for yourself.

Sending out bulk emails

My second listener contribution comes from the forum. It is a question from Richard about sending bulk email.

Richard writes: I need to send out bulk emails to approx 200k registered (opted in) users on a monthly basis.

Does anyone have any recommendations for an outsourced bulk email provider?

As with the previous contribution I want to focus on your responses rather than my own. This is what the Boagworld community had to say…

Jamie was the first of a number of people to recommend Campaign Monitor. Judging by the feedback from the forum they offer an excellent service but are expensive when compared to others.

As well as recommending Campaign Monitor Nick mentions Silverpop, which he described as ‘an enterprise affair’. Apparently it is not as polished as campaign monitor but considerably more powerful.

Phil recommended two more, Mail Chimp and Mad Mini. He hasn’t used them personally but the prices look good and he says the user interfaces appear polished.

Doug doesn’t recommend a specific service but does refer Richard to a post on Creative Tips which provides a comprehensive review of Campaign Monitor, MailChimp, AWeber, and Constant Contact.

If you have suggestions for Richard or would just like to share your experiences of using bulk email services then contribute to the thread in the boagworld forum.

Back to top

128. Details

Posted in Classic shows on by Paul Boag

On this weeks show I’m accompanied by our Producer Ryan and Researcher Stanton. We Interview Dan Rubin on the Details of Design, and answer your questions on managing a bigger team and terms and conditions.

Play

Podcast: Download (23.0MB)

Download this show.

Launch our podcast player

News and events

Silverback Launches

This week has seen the release of Silverback, the highly anticipated app from the guys at Clear:Left. After months of speculations about what Silverback actually was, the “spontaneous, unobtrusive, usability testing software for web designers” is finally available for download.

We’re sure a majority of you know all about Silverback, but for those of you who don’t, Silverback, which is available exclusively for the Mac, is Clear:Left’s answer to convenient usability testing on the go. Utilising the iSight and screen capture facilities of the Mac, user’s experiences can be recorded and reviewed at a later date, taking away the costly and often difficult to setup up approach of using specialist equipment like multiple camcorders which can lead to hours of time spent trawling through video footage.

PatternTap

Whether you’re a designer or developer, there are many occasions where you go on the hunt for inspiration in interface design. Normal CSS Gallery sites give you great examples of full site design, but usually don’t focus on the small details of interface design. The only site i’ve ever been aware of is Christian Watsons “Elements of Design“, which is a great resource showing examples of elements like comment forms, calendars & date pickers, footers, image captions and so on.

There’s a new site I’ve come across this week called PatternTap.com which also wants to collect these design patterns and focus on specific elements of design and to help you to reference, collect and organise them for your own needs.

PatternTap is shaping up to be an absolute goldmine of inspiration, and looks like it will build into a large resource of design element exmples. There’s currently 46 collections, everything from 404 pages, audio players, pagination and search boxes. It let’s you create your own “lightbox” style user sets, so you can keep your favourite examples organised for future reference.

I’ll definitely be adding this to my toolbox of design inspiration links, and recommend you give it a look too.

Google App Engine Update

This week also sees the release of a small update to the Google Apps Engine. The Google Apps Engine allows developers to build applications on Googles own infrastructure. I have to admit that the Google Apps Engine is not something I’ve developed with personally however that doesn’t stop us talking about it so let’s run through the list:

  • Firstly you can now have up to 10 apps on your account as opposed to the previous limit of three 3, the Engine also limits developers to 1000 files per application, so the increase in the number of apps you can now have is a welcome addition.
  • Time windows for Dashboard graphs: Zoom in on the data in your dashboard to get a more accurate picture of whats going on. You can zoom in to see graphs for the last 24, 12, and 6 hour periods.
  • Log files can now be downloaded in plain text.
  • And finally you can send email as the logged in user: If you’re using the users API, you can now send email from the email address of the currently-logged-in user were as before it was only possible from the administrators account.

S3

So some of you may be aware that Amazon’s S3 service suffered from some 6 hours of downtime recently, this echoes the issues of service availability that happened back in February.

For those of you who don’t know, the S3, or “Simple Storage Service” is a scalable and inexpensive data storage infrastructure, which allows you to store and retrieve any amount of data.

So this is a fantastic idea – in theory, it means that if you’re developing a large website or web app and need lots of storage, you don’t have to pay for huge webhosting plans with lots of physical diskspace, you store your assets “in the cloud” as it were, and you’re charged based on how much storage space you, and how much bandwidth you consume.

Lots of large sites rely on the S3 service for their storage needs, Twitter, BaseCamp and SlideShare to name but 3 and the recent downtime has raised the age old issue, “are we putting all our eggs in one basket?” Jonathan Boutelle put it best in a recent blog post, stating “When S3 goes down, the internet goes down”. Aral Balkan also wrote recently urging people to have contingency plans in case events like this happen again, stating that the Open Source Google App Engine SDK could be the answer.

Back to top

Interview: Dan Rubin The Details Of Design

Paul:Joining me today is Dan Rubin who I recently saw at the @media conference. Good to see you or speak to you again Dan should I say?

Dan Rubin:Good to speak to you Paul.

Paul:It was good to meet up with you at @media. It feels like a long time since we met up and it was great to hear you speaking there. That was a first for me.

Dan Rubin:Thanks. It was a privilege to be able to help out Patrick it being very last-minute.

Paul:Oh was it?

Dan Rubin:He sent me an email about two weeks prior saying someone had dropped out and of course I wasn’t going to say no.

Paul: laughs

Dan Rubin:It’s been over 10 years since my last trip to the UK, so it was a great opportunity.

Paul:Cool. Well I have to say considering you only had two weeks to put together the presentation, it was truly phenomenal. It was an excellent presentation and I really enjoyed it. You were talking about ‘design is in the detail’ I guess was the kind of subject you were tackling?

Dan Rubin: I’ve been talking a lot lately about the level of detail, the attention to detail and the design and I’ve done a couple of presentations with Brian Veloso over the last year on that same kind of topic. This was an extension of that injecting some of my own little personal preferences into the talk and got to cover things like typography and some of the simple practical things that you can improve very easily that result in a big improvement and typography, and little tricks in using grids, not on how to make them but how to actually implement them and how they can help workflow and bring things together and make layouts tighter and better without
that much effort and the same thing with digital transformations in photography and a lot of pixel detail that a lot of people don’t notice and its all about the subtle level of design.

Paul:I got this vague feeling that as you were talking you were a little bit appologetic for some of these manushi that kind of individually you sit there and go ‘how is anyone going to notice that?’, but accumulatively they have this effect on the design don’t they?

Dan Rubin:Well that’s the thing. It comes down more to feeling than seeing but its about as a designer what you feel with your eyes more than anything else and how that translates to what users or viewers or readers also feel but since they don’t know it is there, they are likely to never actually see it, but as a designer you’ll know it is there, you can see it, and the trick is to get it to the point of you can still see it but it is not really visible it is just felt.

Paul:A subconscious expression?

Dan Rubin:Yes.

Paul:You covered loads of tips in your presentation and there was some excellent stuff in there but if you had to pick out one that has the biggest impact on a design, which of the many things you talked about would that be?

Dan Rubin:I think what it would be is to really underscore trusting your eyes and it seems a really simple concept and whenever I put that up on the screen you get giggles from the audience. The truth is many of us don’t actually take the time as designers to just step away and look at what we’re working on. It doesn’t matter whether it is for screen or print. The medium is a material at this point and it is just having faith in what you see and what you feel. That’s what being a visual creative is all about. It is trusting what you see. It is the same as being a good musician comes down to trusting what you hear and sometimes we forget that, and we start getting into designing based on the rules or how we think we are supposed to do things or designing on technical limitations alone. When we do that we stop using our eyes.

Paul:It’s interesting in the presentation you talk quite a lot about some of the details and the mechanics of design. You were talking about font sizes going incrementally up, your heading and your sub headings and there being a mathematical relationship in their sizes. You talked about being consistent in your margins and padding and how all those things inter-relate. Are we saying that design is something that can be learnt and it is a mathematical thing and it’s a set of rules that you just adhere to? Or is there some sort of underlying artistic thing, some people just know how to do it and it’s not something that can be learnt. What’s your opinion on it because I get mixed feelings from you? On one hand you talk about these rules and on the other hand you talk about stepping back and looking at your design and it feels more kind of arty-farty if that makes sense!

Dan Rubin:What a load of questions and rightfully so! It’s something I’ve written about before years ago and had a bit of back and forth on the topic with Paul Scrivvens of 9 Rules, with him arguing that you don’t need any natural artistic ability because he didn’t think he had any, yet he was clearly doing things that looked good. I was arguing the opposite but when it comes down to it it’s really not something that you can say definitively either way. Just as there are people who naturally seem to be good musicians or good athletes or good at math and programming, there are people who seem to naturally be good at design and any kind of creative endeavours. It is really difficult to tell whether that seeming innate ability has come from something that happened in very early childhood development or if they were born with it. I do think that however difficult it is to put a finger on it, once you get old enough, especially to the point w here probably most of your listeners are doing what your doing for a living already or you are thinking of changing from one thing to another, you’re past that point of subconscious development where you need to put conscious effort into something and you can. I think you can be trained to do most of the things designers do. You can even train yourself to see the way that creatives see. The older you get the harder it becomes to incorporate into the way you view the world. That is a big part of it. That comes down to sometimes the different personalities. How hard is it to put a finger on what makes you ‘you’. I would say as a teacher, and I spend a lot of time teaching high school students over here about music as well, since that’s my other passion, and it’s specifically not just playing music but it’s specifically singing which is one of those things that you can either carry a tune or you can’t. I’ve also seen kids who can’t carry a tune when they start singing learn how they train themselves. They learn the proper muscle memory, and it’s amazing to see what people can actually accomplish when they put their mind to it. If you are listening out there and you want to become a better designer or maybe you’re not a designer and you’re a programmer or a web standards junkie, and I can say that because I am one too, and there isn’t any reason that you can’t become a better designer, or become a designer from scratch if you realy really want to.

Paul:I think that’s really important to say because I think so many people are intimidated from getting involved in design because there’s almost a bit of snobbery. If you’re not artistic, you’re not artistic there’s nothing you can do about that. I personnaly don’t believe that that’s true. Like you say I think there are some people that are naturally inclined that way but I think a lot of the principles that you were talking about in your presentation pretty much anybody can pick up on and do, which is what encouraged me so much hearing you talk.

Dan Rubin:That is one of the reasons why one of the reasons I say one of the most important thing is to trust your eyes and that’s instinctual. These rules, as a good teacher you have to teach these rules. When you start learning any discipline the first things that you are taught are the basics.The basics are things that many people, once they learn enough, don’t conciously think about, but what you find if you deconstruct their work is that they are doing them, they have incorporated into their flow into their process so it’s second nature to them. What we think of as instinct is really just experience.

Paul:Yeah. One of the things you did mention in the presentation that grabbed my attention is you talked a lot about texture and adding more texture to your design and about how that creates a real feel. There seems to be a slight skism, I don’t know if that is the right word, but like 2 different camps in design at the moment. People like yourself, Elliot Jay Stock is another example that does very rich, very textured design. It’s absolutely gorgeous. At the other end of the extreme you’ve got people like 37signals doing this minimalistic functional design. How do you feel those two sides fit togeth
er? Is there a role for one or the other or have they both got their place

Dan Rubin:I really think that both have their place and more than that it’s popular to create divisions. Not just these days, if you look at any industry that spends a lot of its time looking at itself, like we do, you start to find reasons to create little clicks within it or factions or what have you. If you just ignore those splits that happen because we spend way too much time looking at what we do and try to deconstruct it and answer that question of ‘why’. What you find is that it’s all the same thing. When I talk about texture it is important to understand that it doesn’t just mean rough or ??bulap or brick. Texture can also mean smooth and polished and speaking directly about 37signals for instance. I’ve used their apps and I’ve loved them since the first time they came out. If you look at the first versions of Base Camp and Backpack, before their incremental re-design they’ve actually added the little drop shadow over time. If y ou look at it as a designer you see the flaws in the way they’ve done it because it doesn’t look real and it just ends at some edges, it has hard edges, but that’s not the point. The point is they added it because it created a separation, they added it because they felt it needed it. The rest of the interface doesn’t need any other texture because it isn’t supposed to have a feel to it. It’s actually supposed to totally get out of the way and there are different approaches to minimalism. You can use minimalism in subtle detail where you add in things like I was showing in my presentation, or you can use minimalism where you keep taking away and 37signals apps feel right, they always have felt right to me so as far as I’m concerned that means they’ve hit the nail on the head. It shows when you see people trying to recreate the application interface and theat style that 37signals uses and they get stuck in this pattern of adding things, like they feel ‘well, that’s 37siganls l ook so I think we have to add things to make it better, to make it better, and they never work as well because it’s not just about that. So the answer is, and I try to underscore this when I talk to people about this or present about it or even write about it, as much as these things can be presented as rules and definitive this is the way to do something. the fact is you have to do what works best for you and your particular project or circumstance or situation, and you also have to be open to the fact that what works for you right now might change. It might be different next year, next month or next week, and being able to adapt to your situation as a designer specially is really important, because you have to adapt if you’re doing client work, you have to adapt from project to project, because your style might work for one client but you might need to tweek your style to do what’s best for another client. If your working on your own applications, what works for your users now might not work for your users once they become users that have used your app for a year and they’re experts now.

Paul:You talk about tweaking your style. How easy is that, do you think, to do in reality? I mean I’ve got a very strong style in my design, and I really struggle and I look at someone like Cameron Moll’s style and I just love it. I love the light-handed feel, he’s very delicate, beautiful design, and I wish I was more like that, but there is no way I can make myself become like that, or can I? Is there a way of changing your style?

Dan Rubin:I think we’re all naturally mimics. I’m not going to dig into my opinions on human adapability too much. I spend a lot of time thinking about that as far as evaluating how people use things, whether it’s interfaces or products and it’s interesting to start to see those patterns but you can see it on a global scale too. Historically human beings are species very, very adaptable and that happens on macro and micro levels. If you want to adapt your style you can. You look for the inflences you want to model yourself after. This is just how people learn to be designers when they’re starting out, or learn to be artists. When I took my first watercolour and oil painting classes when I was 11 or 12, the way we learnt was to recreate examples that were painted by masters. So learn how to use the brush strokes they use, to learn how to mix colours the way that they use them, to learn how to use the tools the way that they use them becau se you only discover your preferences and your style by mimicing, copying others. You find out what works and you decide what works for you and what doesn’t. So changing how you design and how you see is not necessarily easy, because at a certain point you’re reprogramming muscle memory and from my experience with singing I know how difficult that is to do. Once muscle memory has been built up to the point where you don’t think about it and you just react, it’s very difficult to break that down and re-build it. Difficult does not mean impossible.

Paul:That’s really interesting that you say that because I’ve always very much struggled to design in any other way than I already do, but I obviously need to push myself in this area. Talking of 37signals, I’m sure you have been following their recent post and various reactions to it about skipping Photoshop, and how they move straight into building with HTML and CSS and I just wondered what your opinion was on that.

Dan Rubin:I know I’d get roped into this discussion somehow. There has already been some great responses from people like Jeff Croft and Mark Boulten to the 37signals post on that, and even interestingly enough a follow-up post sourced by 37signals announcing that they were looking for an additional designer for their team that can push them into different directions that they havent been going naturally. That comes back to the whole adaptability and willing us to change and being open to it. In the argument itself I can’t say I always start in Photoshop or Fireworks or some sort of visual tool. I think Jeff said 37signals starts with a visual tool, it’s pencil and paper. I think even if your tool is a marker on a whiteboard to a certain extent everybody tends to start there, even if you don’t start there you start with a picture in your mind. So there’s some level in the process where a visualisation is occuring, if that’s fair to say. When it comes down to it why does the tool that you’re using to visualise really matter? It starts in your head if you’re a primarily visual person you can either realise that vision by programming it and seeing it in the browser or using Photoshop as a tool. All of these are just tools when it comes down to it, they’re not the end result. They’re just part of the process. I’ve done both. I’ve built straight from XHTML and CSS many times and I do tend to find that most visual designers that have weighed in on this conversation also find that in my opinion the result ends up being more simplistic. that’s not necessarily to say bad. It’s just different and you’ll find that the tools that you use as a visual creative influsence the end result because that comes down to constraints. 37signals of course is huge on constraints and you do save time when you’re doing straight HTML and CSS, you skip a lot of the temptation to play around like I know I do with layers and layer setting s and percentages of opacity. I spend a lot of time playing when I’m in Photoshop, I don’t think that’s bad. That’s part of the creative process when using that tool. When I used to paint which I havent done in way too long. I would play with my
palatte, when I was doing oils my palatte and my palatte knife was tool before I got to the canvas, and I would play with mixing my colours ‘and that’s not quite right’ and ‘wait and go over here’ and sometimes you get it onto the canvas and it doesn’t look the way you want it to and have to wait for it to dry and then you paint over it because that’s what you do with that tool. When you’re doing watercolours you don’t have that forgiveness of the tool, you have extra constraints, so you don’t experiment as much putting it on the paper, putting the paint to paper because you know once it’s dried and there you can’t go back. you can’t paint over it. So you adjust your style depending on the tools and the workflow and it’s all good, it ‘s just all different and you have to I think do yourself a favour and experiment to find which works best for you and don’t be afraid if you’re working on a project and you think ‘this doesn’t feel like it needs a lot of subtle gradients and lines and shadows and Photoshop work. I might just be able to build this without using Photoshop at all’. So do it if it feels like that will work best go that route. If you feel the opposite go the other route. If you feel like it should involve a lot more natural media pull out your watercolour pad and paint something and scan it in and incorporate that

Paul:It really down to the right tool for the job thought process.

Dan Rubin:Exactly. The thing that 37signals does really well is stick to their guns. They state their opinion so firmly that people can easily interpret it as law and I think that’s very important. In any industry it’s very important to have people who do that, who can stick to what they believe so strongly and apply it so universally that it creates this set of rules, but it doesn’t mean that they have to be followed or cant be partially followed or bent or broken and you find just as much as 37signals is enfatic about skipping Photoshop. There are other people who would never in a million years go straight to HTML and CSS, doesn’t mean that either camp is right.

Paul:OK. One last question just to wrap this up. We’re running out of time but there’s something I wanted to ask you which is: We’ve been already talking about that there are people that may be want to learn to be better designers, to find their style and to move into this area, perhaps they’ve been a developer background and they’ve been previously put off exploring design because they have been made to feel inadequate. What kind of resources would you encourage people to look for or look at in order to get going I guess?

Dan Rubin:Whether you’re starting from scratch or just trying to improve what you already have it’s important to touch on a couple of specific areas, and those are typography, layout and working with colour. This applies just to design because it’s worked whether you’re designing on the web or designing in print or branding or whatever you’re doing. Typography is kind of my first love with design and if you want to learn about typography you have to go out and buy ‘The Elements of Typographic Style’ by Robert Bringhurst. It’s the bible for typographers. It’s really easy to read too because he’s a well respected Canadian poet as well. He just happens to be an excellent typographer and book designer, so if you are in a rush, you cant get to the book store or Amazon right away Mark Boulton’s series ‘Five Simple Steps To Better Typography’ is a great place to start as well and he references a ton of other good resources. Start there if you a re going to start online but no matter what buy ‘The Elements of Typographic Style’. When it comes to layout there are a lot of things that you can learn about layout but you’ve got to learn about grids, even if you never use them. Do yourself a favour of learning and I’ll reference Mark again, actually I’ll reference Mark in all three of these. He’s got great starter tutorials about this stuff so ‘Five Simple Steps To Designing Grid Systems’ is really a great place to start. Cameron Moll has written about Griding The 960 and read up over on Khoi Vinh’s site about grids. ‘Grids Are Good’ is a great demonstration as well, and if you want to get a physical book to hold ‘Grid Systems In Graphic Design’ is a great, great phyisical book and I think it’s important to as web designers to also reference ‘Print’, because Print is where all these design rules come from and typography rules and colour rules, so learn from these different implem entations and you’ll figure out things that you can do that you didn’t think about, because you haven’t seen them on the web. So ‘Grid Systems In Graphic Design’ is by Josef Müller Brockmann I believe would be the pronounciation, look that up. Colour, and this is something that’s very preferential maybe but read up again Mark Boulton’s ‘Five Simple Steps To Designing With Colour’. He’s great at teaching, he’s great at communicating all these things. Also play around with some of the online tools like Adobe Kuler, is fun. Look at what other people are putting together, look at combinations, again feel is important. Whatever feels right for what you’re trying to do. Another cool tool is Colorjack. You got a couple of ways of mixing colours and it’s really, really cool to look at. Finally on the topic of colour whenever using colours in an interface please be aware of the different types of colourbl indness that exist, and there are lots of tools online. Photoshop CS4 will have some tools built in as well but there are plug-ins that you can get right now for all sorts of tools and online tools as well that allow you to see what you’re designing, or even just a colour palatte. See them through the eyes of someone that has these various colourblindness afflictions and make sure that whatever you do doesn’t render something unuseable to what ends up being a large percentage of the viewing public when it comes down to it.

Paul:WOW !! That’s a good set of resources !! My word.

Dan Rubin:You didn’t think I’d be that prepared did you?

Paul:That’s a superb list. I certainly didn’t know about all those posts from Mark Boulton. there was some great stuff in there – Thank you very much Dan. Just to say that Dan’s talk at @media will be no doubt going live at some point and you’ll be able to download it and listen to it. Definitely do that, it was superb. So check that out. You will be able to go the shownotes for this episode for all those links that will be useful as well. No doubt you won’t be able to remember them all. Dan thanks for coming on the show, it’s very much appreciated and we will get you back on in the future.

Dan Rubin:Thanks very much for having me Paul. It was a pleasure.

Thanks to Sarah Galley for transcribing this interview.

Linkage

You can find Dan Rubins site, Superfluous Banter here.

Typography
Layout
Colour

Back to top

Listeners feedback:

Managing a Bigger Team

Jon asks: We are a company of 4 people – myself (owner, design lead and general business development/project management person), one designer, and 2 developers.

We are hopefully about to merge with a slightly larger company in a neighbouring town who have slightly more staff than we do (7 in all), and who have more of a project management structure – 2 project managers, using the services of 1 designer, 3 developers, and 1 designer/developer. I would end up as owner/MD of the enlarged company.

My question is really about project management? What do you think is the best organizational structure for a company of 11 people? I was feeling pushed on the project management side before this merger came along, and the merger will bring 2 project managers with it. How does Headscape do it for example – I think you have project managers there – do the designers and developers report to project managers, or do the project managers pick from a pool of design and development resource as required? What are your thoughts generally on the whole project management side of things.

A-ha… this is part two to a question I answered a few weeks back relating to pricing work after two companies merge. I wanted more detail at the time and now I have it!

Comparing to Headscape, we have 4 designers, 4 developers, 3 project managers, 2 business development/analysts and 1 lazy good-for-nothing called Paul … seriously though, Paul effectively markets Headscape and I have to say he’s rather good at it (ungrits teeth…)

Following the merger Jon will have a team of 11. As he is new MD, I think it is imperative that he much reduces the design and PM aspects of his role and concentrates on bringing in business as there are quite a few more mouths to feed.

That leaves roughly 3 designers, 5 developers and 2 PMs. Depending on the work you’re doing I think that is ok especially considering Jon can bolster both the design and PM groups if needed.

Regarding the allocation of work, project managers should rule the roost. Full stop.

It is their job to manage resources. Delivering projects effectively and on time means that they must know that they are in charge regarding who does what and when they need to do it by. A certain amount of fitting the right person to the job should be done but generally, the rule should be that the next piece of work goes to the next available person. This would be particularly useful advice in a merged company where it would much easier to keep going back to ‘your’ guys because you trust them.

One thing that has worked really well for us is to set invoicing targets for the project managers. We don’t operate performance related targets but it still really helps to focus minds on hitting milestones at the end of months.

Terms and Conditions

Adam writes: I am developing my own web application. In summary, it’s a site with user submission of content in a social networking format with video uploads. Anyone can register an account.

I of course have to try and write Terms of Service for this and I am getting stuck. I am wondering what Headscape uses, especially for Getsignoff, and whether you found a pre-written terms of service, or had a specialist write one.

What’s your solution to the problem, and what should / should not be included.

I have to confess to conferring with Headscape’s fount of all legalese knowledge on this – our MD Chris Scott. I tried to get him on the show but he’s still a little jittery after the last time all those years ago… anyway, Chris put together the TOS for Getsignoff and these are his thoughts on it:

For Getsignoff I looked at the TOS of other online services like Harvest, Basecamp, Youtube and Flickr. I’m not a legal person, but this gave me enough material to be able to identify the key issues that I thought we needed to cover in our TOS.

I assembled this into a brief for our legal adviser that was part overview of what we wanted to achieve and part draft TOS using adapted clauses from other TOSs.

Our legal adviser pretty much re-wrote what I had given him but this was from a position where he had a good understanding of how we wanted Getsignoff to work.

The bottom line with this sort of thing is that you really need to get a professional legal person to assist.

Back to top