Testing two WordPress Gutenberg prototypes

I carried out three user testing sessions of two prototypes of the new editor for WordPress. Users were confident in their digital skills. Here are the results.

Key observations

  • The overall design approach is successful. The idea of thinking of content in blocks was easy to grasp, and the context-sensitive popup options were intelligible.
  • The function of the up and down arrows to the left of content blocks wasn’t immediately clear.
  • Users quickly worked out what the block-positioning arrows did, and were happy with this approach. To my surprise, no one tried to drag and drop blocks after finding out how to reposition blocks.
  • In 1.0 people weren’t sure what the S means – is it strikethrough or the option to remove a link? Perhaps we can use proximity to make this clearer – associating this options clearly with presentation controls (underline, italics) or functionality controls (hyperlinks).
  • Some users tried to drag and drop images. How will dragging and dropping work – within and between blocks?

Full notes of the testing sessions

First prototype

Tester 1

Prototype: “A beautiful thing about Apple…”

prototype of a new editor for wordpress

Clicks into a block and sees and instantly understands the alignment options.

Clicks on the ¶ symbol but it doesn’t do anything.
Expected it to be interactive as the mouse changed to a hand pointer, suggesting interactivity.

Clicks on the up arrow, but nothing happens. Wasn’t really sure what it would do.

Enters a line return to break up the paragraph of text, but isn’t able to do this.
Thought that the arrows up and down might allow the insertion of a line break.

Highlights text to make it bold and expects to have to use a keyboard shortcut as nothing was previously visible on the screen. But then notices the popup.
Comfortable with the options presented.

Heading and blockquote options don’t behave as expected. (Dismissed as a limitation of the prototype)

Clicks on the image. The up and down buttons don’t do anything.
Tries to click on the image icon but it doesn’t do anything.
“I don’t understand what the symbols are. I expected them to be interactive but they don’t do anything.”

Tester 2

Understands the split between code and visual editors.

Likes the context-specific editing as it keeps the screen more focused.

Tester 3

Understands the concept of HTML and editable preview.

Wonders how valuable constant HTML visibility is to the average content editor. Will they break things?

Second prototype

“1.0 Is The Loneliest Number”

An early prototype of the new wordpress gutenberg editor

Tester 1

Clicks on the heading text block.
Then clicks on the up arrow.
Then clicks down on it. “Ah, that’s interesting. So these up arrows change the order of the boxes”

User expected to be able to insert line breaks. (How do we want to handle line breaks within paragraph blocks? Do we permit them, meaning that we have big text blocks, or do we take line breaks as denoting the start of a new paragraph block? I’d prefer the latter but haven’t thought about all use cases.)

Clicks on the paragraph of text, and then clicks on the ¶ symbol.
Changes to blockquote style from paragraph style.
Understands that this allows him to apply style to the block.

Scrolls down the page and clicks on the plus sign at the bottom.

When asked what this is, says that it’s “a shortcut to functions that you use regularly.”
When asked: what would you expect to happen if you clicked on one of these items?
“It would create a new paragraph, image, heading, quote, etc, which you’d then populate with content.”

Tester 2

Clicks into the block and manipulates alignment options – working as expected.

Clicks down arrow and block responds as expected. Understands that the page is made up of blocks and that these can be repositioned.

Clicks on “+” and explains that this will add a new block. More options than expected: expected just image or text, so the extra options are “a nice surprise”.

Drags and drops an image into the content block. (Is this something we want to design for?)

Clicks on ¶ – and changes the text block type

Tester 3

Clicks into the text block.
“I wonder what the arrow does” Clicks on the down arrow and sees the block move down. Understands it fine.

Looks at alignment options. “Does that do the whole block? It does”

In general, understands the popup text formatting and link options.
Not sure what the popup “S” with strikethrough icon means.

Clicks on image and manipulates text flow options.

Clicks image icon, but “nothing here”. Expects caption controls.

Clicks on the “+” sign, and understands that this is for adding blocks. Wonders how lists will interact with paragraph blocks, and how we could set levels of header.

Easily understood the blocks concept. Wondered how well this would handle more complex page layouts.

10 tweet summary of NFP Tweetup 32 – Getting off to a good start

Here’s a 10 tweet summary of last night’s NFP Tweetup. I wasn’t actually able to attend the event, but that’s no barrier to summarising the best material from twitter 🙂

Innovation and failure – Luke Williams, RNLI

Set up organisation-spanning structures to keep an eye on trends:

Formal approval processes often take longer than tech implementation work:

How to decide whether to jump a trend:

  • Where is this trend in its lifecycle? Has it peaked? Which other organisations have taken part?
  • Can we relate it to our organisation?
  • What time/resources do we need to make it happen?
  • How do we make it authentic to the trend?
  • Does taking part present any risks?

RNLI are using chatfuel.com to build a Facebook bot.

Tips for getting buy-in for an idea:

Writing a digital brief – Jonty Sharples, Hactar

Headline structure for a digital brief:


Start with the user need and then think about the product. Not the other way round.

Digital Transformation approaches and language – Joe Freeman, Breast Cancer Now

Some resources on transformation recommended by Joe:

https://medium.com/@gadsby/a-digital-transformation-manifesto-1441faef7124#.h929p17e4

https://medium.com/@lucia_adams/my-love-hate-relationship-with-the-term-digital-transformation-db27bec0cde#.ss65flj3r

Eat Sleep Work Repeat podcast

Transformation but not as you know it…

Digital transformation is mostly about culture and ways of working, not technology

Tactical tip: fundraising events teams are usually very receptive to new ways of working to help them reach their goals:


(This has been my experience too.)

Two tools for measuring how well your organisation is doing digitally:

Digital Maturity Matrix

What I learnt by building a side project

A few years ago I completed Harvard’s CS50x – an online computer science course. My favourite part of the course was the self-directed final project. I’m interested in the question of how we can filter the mass of digital information that we’re confronted with every day, so that we can enjoy the best bits without spending all of our time scanning and processing. In my own life, I’d found that I had less time to keep an eye on Twitter. I really wanted a tool to keep an eye on Twitter activity while I was at work. So I explored this in my final project. I built a tool to monitor Twitter activity for a given list.

What I built was pretty basic – it was running on an old laptop in the corner of my flat, and it just displayed the results on the laptop screen after a specified number of hours had elapsed. A solid final project, but it only ever felt like a proof of concept. I wanted to build this into something I could use, and something that other people could use too.

So over the next few years I built it from the ground up as a full-blown web application that has now been launched as a real business: MySocialSummary.com.

The CS50 class was great, but once you have the foundations of understanding in place, there’s no substitute for the motivation, exploration, imagination and excitement of working on your own idea. Subject to having a reasonable grounding (and I’ve taken a number of computer science classes, not just CS50), I’d definitely recommend taking on a project that excites you, to take your knowledge to the next level.

Going through this process I’ve learnt a lot. I’ve divided this into two lists: one on specific computer science and business insights, the other on more overarching observations.

Specific computer science and business lessons

  • How to embed tweets in emails. (Or, rather, why you can’t embed tweets in emails.)
  • Working with APIs. An API is a tool for communicating between two pieces of software. Manipulation of data from the Twitter API is the foundation of My Social Summary.
  • How to load test your website. If you’ve built a website, you probably want people to visit. You commission your infrastructure with a certain number of visitors in mind. It’s worth testing how your site would behave if an unexpectedly large number of visitors arrive at once. This is called Load Testing. I used a tool called Load Impact to check out the load speed of My Social Summary when you add more users. As you can see, the homepage performs well with 50 visitors loading it at the same time.
    load test of mysocialsummary.com with 50 users
  • Working with code libraries. When you’re programming, you’ll often want to do something that other people have done before. For example, loading a web page, sending an email, or accessing the Twitter API. If you’re trying to solve a common problem, often there’ll be a code library of pre-written code that will help you with this. By using a code library, rather than producing this code from scratch, you can save a lot of effort, and the quality of the code is probably much higher quality than what you’d produce yourself. This frees you up to focus on the problems that no-one else has worked on yet. Not all code libraries are created equal – some are better documented and more fully-featured than others – so picking the right one is an important decision.
  • How to set up login via Twitter. A good example of when a code library can be useful. I’d have definitely struggled to build this authentication myself if I’d had to do so from scratch. Check out this documentation.
  • A diagram of part of the Twitter authentication process

  • How to securely handle user input. If you take any form of user input on your website that ends up interacting with the web server, it’s a security risk. So you need to build your site from the ground up with security in mind. I really enjoyed learning about how you can protect yourself using paramaterised database queries and other security best practices.
  • How to set up a LAMP server. That’s a common infrastructure stack, using Linux, Apache, MySQL and PHP to run a website. I’ve learnt how to manipulate the different parts of the machine (e.g. updating Apache settings) and use the different management tools.
  • The complexities of daylight saving time. When the clocks changed, I noticed that my daily summary emails were arriving an hour early. Fixing this problem for myself was trivial, but solving it for all different users – and possible future users – was harder. I’ve built functionality that checks every evening against the very latest timezone rules, and supports every timezone. (Check out the exhaustive dropdown list on the page where you can sign up for a free trial account)
  • The benefits of refactoring code. If code is a set of instructions, refactoring is rewriting those instructions to be more efficient. Refactoring your code isn’t fun – it’s hard work, and involves trawling through your past self’s logic and trying to improve upon it. But it can make massive performance improvements. In one case I was able to decrease the number of database interactions by a factor of a thousand through batch operations.
  • Are you too busy to improve your process?

  • The importance of good version control and deployment infrastructure. Any halfway competent programmer will tell you that good version control is really important. At the start of this project I was evidently not a halfway competent programmer. If you operate version control you can review the history of your code over time, and quickly track down errors. Your deployment infrastructure is the process from getting your code from a code file on your computer to your web server so that it can start doing real work. Ideally you want this to be as frictionless as possible. In the early days, I had no version control, and my deployment procedure was copying and pasting the contents of one text file into another. I can confirm that this was not a good process.
  • How to work with a front-end framework. Specifically Twitter Bootstrap. My interest in this project has been more in concepts and engineering than in presentation. But any website needs to have a front-end of HTML and CSS. Rather than building all of this from scratch, I used the Bootstrap library to help set up the structure and presentation of the site. This made it quite easy to make the site mobile responsive.
  • How to integrate with a payment provider. Taking money from people online has a few complexities. The main one is passing data from your payment provider to your web server, so that you know who’s paid what. This entailed using Paddle’s Webhooks. Handling all the different types of event that might happen – and doing so securely – was quite a bit of work, but it was very satisfying to get this set up and working correctly, and see money come in to my account.
  • How to use cron jobs. On a Linux machine, you can instruct the computer to automatically carry out tasks at certain times. These are called ‘cron jobs’, and they can be really powerful. I make extensive use of these for My Social Summary, for the management of user accounts, and the sending out of communications to new users.
  • How to satisfy the Twitter brand guidelines – and how to update things when they change…. Before coming up with a name, I carefully checked Twitter’s brand guidelines to make sure that the name was consistent. I felt pretty smug about having thought to do this. However, Twitter changed the goalposts, and later the name I’d chosen was no longer acceptable. So I had to come up with a new name for the service – and then roll out this change to all aspects of the service.
  • How to set up HTTPS security. Privacy and security are important, so I wanted to make sure that the site I was building used HTTPS encryption. Setting this up was quite easy – much of the process was done by my web hosting provider who installed the certificate. Seeing the little padlock next to the domain name for the first time was very satisfying.
  • How to set up email authentication via DKIM. I had no idea what this was. But one day I saw a little question mark when I opened up one of my summary emails, next to the sender name. Gmail said that it wasn’t sure if the sender of the email was actually who they claimed to be. Google had instructions on what to do next if you were the person sending this email, and I didn’t even need to follow all of these – pretty much all I had to do was change a setting with my hosting provider and this was resolved.
  • Example of an image without DKIM

  • How to migrate hosting provider. Part-way in to the project, it became clear that I’d outgrown the infrastructure I’d started on and needed a more robust web server. The process of transitioning from one provider to another is something I’ve overseen before in my day job, but to do all the work myself was useful. Before I started the process, I had to choose a provider with the right infrastructure and an SLA to match my needs. (I chose TSOHost and I’m happy with them.) Then I had to commission a new database within slightly different constraints, migrate the data over, and switch the traffic over to the new site. None of this was massively difficult, but the work took about a day nonetheless.
  • How to register a business in the UK.
  • How to track business finances. Fortunately the costs of a digital business are easy to track, so a simple Google spreadsheet suffices.
  • How to complete the year-end tax process. I haven’t had to do this yet – this is likely to be the real learning curve.
  • How to comply with the EU’s new rules on VAT for digital services. The EU changed its laws for how VAT is charged on digital services. Rather than being charged at the rate of the country of the person selling the service, VAT has to be charged at the rate in the country of the person buying the product. You have to collect evidence to prove this, and you can either make VAT payments to each EU country separately, according to its own processes, or you can submit centralised returns through the UK VAT MOSS service. Both of these processes seemed tedious and onerous, so I opted instead to use a payment provider that would handle VAT on my behalf. This meant switching from Stripe or PayPal (I’d been testing out both) to Paddle. Here’s a blog post on Paddle’s site discussing the changes.
  • How to set up a Content Delivery Network. Your website lives on a web server – a computer probably sitting in a datacentre somewhere. If someone the other side of the world wants to see one of your web pages, the information has to travel all the way to your web server and back. It would be quicker if they were requesting the web page from somewhere closer to home. A CDN is a network of web servers around the world that does this task. A CDN can also help take some of the load off your web server – if people are getting pages from the CDN rather than your web server, the web server doesn’t have to work as hard. I use the Cloudflare CDN. It has a free tier and is easy to set up. It disrupted the site’s HTTPS setup for a few hours while I turned it on, but other than setting up the CDN was easy. Cloudflare CDN usage graph example

Broader lessons

  • Starting with a Minimum Viable Product is the right way to build a digital product. I started off by building the simplest possible tool to meet my needs, and then I successively improved it from there until I had a product that was ready to launch. Had I tried to build everything in one go, I wouldn’t have known how to do it, or what I was trying to build. And I wouldn’t have had the motivation and insights provided by being able to use the tool myself from early on in the process. It would not have even been clear if what I was building was going to useful to anyone until after it had been launched. The Lean Startup methodology, and the Agile Manifesto, tell us that you should start off by building a Minimum Viable Product – the most basic version of your product that is still valuable. I already had an intellectual appreciation of this, but it’s always useful to test theories out in practice.
  • How to build a minimum viable product diagram

  • The power of self-directed learning. This project has been entirely self-directed. This has been thrilling – a chance to follow my interests and try and constantly reassess what I need to do – or learn – next. You learn a lot more, and a lot more broadly, if you’re learning to achieve a personal goal.
  • The power of emergent possibilities and understanding. When I started this project, I didn’t know everything upfront. I didn’t even know what I would need to know. The Agile approach to projects – release early, and iterate based on evidence of real user behavior – is thoroughly vindicated by this. Working on this project has strengthened my understanding of, and commitment to, the Agile and Lean principles of releasing early and improving your product from there.
  • The importance of knowing how you work – and what you need to work. Working on a side project is different to working your day job. I’ve become more sensitive to when I’m in the right headspace to do some work on My Social Summary, and when I just need to take it easy. I also have a better understanding about the conditions I need to get work done outside of my normal job. I tend to work quite well in uninterrupted stretches of at least an hour and a half – especially if I’m programming.
  • The importance of dealing with changing circumstances. The external environment changes. During this project, VAT legislation changed and Twitter’s naming rules changed. You need an approach to building a product that is not only resilient to this, but embraces it. Again, if I hadn’t been following an Agile approach, I’d have been in trouble.
  • To trust in your ability to keep learning and know more than you know now. Taking on a self-directed project builds self-confidence and self-reliance.

I’d love for you to check out what I’ve been working on. My Social Summary has a free one month trial, with no card details needed, so see if it can help you get more from Twitter.

What’s next for me?

  • Spread the word about My Social Summary.
  • Explore how we might overcome the ‘filter bubble’ that seems to be narrowing our political and cultural discourse. This feels like an important design challenge.
  • From a computer science perspective, I’d like to build something in a different, and more modern, technology stack. Probably Node.js / Foundation (for front-end) / PostgreSQL / Heroku.

Paul Boag: Digital Adaptation – summary

Paul Boag outlines how to adapt your organisation to meet the challenges presented by digital, in the excellent Digital Adaptation. Further resources are available on the Digital Adaptation website.

User expectations are set by industry leaders

“You may not be in direct competition with Twitter, Google, or even MailChimp, but users will expect the same ease of use that they have been given on those platforms.”
“You are not just up against your competitors, you are competing with the user experience of every digital player out there.”

If you don’t change to meet evolving user expectations, a competitor will appear and render you irrelevant

Napster changed users’ expectations of how they could consume music. People now expected cheap, fast (digital and on-demand) music, focused on tracks not albums.

“The music industry fought hard to have Napster closed down, but the damage was done. User expectations had changed and there was no going back… Instead of adapting to this change in user expectations, the industry failed to act.”

Between 2000 and 2010, record store sales fell by 76%. HMV and Tower Records “crumbled.”
Apple created iTunes and delivered what people wanted. “The music industry lost an unprecedented opportunity because of its failure to adapt to the changing landscape.”

Similarly, Blockbuster had numerous opportunities to purchase Netflix for as little as $50 million. But Blockbuster didn’t accept that customers didn’t want to visit a physical store. By the time it accepted this and began offering a postal service, Netflix was already transitioning to digital streaming. Once this infrastructure was set up, Netflix’s costs plummeted. Blockbuster, with its physical stores and associated costs, could not compete.

Use digital to help you adapt strategically to a changing environment

“With such a rapid rate of change, creating a three to five year strategy is impossible…
Instead, a digital strategy should help the organization become flexible enough and properly prepared to adapt to new challenges and innovations as they arise. The digital strategy should create a digital team capable of thinking strategically on a daily basis.”

“The web requires fast adaptation and close collaboration of people with very different skills. This means that it is the people working with digital every day who have to make rapid, informed decisions. They can’t wait for senior management’s consent.”

Boag recommends: “Good Strategy/Bad Strategy” by Richard Rumelt. A framework for creating a strategy:

  1. A diagnosis
  2. Guiding principles
  3. Coherent actions

“a responsibility matrix is considerably more effective than a web steering committee”

“Web steering committees are not a bad idea in principle, but in practice they can often significantly slow the agility of an organization in a realm where responsiveness and adaptability are crucial.”

“Important decisions are often delayed until a date can be found for the entire steering committee to meet. The meetings themselves often focus on endless internal discussion, rather than basing decisions on data and user testing. Finally, and probably most significantly, most of the people in the room are not qualified to be making decisions on the subjects being discussed.”

Instead, Boag recommends using a RACI framework, showing who is responsible/accountable/supports/is informed about each area of decision-making:
Responsible: doing the work
Accountable: formally accountable for the success of the work
Consulted: consulted about the work
Informed: told of the outcome

Better collaboration across teams

You don’t need to remove departmental structures – just make the edges of departments “fuzzier”.

“Departments should not be the only structure within an organization; there should also be working groups and other smaller teams that work across these departmental divides. But do not mistake this for more committees. I am not talking about interdepartmental committees. I am talking about real teams made up of people from multiple departments who sit and work together.”

Policies promote good decision-making

Standard sources of conflict experienced by digital teams: homepage space, requests for low-value web content, hostility to removing out-of-date content.

Save time on these arguments by having policies. These are impersonal, and have organisational buy-in.

“Instead of saying no to somebody who wants content on the homepage, you are just implementing a policy. Instead of removing somebody’s content, you are just following the rules. It’s not personal, it’s policy.”

Martha Lane Fox’s strategic review of the UK government’s digital offering, 2012

  • Manage digital centrally. Commission content from departmental experts as required. (“complete reversal of the previous policy”)
  • Focus on user needs and the delivery of online services – not just communicating information.
  • Radically simplify the government’s digital footprint.
  • Move from large technical projects to a more agile, iterative approach based on extensive testing.

Predicted that if the government moved 30% of its interactions with citizens online, they could save more than £1.3 billion.

If the UK government can make radical change, so can your organisation

“If an institution with as much inertia and legacy as the UK government is willing to consider such fundamental cultural change, then it demonstrates that this is possible for the vast majority of organizations.”

Common cultural characteristics of effective digital organisations

  • Collaboration.
  • Agile, iterative development.
  • Digital by default.
  • Innovation.
  • Service-oriented

Generic advantages of digital approaches

  • Cheaper
  • Faster
  • More flexible
  • Easier to monitor
  • More targeted

Senior management need to completely re-evaluate the business in the light of digital

Boag recommends Osterwalder’s Business Model Canvas as a tool to achieve this. Explore the following areas:

  1. Customer segments.
  2. Who are your customers, and can digital help you better understand them?
    Could you personalise your offering?
    Could you serve niche audiences that were previously not cost-effective?
    Could digital help you broaden your customer base?

  3. Channels.
  4. Can you deliver your offering through new channels?
    Could you move to a purely digital offering?
    Can digital enhance existing channels – by making them faster, more efficient or cost-effective?

  5. Customer relationships.
  6. What expectations do different customer segments have of you, and could digital help you surpass these?
    Could digital help your manage customer relationships more cheaply?
    Could you transition from telephone support to self-service?
    Could customer service be automated?
    Could you build online communities where customers support each other?
    Could you use digital to enable customers to take a more active role in evolving your products or services?

  7. Value proposition.
  8. What problems does your organisation solve? What needs are you satisfying?
    Could digital help you make your offering more valuable?
    Could digital allow you to solve new problems for your customers?

  9. Revenue streams
  10. Could you collect revenue more efficiently, and more easily for customers, using digital payment?
    Could you introduce new revenue streams, e.g. a subscription model?

  11. Key resources
  12. What are the foundational resources for delivering your offering?
    Could digital be used to replace these or lower their costs?
    e.g. do retailers need physical store fronts?

  13. Key activities
  14. What are the foundational activities required to deliver your offering?
    Could digital streamline these, or even automate them? (e.g. automated dispatch management of ecommerce orders)

  15. Key partners
  16. Who are your key partners or suppliers?
    Could digital tools help you manage them, or even replace them? e.g. self-publishing.

  17. Cost structures.
  18. What are the most important costs associated with your business model?
    Could digital lower these costs?
    e.g. lowering distribution or customer acquisition/retention or stock management costs.

    Looking at competitor organisations during this process can help motivate senior managers.

Embrace failure as integral to improvement

“we need to see failure as a necessary step towards success. Only by failing do we understand what success looks like.”

“In an industry so young and dynamic, the only way to innovate, the only way to progress, is to experiment and that will inevitably mean failure. We need to nurture a culture where failure is acceptable and, in fact, expected.”

“You might think that building something that fails is a waste of time and money. That is true if huge amounts of both have been sunk into its development. However, if you are working within an iterative process, centered around rapid prototyping this will not be the case.”

Test your assumptions by building something and seeing if it works. If it doesn’t, you can learn and improve.
If you have competing ideas, you can test them against each other at prototype stage. This de-politicises decisions, and helps organisations make decisions based on evidence.

Allow staff some time to explore new ideas, and create a culture in which new ideas can be expressed and valued.

Unless you are solely competing on price, you need to embrace customer-centricity

Marketing departments are still focused on broadcasting to mass audiences with mass media, rather than focusing on connections.

“One step in the right direction would be to make user testing a permanent and ongoing feature of your company’s culture.”

Twitter have an ongoing programme of usability testing that is open to anyone.
“user testing is not a periodic event confined to a small team, but an ongoing company-wide policy.”

In “Rocket Surgery Made Easy”, Steve Krug recommends a rolling programme of monthly testing.

How to get the most out of your digital team

“The most important factor is to give your digital team the freedom to do its job and set its own direction. This is a new and very light form of leadership based on respect rather than authority. It means relinquishing control and allowing people to direct their own roles.
I am not suggesting staff should be given the freedom employees of Valve or GitHub have. Although this might ultimately be beneficial to both employee and organization, it requires such profound organizational change that it is beyond the reach of most companies.”

The Harvard Business Review advises that “Senior leaders need to get used to the idea of abandoning absolute control”

“digital workers… need to see the reason for management decisions. It is not enough to tell them how things are going to be; they need to understand why a direction has been chosen.”

Your digital team needs the right tools

“You wouldn’t expect a professional tradesman to work with the same DIY tools we buy from B&Q, so why do so many companies insist that their digital teams use the same technology as the rest of the organization? What some perceive as luxuries such as smartphones, high-end computers, and tablets, are in fact tools of the trade for a digital professional. They shouldn’t have to fight to get these tools, they should just be provided.”

The knowledge of your digital team is their biggest asset. You need to actively invest in growing this

“you need knowledge. You need experts in creating digital solutions. They are your most valuable asset.”

Give staff the time and opportunity to continually strengthen and update their skills.
Digital professionals are motivated to do this – you just need to give them the time to learn and experiment, and to meet with peers.

Traditional project management approaches don’t work well in digital

Digital projects are so complex that traditional project management approaches struggle to scale.

For example, “It can prove nearly impossible to accurately specify large web projects due to the huge number of variables and complexities.”

Incremental change is better than big projects

Monitor user behavior to identify problems, prioritise areas for action (based on value of fixing and ease/cost of fixing), invest in a small incremental improvement, test and iterate, then go live. This reduces the risk of failure.

“This significantly reduces the amount of planning required and acknowledges the uncertainty inherent in running any large website.”

“The idea is to establish a rhythm of building, measuring, learning, and improving so that the site naturally evolves over time.”

Focus development work on user needs, not stakeholder requests

Any development work should be based on a user need, and to serve a defined persona.
“These personas need organization-wide approval and they become the bedrock on which your applications are built. Only tasks that meet the needs of these personas should be considered, and no task should be built that prevents a persona from completing one of their key tasks. They act as a filter for deciding which user stories will be accepted into the backlog of work to be developed.”

Tips for building grassroots change

Support each other, to build a safe and fun environment where you have the support of your team to take some initiative.

Try implementing small improvements to how you work. This will build your confidence and get management used to the idea. Don’t ask for permission – just make the change. Have a clear rationale ready in case you’re challenged. This should be focused on how it benefits the organisation.

Always educate and build bridges. So don’t just say “no” to requests.
Make sure you aren’t seen as a blockage that needs to be worked around.
Help colleagues think through alternatives – focus on their underlying need, rather than on the particular approach they’ve advocated.

Educate colleagues:

  1. Highlight best practice. (Competitor examples are particularly effective)
  2. Destroy preconceptions, using evidence. (e.g. any weird beliefs that people have)
  3. Promote your successes. Explain why things worked – builds the credibility of your team, and the organisation’s understanding.
  4. Explain failures. Explain why they happened and discuss how they could be avoided in future. Build a culture that embraces failure.

Senior management

“Getting frustrated with them will not help, but learning more about them will.”

Understand their objectives/targets and needs.

Focus on:

  • topics that they are interested in
  • their broader objectives and targets
  • return on investment
  • threats

A spreadsheet to assign jobs to sprints

If you’re following an Agile project management process, and know your development capacity, I’ve made a spreadsheet that automatically assigns a prioritised backlog to sprints, and gives ETAs for each job. You’re welcome to download this semi-automatic-sprint-planning-spreadsheet. (ODS format)

The spreadsheet uses a macro to automatically allocate jobs to sprints.

You need to tell the spreadsheet:

  • How many hours each job requires. (This could be changed to story points if that’s how you work.)
  • Your prioritisation of work – i.e. you need to rank the jobs in descending priority.
  • When your sprints start and finish; and how many hours you have in each sprint. (There’s a separate tab for this)

This spreadsheet could make your life easier by:

  • Saving you time allocating jobs to sprints.
  • Saving you time in updating information when details change (e.g. you add new high priority jobs and want to see how this affects the eta of various pieces of work)

The spreadsheet allows you to answer questions like:

  • When will job X be completed?
  • Is it worth expediting estimation of this small job so that we can fit it in to the next sprint?
  • Why aren’t we working on job Y yet?

If you know the number of hours (or story points) required for a job, and you have an ordered prioritisation, just press the button and jobs will be assigned to sprints.

The basic logic is: For each sprint, start from the top priority job that hasn’t yet been assigned. If there’s room, put this job in this sprint, reduce the number of hours available in the sprint, and write down the estimated delivery date for that job. Then look at the second highest priority job and see if there’s room for it in the sprint. Keep going down until you run out of hours remaining in the sprint.

Then move on to the next sprint, and do the same thing again. Continue until you run out of sprints.

This tool can help when you have a large backlog of development work. The ideal is, of course, to not have a large backlog. A backlog of development work incurs inventory costs: Understandings of requirements fade, developers move to different tasks, internal stakeholders have to be kept updated, motivation and excitement around each job diminishes.

This spreadsheet is shared under the MIT License.

Could you be a Digital Superhero? Julie Dodd (Camp Digital 2016)

Here’s a summary of Julie Dodd’s talk at this year’s Camp Digital conference.

Julie argues that digital superheroes…:

  1. Make products that really help people

    e.g. the Ugly Mugs app, built for sex workers to make sex work safer, or Tony Canning’s use of 3D printing to reduce the cost of prosthetic limbs from £20,000 to £40 – and reducing production time from months to hours – and making the technology available open source so that others can use it.

  2. Use any tool or platform that they can get their hands on

    BBC used Whatsapp at the height of Ebola in Sierra Leone in 2014 to provide information. Largely simple infographics. Reached 20,000 people in the first 3 months who were otherwise very hard to reach. The BBC used existing technology as it was cheaper, quicker and more effective.

    Crowdfunding sites – e.g. Kickstarter and IndieGogo – reduce the need for people to go through a middleman. Charities need to think what this means for them.
    Girl Scouts in America used IndieGogo to raise funds after rejecting a $100,000 transphobic donation. The troop of Girl Scouts that turned down the donation started a crowdfunding campaign that raised 4 times as much money – and sent out a powerful message about inclusion. They also changed the policy of how the national organisation takes donations.

  3. Aren’t frightened to try new ways to do things

    For example bringing in service design thinking or agile methogologies.

    St Mary’s Hospital in Paddington has created the ‘Helix centre’ innovation lab. It’s focused on lean, iterative solutions and combines online and offline. Projects include increasing rates of bowel cancer screening, offering guidance for clinicians on how to communicate about end of life care, asthma management tools for kids, dealing with storage of IV fluids.

    The Town of Jun use twitter for civic discourse and interactions – e.g. booking an appointment with the doctor or making a complaint. Everyone was trained. Saw a rise in public workers being thanked.

  4. Can be found anywhere – not just in tech.

    Google studied a favela in Rio, and didn’t expect to find much technology. Rather, they found a flourishing ecosystem, with radio stations, cyber cafes. With the proliferation of smartphones, people often ‘leapfrog’ the desktop ‘stage’ of development.

  5. Can have significant impact on organisations.

    The British Library now conceives of itself as a data institution, rather than a custodian of physical objects.
    The #tweetmythesis movement encourages academics to share their thesis in a tweet.

  6. Can have an impact in major commercial brands too.

    Barclays set up Digital Eagles programme, driven by the profit motive of reducing cost by moving more people to online banking. (And thereby reducing interaction costs, e.g. staff and branch costs). So it has trained 20,000 staff to train people across the community.

  7. Should work for organisations interested in changing.

    When researching “The New Reality” Julie found some organisations just weren’t interested in digital transformation. She thinks many won’t survive a decade, and wants to spend her energy not fighting those ones to change, but working for the ones that do want to change.

A few miscellaneous recommendations:

  • Recommended meetups: Citizen Beta and Tech for Good
  • “Apps without marketing are pointless”
  • If you do pro bono work with a charity, make clear the equivalent financial value. Otherwise they won’t value it because it’s free.
  • “Asking people to experiment is easier than asking them to commit”

Design for Real Life

Design for Real Life argues that we need to take accessibility more seriously. This goes beyond just conforming to a set of content presentation guidelines (e.g. the W3C standards), and goes to your overall design process. You can buy the book from A Book Apart

  1. Identify and challenge assumptions

    Think about what assumptions you’ve built into what you’re designing. What will happen if someone falls outside these?

    Facebook’s Year In Review – a feature designed to help people celebrate and share their great year – wasn’t designed with the experiences of people who’d not had a great year in mind.

    Inappropriate Year In Review images included:

    • a photo of the user’s apartment on fire
    • a photo of an urn containing the user’s father’s ashes
    • a sonogram of a pregnancy that later ended in miscarriage
    • a photo of a friend’s gravestone

    Facebook’s design team had a narrow vision, and so excluded all of these users. Meyer and Wachter-Boettcher challenge us to bring “edge cases” to the centre. “Instead of treating stress situations as fringe concerns, it’s time we move them to the center of our conversations – to start with our most vulnerable, distracted, and stressed-out users, and then work our way outward.” All users will benefit from this more focused, understandable and empathetic approach.

  2. Make space for real people

    Give people “enough room within our interfaces to be themselves.” For example, gender is often presented as a binary choice between male and female, which doesn’t fit with our current understanding of gender. Facebook is an example of best practice here, allowing people to choose male, female or custom – which is a free text field with a list of common choices as prompts.

    Other examples of systems not giving people space to be themselves include systems that can’t handle names longer than a certain length (e.g. 15 characters), systems that don’t accept hyphens in names, or ones that don’t accept names that don’t pass culturally-specific test of validity. (e.g. Facebook rejecting Shane Creepingbear’s name as not real.)

    Organisations often make assumptions about what matters to users, or about who they are. The ‘Apple Health’ app didn’t include period tracking on launch, even though it boasted that it tracked ‘all of your metrics that you’re most interested in’. Its implicit focus was on men. And period tracking apps themselves often have a bias towards straight, sexually active, partnered people.

  3. Incorporate stress cases

    A DIY and home appliance retailer was looking to improve its product guides. Originally these were written in a chirpy, positive tone, for happy, confident home-improvers. But sometimes users are more stressed when carrying out these tasks. The team found that there were two general categories of use: “urgent” and “upgrade”. They updated their style guide to write for the urgent case. This improved the guides for all users, as the clarity of information increased. Guides now feature installation availability and time-frames, estimated cost ranges, greater user of subheadings to allow for easy skimming, one-sentence summaries, reassuring tone.

    You can incorporate stress or crisis cases in usability testing. And you can test how a product performs in a cognitively-demanding environment by either testing in that environment, or by tiring people out mentally before the testing – e.g. by giving them some maths tasks to carry out.

  4. Only ask necessary questions in forms

    Organisations are often pushy to obtain as much information as they can from every web form. Often this is done with a total disregard for the user’s experience. Caroline Jarrett has a protocol for evaluating each question you want to include:

    1. Who in the organisation will use the answer?
    2. What will the answer be used for?
    3. Is the answer required or optional?
    4. If the question is required, what happens if the user enters rubbish data just to get through the form

    This question protocol can help open up a discussion about the true business value of each question.

  5. Learn from users

    Work to understand how your users see the world. This goes deeper than just testing top tasks on your website, or discussing product features.

    Steve Portigal recommends three types of question:

    1. Gather context and collect detail. e.g. asking about a sequence (Describe a typical workday) or specific examples (What was the last app you used?)
    2. Probe. e.g. ask for clarification of how a system works.
    3. Draw out contrasts. Useful for uncovering frameworks and mental models. e.g comparing processes or approaches.

    Open-ended research is about opening up questions and ideas, expanding your vision and the types of question you ask. This helps you move towards a design process centred aroudn real people and their needs.

    Customer mapping can help you identify pain points, broken flows, and content gaps, through analysis of lenses, touchpoints, channels, actions, thoughts and feelings. Adaptive Path have produced a guide to customer experience/journey mapping.

  6. Making the business case for accessibility

    Karl Groves, an accessibility consultant, argues that there are only three business cases for anything. Here’s how to argue for accessibility for each of these:

    1. It will make money. You can use accessibility to stand our from your competitors. e.g. Slack gaining users through ease-of-use. You can reach new audiences if more people are able to use your product.
    2. It will save money. You can cut customer service costs. The UK government found that as of 2011 it was receiving 150 million avoidable calls a year – calls for which an online service existed. This represented a possible annual saving of around £4 billion a year. Improving accessibility saves you money by increasing user retention – which is between 5 and 25 times more cheaper than acquiring new customers.
    3. It will decrease risk. Accessibility helps you avoid negative experiences and associated backlash – e.g. Facebook’s year in review generated a lot of negative press.

Agile Planning – How to plan quickly and collaboratively

A summary of the 02/08/2016 Digital Project Managers meetup on agile planning.

Estimating jobs

Gather the whole team for this exercise. e.g. UX design, developers, testers, product managers.

Discuss each job, and collaboratively rank the jobs in order of complexity.
Think about all the work that will be required to get this job ready to go live.
For each job, some elements might take more time. E.g. UX might take a long time on one job, but the development might be quick. So you need the different perspectives involved in the discussion.

Groups similarly-complex jobs together.

Apply T shirt sizes to these groups: S, M, L, XL, XXL.

If you have anything XXL, break it down into two or more different jobs.

Assign ‘story points’ to each job, based on its size:
S: 1
M: 3
L: 5
XL: 8

(If you’re running an internal team, there’s no need to think in terms of hours.)

Estimating velocity (how much work you can get done in a given time period)

If you’re an established team, you’ll know from experience how much work you can complete in a given period.

If you’re a new team, you’ll need to collaboratively estimate how much work you think you could carry out in a given development sprint (e.g. 2 weeks).

Work through each of the jobs, and combine them into groups showing how much you think you could carry out in a sprint. Then total the number of story points of the jobs in that group.

Once you’ve done this 10 or more times, work out the average number of story points per sprint. (Round down to the nearest whole number if required.)

This is your estimated velocity – the number of story points you think you can complete in a given sprint.

Prioritising jobs to build a plan of work

Prioritise all jobs using force rank: the highest priority job goes at the top, the lowest at the bottom. Nothing has equal priority.

You can also factor in sequence dependencies (and so promote some jobs that are required to allow you to complete other, higher priority jobs) and highlight milestones (points where you’ll have something specific to show off).

Once you have this ordered list of jobs, lay them out into separate sprints, totaling the number of story points you think the team can achieve during that sprint.

N.B. to adjust for team size, the time of year (e.g. holiday season), and to leave a % for emergent scope: between 20 and 40%, depending on how confident you are that the stories are comprehensive.

Managing ongoing development

Scope is variable. The plan isn’t fixed – it’s an overall route map.

Estimates are fixed. Once an estimate has been made, don’t change it. Retroactively changing estimates wastes time, and disrupts your statistics.
If you aren’t able to get through as much work as you expected, you need to have this recorded in the sprint burn-down. Reflect on this in the sprint retrospective. Re-project your velocity as required.

New stories are sized with reference to existing stories. This makes it quicker to estimate new jobs, as you can compare them to other similar jobs already completed. So estimation becomes easier and quicker over time.

Update your burn-up chart each sprint.Track the speed to the target (velocity) and the distance from the target (scope).

Once your team is established, use their measured velocity and use this to re-project timings. This is more useful than the estimated capacity that you use when planning with a new team.

Agile Foundation – some key insights from the BCS course

In May this year I obtained the BCS Agile Foundation certification. Here are some of the key insights.

Project variables

In a digital project, you have a set of variables that you can control.

Project-triangle-en

In a Waterfall project, you attempt to deliver a fixed scope. To meet this, the time and cost may have to change – and the quality may also be undermined.

In an Agile project, you fix the time and cost, but vary the scope. You might deliver less, but the quality is high, costs are controlled, you release on time, and what you do deliver is the most valuable work you could have been doing.

(A lean approach is slightly different. You look at flow rather than time, value rather than cost (if you only focus on cost you might not get a good ROI), and quality rather than scope (deliver less, deliver right).)

Agile focuses on the early delivery of value to the customer

A Waterfall project only generates value at the end of the project. You start off by specifying what you want to build, then you carry out design, then development, then testing, before launch. It is only at this last step that any value is experienced, because this is the first time that a working product is used by the customer.

An Agile project goes through all these stages during each sprint. At the end of each time-bound unit of development, something valuable is released to the customer. This means that value is delivered more quickly than in a Waterfall project.

Agile focuses on frequent delivery of value

Working in short sprints allows you to regularly release valuable software.

In 2010 Etsy re-engineered their backend systems to reduce the time taken between software releases. The aim was to be more responsive to customers. In 2011, Etsy released new software 12,000 times – about 30-40 releases per day. They tested code on a small percentage of the audience, and then ramped this up to 100% if it tested well. This new approach reduced defects on go-live by 90%, as each change was so small that it was very difficult to make mistakes.

Think about how you could make a small, safe change to your business process. This would help with risk management, and with agility.

Agile values emergent solutions

It’s important to plan, but plans always have to change. Our competitive advantage is in how well we respond to change.

Management must provide vision and purpose, so that self-organising teams know the overall direction they should be travelling in, and management must clear out blockers for teams. This is different to command and control, with management closely directing the actions of teams.

We must design for uncertainty and change

The ‘Waterfall’ model of developing software was codified by Royce in 1970. He noted that it is “risky and invites failure”. The waterfall approach assumes a simplicity which does not exist in a complex environment like software development.

When applied to software development, a waterfall approach makes three dangerous foundational assumptions:

Myth Reality
The customer knows what they want We don’t know what we want until we interact with it and see if it works for us or not.
We know how to make the software. Generally developers don’t know how exactly to build the software until they start building.
Nothing changes during the project Understanding and requirements change over time. Barry Boehm found that estimates given at the start of the project could be 4 times too high or 4 times too low. Later estimates are more reliable.

In an uncertain environment, use an empirical approach

Build something -> Measure how well it works -> Learn

Loop through this cycle as rapidly as you can.

Regular delivery allows for the earlier realisation of value, and for learning and customer feedback that can improve the product. This reduces the risk of building the wrong thing.
If you take a single, late delivery approach (as in Waterfall), the risk of not delivering the right product only starts to fall later on when you do UAT or go live. It’s better to deliver regularly, and to focus on high-value and high-risk areas, delivering these early to de-risk the project.

The economic case for Agile vs Waterfall

The Standish Group’s 2002 Chaos Report found that:
– Around 1/3 of projects are never completed.
– For completed projects, 45% of features were never used. So almost half of all time, effort and cost was for no reason.
– 19% of features were rarely used.
– 64% of time/effort/cost was wasted. So you need to focus on value, and you need to validate ideas with customers.

The Cutter Consortium found the following improvements due to the adoption of Agile practices:
– 61% cheaper.
– 24% faster.
– 83% fewer defects.
– 39% smaller teams.
These gains are a result of simplicity – maximising the work not done.

The key learning is: make your products smaller and more focused on the core value.

Report on the delivery of actual value, not on progress through project stages

In traditional project reporting, you monitor progress through different stages of the project. But there’s no value delivered until you go live.

Instead: look at revenue, impact, satisfaction levels, customer calls, organisational KPIs.

A 10 tweet summary of NFP tweetup 30

After a short hiatus, the NFP tweetup has returned – this time held at JustGiving’s offices. Here are ten particularly illuminating tweets from the event:

@LucyCaldicott & @Skipinder set up a Facebook group for fundraising discussion

They’ve found it helpful to extend their network.


Recently I attended an event for digital project managers. Most attendees were definitely not in the charity sector. Hearing quite different perspectives – and different acronyms and terminology – was refreshing and illuminating.

The group set up by @LucyCaldicott & @Skipinder is self-organising and non-hierarchical

Modbods is another recommended self-organising digital community. It’s on Google+ and is focused on community management:

@charlotte_cox worked to move somewhereto_ from charity to self-sustaining social enterprise

They had to work on selling their core offer to customers. Potential customers weren’t actually interested in their core vision/mission, so they had to package things differently.
They’ve boiled their key message down to: “Rent space from us, support creative young people”. (Or, to actually quote their website: “Find and book the perfect space to pursue your ideas and ambitions while giving a young person the chance to pursue theirs.”)

Tips on event social media from @dansmythphoto of @TeenageCancer

Covering a live event on social media doesn’t need to entail incessant posting of content:

Facebook will penalise you for sharing too much similar content.

It’s not always sufficient to just share supporter videos. You need to invest in editing them:

Teenage Cancer Trust re-cut their longer videos to appropriate length for social.
Don’t just cross-post your video on all the different social networks: adapt your content for each.
If you want to share a video on social, make it shorter, and make it work with no sound.

Just Giving’s #poweredbypeople campaign at the London Marathon

Just Giving created fundraisers a personalised page that they could share with their friends. It showed how much they’d raised, and pulled in information on their cause.

“People like presents more than asks.”

Also, snapchat filters are apparently quite cost-effective at the moment, with a good ROI.