Many companies are betting big that electronic book readers will be one of the main ways people read long-form text in the future. However, such products will succeed only if the reading experience is much better than the misery of reading from PC monitors.

Various types of tablets ought to do better than desktop computers because they offer higher-resolution screens and a more comfy reading posture. But are tablets as good as printed books?

To find out, we conducted a readability study of people reading fiction on the two highest-profile tablets: Apple's iPad (first-generation) and Amazon's Kindle 2.

In contrast to our previous study of iPad application usability, we didn't study a range of user interfaces. Instead, we tested only the default iBook app. Testing a single iPad reader let us more easily compare it with the Kindle, which has only one user interface.

Also, in contrast to our previous analysis of Kindle content usability, we didn't consider non-linear content, such as Web pages or newspapers. Instead, we specifically focused on testing linear, narrative content because it's the primary use case for e-book readers.

Finally, we didn't test the many issues related to choosing and installing reading software, nor did we test the reading UI's learnability. We taught participants how to use the readers before we started measuring their reading speeds.

In This Article:

Method

We ran a within-subjects study, testing each user on all 4 reading conditions — printed book, PC, iPad, and Kindle — rotating the sequence in which we exposed users to each device.

In usability research, it's often better to use a between-subjects study, in which each person tests different conditions to avoid the transfer of learning from one system to the next. But in this reading study, we didn't test learnability to begin with, and users definitely had substantial previous experience with the main skill we tested: that of reading. One major benefit of within-subjects testing is that it minimizes the effect of individual variability among the test participants.

On each device, we asked each user to read a short story by Ernest Hemingway. We picked Hemingway because his work is pleasant and engaging to read, and yet not so complicated that it would be above the heads of users.

On average, the stories took 17 minutes and 20 seconds to read. This is obviously less time than people might spend reading a novel or a college textbook, but it's much longer than the abrupt reading that characterizes Web browsing. Asking users to read 17 minutes or more is enough to get them immersed in the story. It's also representative for many other genres of interest, such as whitepapers and reports.

After users read each story, we gave them a brief comprehension questionnaire to test their understanding of the story. Our test participants got almost all the questions right, regardless of device, so we won't analyze this data further here. The exam's main purpose was to ensure that people would take the reading task seriously because they knew they'd be tested on it.

We didn't ask participants to think out loud, so we were able to use multiple-user simultaneous testing (MUST) and test several users at once. Participants simply sat quietly and read, the way most people do at home. We provided comfortable easy chairs to emulate the typical reading experience with a tablet. (We also tested users reading from a PC monitor at a test station arranged like a normal office, with the computer on a desk and the user seated in an office chair.)

Users

We tested a total of 32 users: 5 for a few rounds of pilot testing and 27 for the main study. Unfortunately, we had to discard the quantitative data from 3 users due to measurement flaws, so our reading-speed statistics are based on the remaining 24 users.

We recruited participants who like reading and frequently read books. This is obviously a biased sample compared with the entire population, but we felt that narrowing the target audience was reasonable for a study of e-readers.

At the beginning of each session, we quickly assessed the study participants' reading skills by administering the REALM literacy test. (This test asks people to read words of varying difficulty and scores them based on the number they mispronounce. In our study, most users got all the words right; 2 people failed a single word, which indicates at least a high-school literacy level.)

We deliberately didn't include any lower-literacy users in this study. Again, this was because we wanted to focus on the people most likely to actually read long texts on tablets.

Results: Books Faster Than Tablets

The iPad measured at 6.2% lower reading speed than the printed book, whereas the Kindle measured at 10.7% slower than print. However, the difference between the two devices was not statistically significant because of the data's fairly high variability.

Thus, the only fair conclusion is that we can't say for sure which device offers the fastest reading speed. In any case, the difference would be so small that it wouldn't be a reason to buy one over the other.

But we can say that tablets still haven't beaten the printed book: the difference between Kindle and the book was significant at the p <.01 level, and the difference between iPad and the book was marginally significant at p =.06.

User Satisfaction: iPad Loved, PCs Hated

After using each device, we asked users to rate their satisfaction on a 1–7 scale, with 7 being the best score.

iPad, Kindle, and the printed book all scored fairly high at 5.8, 5.7, and 5.6, respectively. The PC, however, scored an abysmal 3.6.

Most of the users' free-form comments were predictable. For example, they disliked that the iPad was so heavy and that the Kindle featured less-crisp gray-on-gray letters. People also disliked the lack of true pagination and preferred the way the iPad (actually, the iBook app) indicated the amount of text left in a chapter.

Less predictable comments: Users felt that reading the printed book was more relaxing than using electronic devices. And they felt uncomfortable with the PC because it reminded them of work.

This study is promising for the future of e-readers and tablet computers. We can expect higher-quality screens in the future, as indicated by the recent release of the iPhone 4 with a 326 dpi display. But even the current generation is almost as good as print in formal performance metrics — and actually scores slightly higher in user satisfaction.

More on Reading and Tablets

Full-day seminars at the annual Usability Week conference:

293-page report on Usability of Mobile Websites and Applications with 210 design guidelines and 479 screenshots is available for download.