Helsper (2012) developed a framework which theorised about the pathways between specific types of social exclusion and specific types of digital exclusion. She argued that access, digital skills and motivations mediate the relationship between specific offline characteristics and engagement with technologies. The research presented here is a key part of a wider research project that used this framework as a starting point and aimed to develop measures of outcomes of Internet use, different types of engagement with the Internet and the skills needed to engage in this way. This particular report focuses on testing whether reliable and valid scales could be developed that measure digital skills.

 

To come up with such an instrument, we took a critical look at the existing digital skills literature. Moreover, our own experience and work related to digital skills helped us in building an elaborate skills framework including specific skill indictors. Two main theoretical approaches the proposed framework was built on were the skill distinctions as set out by Van Deursen and Van Dijk (2009a, 2009b, 2010; Van Dijk & Van Deursen, 2014) and measures tested by Helsper and Eynon (2013). For several types of skills proposed in these approaches we were able to define corresponding items. We ensured that all items reflected typical Internet use that everyone might imagine him or herself doing. Furthermore, we avoided contextual items related to specific platforms or activities. This should allow these items to be used for a considerable amount of time because they are not dependent on what type of activity is trending or on new platforms becoming popular. The only exceptions are the items that were introduced regarding mobile skills, as a consequence these items might have to be adjusted or integrated into other skills as mobile platforms become more mainstream.

 

All items used a scale that gave statements about things that a person was able to do with answer formats that ranged from ‘Not at all true of me’ to ‘Very true of me,’ and furthermore included a ‘I do not understand what this means’ option.

 

After the development of a first full survey instrument, we used a three-fold approach to test the validity and reliability of the latent skill constructs and the corresponding items. The first step consisted of cognitive interviews held in both the UK and the Netherlands. Based on the cognitive interview results, we made several amendments to the proposed skill items to improve clarity.

The second step consisted of a pilot survey of digital skills, both in the UK and in the Netherlands (at this stage we also measured uses and outcomes of Internet use, the results of this will be reported later in 2014). The result of the second step was a final theoretical, empirically and cross nationally consistent framework consisting of five types of digital skills: Operational, Information Navigation, Social, Creative and Mobile skills. We suggested longer scales for most of these, consisting of between six to ten items, and shorter scales consisting of five items. The Mobile skills scale (the least theoretically grounded) consisted of only three items.

 

Proposed items and factors to measure Internet skills

 

Skill Item
Operational I know how to open downloaded files
 I know how to download/save a photo I found online
 I know how to use shortcut keys (e.g. CTRL-C for copy, CTRL-S for save)
 I know how to open a new tab in my browser
 I know how to bookmark a website
 I know where to click to go to a different webpage
 I know how to complete online forms
 I know how to upload files
 I know how to adjust privacy settings
 I know how to connect to a WIFI network
Information Navigation I find it hard to decide what the best keywords are to use for online searches
 I find it hard to find a website I visited before
 I get tired when looking for information online
 Sometimes I end up on websites without knowing how I got there
 I find the way in which many websites are designed confusing
 All the different website layouts make working with the internet difficult for me
 I should take a course on finding information online
 Sometimes I find it hard to verify information I have retrieved
Social I know which information I should and shouldn’t share online
 I know when I should and shouldn’t share information online
 I am careful to make my comments and behaviours appropriate to the situation I find myself in online
 I know how to change who I share content with (e.g. friends, friends of friends or public)
 I know how to remove friends from my contact lists
 I feel comfortable deciding who to follow online (e.g. on services like Twitter or Tumblr)
Creative I know how to create something new from existing online images, music or video
 I know how to make basic changes to the content that others have produced
 I know how to design a website
 I know which different types of licences apply to online content
 I would feel confident putting video content I have created online
 I know which apps/software are safe to download
 I am confident about writing a comment on a blog, website or forum
 I would feel confident writing and commenting online
Mobile I know how to install apps on a mobile device
 I know how to download apps to my mobile device
 I know how to keep track of the costs of mobile app use

 

Note I. Items in black make up the proposed short scales since they have the highest factor loadings, blue items are added for the long scales.
Note II. There is also a set of Critical (literacy) skills that are not included because they were shown to be individual context dependent and not easy to measure in general population survey research
Note III. The information navigation items are all negatively formulated. We recommend that future research use positively formulated items measuring the same skills.

 

We recommend the use of the shorter five item scales in larger research projects that need to include a variety of skills. However, if researchers want to focus on skills only or on a specific skill in relation to other variables, they should use the longer scales which offer more variance and the opportunity to study them in detail by, for example, distinguishing different operational skills. If researchers or evaluators do not care about distinguishing different skills, it would be possible to take two items (the highest loading or most relevant ones) from each short scale and create one ten item scale. However, we strongly recommend against this approach. Past research shows that analysis of the causes and consequences of digital literacy is complicated when using these limited scales (Helsper & Eynon, 2013).

 

During the final step, we examined the consistency of the five Internet skill scales and their characteristics when measured in a representative sample survey of Dutch Internet users. This step revealed that reliability and validity of the scales as well as indicators of convergent and discriminant characteristics were good. As a result of this work, we recommend the use of the Operational, Information Navigation, Social, Creative and Mobile skills scales in general population research. While we believe these scales are a significant contribution to research that measures digital skills, it is important that further research is carried out to understand the different relationships these skills have with one another within different socio-demographic groups. As noted in section 5, we suspect that there might be underlying differences in experience and meaning between these groups that underlie slight variations in the digital skills landscape, even when all the skills are valid ways of classifying the abilities people need to use Information and Communication Technologies.

As noted above, the key purpose of this report is to put forward a set of valid digital skills measures that are of value to survey researchers working in this field . We very much welcome feedback and comments from readers who are interested in testing these scales in a range of countries.