This report aims to address the lack of conceptualisation and the absence of good measurements for outcomes of Internet use. Up until now uses or different ways of engaging with the Internet have often been used as proxy measures for outcomes. This is unsatisfactory because it is by no means clear that engaging with an activity online brings tangible offline benefits to all those involved. In addition, when outcomes are measured they are often not tangible (that is concrete, specific and observable by an external evaluator) but measure attitudes or opinions towards the digital activity rather than the offline outcome itself.

 

Measurement
The first issue we had to tackle was how to measure outcomes in general. We settled on and tested the validity of scales and dichotomous items and concluded that scale items that measured quantity (achievement of the outcome) and quality (i.e. satisfaction with the outcome) are the best way to measure tangible outcomes of Internet use.

This is the proposed answer format for the achievement and satisfaction items:

 

Achievement Satisfaction
Thinking about your online activities in the past year, how much do you agree or disagree with the following statement? If you did not undertake the activity the question is about, just select Not Applicable (NA)  Thinking about your online activities in the past year, how satisfied or unsatisfied were you with the following? If you did not undertake the activity the question is about, just select Not Applicable (NA)
Disagree strongly (1)  Very dissatisfied (1) 
Somewhat disagree (2)  Somewhat dissatisfied (2) 
Neither agree nor disagree (3)  Neither dissatisfied nor satisfied (3) 
Somewhat agree (4)  Somewhat satisfied (4) 
Agree strongly (5)  Very satisfied (5) 
Not applicable (NA) (6)  Not applicable (NA) (6) 
Don't know (7) Don't know (7)

 

In this report we developed an instrument that tested a series of questions designed on the bases of the corresponding fields framework which conceptualises outcomes in four fields of resources: economic, cultural, social and individual well-being. The report showed that this framework was very helpful in organising and developing lower level indicators. The most difficult to design were measures related to cultural and social resources.

We suggest that any research that tries to measure outcomes and wants to understand not just very specific outcomes that relate directly to the object of study or the goals of an intervention should include the following categories of items (and use the answer scales described above).

 

Explanations for differences in outcomes

When it comes to explaining achievement of and satisfaction with outcomes from Internet use, there are three important preliminary conclusions that can be drawn based on the material presented in this report.

1. Engagement with a specific online activity does not necessarily lead to tangible offline outcomes. However, once an outcome is achieved the satisfaction with these outcomes tends to be high. This is the case for most outcomes apart from the cultural ones.

2. Those who are able to achieve an outcome in one area are not necessarily able to achieve an outcome in another area. So while there might be transfer between one type of digital engagement and another it is by no means the case that once a person is online and reaping the benefits in one way they are digitally included in all other ways as well. Being able to achieve benefits at a personal level does not mean that the person is able to achieve benefits at the social or economic level.

3. To translate engagement with an online activity into a tangible outcome digital skills are essential. Differences in digital skills between different socio-economic and socio-cultural groups lead to lower levels of achievement of and satisfaction with outcomes of Internet use.

 

The study presented in this report is a first attempt at measuring and explaining outcomes of digital engagement. There will be future publications based on this study that explore the pathways from social exclusion to tangible outcomes of Internet use through means of digital access, literacy, motivation and engagement. More population-based research is needed to understand what explains differences in outcomes in different socio-cultural contexts. Further, qualitative work is vital to understand how satisfaction and achievement of outcomes are culturally and socially constructed and to explore the cognitive, quantitative aspects of outcomes as well as the affective, qualitative aspects of how digital engagement translates into real benefits in everyday life.