The person promoting data sharing in the organisation
This section describes why and how the organisation can monitor the use of its opened datasets.
Investment decisions in open data publications often need to be justified by value creation, i.e. the benefits derived from the investment. The organisation may also have set quantitative targets for such benefits. It is important to measure the benefits created by open data, making it possible to report them and to use them to justify future investments.
Organisations that have already opened their data are often interested in finding out the extent to which their datasets have been viewed or downloaded in the data portal. It is important to note that while the viewing and downloading data concerning datasets published on the portal or other online service do not indicate if the dataset has actually been used and how, they do give an indication of the interest attracted by the opened dataset.
Measuring open data is difficult, but the scale of its use can be measured and estimated. You can choose from a number of indicators, such as view and download statistics, or API use metrics. In addition, the utilisation of data can be measured with other factors, such as media visibility and the number of organisations and applications making use of your data. Information on open data use can, for example, be collected by means of media monitoring (keywords) and surveys or by offering various incentives to report on data use. The table below contains examples of different use-related factors and indicators.
Examples of measurement methods
| Measured factor | Example indicator |
|---|---|
| Downloads | Number of files downloaded |
| Views | Number of files browsed |
| API use | API usage statistics |
| Data utilisation | Media monitoring and press articles |
| Number of organisations using the data | |
| Number of applications using the data | |
| Surveys and interviews | |
| Use of data in hackathons | |
| Contacts by higher education institutions | |
| Data quality | Feedback: contacts, number, content, background organisations |
Monitoring and measuring the use of opened data also depends on the format it was published in. Different statistics can be obtained on the use of data published in a dynamic form, often through an API, and its use is reasonably easy to monitor. On the other hand, monitoring the use of static data (files) is more challenging.
Static data is usually downloaded only once, after which the user has unlimited access to it. Static data downloads do not necessarily convey any information about the extent of data use or the impact of its publication. In addition, open data licences permit the re-sharing of data, which means that other parties can also share the file in their services. For example, an individual media outlet may download the dataset once and publish a news item based on it, which spreads to several other media outlets. Consequently, even one download can have a high impact.
It is possible to collect analytics on API use, which can then be used to develop the API and possibly also to improve data quality. Statistics can be compiled using various applications, such as Matomo. For example, analytics can be collected on the general volume and methods of data use, API requests and user locations.
If the use of the API requires an API key, this key can be used to distinguish between data users and to monitor the API’s utilisation rate. API keys can also be distributed anonymously, allowing data users to retain their anonymity, and the use of the API key does not undermine the basic principles of openness. If someone misuses an open data API that requires an API key, the key can be used to differentiate between users and, if necessary, limit server loads.
This section describes how the organisation can monitor the use-related benefits and impacts of its opened datasets.
After opening your data, it is a good idea to monitor and measure the benefits and impacts that this process has had. The benefits and impacts of opening data are impossible to estimate accurately, as the opening of data has many indirect benefits that are difficult to measure. For more information on the benefits of opening data, see step 3.
No established set of indicators exists for evaluating the impact of open data, as the development of research and measurement tools is still in its infancy. However, the measurement of the impact of open data has been carried out both at the organisational level and internationally. Examples of impact studies can be found at the end of this section.
The following methods can be used to obtain information on the impact of the opened data:
When assessing the impacts, it is advisable to take into consideration the entire life cycle of the data and the impacts of opening it from a wider societal perspective. Assessing the impacts can help the organisation to improve its effectiveness, fulfil its public mission better and increase the transparency of its work.
This section contains support material related to the topics discussed in this step.
Training courses on the data.europa.eu website: