Skip to main content
Making best use of data
  • With site investigation data costly to acquire are you making best use of existing and newly collected ground investigation data?
  • Are you best exploiting the readily available open source datasets and other information? 
  • Do you acquire, process and store site investigation data for the now only?  
  • Or do you think about the data storage and accessibility with respect to on-going value of the site data collected? 

Open source data sets
The UK is data rich when it comes to historical sub-surface information. However, the accessibility, reliable usability, common formatting and continued gathering and sharing of this information historically has been challenging, to such an extent that much of this information remains untapped, along with the economic benefits many believe it should bring. 

  • Geospatial Commission
    For the site investigation industry, developments funded by the Geospatial Commission are aimed at better accessibility, sharing and confidence in 3rd party and historical ground investigation data. The launch of a new Data Exploration Licence gives access to open source datasets held by the BGS, Coal Authority, HM Land Registry, Ordnance Survey and the UK Hydrographic Office through a single licence, which is free for R&D and Innovation purposes, find out more here  Centralised mapping of underground gas and water pipes, electricity and phone cables into a UK wide Underground Assets Register, starting with pilot projects in London and the North East is underway. The Commission estimate that accidental strikes on underground pipes costs in the region of £1.2 billion, along with the associated risk of serious injury or death. Benefits of an asset register to site investigation is likely to be significant if the challenge of getting this data into an accurate and common format can be achieved.


  • BGS - OpenGeoscience
    The British Geological Survey (BGS) holds an extensive range of datasets which it makes available for use, with increasing numbers of these open access and made available under OpenGeoscience
    Pages where detailed data can be sourced can be found here. Descriptions of the nature of the dataset and how the data is accessed is also provided.

    BGS has long been established as a source of geological map data and borehole records but it also holds data on a vast number of topics that are sometimes less well known. These non-exhaustively include: the national landslides database, geochemical data, GeoSure, hydrogeological data, 3D geological models and the civils bundle of datasets relating to ground conditions such as excavatability, corrosivity and engineered fill. 

    The BGS website is also home to a number of app’s which can be used in the field, such as iGeology and iGeology 3D, giving you access to interactive geological map data. 

    Groundhog desk top
    Also available via the website is Groundhog Desktop, a free-to-use programme developed by the GeoAnalytics and Modelling directorate of BGS that allows users to import their own site data and then display and edit borehole logs, digitize geological cross-sections, and export interpreted data. Groundhog Desktop also supports a range of baseline data such as geo-registered images and digital elevation models. 

    Data sharing 
    Whilst many of the datasets held by BGS reflect work carried out by the survey, BGS also acts as custodian for data supplied by external stakeholders. Borehole records, geophysical logs and geotechnical data, for example, are deposited with the National Geoscience Data Centre (NGDC) at BGS and made available to users through GeoIndex. 

    BGS is a promotor of data sharing and the borehole database would not be possible without a willingness to do this within our community. In recent years initiatives such as ASK Networks and Dig to Share have been effective in increasing standardised data sharing and BGS has seen a rise in data deposition. 

    Up to 50% of construction project delays and overspends are due to unforeseen ground conditions which can be reduced through the reuse of archive ground investigation data. Through outreach activities there has been a significant rise in the number of data deposits with BGS but challenges remain. Reasons why some organisations do not share data in this way include uncertainty over data ownership and fears concerning liability. 

    The new NGDC terms and conditions now address the latter concern, while the former is being addressed by adding a clause to contracts that state data will be deposited with the NGDC at the end of the contract. BGS is able to place data under an embargo period so deposited logs can be subject to a time-delayed release at the request of the depositor, subject to site-specific/client-specific requirements. BGS and Dig to Share are currently working on a pack that explains these issues which will hopefully settle such concerns.

    An advantage of data sharing is that is leads to standardisation within the industry. The NGDC’s preferred digital data format is AGS , which can be easily uploaded and which is readily accessible by many of the potential users of the data. Both the message of good practise in data formats and data sharing is being heard and there has been an increase in willingness to deposit data with the NGDC. 


  • DINGO (Databases to INterrogate Geotechnical Observations)
    A database of pile load test information that has been built as part of the Engineering and Physical Sciences Research Council (EPSRC) funded project EP/P020933/1: DINGO which ran between 1 July 2017 and 9 June 2019. The database is populated with data digitised from the literature as well as datasets supplied by contributors from the geotechnical engineering industry in the UK. You can find a summary report of how DINGO was compiled and what it covers here. The database itself can be downloaded here

Building Information Modelling (BIM)
Above ground the implementation of BIM is fast becoming routine but such modelling often excludes below ground information.  Tawelian L.R et al from Glasgow Caledonian University, reported  in their 2016 paper ‘The Implementation of Geotechnical Data Into the BIM Process’ some of the current hindrances they identified in incorporating geotechnical data into BIM for transportation projects in Scotland.  This included limitations of the current AGS Data Format which does not take account of historical data, sub-surface structures and buried utilities. 

There are a number of UK and international standards on how to  to embed knowledge and experience of the processes necessary to provide a collaborative way of working. These include:

  • ISO 19650-1: Organization and digitization of information about buildings and civil engineering works, including building information modelling (BIM) 
  • ISO 19650-2: Organization and digitization of information about buildings and civil engineering works, including building information modelling (BIM) 

These international standards published January 2019 supersede BS1192 (principles) and PAS 1192 part 2.  For more information on ISO 19650 Transition please see BSI FAQ .

  • BS 8574:2014 Code of practice for the management of geotechnical data for ground engineering projects

The Centre for Digital Built Britain (CDBB), has information on the digitisation of the construction cycle, not just site investigation. This includes recent  CDBB reports such as reports Adaptive design of supported excavations Final Report and A Bayesian definition of ‘most probable’ Parameters.

‘Big’ data and machine learning 
It is not just within the political arena that big data analytics is changing the landscape of our society with many forward-looking companies using technology to extract hidden value from their data.  For example chemical test laboratories have helped inform the scale/significance of asbestos in soils in the UK by sharing raw data with SoBRA’s working group on asbestos in soils 

The International Society for Soil Mechanics and Geotechnical Engineering (ISSMGE) has set up a forum for their membership to explore ML techniques to solve problems relevant to soil mechanics and geotechnical engineering.  LinkedIn has over 700 professionals who have joined the group ‘Data Science and Machine Learning application in Geotechnical Engineering’.

Use of ML in combination with remote sensing data is happening already.  For example, Norway is reducing direct ground investigation costs by using a combination of geophysical mapping (Airborne Electromagnetic Measurements (AEM)) and ML to interpret the data generated from the AEM measurements for large infrastructure projects. For further information click here

Applications of ML in the environmental arena also are beginning to be seen in the areas of tracking air and groundwater pollution, and in understanding the near surface geochemistry.  In fact, the BGS has been looking at what big data analytics and ML can do for geological and environmental mapping, for example combining aerial and ground survey information through using an artificial neural network (ANN) has produced detailed mapping of arsenic enrichment in south-west England, providing valuable prospecting and environmental information.