Meeting Venue: NV Zebrastraat, Zebrastraat 32, Ghent, Belgium

Date: Tuesday 03/09/2019 - 13:00 to 17:30

Participants

Summary of action points TWG-6

Meeting Minutes

Updated Terms of Reference

The terms of reference (ToR) were updated with the input of DG MARE. Additional technical monthly meetings will be held between VLIZ and EMODnet secretariat, which can be joined by the EMODnet thematic portals. Reports of these meetings will be reported.

• Action: Secretariat to circulate the updated ToR
• Action: All lots have to validate the names of the persons that will lead and support the Technical Working Group

Review of EMODnet priorities

Web services documentation

The EMODnet Web service documentation was published on the OpenSeaLab website. The documentation provides the urls for the predefined download of data, OGC Web services and catalogue services for all portals. The documentation needs to be checked by portals.

• Actions: Portals to initially review and update the Web service documentation on the JIRA and keep it up to date as Web service endpoints are added or modified.
• Action: EMODnet secretariat will report encountered issues on Web services of the thematic portals through JIRA. Reported issues to be addressed by thematic portals and difficulties and experiences on the technical implementation are to be put on the JIRA

Harmonisation of the portals menus and look and feel.

Some harmonization of the portals look-and-feel has been achieved based on previous actions but the user experience remains disparate across portals. The current contracts require a common look and feel but EASME would like to go further in the following contracts to ensure a similar user experience across portals. This does not mean that portals can only have a limited set of menu items but the common sections need to be the same. It also does not require that map-viewers are the same.

EMODnet Secretariat comments that an efficient way to achieve a full harmonization process would be to use the same Content Management system (CMS) as the existing discrepancies mainly result of different CMS technologies. The Web look & feel template for EMODnet has been developed base on the DRUPAL CMS. A complete harmonizing of the Web sites would require some of the lots to migrate to DRUPAL. The EMODnet Secretariat does a tour de table to discuss if migration to the DRUPAL CMS would be an option. Thematic portals state that this would require significant time investment and that this not a priority. This could be achieved if such a CMS migration is specified in the next contract and tender.

An alternative approach is to strengthen the harmonization effort with a second set of guidelines letting the portals with their current CMS.

Trust-IT would be able to create these guidelines which will be communicated on JIRA allowing the portals to review them and provide feedback.

• Action: Trust-IT, with input from DG MARE, to develop guidelines on how to achieve a similar user experience across portals.

Use cases

The EMODnet use-cases need to be monitored efficiently (i.e. each use case needs a separate MATOMO tracking script so that they can be monitored individually). The TWG discusses that either all the use-cases are hosted on the Central Portal and the thematic portals show a filtered selection relevant to their topic (using a mirror url/embedded). This has the advantage that they are more centralized and thus may reach more users. This is important for the Secretariat. Or alternatively, the thematic portals host their relevant use cases and the Central Portal only provides a highlight with a link to the respective portals. There is also an increasing need to develop more use cases in a “story telling” format with visuals and quotes.

• Action: Trust-IT, with input from Secretariat and DG MARE, to develop a guidelines/plan for a “use cases” structure and format (more visual, quote, image, text)
• Action: Trust-IT will ask for a more visible link (e.g. clickable EMODnet portal icon) for each use case

INSPIRE compliancy

Pascal Derycke points out that the Central Portal GeoNetwork harvest metadata from all the lots in different ways. No coordination has been achieved or no coordination has been organised so far. Some lots have a catalogue service based on GeoNetwork, some others no. For some lots, the metadata are harvested from their view services and some others provide the INSPIRE XML metadata records.

All thematic portals need to provide a CSW endpoints that can be harvested so that the GeoNetwork can be kept up to date efficiently. Comments from the TWG show that maintaining a GeoNetwork is not always straight forward. Bart Vanhoorne reminds that there is a GeoServer plugin for INSPIRE compliancy but it may be unstable so the Geoserver version needs to be checked.

• Action: thematic lots to implement INSPIRE compliant CSW endpoint and add it to the Web service documentation on JIRA.

Harvesting of GeoNetwork by EU Open Data Catalog

Top priority action: The EMODnet Central Portal GeoNetwork needs to be updated to the latest version so that it can be harvested by EU Open Data Catalog (An RDF GeoDCAT-AP endpoint is required)

• Action Update the Central Portal GeoNetwork version so that it can be harvested by the EU Open Data Catalog.

Query Tool

Paula Oset (VLIZ) performs a demonstration of the Query Tool which is live but needs to be hyperlinked in the portal.

The Query tool allows user to select an area and search through some data layers available in the different portals. Statistics from the layers are produced for the selected area. Layers of interest can be pinned and will remain in the viewer. Some of the information is cached and some of it is requested in real time and hence the duration of the request depends on the data access speed which might be slow.

Specific issues requiring action were highlighted:

EMODnet chemistry does not offer WFS or WCS which is a requirement for the integration in the Query Tool and blocks the IP address after several attempts to access data (not INSPIRE compliant!).

• Action: EMODnet Chemistry to make sure OGC Web services (WFS and WCS) are enabled for EMODnet Chemistry and add these endpoints to the Web service documentation on the JIRA

No WCS service for gridded products of EMODnet Physics and this is required for integration in the Query Tool:

• Action: EMODnet Physics to enable WCS service for gridded products

From EMODnet Geology, some layers show up in the spatial query even though there is not data available.

Icons of the thematic portals are different than the official ones

• Action: VLIZ to update thematic lots icons in the Query Tool
• Action: VLIZ to finalise Query tool v1 and link to it from the Central Portal

HTTPS on thematic portals

Most portals have enabled HTTPS (mandatory) :

EMODnet Human Activities: Yes

EMODnet Geology: Yes. Explained the use of “Let’s encrypt” a free service

EMODnet Chemistry: Yes for the portal but still needs to implemented for Web services

EMODnet Bathymetry: Yes

EMODnet Physics: Yes, required an update of their window server

CP MATOMO: Yes

EMODnet Biology & Central Portal: VLIZ will implement in October-November

• See action from TWG4

The Portals also need to evaluate and resolve the BREACH vulnerability as per request of EASME:

• Action: Resolve the BREACH vulnerability, EMODnet Secretariat will create JIRA task. Portals to take required actions.

Centralized/automatized monitoring Web metrics on thematic portals

The implementation by VLIZ of a central MATOMO instance to collect Web metrics for all portals has demonstrated its important role as facilitating the progress reporting. To facilitate the access to the Web metrics it is possible to develop a dashboard on top of the MATOMO API. The dashboard will allow DG MARE to easily access and visualize the Web metrics for the different portals. DG MARE could provide feedback on the Web metrics they want to see displayed. Furthermore, TRUST-IT needs more minterpretation of the metrics by the thematic portals for the progress indicator reporting

• Action: TRUST-IT to develop a dashboard to display MATOMO Web metrics
• Action: Thematic portal to provide interpretation of the web metrics so that the progress indicator reporting can be improved

Monitoring usage of the Web services

Secretariat: monitoring web services

The EMODnet secretariat reports on a potential technical solution based on the GeoNode monitoring application to collect centrally metrics of the EMODnet Web services. The GeoNode monitoring application relies on the Geoserver audit API. This API is part of the GeoNode monitoring extension. The version of Geoserver is important to get this working. The secretariat and VLIZ inform the TWG that they will further evaluate the solution and may require the support of the thematic lots in evaluating the monitoring application. Initial test with the Geoserver from EMODnet Geology indicates that it works for version 2.11.1. Other portals need to check their Geoserver version and report.

Physics and Chemistry portals inform that they wrote their own script to parse the apache log files to get the metrics.

• VLIZ and Secretariat to further investigate feasibility and what GeoServer versions are supported?
• Physics and Chemistry to report on how they get the metrics of Web services.
• All portals to provide information on the version of Geoserver in production. Report if the monitoring extension and configure audit logging can be enabled.

Brexit Preparations

All EMODnet domains with the .eu extension need to be owned by solely EU companies, institutions, …. There was a short discussion about the transmission of ownership of the EMODnet Seabed Habitats domain to Ifremer.

Future EMODnet priorities “EMODnet as a service provider

Centralized Helpdesk (JIRA service desk)

There is a need to centralise the information and feedback coming from users so that it can be easily proceeded and analyzed.

• The Secreteriat to investigate if the JIRA service desk can be a solution to manage a centralised helpdesk service.

Monitoring SDIs / Stress-test (GeoHealthCheck)

As a service provider with distributed services (i.e.: multiple Spatial data infrastructures), we need to know the status in real time (up or down time) of our services. GeoHealthCheck is a Python application to externally monitor OGC Web Services uptime and availability. GeoHealthCheck is able to send notification emails in case of issues. A live demo was shown during the meeting.

• Actions: All thematic portals have to submit the email of the persons to be notified in case of downtime.
• Action: VLIZ, with support of Secretariat, will set up the GeoHealthCheck application as a service of the Central portal.

EMODnet data providers Web catalogue:

EMODnet data store: Portable shopping applications

Secretariat: EMODnet data store proof of concept

Pascal Derycke presented a proof of concept for an EMODnet data store. It will help users find, view and provide access to all EMODnet data products in one-go using a one-stop shopping platform. The idea is built on the experience gathered in the European Atlas of the Seas project, which managed to build a successful extensive geodata catalogue from a wide variety of data providers using Machine2Machine communication. The data store would go even further and would provide an integrated mapping application for experts. Data will be central and linked to added value services and information (i.e: use cases, code to process the acquired data, data policy, data providers,…). The aim of the data store is to secure a user friendly access to the data and turn visitors into users with a profiling strategy.

The concept was debated by the different TWG members. A question was raised on how this Data Store would be different from the Query tool and if these services could be combined. There was confusion for the Chemistry and Bathymetry technical coordinator about the concept of a Data store. In order to improve following TWG discussions, a white paper of the Data Store could be developed, circulated and reviewed by TWG members in advance of the next meeting so that the discussions can be more constructive.

• Action: the EMODnet Secretariat will develop a white paper of the Data Store concept

More EMODnet data products (Use of DIVA)

EMODnet needs more data products and a discussion follows on whether Data-Interpolating Variational Analysis (DIVA) can be used to create additional EMODnet data products. DIVA allows realistic interpolation based on physics of ocean boundaries & circulation. Chemistry and Biology are already using this tool to create data products and Physics is starting to work with Charles Troupin (EMODnet Chemistry) for doing DIVA Interpolations. This interpolation is not suited for Bathymetry data.

• Action: compile information on data products vs interpolation methods vs data product levels.

Support of time/depth dimension in Web services

Support of the WCS time dimension is required towards the future, in particular for the vessels density maps, as the number of raster layers for each month/vessel type is growing rapidly and it is not user friendly to host them as separate services. Pascal Deryckeinvestigated the possibility for geoserver to support the time for raster time series and reported in the JIRA (https://webgate.ec.europa.eu/CITnet/jira/browse/EMODNET-6)

• Action: Implement time series dimension in Geoserver using the vessel density data (Human Activities) as a test case https://webgate.ec.europa.eu/CITnet/jira/browse/EMODNET-6

Quality Control/Quality Assurance for data products – how to inform on missing data?

Bilbomatica, have a tool identify where data is missing (ref: Corda project). Important to inform the user (e.g. in Human activities)

Technical support & other updates

Blue Cloud

Trust-IT: BlueCloud

The Blue cloud project aims to connect several data infrastructure (Copenicus, EMODnet, ..) and provide services to search and access data. It will also provide cloud computing facilities in a virtual research environment. A pilot with 5 demonstrators will be launched which should indicate that it is possible to combine data from different infrastructures to derive knowledge from data.

MarineID

Dick Schaap: MarineID

Marine ID is an authentication, authorisation, and accounting (AAA) service which can be used to keep track of what users are doing with the data. This will be applied in the blue cloud and is already implemented in some of the EMODnet portals (Chemistry, Bathymetry).

• Action: Thematic portals to implement the MarineID authentication system in the course of the Blue cloud project

EMODnet Open Sea Lab II Hackathon

Successful edition with 80+ participants and 28 coaches.

Presentation of hackathon package

The hackathon package is presented to the TWG. It is an important document to make the OSL event successful as it includes the Web service documentation for EMODnet and R/Python tutorials on how to search and access EMODnet data using OGC Web services. The examples show how the data can be accessed, processed and displayed. The hackathon package also includes the list of the coaches with their specific skills, the list and locations of the workshops, a survey on EMODnet…

Role of the coaches

Pascal Derycke explains the role of the coaches during the Open Sea Lab hackathon. The coaches will provide a “close” support to the teams by providing info on how to search, access and process EMODnet data for their application. All coaches will receive an OSL t-shirt to identify them as a coach. There is a coaches room in the venue where participants can come and ask for support. After each day there will be a debrief meeting with discussion between the coaches on how the teams are performing and how we can improve our support. A Whatsapp group will be set up to improve communication between coaches.

Wrap up, actions, next meeting

There was a question if Marine Copernicus would be invited to the next TWG. There is already a dialogue with Copernicus. A common documentation of Copernicus and EMODnet Web services has been published for Open Sea Lab. EMODnet and Copernicus have coordination meetings. It would be an idea to exchange on state-of-the-art and maintenance of the Web services.

• Action: TWG members were asked to provide idea’s for the next venue to the EMODnet Secretariat
• Action: EMODnet Secretariat to organize and circulate date and venue of next TWG.

Annex I - Agenda

Tuesday 3 September 2019

13:00-13:15 - Welcome & introduction (Jan-Bart Calewaert)

• Welcome to participants 5’
• Practical arrangements 5’
• Tour de Table 5’

13:15-13:45 - Progress & objectives (Jan-Bart Calewaert)

• Updates since last meetings (revision of action points from previous meetings) (Paula Oset) 20’
• Expected outcomes of this meeting (Pascal Derycke) 10’

13:45-14:10 – Updated Terms of Reference (ToR)

• Walkthrough of ToR updates 15’ (Jan-Bart Calewaert)
• Reporting of EMODnet CP technical monthly meetings between VLIZ & EMODnet Secretariat 10’ (Pascal Derycke)

14:10-14:55 – Review of EMODnet priorities

• Web services documentation, harmonization, INSPIRE compliancy and updates 10’ (Pascal Derycke)
• EMODnet central Geonetwork harvest by EU Open Data Portal 5’ (Pascal Derycke)

• EMODnet QueryTool 2.0 5’ (Paula Oset)
• HTTPS on thematic portals – EMODnet portal status: EU websites? 5’ (Bart Vanhoorne)
• Centralized/automatized monitoring Web metrics on thematic portals (using MATOMO API) 10’ (TrustIT)
• Monitoring usage of web services 10’ (Tim Collart)

14:55-15:15 – Coffee Break

15:15-15:45 – Future EMODnet priorities “EMODnet as a service provider” (Pascal Derycke)

• Centralized Helpdesk (JIRA service desk)

• Monitoring SDIs / Stress-test (GeoHealthCheck)

• EMODnet data providers Web catalogue
• EMODnet data store
• More EMODnet data products (Use of DIVA)
• Support of time/depth dimension in Web services
• Quality Control/Quality Assurance for data products – how to inform on missing data?

15:45-16:15 – Technical support & other updates

• Specific questions & technical issues 10’
• Other updates 10’
• MarineID / Blue Cloud 10’ (Dick Schaap)

16:15-17:15 - EMODnet Open Sea Lab II Hackathon (Pascal Derycke)

• Presentation of hackathon package (instruction for coaches) 30’
• Role of the coaches 10’
• Participation of Marine Copernicus, ICES, OVH, Imec 20’

17:15-17:30 - Wrap up, actions, next meeting (Secretariat)